Re^5: putting text into array word by word

OK, now that's solved, let's look at the definition of 'word' (yes, things are going to get hairy...). Take this sentence, for example:

"No, he said."

The 'words' that your current code would extract are:

"No,
he
said."
[download]

Is that really what you want? Or would you prefer:

No # or, better, 'no'
he
said
[download]

Comment on Re^5: putting text into array word by word Select or Download Code

Replies are listed 'Best First'.
Re^6: putting text into array word by word by jms53 (Monk) on Jan 09, 2012 at 23:19 UTC
during the foreach loop I remove punctuation and set to lowercase (such that No and no are the same word. `while (<FILE>) { my @these_words = split(' ', $_); foreach my $this_word (@these_words) { $this_word =~ s/[[:punct:]]//g; $this_word = lc($this_word); push @all_words, $this_word; } }` [download]	[reply] [d/l]
Re^7: putting text into array word by word by Not_a_Number (Prior) on Jan 10, 2012 at 10:35 UTC
`$this_word =~ s/[[:punct:]]//g;` The only problem with that approach is that it removes internal punctuation (ie apostrophes) as well, so that I'll becomes ill, she'd becomes shed, etc. ('Why was Virgina Woolf so obsessed with sheds?' I hear someone ask.) I'd use this instead: `$this_word =~ s/^[[:punct:]]+//; # Remove leading punct. $this_word =~ s/[[:punct:]]+$//; # Remove trailing punct.` [download] Update: added Virginia Woolf sentence.	[reply] [d/l] [select]
Re^8: putting text into array word by word by jms53 (Monk) on Jan 10, 2012 at 19:47 UTC
hadn't thought about that. Good catch, thanks!	[reply]
Re^9: putting text into array word by word by Not_a_Number (Prior) on Jan 10, 2012 at 20:02 UTC
Re^10: putting text into array word by word by jms53 (Monk) on Jan 10, 2012 at 20:14 UTC