G'day sbrothy,
It sounds like you just need "split /\W+/, $string";
although, I don't know what the "C++ Boost" connection is.
$ perl -E 'say for split /\W+/, " THIS is a variable number of words,
+spaces and punctuation. "'
THIS
is
a
variable
number
of
words
spaces
and
punctuation
Is the string you showed representative of your data?
Do you have words with hyphens or apostrophes?
Are sentences with leading whitespace normal?
The simplicity of my solution may be invalid. Perhaps you need something closer to:
$ perl -E 'say for split /[ ,.]+/, " THIS is a variable number of word
+s, spaces and punctuation. " =~ s/^[ ,.]*//r'
THIS
is
a
variable
number
of
words
spaces
and
punctuation
|