If all the words are correctly spelt and are in your dictionary, then this appears to make a good attempt at many inputs:

#! perl -slw
use strict;

my @w = do{ local @ARGV = 'words.txt'; <> };
chomp @w;

my $s = 'couldsomeonerecommendaworkingperlmoduletosplitconcatenatedwor
+ds';

my @subset = grep{ $s =~ /$_/ } 'a', 'perl', @w;

my $re1 = join '|', sort{ length( $b ) <=> length( $a ) }@subset;
my $re2 = "($re1)?" x 11;

print for grep defined(), $s =~ /^$re2$/;

__END__
C:\test>junk
could
someone
recommend
a
working
perl
module
to
split
concatenated
words
[download]

But note: I had to add 'a' & 'perl' which don't appear in my dictionary; and I cheated by hardwiring the number of words (11) to look for.

If I change that to a larger number (say 100), then the results are less good:

C:\test>junk
could
someone
recommend
aw
or
king
perl
module
to
split
concatenated
words
[download]

However, if I removed the iffy non-word 'aw' from my rather permissive dictionary, it once again produces the right output, but that just goes to prove how sensitive and dependent the results would be on a good dictionary and correct spelling.

Thanks, great code. This is really something and I will probably take advantage of this.

However I would prefer to have some probabilistic approach so that no need in guessing between 11 or 100. There should be some NLP related module, I think.

However I would prefer to have some probabilistic approach so that no need in guessing between 11 or 100.

The value of 100 is just an arbitrary value that just needs to be large enough to encompass the number of words in the input.

I'm not sure you'd get much better results with NLP, but its been a long time since I did anything with that and the art moves on. Good luck.

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

The start of some sanity?

This is not working for me.
I do not understand how

$s =~ /^$re2$/;
[download]

can match from the beginning to the end for

my $s = 'couldsomeonerecommendaworkingperlmoduletosplitconcatenatedwor
+ds';
[download]

and

## $re2 = (recommend|someone|working|module|words|split|comme|could|..
+.
[download]

I do not understand how $s =~ /^$re2$/; can match from the beginning to the end for my $s = 'couldsomeonerecommendaworkingperlmoduletosplitconcatenatedwords'; and ## $re2 = (recommend|someone|working|module|words|split|comme|could|...

If that's what $re2 looks like, then you are not running the code I posted!

It should look like:

(concatenated|concatenate|catenated|recommend|catenate|commend|someone
+|working|catena|module|could|enate|split|words|perl|cate|king|mend|so
+me|word|work|ate|cat|con|daw|end|eon|ere|kin|let|lit|men|mod|one|per|
+rec|som|ted|ten|at|aw|ed|en|er|et|in|it|ki|li|me|mm|mo|na|ne|od|om|on
+|or|os|pe|re|so|to|wo|a)?
... repeat 11 (or more) times.
[download]

The way the code:

#! perl -slw
use strict;

my @w = do{ local @ARGV = 'words.txt'; <> };
chomp @w;

my $s = 'couldsomeonerecommendaworkingperlmoduletosplitconcatenatedwor
+ds';
my @subset = grep{ $s =~ /$_/ } 'a', 'perl', @w;
my $re1 = join '|', sort{ length( $b ) <=> length( $a ) }@subset;
my $re2 = "($re1)?" x 11;

print for grep defined(), $s =~ /^$re2$/;
[download]

Works is:

1. First it loads the dictionary into @w.
2. Then it filters that array against the input string with grep, to produce @subset

   This is the subset of all words in the dictionary that appear somewhere in the input string.

3. It then build $re1, which is an alternation of all those words, longest first.

   So $re1 looks like this: longestword|longword|shorter|short

4. It then builds $re2 which looks like this:

   (longestword|longword|shorter|short)?(longestword|longword|shorter|sho
   +rt)?(longestword|longword|shorter|short)?...
   [download]

   The effect is that (provided every word in your input is spelt correctly, and appears in the dictionary), is that it will match each longest word in turn.

   Because the repeated regex is conditional ($re1)?, if the number of repeated elements is longer than the number of words in the string, it will just stop matching when it reaches the end of the string, and the redundant captures will return the null string.

   Hence the grep defined().

The results will rarely be perfect, but it depending upon the source of your strings, it might form the basis for further, perhaps statistical, analysis.

I'm curious as hell about the source of the data and the purpose of the exercise?

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

The start of some sanity?

"Concatenated" how?

• simplyruntogether (using concat operator, .)?
• (loosely speaking) "concatenated" with spaces-or|other:punctuation between words
• something else?

And - - if the first - - what criteria do you wish to impose where a leading word might be a base word either with modifiers such as "ed" or "ate" or "ly" appended or where the base is followed by the beginning of a new word? eg:

fortuneatelysergicacid...

^*1 An unfortunate spelling error but it will suffice for an OTTOMH example

If you can come up with a list of all words that appear in compound words (not, I think, an easy job), here's one approach. I couldn't think of a triple- or quadruple-word compound to test with. Also, ambiguities in compounding, as exemplified in the reply of ww, are not addressed by this approach.

>perl -wMstrict -le
"my @elements = qw(stop sign light painter porch store front);
 my ($elem) =
   map qr{ $_ }xms,
   join q{ | },
   @elements
   ;
 my $plural = qr{ e? s }xms;
 my $compound = qr{ \b $elem{2,} (?= $plural? \b) }xms;
 ;;
 my $str = qq{a signpainter wanted the stopsigns in front \n}
         . qq{of all his storefronts` frontporches replaced \n}
         . qq{by stoplights to reduce accidents in front \n}
         . qq{of his stores}
         ;
 print qq{[$str]};
 ;;
 my @compounds = $str =~ m{ $compound }xmsg;
 printf qq{'$_' } for @compounds;
"
[a signpainter wanted the stopsigns in front
of all his storefronts` frontporches replaced
by stoplights to reduce accidents in front
of his stores]
'signpainter' 'stopsign' 'storefront' 'frontporch' 'stoplight'
[download]

Update: There must surely be (tens of?) thousands of words that are parts of compounds. The regex alternation formed from all these words will be very long. Even so, my guess is that the regex will compile and execute. OTOH, speed of execution is another matter.

Update: Correction: Regex  qr{ $_ \b }xms in original example code should have been  qr{ $_ }xms – fixed.

Update: Improved example code to recognize plurals – at least in English.

Trying to do it with a regex could be pretty time consuming if the dictionary or the subject text got very long. I worked on a project in a natural language translation class way longer ago than I care to think about, and the approach to making the dictionary was to make a linked tree where each letter was a node, with the possible subsequent letters as the words being child nodes. At the end of each complete word you put a flag node that says "end of word", but for a true compound word you'd have a child node with the next letter and another child with the "EOW" flag.

Kind of like this (where "." is end of word)

                 T
                 /\
                H  O
               /\  /\
              E  I . N
             /\  /\   \.
            N  .

etc.
[download]

This dictionary Includes "To","Ton","The","Then" and starts to spell out "this". It makes finding the combined words fast and straightforward, but it doesn't help with distinguishing true compound words (e.g. "bookkeeper") from things like "theme" which could be "the me"(updated here to correct my bad choice of example). If you're really clever you might use some sort of Markov chain tool to guess that.

But I don't know the NLP modules well enough to know if there's something kicking around in CPAN. If you have a dictionary to slurp, you could do it yourself fairly easily.

Update:You might even get by ok with something like Text::SpellChecker

Improved example code to recognize plurals – at least in English.

Weird... I added 'child' to your list but 'children' wasn't recognized. Then I tried 'goose' and still no luck. None with "hobby" either, I'm afraid.

. . .

Okay, I didn't really try them.

-sauoq
"My two cents aren't worth a dime.";

Not quite sure what you mean. Is 'stopsign' a compound word ('stop' and 'sign')?

yes, like stopsign or bookreader

What about words like "strawberry"?

What aboutest it :)

C:\test>junk
whataboutwordslikestrawberry
what
about
words
likest
raw
berry
Took 1.008725 seconds
[download]

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

The start of some sanity?

I did this using a trie a few years back. It wasn't rocket surgery and I believe it was tons faster than the regex solution (and it scored multiple solutions in order to more often avoid the common pitfalls already discussed). But Clinton paid me for it (and I'd have to find it on backup) so I'll wait for his permission before posting it.

But it also wouldn't be a lot of work to do over. See Re: find all paths of length n in a graph (Boggle solver) for an example of making and using a trie for a somewhat similar purpose.