I do not understand how $s =~ /^$re2$/; can match from the beginning to the end for my $s = 'couldsomeonerecommendaworkingperlmoduletosplitconcatenatedwords'; and ## $re2 = (recommend|someone|working|module|words|split|comme|could|...

If that's what $re2 looks like, then you are not running the code I posted!

It should look like:

(concatenated|concatenate|catenated|recommend|catenate|commend|someone
+|working|catena|module|could|enate|split|words|perl|cate|king|mend|so
+me|word|work|ate|cat|con|daw|end|eon|ere|kin|let|lit|men|mod|one|per|
+rec|som|ted|ten|at|aw|ed|en|er|et|in|it|ki|li|me|mm|mo|na|ne|od|om|on
+|or|os|pe|re|so|to|wo|a)?
... repeat 11 (or more) times.
[download]

The way the code:

#! perl -slw
use strict;

my @w = do{ local @ARGV = 'words.txt'; <> };
chomp @w;

my $s = 'couldsomeonerecommendaworkingperlmoduletosplitconcatenatedwor
+ds';
my @subset = grep{ $s =~ /$_/ } 'a', 'perl', @w;
my $re1 = join '|', sort{ length( $b ) <=> length( $a ) }@subset;
my $re2 = "($re1)?" x 11;

print for grep defined(), $s =~ /^$re2$/;
[download]

Works is:

1. First it loads the dictionary into @w.
2. Then it filters that array against the input string with grep, to produce @subset

   This is the subset of all words in the dictionary that appear somewhere in the input string.

3. It then build $re1, which is an alternation of all those words, longest first.

   So $re1 looks like this: longestword|longword|shorter|short

4. It then builds $re2 which looks like this:

   (longestword|longword|shorter|short)?(longestword|longword|shorter|sho
   +rt)?(longestword|longword|shorter|short)?...
   [download]

   The effect is that (provided every word in your input is spelt correctly, and appears in the dictionary), is that it will match each longest word in turn.

   Because the repeated regex is conditional ($re1)?, if the number of repeated elements is longer than the number of words in the string, it will just stop matching when it reaches the end of the string, and the redundant captures will return the null string.

   Hence the grep defined().

The results will rarely be perfect, but it depending upon the source of your strings, it might form the basis for further, perhaps statistical, analysis.

I'm curious as hell about the source of the data and the purpose of the exercise?

It actually works. I created a dictionary of 50000+ words sorted by popularity. The problem was that I ran your example but I did not have "concatenated" in there.
Thanks for your explanation. I will spend some time to understand this part

print for grep defined(), $s =~ /^$re2$/;
[download]

in details. Looks like genius. In terms of both, Perl flexibility and your implementation. Actually it needs

print "$_\n" for grep defined(), $s =~ /^$re2$/;
[download]

to print line-wise.

I'm curious as hell about the source of the data and the purpose of the exercise?

The dictionary is the side affect of phrases I retrieved from crawling for my applications. The purpose of the exercise is the following. I created a keyword generator which is a web application. From the logs I found that some people merge words in a seed phrase and my resulted keywords filter fails to establish similarity in these cases. So I need to split. But usually it is only a two reasonable words split so that looks like your algorithm with my dictionary works well.

I can send you a link to the tool, but I am not sure it's a right way to do it through the forum.

I will spend some time to understand this part print for grep defined(), $s =~ /^$re2$/;

Maybe this will help:

#! perl -slw
use strict;

my @w = do{ local @ARGV = 'words.txt'; <> };
chomp @w;

my $s = 'couldsomeonerecommendaworkingperlmoduletosplitconcatenatedwor
+ds';

my @subset = grep{ $s =~ /$_/ } 'a', 'perl', @w;

my $re1 = join '|', sort{ length( $b ) <=> length( $a ) }@subset;
my $re2 = "($re1)?" x 20;

my @found = $s =~ /^$re2$/;

print join '|', @found;

my @undefsRemoved = grep defined(), @found;

print for @undefsRemoved;
[download]

Note: I've increase the number of repetitions from 11 to 20 to allow for sentences with more words. The side effect of that is that the unmatched captures now return undef, so If I just printed out @found, I get this:

c:\test>junk
Use of uninitialized value $found[12] in join or string at C:\test\jun
+k.pl line 15, <> line 178691.
Use of uninitialized value $found[13] in join or string at C:\test\jun
+k.pl line 15, <> line 178691.
Use of uninitialized value $found[14] in join or string at C:\test\jun
+k.pl line 15, <> line 178691.
Use of uninitialized value $found[15] in join or string at C:\test\jun
+k.pl line 15, <> line 178691.
Use of uninitialized value $found[16] in join or string at C:\test\jun
+k.pl line 15, <> line 178691.
Use of uninitialized value $found[17] in join or string at C:\test\jun
+k.pl line 15, <> line 178691.
Use of uninitialized value $found[18] in join or string at C:\test\jun
+k.pl line 15, <> line 178691.
Use of uninitialized value $found[19] in join or string at C:\test\jun
+k.pl line 15, <> line 178691.
could|someone|recommend|aw|or|king|perl|module|to|split|concatenated|w
+ords||||||||
[download]

So I use grep to remove any undefined values:

my @undefsRemoved = grep defined(), @found;
[download]

Actually it needs print "$_\n" for grep defined(), $s =~ /^$re2$/; to print line-wise.

If you look at the first line of the code I posted: #! perl -slw, the l in the -slw has the affect of adding "\n" to print statements automatically. Saves me having to type them all the time.

I can send you a link to the tool, but I am not sure it's a right way to do it through the forum.

Just post the link as text, or wrap it in brackets: [ link here ] to make it an active link.

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

The start of some sanity?