Re^2: Splitting strings into words when there are no separators

in reply to Re: Splitting strings into words when there are no separators
in thread Splitting strings into words when there are no separators

This treatment compares each entry in the dictionary to the data and stores matching positions. All of the possible matches are then reconstructed in the printwords function.

The words are read from the dictionary and compared without needing to be stored in memory.

#! /usr/bin/perl
use strict;
use warnings;

my $data = lc "howdoesonehandlethissensibly";
#my $data ='thesuperuserwascalledderekuser';

my %foundwords;
my $position;
while (<>) {
    chomp;
    next unless length>2; # remove 1 and 2 letter words
    $position = 0;
    while (index($data,lc $_, $position) >= 0) {
        my $found = index($data, lc $_, $position);
        push @{$foundwords{$found}}, lc $_;
        $position = $found + 1;
    }
}

# Print the word combinations
printwords('', -1);
sub printwords
{
    my $string = shift;
    my $position = shift;
    
    while (++$position < length $data) {
        if (exists $foundwords{$position}) {
            foreach my $word (sort {length $a <=> length $b} @{$foundw
+ords{$position}}) {
                printwords (join(' ', $string, $word), $position + len
+gth($word) -1);
            }
            last;
        }
    }
    print "$string\n" if $position == length $data;
}
[download]

In Section Seekers of Perl Wisdom