Here is a benchmark:

#!/usr/bin/perl

use strict;
use warnings;
use Benchmark qw /timethese cmpthese/;

my $file = "/usr/share/dict/words";

chomp (our @large_wordlist = `cat $file`);

our ($ngram_f, $ngram_a, $ngram_y);

cmpthese -60 => {
    fatvamp  => '$ngram_f  = {};
                 for my $w (@large_wordlist) {
                     $word = "_" . $w . "_";
                     my $len  = length($word);
                     my $flen = $len;
                     for (my $i = 0; $i < $flen; $i ++) {
                         $$ngram_f {substr ($word, $i, 5)} ++ if $len 
+> 4;
                         $$ngram_f {substr ($word, $i, 4)} ++ if $len 
+> 3;
                         $$ngram_f {substr ($word, $i, 3)} ++ if $len 
+> 2;
                         $$ngram_f {substr ($word, $i, 2)} ++ if $len 
+> 1;
                         $$ngram_f {substr ($word, $i, 1)} ++;
                         $len --;
                     }
                 }',
    abigail  => '$ngram_a = {};
                 "_${_}_" =~ /(.{1,5})(?{$$ngram_a {$^N} ++})(?!)/
                            for @large_wordlist;',
    ysth     => '$ngram_y = {};
                 for my $word (@large_wordlist) {
                     ++ $$ngram_y {$_} for grep defined,
                     "_${word}_" =~ /(?=(((((.).).).).))|(?=((((.).).)
+.))|
                                       (?=(((.).).))|(?=((.).))|(?=(.)
+)/gx;
                 }',
};


my @keys_f = sort keys %$ngram_f;
my @keys_a = sort keys %$ngram_a;
my @keys_y = sort keys %$ngram_y;

die "Wrong keys\n" unless "@keys_f" eq "@keys_a" &&
                          "@keys_f" eq "@keys_y";
    
my @values_f = @$ngram_f {@keys_f};
my @values_a = @$ngram_a {@keys_a};
my @values_y = @$ngram_y {@keys_y};
                     
die "Wrong values\n" unless "@values_f" eq "@values_a" &&
                            "@values_f" eq "@values_y";
__END__
        s/iter    ysth abigail fatvamp
ysth      7.08      --    -48%    -62%
abigail   3.70     92%      --    -28%
fatvamp   2.68    164%     38%      --
[download]

Abigail

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.