I use this code to read my dictionary:

You are using far more (double maybe even triple the memory requirement) because of the way you are returning the data from your subroutine.

It may not be enough to relieve your out-of-memory situation, but try this before you seek other more complex and inevitably slower solutions:

sub read_dict{
    my $file = shift;    
    my %dict;
    open( my $fh, "<:encoding(utf5)", $file );
    while( <FILE> ) {
       chomp; ## no need to chomp twice
       my ($p1, $p2) = split /\t/;
       push( @{ $dict{ $p1 } }, $p2 );
    }
    close $fh;
    return \%dict; ## main space saving change; return a ref to the ha
+sh
}

...
my $dict = read_dict( $dict_name );

...

for my $next_phrase ( @{ $dict->{ $key } } ){
    ...
}
[download]