in reply to Mysterious slow down with large data set

You can shed another chunk of time by norm()'ing your vectors once as you load them, rather than repeatedly:

while(<>){
    chomp;
    my ($wrd, @data) = split;
    $kernel{$wrd} = norm( pdl(@data) );
...
        my $sim = inner( $kernel{ $w1 }, $kernel{ $w2 } );

[download]

But the PDL -- now down to just the inner()s -- still exhibits the slowdown behaviour, despite that it doesn't do so when run in a tight loop on two similar vectors in a standalone test.

At this point you need some serious PDL knowledge to move forward I think.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

The start of some sanity?

In Section Seekers of Perl Wisdom