I can't really see anyone else being able to use this. The original implementation is only faster because I know that I'll be comparing the same vectors many times, so I can precompute the expensive things (like the z scores and the standard divs). It's only useful in a small number of cases and is a trivial thing to implement.

As for the C stuff, I don't know anything about XS (though it seems like it's not easy) so I couldn't really provide a Perl interface to it. What I am doing is preprocessing the data into an easily digestible format in Perl then just packing it and piping it to the script I posted; the script will also need to collect the results, do a bunch of sorting - I only need the top couple hundred values for each vector - and probably dump them to a file that a perl script then will load into a database.

So, I minimize the time I spend in C and still remove the huge (quite an understatement there) bottleneck. I'm afraid I don't have the time or the knowhow to package this for external consumption.