http://qs1969.pair.com?node_id=282723

atlantageek has asked for the wisdom of the Perl Monks concerning the following question:

I am looking for information on data analysis applications that use perl. Not just parsing text files but actually doing heavy math. I am writing an article about data analysis in Perl and am wanting to add some more diverse examples. Either send me an email to tj@atlantageek.com or reply to this node. Your help is greatly appreciated.
----
I always wanted to be somebody... I guess I should have been more specific.

Replies are listed 'Best First'.
Re: Data Analysis
by Zaxo (Archbishop) on Aug 11, 2003 at 04:20 UTC

    The suggested search will miss the great PDL distribution (PDL is the acronym for Perl Data Language). It is a fine general-purpose module for mathematical data handling. It is fast and handles arrays of data easily.

    After Compline,
    Zaxo

      Agreed! An example of the very heavy maths processing possible with PDL is Photonic, a PDL-using module used for real physics research into metamaterials (materials made from mixed substances). Specifically, and not surprisingly given the module name, their optical properties.
Re: Data Analysis
by simonm (Vicar) on Aug 11, 2003 at 03:28 UTC
    A good first place to look would be the various Math and Statistics modules on CPAN.

    Update: For those of you keeping score at home, that's well over two hundred matching CPAN modules, in over a hundred distributions... Perhaps someone has assembled a good introductory tutorial somewhere? I spent a few minutes SuperSearching without finding anything local..

Re: Data Analysis
by waswas-fng (Curate) on Aug 11, 2003 at 04:03 UTC
    I don't know of very many projects that do "heavy math" in perl. usually high end math problems run in very long times, forcing peop,e to over optimize the application that is solving the problem. You tend to see many long running math apps as C with a lot of inline asm to optimize the time the program takes. For instance I don't think you could even come close to many of the C/ASM PI calculators out there in perl (even if you are using XS/inline C. I would think that there are many proof of algo consepts done in perl -- but most are converted to optimized C/asm for production runs.

    -Waswas
Re: Data Analysis
by diskcrash (Hermit) on Aug 11, 2003 at 08:48 UTC
    Dear atlantageek,

    Perhaps what is more important is the wide range of apps that Perl can be applied to, at really low cost. I take radiological data from a Geiger Counter in Perl. Once the data is in the clutches of the Perl script, really anything is possible. Some of my recent code simplifies downstream processing by taking a simple sum of three data points, which happens to produce data samples that correspond to micro Roentgens per hour. The script then inserts the summed values into a MySQL databse. Other code then reads the data from the table. You can do any type of algebra in between, but more critically you can move the data around, reformat it, do side calculations, and then FTP it or make HTML or PNG's from it. Matlab wants $3,500 US for their "web page formatter". There is no easy to use FTP module for Matlab, either. (You can exec out etc., but this is not very pure.)

    In CPAN there are wonderful statistical modules, Fast Fourier Transforms, and even Neural Networks, for serious data grinders. Look in the Math section and stand back, cause you might get hurt with all of the output.

    Good luck!

    Diskcrash

A reply falls below the community's threshold of quality. You may see it by logging in.