Building a better perl math library?

Obviously, math has never been perl's strong point, though it is quite capable of doing just the same mathematics calculations as other languages; however it cannot benefit from the high optimizations typically assoicated with, say, FORTRAN. There's been a few modules in the Math:: tree of benefit, but I noticed several lacking areas, such as advanced matrix manipulation, optimization, etc. For example, I needed a matrix decomposition routine that would have been great to have had in perl since I was smoozing data from other sources already in perl, but instead I had to import the data to another program to get results in a timely manner.

I've been thinking of possibly writing a ground-up mathematics library for perl piece by piece, but with two specific goals in mind:

First, the library should be as drop-in as possible. Need a fast way to multiple two matrices together? You should only need to use one module, and you should be all set. With perl, this isn't a large concern, but it is important later.
Second, a bit more trickier, each function of the library should have a native perl codebase as well as a version that uses library hooks to use pre-compiled routines. For example, a matrix multiplication library "Math2::MatrixMultiply" may have subclasses "Math2::MatrixMultiply::Native" and "Math2::MatrixMultiply::MATHLIB", where MATHLIB is one of the numerous free FORTRAN and C math that are around. This allows one that might start by using the native perl features to develop quick programs, but then easily switch to a compiled base to gain speed benefits if the program becomes heavily used. By default, if you only "use Math2::MatrixMultiply", you'd get the perl native version, thus keeping in step with item 1 above.

Certainly there's a lot to consider here, and I've only just prodded at the idea myself. But do people think there might be enough interest into developing this type of library?

-----------------------------------------------------
Dr. Michael K. Neylon - mneylon-pm@masemware.com || "You've left the lens cap of your mind on again, Pinky" - The Brain
It's not what you know, but knowing how to find it if you don't know that's important

Comment on Building a better perl math library?

Replies are listed 'Best First'.
Re: Building a better perl math library? by Zaxo (Archbishop) on Aug 25, 2001 at 00:33 UTC
Have you looked at the PDL:: hierarchy, particularly for matrix operations? After Compline, Zaxo	[reply]
Re: Building a better perl math library? by ChemBoy (Priest) on Aug 26, 2001 at 10:20 UTC
I don't know precisely what issues you're facing in terms of number and size of matrices (and hence, what your definition of a timely manner is), but I have had reasonable success with small-scale matrix operations using Math::MatrixReal (where by small-scale I mean inversion and multiplication of a relatively small matrix). Certainly, if you're planning on expanding that specific area you should contact Steffen Beyer and see what if anything he has planned there. In the more general sense, I think this is a great idea, in particular with reference to the idea of having a base module in pure Perl with an optional FORTRAN/C/X backend. If God had meant us to fly, he would never have given us the railroads. --Michael Flanders	[reply]
Re: Re: Building a better perl math library? by Masem (Monsignor) on Aug 26, 2001 at 16:31 UTC
Well, specifically, I was looking at things like eigenvalue calculations, triangularization, etc. There's also things lacking like Intergration, Differentition, etc. that aren't necessarily matrix methods, but are important mathematical concepts. One of the concerns that I had with the possibility of back-ending the module with standard math libraries was that the way the data was stored in a Matrix/Vector class would be such that if one was using the math library backend, the data would be optimized to avoid having to do any conversion from a data structure stored in perl to what is needed by the external library. This would help speed up execution by avoiding this step when it's needed. ----------------------------------------------------- Dr. Michael K. Neylon - mneylon-pm@masemware.com \|\| "You've left the lens cap of your mind on again, Pinky" - The Brain It's not what you know, but knowing how to find it if you don't know that's important	[reply]
Re: Building a better perl math library? by VSarkiss (Monsignor) on Aug 27, 2001 at 19:00 UTC
You may want to consider symbolic versus numeric computation separately. Numeric is difficult to port to new hardware and compilers correctly (especially if the port is to specialized hardware) and may hinder future growth of your module(s). But symbolic would be more in Perl's line. You could either build an interface to something like Mathematica, or roll your own. Second, I'm not sure how big the audience would be for something like this. On the other hand, that never stopped anyone from building a cool new module!	[reply]