Re: OpenMP from Perl?

Does anyone know of an implementation of OpenMP with Perl?
Looking around, the only references I found for Perl & OpenMP are those relating to Perl's use in running the OpenMP test and validation suite--so the answer to your first question appears to be: No.
Could there be a Perl implementation the OpenMP API?
I'm talking at the Perl level, rather than within the compiler which I think is a non-starter.
On the basis of the examples I've seen, there would be little problem hand coding the types of constructs used.
Making these turn-on and offable (with apologies to EE), via say, the inclusion of a module is probably possible.
The module could use a source filter to expand/enact comment-embedded directives. Comment out the use statement, you have a single-threaded program. Uncomment the use statement and it becomes parallel.
Smart::Comments works in a similar way.
Would this be a useful thing to do?
It would depend upon what you were using it for, but my gut feel is no.
On the basis that all of the examples of using OpenMP I found, were for cpu intensive math. Usually on large, repetative data structures. And whilst Perl is fine for the odd bit of math here and there and can certainly handle the large data structures, it is never going to be good at performing intensively repetative math. Natively that is.
However, PDL is great at performing this kind of thing, and the speed ups obtained from switching to using piddles rather than Perl arrays for this type of work will far exceed any benefits that you could derive from threading Perl native math.
The next logical question is could OpenMP directives (at the Perl level) be used in conjunction with PDL and piddles to distribute piddle processing across multiple threads?
On the basis that piddles are opaque black boxes at the Perl level, I would have to say I think not. Hopefully, lin0 will breeze by and give an answer with more authority on that than I can. Ie, Some authority.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Comment on Re: OpenMP from Perl? Download Code

Replies are listed 'Best First'.
Re^2: OpenMP from Perl? by lin0 (Curate) on Jun 22, 2007 at 06:35 UTC
Hi BrowserUk, There is some work related to using OpenMP with piddles by Darin McGill, the author of PDL::Parallel::MPI. However, I am not aware of any concrete result yet. Cheers, lin0	[reply]
Re^3: OpenMP from Perl? by BrowserUk (Patriarch) on Jun 22, 2007 at 08:10 UTC
Thanks lin0, that looks interesting. The MPI standard appears to be distributed processes controlled through function calls and 'sharing' data via tcp. Whilst distinctly non-trivial to implement, in Perl's terms, calling functions that fork processes and pass chunks of data back and forth via sockets is at least 'known technology'. There is some mention of threads in MPI-2, but they (seem) to relate to whether the programs using an MPI implementation can be threaded, and whether they can call the library routines from more than one thread serially or concurrently. Ie. The work is done by forking processes, but those processes might be multi-threaded. Looking at the OpenMP standard, it appears (at a fairly cursory reading) to use threads and shared memory to do the work. It also specifies that the spawning of threads and the subdivision and sharing of data be done via directives embedded in comments. The idea appears to be that by disabling the directives, the same program shoudl be able to run to completion as a single threaded application. And by enabling them, it completes, but more quickly--assuming the availability of multiple cpus/cores. The directives bit is easily, if controversially, achievable using a source filter. The most problematic part is that the API assumes an implicitly-shared memory environment--unlike Perl's iThreads which are explicitly-shared only. The problem is that the directives are geared to indicating to the preprocessor which variables need to be explicitly protected and/or later accumulated. In Perl's explicitly shared environment, that makes the information provided by the programmer on the directives almost the exact opposite of what is required. All of that said, as piddles are opaque objects at the perl level may mean that they can be implicitly shared by threads? Also, the fact that piddle slices can be referenced in-place may make the implementation of the parallel for directive easier than it is for perl arrays. I guess the real question is whether the PDL libraries are thread-safe? Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^4: OpenMP from Perl? by lin0 (Curate) on Jun 22, 2007 at 13:33 UTC
Hi BrowserUk, There is a group of PDL libraries that are not thread-safe. I am referring to those associated with dimension manipulation as indicated in the PDL::Internals documentation: Certain types of transformations can be optimized very efficiently obviating the need for explicit readdata and writebackdata methods. Those transformations are called pdl_vaffine. Most dimension manipulating functions (e.g., slice, xchg) belong to this class. The basic trick is that parent and child of such a transformation work on the same (shared) block of data which they just choose to interpret differently (by dusing different dims, dimincs and offs on the same data, compare the pdl structure above). Each operation on a piddle sharing data with another one in this way is therefore automatically flown from child to parent and back -- after all they are reading and writing the same block of memory. This is currently not perl thread safe -- The same documentation goes on to indicate that "no big loss since the whole PDL core is not reentrant". Therefore, the PDL core would not be thread safe Cheers, lin0	[reply]