thekestrel has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,
I have a module I've written which can be quite computationally expensive depending on how complex the operations used are. You can obviously get speed improvements by working with straight compiled C (depending on what you are doing, assuming same algorithms etc.), but I would like to explore speeding my code by using Perl/C options to see if I can't get it to run like a scared cat in heat (*). I'm not wanting to focus in this post about optimizing a specific routine, but the best information if one wishes to proceed with using Perl/C. From a brief bit of investigating the 3 choices appear to be..

1) XS
2) Inline C
3) SWIG


My question(s) are which is the preferred method and why? Inline C appears to be the 'quick' approach if you've absolutely gotta have some other language code involved and just want to slap it straight into the script. XS appears to require 'a lot' more setup and background reading, but from what I can gather is the way lots of modules seem to be heading.
What do you, the experienced perl community view as the pivotal documents (books, FAQs, posts) on these subjects? I couldn't find an o'reilly animal book on the subject (I really enjoy their format and have about 10 of them).
Basically, I would appreciate any comments on transitioning from straight perl to Perl/C implementation.

Regards Paul

(*) No, I don't have any data to show that this is faster than a normal cat, but I'm postulating that there would be some degree of haste involved =P.

Replies are listed 'Best First'.
Re: Perl & C
by Limbic~Region (Chancellor) on Mar 28, 2005 at 18:58 UTC
[reply]
Re: Perl & C
by dragonchild (Archbishop) on Mar 28, 2005 at 18:59 UTC
    XS is the way you hooked from Perl to C from the beginning. Inline::C is a relatively new way to wrap around XS - it actually generates, then compiles the appropriate XS file(s).

    I would feel more comfortable if my tools used hand-rolled XS as opposed to Inline::C. However, that's a preference with nothing to back it. Of course, Inline::C is an Ingy-module. I've never broken an Ingy-module. They are way cool ... sometimes too cool - kinda like Damian-code. You'll never understand it and it should never break. But, if it does ... you're screwed.

    Now, Inline::C can do some really neat things, like compile C expressions at runtime, kinda like an eval for C. There's penalties, but I don't think they're insurmountable.

    Being right, does not endow the right to be rude; politeness costs nothing.
    Being unknowing, is not the same as being stupid.
    Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
    Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

[reply]
Re: Perl & C
by samtregar (Abbot) on Mar 28, 2005 at 22:59 UTC
    I think you'll find that Inline::C is the best tool for this job. XS can be marginally easier if you have a large API to wrap, but the learning curve is steeper. I'd reserve SWIG for cases where you want to wrap a large C library for use in more than one scripting language.

    You might be interested in checking out my book, Writing Perl Modules for CPAN. I wrote a chapter each about the Perl C API, XS and Inline::C. You can even download a free copy!

    -sam

[reply]
      Sam,
      Thanks for the Links, that is exactly the sort of thing I was after, a good ease into both Inline and XS so I can then take on all the man pages and have an idea what they're on about.

      =)



      Regards Paul.
[reply]
Re: Perl & C
by ambs (Pilgrim) on Mar 28, 2005 at 18:56 UTC
    I would suggest you to go into XS or Inline C. Just remember that Inline C is XS anyway. It is just an easier way to write them.

    If you go for XS, h2xs is a good start point. Don't be afraid. It seems harder than it really is.

    Alberto Simões

[reply]
Re: Perl & C
by Roy Johnson (Monsignor) on Mar 28, 2005 at 20:11 UTC
    Depending on what you're doing, PDL is another option that might scare your cat.
    Caution: Contents may have been coded under pressure.
[reply]
Re: Perl & C
by jsegal (Friar) on Mar 29, 2005 at 17:58 UTC
    Like others above, I would recommend Inline::C as an "easy" way to do XS. There are some tricks you can even do to let vanilla perl handle your memory management -- see for example the snippet at Pattern for Inline-C-based crunching. That snippet talks about using a packed perl string as the data source to pass data in to a C function, but the exact same technique could be used to get data out (if you know how big the result will be).

    For example, if you knew you were passing in an array of 15 doubles and were getting back an array of 15 doubles (with the obvious extension if the sizes were not the same), you could do: 

    $inval = pack("d*",1,2,3,4,5,6,7,8,9,10,11,12,13,14,15)
$results = pack("d*",(0)x15)
$summary_result = c_function($inval,$results,15)
@real_results = unpack("d*",$results)

use Inline C => <<'END_OF_C_CODE';

double c_function(char * the_data, char * results,int nelems) {
    double *real_data = (double *) the_data;
    double *real_results = (double *) results;
    double summary_result;
    /*  do your fast calculation here, storing elements in *real_resul
+ts  */
    return summary_result;
}
END_OF_C_CODE

    [download]


    --JAS
[reply]
[d/l]

Back to Seekers of Perl Wisdom