Looking at the XS, there seems to be very little code between your perl and the C library. About the only way you might speed that up is to write some more XS that took an array of sets of parameters and called the underlying C function in a tight C-loop accumulating the results before passing them back to Perl.

But for that to be beneficial you would have to be calling the same function many (many) times with different parameters sets. And the logic of your program would have to lend itself to it. Eg. You would have to know the parameters for many calls at the same time (as opposed to subsequent sets of parameters being derived from previous results sets).

Which function are you calling and how many times that this is the bottleneck?