Here's what I did when I got bitten by segfaults in an XS module I'm writing:

First, compile perl with -DDEBUGGING *). Then recompile the XS module (which will then also have DEBUGGING support).

Run valgrind perl program-that-crashes.pl

With a bit of luck, you'll get pretty precise indications of where the problem lies.

get valgrind here if it's not available in your system's package repository.

*) This flag is automatically set if you include -g option when Configure asks you about optimizer/debugger flags.