GDB journey into perl internals

cosimo has asked for the wisdom of the Perl Monks concerning the following question:

Recently I've been working on a proprietary XS extension to speed up HTML generation from data structures in a web framework.
The XS code pretty much works and performs from 2 to 50 times better than the same implementation in Perl.

But... There's a strange test case that happens to segfault badly with an error similar to this:

Attempt to free unreferenced scalar: SV 0x01234567 at ... line xyz
[download]

Then I compiled a debugging perl. I wanted to use gdb to try to understand what's wrong. And I did. This was the result.

(gdb) b XS_MyMod_myfunc
Breakpoint 2 at 0x2a97e2fdc0: file MyMod.c, line xxx.
(gdb) ...

Program received signal SIGSEGV, Segmentation fault.
Perl_sv_unref_flags (sv=0x833a80, flags=1) at sv.c:7972
7972        if (SvREFCNT(rv) != 1 || (flags & SV_IMMEDIATE_UNREF))
(gdb) backtrace
#0  Perl_sv_unref_flags (sv=0x833a80, flags=1) at sv.c:7972
#1  0x00000000004ae782 in Perl_leave_scope (base=47) at scope.c:925
#2  0x00000000004b407e in Perl_pp_return () at pp_ctl.c:1912
#3  0x000000000048659e in Perl_runops_standard () at run.c:38
#4  0x000000000043ca28 in perl_run (my_perl=Variable "my_perl" is not 
+available.
) at perl.c:1934
#5  0x000000000041c66d in main (argc=3, argv=0x7fbffff868, env=Variabl
+e "env" is not available.
) at perlmain.c:98
(gdb)
[download]

Now, I'm definitely not a gdb expert. Any idea on how to get some more information? Now I'm going to try breakpointing and single-stepping. I suspect the problem is way before. Probably in some SV which is sv_2mortal()-ized more than necessary. When sub finally returns, the SV gets freed but it doesn't need to.

The strange thing here is that this only happens on a single x86-64 system with perl 5.8.6, while every other system I've tested the code on, i386, x86-64, with perl-5.8.6 or newer, it works perfectly...

Comment on GDB journey into perl internals Select or Download Code

Replies are listed 'Best First'.
Re: GDB journey into perl internals by Joost (Canon) on Mar 17, 2008 at 00:29 UTC
I'm not a GDB expert either, and I have only a general knowledge of the perl internals, so what I do to catch these errors is to run the code through valgrind with increasing levels of verbosity until I get to the point where the errors start to make sense. (As usual, it really helps here if your code is split up in fairly small functions) I usually don't go stepping through the code, since that takes a long time and is generally boring. I just throw in a lot of assert()s and log/warn marcros in my XS code on the suspicious parts. That way I can just compile with NDEBUG or not depending on if I'm developing or compiling production code. "What should it profit a man, if he should win a flame war, yet lose his cool?"	[reply]
Re: GDB journey into perl internals by BrowserUk (Patriarch) on Mar 17, 2008 at 00:49 UTC
I've never used GDB. Does it allow you to place watch statements on breakpoints? If so, and if the address of the unreference scalar is consistant, then put a breakpoint on the return statement from malloc with a watch looking for the breaking address. That should allow you to backtrack to the routine that allocates the SV. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^2: GDB journey into perl internals by cosimo (Hermit) on Mar 17, 2008 at 07:51 UTC
Good idea! That remembers me my soft-ice sessions of many years ago :-) I just don't know how to do that in `gdb`. Of course, I should RTFM. Maybe I should try `ddd` once again...	[reply] [d/l] [select]
Re: GDB journey into perl internals by ForgotPasswordAgain (Vicar) on Mar 17, 2008 at 14:23 UTC
Probably in some SV which is sv_2mortal()-ized more than necessary. For what it's worth, I think that is very likely the cause though I don't know why. (See the note in http://search.cpan.org/src/SLANNING/Ogre-0.35/Ogre.xs (search for mortalized).) So what did you do to speed things up so much?	[reply]
Re^2: GDB journey into perl internals by cosimo (Hermit) on Mar 18, 2008 at 20:26 UTC
About the speed up: this is a custom module in which I reimplemented already existing Perl code in C/XS. The main task of this code is to "serialize" to string whatever you throw at it, with custom rules. For instance, strings are kept as strings, arrays are just joined, hashes are output as `HASH(0x...)` unless they have special keys, and objects are serialized with their `toString()` method, if they have one... All these conversions are pushing CPUs. I made a quick prototype where I saw 100% speedups. So I decided to go for it and here it is... For the curious, this is part of a proprietary large scale web framework that automatically generates themed HTML/CSS code from high-level "GUI" widgets/controls implemented as Perl classes.	[reply] [d/l] [select]
Re: GDB journey into perl internals by jeroenes (Priest) on Mar 18, 2008 at 03:14 UTC
This thread reminded me of an old node about debugging and the use of debuggers. And I can only dream of a shred of the skills of some mo nk s in that thread. About your problem: perhaps it has little to do with your code or the library you are working with. Are you interested in the system you find the segfault with, i.e. is it important to you? Perhaps it is a heavily patched, manual installation as opposed to a freshly installed popular distribution. Perhaps you cannot even repeat the segfault with the same distribution on slightly different hardware. Hope this helps, Jeroen "We are not alone"(FZ)	[reply]
Re: GDB journey into perl internals by locked_user sundialsvc4 (Abbot) on Mar 19, 2008 at 13:12 UTC
As they said in the movie, “Zed, we've got a bug!” Simple as that. Thank goodness you found it. What you're basically looking at is a “double free,” and it is occurring when Perl is leaving the scope of a function or block. The target variable, which is at a known location in your Perl program (“line xyz”) must be the culprit. The error, though it may be difficult to find, is legitimate. It won't go away. Look carefully at your XS code and see how you're handling memory and reference-counting within your own code. Maybe insert some debugging statements when you release memory. If you're using a compiler that provides its own garbage collection, make sure you know how that works. Most importantly of all, when you release anything, always* explicitly set every pointer to that thing to `NULL`* and do it in the very next statement! (Failure to do that, albeit in vendor-supplied code, once cost me $10,000.00 in real money, but that's another story.) The problem isn't in “Perl internals.” It is, I'm sorry to say, entirely and directly in your code. Definitely a b**ch to find...