in reply to stack trace from thread exit

To get a core dump when a warning is issued, set $SIG{__WARN__} to produce a core dump. For example:

use warnings;

my @x = qw(a b c);

$SIG{__WARN__} = sub { print "got a warning: $_[0]\n"; CORE::dump; };

for (0..3) {
    print "$x[$_]\n";
}

[download]

Remember to set your process limit to allow the core file to be produced (ulimit -c if you are using bash, see the man bash for details of ulimit).

Replies are listed 'Best First'.
Re^2: stack trace from thread exit
by locked_user sundialsvc4 (Abbot) on Oct 25, 2010 at 22:18 UTC

    So, ig, you’re saying that you find core-dumps to be a useful diagnostic technique with regard to Perl programs?   Interesting...   Could you elaborate on (or reference to) how you use them in this context?

      Could you elaborate on (or reference to) how you use them in this context?

      I am not sure which context you refer to by this context.

      With regard to Perl programs in general, I have rarely used gdb and, even less often, core dumps: to investigate perl initialization, lexing and parsing, to study and customize the regular expression engine and once to track down a bug in PerlIO that produced a core dump during initialization when attempting to read a UTF-16 encoded file. I don't recall any other use at the moment...

      Much more often, I use logging and test cases to investigate Perl programs and programs in general. I developed my debugging habits about the time Brian Kernighan wrote: The most effective debugging tool is still careful thought, coupled with judiciously placed print statements., and I remain very much in that camp. But there have been times when I have found debuggers and core dumps very helpful. For example, I once found a bug in the perl tokenizer that could not be isolated with test Perl programs and print statements, and tracing the execution was much more effective that merely reading the source in toke.c.

      I haven't done much with XS, but once outside the realm of the Perl interpreter gdb can be a helpful tool.

      When I do use them with Perl programs (gdb and core dumps) I do so as I would for any other program. There are various manuals and tutorials available on the net, including http://www.gnu.org/software/gdb/documentation/ and http://www.cs.cmu.edu/~gilpin/tutorial/. I always build perl from source with -g flag and generally investigate the issue by other means first. While it is possible to attach gdb to live programs, I very much prefer small test cases run in isolation.

      If by this context you mean the situation of the OP, I have not used gdb or core dumps to investigate crashing threaded Perl programs using possibly buggy XS modules. I would try other means to isolate the fault first, but might use gdb if core files were being produced, at least to get some idea where in the code the fault was occuring, if other approaches proved unsatisfactory.

      I was merely responding to the OP who said: I'd like to trap the fault in gdb. Given that specific objective and the warning, I thought a core dump produced in the context of the warning might be what the OP was wanting.

      The test program I provided produces a core dump, if resource limits permit (I am assuming *nix system). It is easily accessed in gdb with:

      gdb /usr/bin/perl core

      [download]

      (Substitute the path to the relevant instance of perl on your system.)

      The C stack is easily displayed with the bt command. Beyond that, one would have to know perl and the relevant Perl and XS programs in some detail to know where to look.

[reply]
[d/l]
[select]
        I recompiled perl with -Doptimize=-g, but I intentionally did not carry forward my hacks to stub sigaction() calls. With ig's $SIG{__WARN__} function I still get a trace that does not include the root cause. 
        Thread 2 (Thread 18154):
#0  0x00007f96d1155d57 in kill () from /lib/libc.so.6
#1  0x000000000046a9bb in Perl_my_unexec (my_perl=0x3229c40) at perl.c
+:3552
#2  0x00000000004cb36f in Perl_pp_goto (my_perl=0x3229c40) at pp_ctl.c
+:2522
#3  0x0000000000445bb0 in Perl_runops_debug (my_perl=0x3229c40) at dum
+p.c:1639
#4  0x00000000004680ed in S_call_body (my_perl=0x3229c40, myop=0x7f96c
+daa6a20, is_eval=0 '\000') at perl.c:2804
#5  0x0000000000467ae2 in Perl_call_sv (my_perl=0x3229c40, sv=0x336dd3
+0, flags=2) at perl.c:2708
#6  0x00000000004493e5 in S_vdie_common (my_perl=0x3229c40, 
    message=0x3dbdd70 "Thread 1 terminated abnormally: main=HASH(0x3ec
+fb90) at /p4/smframework/app/unit_tests/test_fmwk/tests/mnc-ha/Cluste
+rSuite.pm line 115.\n", msglen=136, utf8=0, 
    warn=1 '\001') at util.c:1283
#7  0x0000000000449f14 in Perl_vwarn (my_perl=0x3229c40, pat=0x7f96d09
+095a8 "Thread %lu terminated abnormally: %_", args=0x7f96cdaa6c10) at
+ util.c:1447
#8  0x000000000044a0e5 in Perl_warn (my_perl=0x3229c40, pat=0x7f96d090
+95a8 "Thread %lu terminated abnormally: %_") at util.c:1480
#9  0x00007f96d0901ecb in S_ithread_run (arg=0x32289b0) at threads.xs:
+480
#10 0x00007f96d14ab9ca in start_thread () from /lib/libpthread.so.0
#11 0x00007f96d12086fd in clone () from /lib/libc.so.6
#12 0x0000000000000000 in ?? ()

        [download]
        I think perhaps my hacks to sigaction() were useful and/or there must be other options. As this is not indicating XSUBs code, I'll try 'perl -d' first.
[reply]
[d/l]
      ig was responding to my request. Because I suspect XSUB C functions, I was looking to get a gdb bt; and here it is: 
      Thread 2 (Thread 25070):
#0  0x0000000000446c34 in Perl_hv_common ()
No symbol table info available.
#1  0x0000000000447fee in Perl_hv_common_key_len ()
No symbol table info available.
#2  0x00000000004d3b3f in Perl_gv_fetchmeth ()
No symbol table info available.
#3  0x00000000004d3d32 in Perl_gv_fetchmeth ()
No symbol table info available.
#4  0x00000000004d5dae in Perl_Gv_AMupdate ()
No symbol table info available.
#5  0x00000000004613cb in Perl_sv_bless ()
No symbol table info available.
#6  0x0000000000461836 in Perl_newSVrv ()
No symbol table info available.
#7  0x0000000000468ff6 in Perl_sv_setref_pv ()
No symbol table info available.
#8  0x00007f6c0548b63e in XS_Net__SSH2__new () from /auto/share/perl/5
+.8.9/lib/site_perl/5.8.9/x86_64-linux-thread-multi/auto/Net/SSH2/SSH2
+.so
No symbol table info available.
#9  0x00000000004547fb in Perl_pp_entersub ()
No symbol table info available.
#10 0x0000000000452d06 in Perl_runops_standard ()
No symbol table info available.
#11 0x00000000004507ce in Perl_call_sv ()
No symbol table info available.
#12 0x00007f6c05aab899 in S_ithread_run () from /auto/share/perl/5.8.9
+/lib/5.8.9/x86_64-linux-thread-multi/auto/threads/threads.so
No symbol table info available.
#13 0x00007f6c0664e9ca in start_thread () from /lib/libpthread.so.0
No symbol table info available.
#14 0x00007f6c063ab6fd in clone () from /lib/libc.so.6
No symbol table info available.
#15 0x0000000000000000 in ?? ()
No symbol table info available.

      [download]
      I'm not sure what to make of it yet.
[reply]
[d/l]

        If you compile perl and your XS code with -g flag, gdb will be able to show you more information, including where and in which file each frame in the stack relates to.

        See Building a debugging perl in the INSTALL file that comes with the perl source for guidance on how to do this for perl.

        This will make it easier to understand, but nothing will make it easy - between perl, XS, SSH and threads, you have quite a complicated situation.

[reply]

In Section Seekers of Perl Wisdom