Oh Perl Monks, I plead for your insight..

My app uses WWW::Curl::Multi (libcurl) to perform web tasks which are handled, among many other things, in a super loop. I've recently added a piece of code which fork()s to perform an occasional long-running task, which I do not want to block my main loop with.

Works great. On Linux.

Upon testing the same code under Windows (Strawberry Perl) with the newly added code which forks, perl crashes. Debugging shows me if I don't fork(), it doesn't crash, and if I don't "WWW::Curl::Multi->new();" it doesn't crash.

Knowing that Windows emulates fork() with interpreter threads, I'm guessing the WWW::Curl library is not thread safe.

This was confirmed with the following bare-minimum test cases:
Windows (wine):

use WWW::Curl::Multi;
my $x = WWW::Curl::Multi->new();

if (fork == 0) {
        #Do stuff NOT related to WWW::Curl here..
        print STDERR "thread sleeping\n";
        sleep 1;
        print STDERR "thread exiting\n";
        exit;
}

print STDERR "main sleeping\n";
sleep 3;
print STDERR "main exiting\n";
exit 0;

[download]

Which yields:
main sleeping
thread sleeping
thread exiting
Free to wrong pool 249ea0 not 241ba8 at testcrash.pl line 9.
wine: Unhandled page fault on write access to 0x00000000 at address 0x
+713c9106 (thread 0031), starting debugger...

[download]

And under Linux:

use threads;
use WWW::Curl::Easy;

my $x = WWW::Curl::Easy->new();

my $t = threads->create(sub{ 
        print STDERR "thread sleeping\n";
        #Do stuff NOT related to WWW::Curl
        sleep 1;
        print STDERR "thread exiting\n";
});

print STDERR "main sleeping\n";
sleep 3;
$t->join();
print STDERR "main exiting\n";
exit 0;

[download]

Which yields (under GDB): 

[New Thread 0x7ffff1689700 (LWP 12664)]
main sleeping
thread sleeping
thread exiting
[Thread 0x7ffff1689700 (LWP 12664) exited]
main exiting

Program received signal SIGSEGV, Segmentation fault.
0x0000003f3fe39d63 in Curl_splay () from /lib64/libcurl.so.4

[download]

Now, I don't need any curl resources within my child/thread, it's only used in the main program (technically another .pm), but the object created is globally scoped and has a member with a WWW::Curl object. So I don't think I can keep it hidden from my child/thread. Curl itself is supposed to be thread-safe, with the exception of the curl_global_init. I'm pretty sure that's not the issue though, since the child does not call it. Unless it's being automatically called a second time by the BOOT: XS syntax on the thread creation?

I believe that when I fork under Win32/thread under Linux, my object may be free()d by the DESTORY XS methods when I exit/join the thread? And I probably need to prevent curl_global_cleanup() from being called when the thread exits in the .pm END{} block... ??

So... this is where it gets fuzzy for me. I'm a strong C programmer, but there's a learning curve with perl's XS. Are there any other modules out there which properly deal with these sort of situations which I may learn / copy from? I've spent all day reading perlthrtut, perlguts, perlmod, Example::CLONE and others and ended up with a nasty headache..

Most humbly,
--Luke

update:

Is the solution simply to (in reference to the above examples):

use threads;
use threads::shared;
...
share($x);

[download]

This seems to prevent perl from crashing, and the double DESTROY, however, with the fork() example under windows, the END{} curl_global_cleanup() is called twice, in the thread and in the parent, where in the thread->create()->join() it is only called once.....

Is this a safe workaround to what I'm seeing?

In reply to WWW::Curl and making it thread safe by lsuchocki

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.