in reply to Re^8: threads + sockets stalling question
in thread threads + sockets stalling question

[accept()] currently works, it's [PerlIO] that has problems.

I don't see how PerlIO can be a problem. It's a system to provide IO layers to handles. Individual functions in Perl's IO library might have bugs, but that doesn't mean PerlIO is a problem.

In fact, this replicates the problem outside of PerlIO (I think): 

#! perl -lw
use strict;
use threads;
use IO::Socket;

my $lsn = new IO::Socket::INET(
    Listen => 5, LocalPort => '12345'
) or die "Failed to open listening port: $!\n";

async{
    print 'accept starting';
    my $c = $lsn->accept;
    print 'accept returned';
}->detach;

sleep 2;

print "-d starting";
-d $lsn;    # Freezes
print "-d returned";

[download]

Replies are listed 'Best First'.
Re^10: threads + sockets stalling question
by BrowserUk (Patriarch) on Mar 18, 2010 at 23:51 UTC

    The problem does not lie in the CRT. This does everything that my sample in the OP and your sample do, using the CRT directly, and there are no signs of any locking:

    <Reveal this spoiler or all in this thread>

    Output:

    C:\test>828831.exe
828831.c(115):_fstat( lsn, &s ) == -1
        failed with error 9 (Bad file descriptor)
828831.c(116):_tell( lsn )
        failed with error 9 (Bad file descriptor)
828831.c(117):_dup( lsn ) == -1
        failed with error 9 (Bad file descriptor)
Connect from: 120
Thread started for client 120
Connect from: 128
Thread started for client 128
Thread completed 4895 echos for client 128
Thread completed 11575 echos for client 120

    [download]

    The locking is occuring within Perl itself, but damned if I can work out where. It must be in Win32 specific code given that other OSs aren't suffering the same problem. But given that OS conditional code is spread throughout the entire code base, it'll take someone clever than me to find it.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]

      Whenever I said accept, I meant Perl's accept. I didn't mean to imply there was a problem with the CRT. There's a problem with how Perl serializes access to the CRT. (At least I presume it's Perl that does the serializing.)

      Granted, at first I thought it was a global lock, but it appears to be much finer grained than that. Perl's accept will not block all stat operations, just those on the socket passed to accept.

      Is that necessary, or can something be done about it? Why isn't linux affected? I don't know.

      But given that OS conditional code is spread throughout the entire code base, it'll take someone clever than me to find it.

      The lock appears to be specific to a handle or descriptor. I would start by finding out which, then finding out where the lock is stored in the relevant structure, then find out what uses the lock.

[reply]
        Granted, at first I thought it was a global lock, but it appears to be much finer grained than that. Perl's accept will not block all stat operations, just those on the socket passed to accept

        Which is where I was coming from. If PerlIO didn't use fstat() to try to determine if a file descriptor belongs to a "regular" file--which is a pretty expensive way of doing it, even if it worked--then that paricular problem would go away.

        Similarly for the ftell() which isn't useful on a socket. All of this takes place as a part of PerlIO_dup() when they are cloning. I don't understand why they can't just use the CRT dup() or dup2()?

        The lock appears to be specific to a handle or descriptor.

        That's partly why I reached the conclusion that it must be in the windows CRT. Why would they lock them for windows and not *nix?

        And tracing backward from Win32sck.c:Win32_accept(), darned if I can see anywhere locking is applied. But once you get into PerlIO.c, there are so many layers of macro and indirection through any one of a dozen different flavours of IO layer dispatch tables, I can quite see that I might have missed something. I don't think I did, but I might have.

        I would start by finding out which, then finding out where the lock is stored in the relevant structure, then find out what uses the lock.

        At this point: be my guest :) I'm done. (For a few days at least.)

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Seekers of Perl Wisdom