Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi wizards,

I am trying to change a single process proxy style Perl program to handle multiple connections. On Windows 2000 and AS 5.6.1 fork() never worked for me when trying to do HTTP/LWP/Daemon code, so I just installed AS 5.8.0 and trying to multi-thread. I am using HTTP::Daemon to accept the connections. Problem: The get_request method is stuck in the thread. Do I need to fix my program or is this a bug? Do you know of example of similar program?

I am on Windows 2000 server and just installed AS 5.8.0 beta using MSI (Windows installer).

The bad behavior is that the first two connects are stalled in the thread at the get_request, the third connect will activate the first two connects and is stalled itself.

To use the program just change the browser setting to proxy on 127.0.0.1 port 3126 and try locations www.google.com, then www.yahoo.com, then www.msn.com

This is just a small code to get past the get_request :(

============================ use 5.008; # 5.8 required for stable threading use strict; use warnings; use threads; # threading routines use threads::shared; # and variable sharing routines use HTTP::Daemon; my $daemon = HTTP::Daemon->new( LocalPort => 3126, Listen => 5, Reuse => 1 ); die "Cannot listen\n" unless defined $daemon; $daemon->autoflush(1); my $client; while( 1 ) { $client = $daemon->accept; print STDERR "got connect\n"; threads->create("start_thread", $client ); # threads->new(\&start_thread, $client ); } $daemon->shutdown(2); exit; sub start_thread { threads->self->detach(); print STDERR "Thread started\n"; print STDERR $_[0] . "\n"; my $client= $_[0]; my $request = $client->get_request; print STDERR "====Got request=Start=====\n"; print STDERR $request->as_string; return; } ========================= output: ----------------------- got connect Thread started HTTP::Daemon::ClientConn=GLOB(0x1f2d640) got connect Thread started HTTP::Daemon::ClientConn=GLOB(0x20e8e08) got connect ====Got request=Start===== GET http://www.google.com/ HTTP/1.0 Accept: image/gif, image/jpeg, */* Accept-Language: en Host: www.google.com User-Agent: Mozilla/4.7 (compatible; OffByOne; Windows 2000) Webster Pro V3.2 ====Got request=Start===== GET http://www.yahoo.com/ HTTP/1.0 Accept: image/gif, image/jpeg, */* Accept-Language: en Host: www.yahoo.com User-Agent: Mozilla/4.7 (compatible; OffByOne; Windows 2000) Webster Pro V3.2 Thread started HTTP::Daemon::ClientConn=GLOB(0x33135d8) ----------------------------------------

Replies are listed 'Best First'.
Re: HTTP::Daemon not working in threads?
by pg (Canon) on Nov 25, 2002 at 02:27 UTC

    Update:

    See A simple web server with HTTP::Daemon and threads, which clearly points out the problem in the original post.

    Original:

    Instead of using HTTP::Daemon you used, I used IO::Socket::INET directly, but don't think that's what caused your problem.
    proxy.pl: use strict; use threads; use IO::Socket::INET; $| ++; my $listener = IO::Socket::INET->new(LocalPort => 3126, Listen => 5, Reuse => 1) || die "Cannot create socket\n"; my $client; my $client_num = 0; while (1) { $client = $listener->accept; threads->create(\&start_thread, $client, ++ $client_num); } sub start_thread { threads->self->detach(); my ($client, $client_num) = @_; print "thread created for client $client_num\n"; while (1) { my $whole_req = ""; do { my $req; $client->recv($req, 700000); return if ($req eq ""); $whole_req = $whole_req . $req; } until ($whole_req =~ m/\r\n\r\n/x); print "client $client_num got req:\n$whole_req"; $whole_req =~ m/Host: ([\.|\w]*)/; my $host = $1; my $server = new IO::Socket::INET(Proto => "tcp", PeerPort => 80, PeerAddr => $host) || die "failed to connect to $host\n"; print $server $whole_req; my $whole_res = ""; do { my $res; $server->recv($res, 700000); $whole_res = $whole_res . $res; } until ($whole_res =~ m/<\/html>/); print "client $client_num got res\n"; print $client $whole_res; close($server); } return; } tester.pl: use strict; use IO::Socket; my $server = IO::Socket::INET->new(Proto => "tcp", PeerPort => 3126, PeerAddr => "localhost", Timeout => 2000) || die "failed to connect\n"; my $req = "GET / HTTP/1.1\r\nHost: $ARGV[0]\r\n\r\n"; while (1) { print $req; print $server $req; my $res; my $whole_res = ""; do { $server->recv($res, 70000); exit if ($res eq ""); $whole_res = $whole_res . $res; } until ($res =~ m/<\/html>/); print $whole_res; }
      Thank you for the reply.
      
      I was just trying to not "reinvent the wheel". 
      
      I now believe that HTTP::Daemon, and probably the entire
      LWP library do not work properly in threads, at least on
      Windows, for one reason or another :(.
      
      I wanted to use HTTP::Daemon so I can access get_request
      and the corresponding HTTP::Request returned object.
      Similarily for the send side. The IO::Socket::INET is lower
      level and I need to "reinvent the wheel" and receive
      and parse the headers and possibly data in POST
      request. Basically re-doing the get_request.
      
      Is $sock->recv() less blocking than $sock->read?
      I think that $sock->recv() loop to receive the data as in
      your example might go into CPU loop waiting for data.
      $sock->read() might not but sometime is blocking too much.
      I think if I use threads I can let socket read/write block,
      but I must find the right number of bytes to read first,
      might be impossible as in waiting for enf-of-http-header mark.
      
      JM.
      
Re: HTTP::Daemon not working in threads?
by dingus (Friar) on Nov 25, 2002 at 17:45 UTC
    Note there are strong hints that HTTP::Daemon doesn't thread well (if at all) at the LWP site and the off-link to kirra-httpd

    If anyone has got this to work its the POE people, but I've never got their webserver to properly with Activestate 5.6.x and Win2k

    Dingus


    Enter any 47-digit prime number to continue.

      Dude! As one of "the POE people", this is the first I've heard of your trouble. Don't make authors hunt you down for feedback. Also see rt.cpan.org for your all your CPAN bug reporting needs.

      -- Rocco Caputo - troc@pobox.com - poe.perl.org