arunb has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks, I have a perl application that is a web-server. As usual, it accepts connections and for each connection, spawns a thread and detaches the thread. The thread subroutine then performs the required operations and exits.

Now this works fine in my application. We have a scenario where we do a file upload to the web server from the front-end. So when the request comes to back-end, i spawn a thread to download the file to the back-end server. Then i spawn another thread from this thread which will do some pre-processing on the downloaded file.After spawning, i detach it as well.( I am not so concerned about the return status of pre-processing.)

When the file size is around 5-6MB, there is no problem. This works fine. When i try to upload a 13MB file, the second thread ( for pre-processing ) does not get spawned. The threads->create() subroutine does not create a new thread. It seems to hang. The interesting part is when i get another request from front-end, the web server spawns a new thread and then i see that the previously hung thread wakes up and starts execution. I am stuck at this stage. Please help.

~Arun

Replies are listed 'Best First'.
Re: perl thread hangs/not spawned
by moritz (Cardinal) on Oct 15, 2009 at 12:58 UTC
    The error is on line 42, obviously. Oh, wait...

    It's hard to tell you anything without seeing your code, or even knowing what kind synchronization you use (semaphores? queues?).

    If you let the thread sleep for as long as the 13MB download lasts, does the same error occur? Do you actually load the file to memory?

    Perl 6 - links to (nearly) everything that is Perl 6.

      I am not using any synchronzation( no queues, no semaphores). And the second thread is spawned after the file is downloaded completely to the web server.

      In the main while loop, i do the following

      foreach (@connections_pending) { my $fh; my $remote = accept( $fh, $_ ); my ( $port, $iaddr ) = sockaddr_in($remote); my $peeraddress = inet_ntoa($iaddr); my $t = threads->create( \&connection, $fh ); }

      So when the request for file upload comes from front0end, i spawn the thread.

      Now, in the connection subroutine, i check if its a POST method and if so, i call a subroutine to handle that

      sub upload_file { my ( $fh, $req_ ) = @_; my %req = %$req_; my $header_ = $req{HEADER}; my %header = %$header_; my ($file, $desc, $dirpath ) = (undef, undef, undef); my $boundary = trim (form_data_boundary( $header{'content-type'} ) +); my $user_agent = $header{'user-agent'}; # Now i will process the post buffer and copy the file #to local mach +ine into a temp directory. I shall also #receive the filename, destin +ation directory and some #other metadata information. ($file, $desc, $dirpath) = process_POST_buffer($fh, $req_, $boundary); # Then i spawn another thread to do the pre-processing on this file my $t = threads->create( \&pre_process, $file, $desc, $dirpath ); }

      The pre-process thread does this

      sub pre_process { threads->detach(); my ( $file_name, $desc, $dirpath ) = @_; my ( $file1, $file2, $file3 ) = ( getTempFile(), getTempFile(), + getTempFile() ); my $metadata = i_PreProcessor->new( FILENAME => $file_name, SEARCHXMLFILE => $file1, DESCRIPTION => $desc, MAPXMLFILE => $file2, TAGXMLFILE => $file3, DIRNAME => $dirpath ); $metadata->build2(); #delete temp files. my @files = ( $file1, $file2, $file3 ); unlink(@files); threads->exit(); }
      ~Arun
Re: perl thread hangs/not spawned
by bart (Canon) on Oct 15, 2009 at 13:34 UTC
    Could it be due to some upload size limit by the webserver? Often there is a limit at about 10MB.

      I dont think so. The file is completely uploaded. the only problem is the thread does not get spawned.

      ~Arun