in reply to Re: perl -Dusethreads compilation
in thread perl -Dusethreads compilation

Blocking certain critical blocks of code.

Replies are listed 'Best First'.
Re^3: perl -Dusethreads compilation
by BrowserUk (Patriarch) on Apr 05, 2010 at 15:13 UTC

    Which critical blocks of code?

    (This would go a lot quicker of you gave us some clue as to what you are trying to do! Ie. Post your code, or at the very least, some pseudo-code of what you're intending to do. As is, your question is impossible to answer.)


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      # create n numbers of threads #create ftp connection/login while (1) { wait msg Q(); 1. dequeue element 2. download file from ftp server. 3. delete file. } # Master Thread #create ftp connection/login while (1) { if 'files exist on ftp server' { enqueue files to download } }

      To sum it, master threads gets list to download files and workers just downloads the file.

      I am worried about multiple threads colliding, race condtions, in NET::FTP module.

        So you don't really want a semaphore (people rarely need that), you just want a Thread::Queue for example, to distribute the jobs among your worker threads. Alternatively, if threads are unavailable to you, see Parallel::ForkManager, as your child/worker threads don't really need to communicate with the master thread.

        Also, you might think about whether multiple threads will really improve the runtime of a process that is basically limited by the available bandwith and not by the available computation power.

        I am worried about multiple threads colliding, race condtions, in NET::FTP module.

        Just create a new instance of Net:FTP in each thread. There should(untested) be no conflicts.

        This untested pseudo-code might get you started:

        use threads use Thread::Queue use Net::FTP my $Q = new Thread::Queue; sub downloader { my $ftp = Net::FTP->new("some.host.name", Debug => 0) or die "Cannot connect to some.host.name: $@"; $ftp->login("anonymous",'-anonymous@') or die "Cannot login ", $ftp->message; $ftp->cwd("/pub") or die "Cannot change working directory ", $ftp->message; while( my $file = $Q->dequeue ) { $ftp->get( $file ); $ftp->delete( $file ); } $ftp->quit } my @threads = map threads->create( \&downloader ), 1 .. $N; my $ftp = Net::FTP->new("some.host.name", Debug => 0) or die "Cannot connect to some.host.name: $@"; $ftp->login("anonymous",'-anonymous@') or die "Cannot login ", $ftp->message; $ftp->cwd("/pub") or die "Cannot change working directory ", $ftp->message; $Q->enqueue( $ftp->ls ) or die $!; $ftp->quit; $Q->enqueue( (undef) x $N ); $_->join for @threads;

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.