in reply to Queuing in multithread context

My problem now is that I can't send each copy task at once on the same server (lower I/Os, increase copy time, may have high impact on overall server performance and so on), so I try to implement queuing in thread.

That sentence doesn't make any sense.

You don't explain what problem "implement queuing in thread" is meant to address; nor how it will help. Actually, not even what "implement queuing in thread" means?

While proceeding like this I bump into the below error:

Two of those error messages have nothing to do with your perl script:

  1. "Process can't access file because it is used by another process"

    That comes from the OS, not Perl.

  2. "Error copying C:\temp\deploy_files.cmd to remote system"

    And that not a OS error, so it must be coming from PsExec.exe. Nothing to do with perl.

Finally, "Free to wrong pool 348d3a0 not 298de8 during global destruction." simply means that you aren't cleaning up your threads properly before ending the script.

We might've been able to help you with that, had you posted a short but complete and working, runnable script; but as is there is nothing we can do.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

Replies are listed 'Best First'.
Re^2: Queuing in multithread context
by Hardin (Novice) on Jan 20, 2015 at 11:47 UTC

    Hi,

    Sorry if I'm not clear, as explained a bit earlier my goal in using queued thread is to ensure only one copy task is processed to remote server at a time.

    I agree, the error is not perl related, but related to my `psexec` instruction that copy the batch script prior executing it. But it's a side effect of having all my copy threads sent all at the same time to the remote server, this I try to bypass using queued thread. Does it make sens to you ?

    Here is the full "working" script:

    And here is a sample of ini file the script is parsing and provided as parameter to the script:

    I hope this sched some light ? Please be patient as I'm still figuring some stuff out but I'll try to better explain my needs in the future.
    Thanks !

      Okay. Now you've shown us the sample config file, it shows that you are running each of multiple DEPLOY sections on each of the servers, each in a separate thread; hence your conflicts.

      The simple solution to your problem is to only start one thread for each server; and call the deployRoutine() multiple times within that thread; serially.

      Ie. Something like this:

      foreach my $targetServer (@targets) { # Ping host to check its available my $p = Net::Ping->new(); if ($p->ping($targetServer)) { my @subs; ### Accumulate the deployRoutines here foreach my $deploySection (@deployments) { if ($flagDeploy) { my $deploySource = "$config{$deploySection}{'Source'}" ; my $deployDest = "$config{$deploySection}{'Destination'}" +; if ($flagNoMove) { ## Add this sub to the array to be executed for this s +erver push @subs, sub { deployRoutine($targetServer, $deploy +Source, $deployDest, "true") }; $threadDetails{$thread}{"Section"} = "$deploySection"; $threadDetails{$thread}{"Target"} = "$targetServer"; } else { ## Add this sub to the array to be executed for this s +erver push @subs, sub { deployRoutine( $targetServer, $deplo +ySource, $deployDest ) }; $threadDetails{$thread}{"Section"} = "$deploySection"; $threadDetails{$thread}{"Target"} = "$targetServer"; } } } ## Now start one thread to execute them all; serially push @threads, threads->new( sub { $_->() for @subs; } ); $p->close(); }

      Note: That is obviously untested code in the context of your application, but in isolation, the principle works:

      { use threads; my @subs = map{ eval "sub{ print $_; }" } 1 .. 10; threads->new( sub{ $_->() for @subs; } ); sleep 10; };; 1 2 3 4 5 6 7 8 9 10

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
      In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

        That's it! Definitely what I want to achieve, thank you

        For now I'm unable to achieve a working implementation of that logic, if I proceed so thread aren't joined and the script exit while all haven't been joined and I need to retrieve their status anyway.

        I added a join for @threads but now I'm stuck on the first thread execution not sure why it's not going further...

        Still working on it and I appreciate your help!