You don't only have the option to use threads. You also have the option of forking off processes (which I know you considered) and letting them run in parallel assuming that the processes are not dependent upon each other's completion for another to start. This method worked well for me for transactions that would take potentially up to 10 minutes a piece. You can do this with Parallel::ForkManager using the following code: 

use Parallel::ForkManager;

 # Begin ForkManager
 my $_max_procs = 5;
 $_pm = new Parallel::ForkManager($_max_procs);

 # Log at process fork
 $_pm ->run_on_start(
   sub { my ($pid, $host) = @_;
     print "Forking process PID: $pid\n";
   }
 );

 # Log at process copmletion
 $_pm ->run_on_finish(
   sub { my ($pid, $exit_code, $func) = @_;
     print "Finishing up process PID: $pid\n";
   }
 );

 # This run_on_wait is currently set to print every 5 sec
 # It can easily be modified to write to a socket every X sec
 $_pm->run_on_wait(
   sub { print "Waiting for children to finish.\n" }, 5.0 );

foreach my $func (@func_list) {
  # Fork off the children and get going on the queries
  my $pid = $_pm->start($func) and next;

  # foo here

  # Closing the forked process
  $_pm->finish;
}

# Ensure all children have finished
$_pm->wait_all_children;
exit(1);

[download]

I am assuming that this module is available on the win32 platform since I just got it off of CPAN.

In reply to Re: How to break up a long running process by madbombX
in thread How to break up a long running process by markh

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.