arunhorne has asked for the wisdom of the Perl Monks concerning the following question:

Monks

I have a subroutine in a package that takes a long (upto 30 mins) to execute and unfortunately I have no way of decomposing it into smaller routines -- theres just loads of data to process and collate. However, I would like to provide the user with feedback as to the percentage completeness of the process, perhaps in the form of something on the console like this:

[############ ] 50%

However, I do not want the subroutine in my package to write directly to the console. I want the script that actually calls the method to control the progress bar, but just need some way of getting feedback from the long-running subroutine as to it progress, to the calling script so the progress bar can be updated.

Can anyone help me with this if it is possible?

____________
Arun

Replies are listed 'Best First'.
Re: User Feedback for Subroutines with Long Execution Time
by BrowserUk (Patriarch) on May 29, 2003 at 12:17 UTC

    This is an ideal application for threading if that option is available to you.

    #! perl -sw use strict; require 5.008; use threads 'yield'; use threads::shared; use vars qw[$ITERS]; $ITERS ||= 10000; $|=1; my $percent : shared = 0; sub progress{ print '[', ' ' x 80, ']', $percent; while( $percent < 1 ) { printf "\r[%s%s] %5.2f%% ", '#' x (80*$percent), ' 'x(80*(1-$p +ercent)), 100*$percent; yield; } print "\n"; return; } sub TakesALongTime{ my $in = 0; my $max = shift || 10000; for my $count ( 1 .. $max ) { $percent = $count/$max; rand () ** 2 + rand () ** 2 < 1 && $in ++ ; yield; } return sprintf '%f', 4 * $in / $max; } my $thread = threads->create('progress'); my $PI_approx = TakesALongTime($ITERS); $thread->join; print "After $ITERS iterations, pi was approximated to be: $PI_approx\ +n"; __END__ D:\Perl\test>261528 -ITERS=100000 [##################################################################### +##########] 100.00% After 100000 iterations, pi was approximated to be: 3.136440

    Not a great demo, but if any of it needs explaination, please ask. (See Monte Carlo approximation of PI for Abigail's Monte Carlo approximation of PI snippet.)


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


Re: User Feedback for Subroutines with Long Execution Time
by Tanalis (Curate) on May 29, 2003 at 11:21 UTC
    I want the script that actually calls the method to control the progress bar

    That could be interesting to do, if the parent script is simply sitting waiting for the long-running sub to return.

    One way to achieve something like this would be to set up pipes, and fork to execute the 30-minute subroutine. That way, it can return data about its progress to the parent process, which can then display it to the user.

    This seems like overkill, though, for what seems a fairly simple problem. It'd probably be easiest (to implement) to have the long sub directly write to the console.

    Maybe you could do something with threads to allow the parent script to continue to update the progress bar while the long-running sub is executing?

    Hope that helps .. just a few ideas really.

    -- Foxcub
    #include www.liquidfusion.org.uk

Re: User Feedback for Subroutines with Long Execution Time
by tachyon (Chancellor) on May 29, 2003 at 11:43 UTC

    You can get a wget style progress bar here. Although the easiest thing is to get the long running sub to write to the console if you want it's parent/some other process to do the progress bar you need some sort of IPC perlman:perlipc.

    The simplest solution is to have the long running widget update a semaphore file with a number ie how many % or whatever it has done so far. The process you want to do the progress bar just reads the file to get the info it needs. Provided the process generating the progress bar knows what this file will be called you have no problem. Delete this semaphore file when finished to indicate the completed status. There are literally dozens of other methods to do IPC, but at the pace you are talking about a file is the quickest solution and will be efficient enough (ie the overhead will not kill you).

    cheers

    tachyon

    s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

      In a similar project, I wanted to run an arbitrary subprocess that wasn't designed with progress in mind. I relied on the parent (progress monitoring) process having either (1) an initial guess as to the number of lines of STDOUT output, or (2) a process-specific way of converting a line of STDOUT output into a percent-complete guess.

      For example, to monitor a long tar jcvf $date.home.tar.bz2 /home process, you can either figure out how many files that tar will have to process and then count actual output lines, or you can try to watch the naturally sorted filenames ("./meetings/..." is roughly halfway through the alphabet) for a gut-check guess.

      --
      [ e d @ h a l l e y . c c ]

Re: User Feedback for Subroutines with Long Execution Time
by PodMaster (Abbot) on May 29, 2003 at 11:47 UTC
    You could try Term::StatusBar


    MJD says you can't just make shit up and expect the computer to know what you mean, retardo!
    I run a Win32 PPM repository for perl 5.6x+5.8x. I take requests.
    ** The Third rule of perl club is a statement of fact: pod is sexy.

Re: User Feedback for Subroutines with Long Execution Time
by zengargoyle (Deacon) on May 29, 2003 at 12:52 UTC

    take a look at the POE_Cookbook/Looping section. you might be able to break your calculations into N discrete steps. then you could do something like:

    my $N = setupSub( %myParameters ); while (defined( my $at = stepSub())) { printf "%d percent done\n", $at/$N; } my $results = resultsSub();

    or, you could have your subroutine take an optional callback and granularity.

    my $granularity = 5; sub showProgress { # will be called $granularity+1 times my $at = shift; if ($at == 0) { print "starting\n"; } elsif ($at == $granularity) { print "done\n"; } else { print "$at/$granularity done\n"; } } my $results = mySub(%myParam, progress => { callback => \&showProgress, granularity => $granularity } );
Re: User Feedback for Subroutines with Long Execution Time
by petesmiley (Friar) on May 29, 2003 at 15:41 UTC
    There are some good answers here, but they strike me as a little complicated. Why not just pass a reference to a sub that handles the info. Then all you have to do in your subroutine is call the reference on each iteration.

    This is good if you're not looking for something pretty, and takes about 3 extra minutes to code.

    smiles

      Re: User Feedback for Subroutines with Long Execution Time by petesmiley
      There are some good answers here, but they strike me as a little complicated. Why not just pass a reference to a sub that handles the info. Then all you have to do in your subroutine is call the reference on each iteration.
      This is good if you're not looking for something pretty, and takes about 3 extra minutes to code.

      smiles


      My thoughts exactly. I did something like this with a Perl/Tk script which compressed VERY large files. I re-wrote Zip.pm to include a referenced variable in which I put the percentage done. Then I just updated the TopLevel widget regularly.
      -Arden.

      For those not familiar with graphical interfaces, this is often called a "callback". In any case, one must assume that your long running process has a way of knowing how far along it is, and you'll have to have various points in the process explicitly call the callback with the current status report.
      sub statusPrint { # code to update status print shift; } sub longprocess { my &status = shift; #do stuff &status("10%"); #more stuff &status("20%"); } call it with: longprocess (\&statusPrint);
Re: User Feedback for Subroutines with Long Execution Time
by arthas (Hermit) on May 29, 2003 at 12:18 UTC
    You may want to work something out with threads. However, as somebody already pointed out, the easiest thing would be to let the soubroutine update the progress bar directly. Is there a particular reason because this can't be done?

    Michele.
Re: User Feedback for Subroutines with Long Execution Time
by zakzebrowski (Curate) on May 29, 2003 at 18:40 UTC
    ... while not exactly what you want, there's always Term::Twiddle ...

    ----
    Zak
    Pluralitas non est ponenda sine neccesitate - mysql's philosphy