elbow has asked for the wisdom of the Perl Monks concerning the following question:

Hi all.

Does anybody have any ideas for this:-

I have a perl script running on a Windows PC that calls an external batch job. The batch jobs takes several hours to complete. I am trying to implement an extra security measure to check the progress of the batch job by taking a periodic count of how many records have been processed in a given time period and see if that corresponds to an average rate.

I'm intending to notify a seperate machine once the process has started. The only idea I have for the periodic checking is to use Windows Scheduler but I'm not sure I want to do that! Has anybody any ideas of a perl script that can do anything like this?

thanks

elbow

Replies are listed 'Best First'.
Re: Periodic checks
by Corion (Patriarch) on Jan 17, 2003 at 12:11 UTC

    I would stay away from the Windows scheduler, as it has some rather nasty quirks (a failed job gets unscheduled automatically) and no nice way of logging / mailing the output.

    I would use Schedule::Cron (or Schedule::Cron::Nofork, a modification of Schedule::Cron I wrote), to run jobs periodically. Both modules do not have the notification features of cron, and have no built-in logging, but it's not hard to implement logging.

    With Schedule::Cron, you can schedule your tasks in a crontab style, to run every n minutes.

    perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ; # The $d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider ($c = $d->accept())->get_request(); $c->send_response( new #in the HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' # web
Re: Periodic checks
by BrowserUk (Patriarch) on Jan 17, 2003 at 13:08 UTC

    It's not clear how you will determine how many records have be processed by the external batch job from external to that batch job. Assuming you can do this (by checking the size of an output file or having the batch process log its progress to a file somewhere), then the next question is what action to you want to take if your calculated average rate is out of spec?

    If the action you want to take can be completed by a third process that performs the check and calculation, then its an easier problem than if you need to pass that information back to the parent script that initiates the external batch job.

    One possibility is that you could fork the parent script just before initiating the batch job and have the child watch the parent (or vice versa) and take the appropriate action.

    If you are using a threaded perl (AS5.8) or similar, then you could use a thread for for the same purpose. This has the advantage of making communucation between the watcher and the script that starts the batch job easier--should that be a requirement.

    You could also use Win32::Process to detach a process to run a second copy of perl and an independant script that uses sleep in a loop to perform the checking and takes the action if it falls out of the loop. Adding a check to make this self terminating after a specified amount of time is easy too.


    Examine what is said, not who speaks.

    The 7th Rule of perl club is -- pearl clubs are easily damaged. Use a diamond club instead.

      Thanks

      The batch job does create a file for each transaction it has processed. I was going to have the check process take a copy of this at certain points and count how many records had completed at that point. The idea is that if the batch job seems to have stopped or slowed dramatically the check process will place a call to the overnight operator's monitor alerting them to a potential fault.

      Both the fork and sleep ideas sound good - but I'm going to have to learn about fork! Thanks for your help.

      elbow
Re: Periodic checks
by John M. Dlugosz (Monsignor) on Jan 17, 2003 at 17:19 UTC
    You could record a progress indicator in a Windows Registry value. Then the Registry can be viewed from a remote machine with existing tools.