in reply to launch long running background program and watch final progress

Well, I was waiting around and working on another part of the code when it hit me. I am still not sure how to do what I want the "right" way (assuming there is such a thing). But I came up with what I think is a pretty reasonable solution:

I redirected STDOUT and STDERR to a file when I run the application. Once the file is created, I use File::Tail to watch the file for the telltale string letting me the database is ready to accept connections and break out of the while loop that File::Tail is conveniently allowing me to use to block with. Here is the working code if anyone happens to ever have a similar problem

Or maybe I am the only knuckle dragger that has never run into this kind of issue before and solved it a long time ago?

sub cge_start { my($arg_ref) = @_; #=+ iterate over provided config and replace defaults while(my($k,$v) = each %$arg_ref) { #=+ Replace each default value if the key exists. Perhaps paranoi +d, but prevent arbitrary/unexpected settings $arguments{$k} = $v if exists $arguments{$k}; } #=+ Make sure we have a bare minimum of settings return undef if $arguments{dataDir} eq '' || !-d $arguments{dataDir} + || $arguments{imagesPerNode} eq '' || $arguments{nodeCount} eq ''; #=+ Create the results directory # make sure that our directory has a trailing slash "/" $arguments{dataDir} .= '/' unless $arguments{dataDir} =~ m/\/$/; #=+ Create the results directory if it does not exist mkdir $arguments{dataDir}.'results', 0744 if !-d $arguments{dataDir} +.'results'; $arguments{resultDir} = $arguments{dataDir}.'results'; #=+ Create a temporary directory to store all the junk files we crea +te for easy cleanup later mkdir $arguments{dataDir}.'temp', 0744 if !-d $arguments{dataDir}.'t +emp'; #=+ Create the logfile string (does not need to exist) $arguments{logFile} = $arguments{dataDir}.'cge_logfile.log'; #=+ Make sure we have both the number of instances and number of des +ired nodes return (undef,undef) unless $arguments{imagesPerNode} > 0 && $argume +nts{nodeCount} > 0; #=+ Create a cleanup script that we will use to kill "this" server s +ession $arguments{cleanupScript} = $arguments{dataDir}.'temp/cleanup.sh'; #=+ Make sure we have a free TCP port to use my $port = 3750; if(check_port($port)) { $port++; while(check_port($port)) { $port++; } } $arguments{queryPort} = $port; #=+ Concatenate all the non-blank command line arguments my @args; while(my($k,$v) = each %arguments) { push @args, '--'.$k.' '.$v.' ' unless $v eq ''; } my $arg_string = join('',@args); #=+ Run the launcher #exec($cge_launch.' '.$arg_string.'> /dev/null 2>&1 &'); my $success = run(command => $cge_launch.' '.$arg_string.' > '.$argu +ments{dataDir}.'temp/stdout.txt 2>&1 &', verbose => 0); #=+ Something went wrong, so just return undef return (undef,undef) unless $success; my $file = File::Tail->new($arguments{dataDir}.'temp/stdout.txt'); while(my $line = $file->read) { last if $line =~ m/Starting port forwarding/; } open(my $CU,'<',$arguments{cleanupScript}) or croak $!; my $text = <$CU>; close($CU); my $pid; if($text =~ m/scancel (\d+)/m) { $pid = $1; } return ($pid,$port); }
  • Comment on Re: launch long running background program and watch final progress
  • Download Code

Replies are listed 'Best First'.
Re^2: launch long running background program and watch final progress
by Marshall (Canon) on Apr 22, 2016 at 16:30 UTC
    If you are waiting for some file to change, then some variant of inotify will tell you the file changed without looping continuously in a tail loop burning Mips to not much effect. You can also get notified if a directory changes, some new file appears, perhaps some long awaited "I am finished and ready for business" file.

    Update: I suppose also if you opened the file for read, then while(<FILE>)... will cause a program pause in the OS waiting for the next line to appear. You can examine each line waiting for the right msg to be written. Of course this could "deadlock" if the if the other program hangs. But if it dies and file closes, then you should get an EOF. The basic rationale behind these 2 ideas is not burn a lot of Mips waiting for something that itself is burning a lot of Mips where your "watcher" slows things down. I haven't tested, but I don't think you get an EOF if somebody else has the file open for "write". I have used inotify before and that will work.

    Update: eatingmyownwords = I now think that that this won't work unless you are using some sort of trick like Tee and reading from a pipe which is left open. I think you get EOF when you reach the current extent of text in the file whether it is open or not for additional writes - that's what tail does.

      So, are you saying my File::Tail approach is OK? Or is there some way that is better? I'm totally open to doing in whichever way is best...the approach I took was simply to "git 'er done"...certainly not because I was positive it was the most elegant solution...

        Well "what works" is good! I'm not saying that it isn't. If you are waiting for something that might take an hour clock time, who the heck cares if you get an immediate answer? And probably some burned Mips doesn't matter at all? file tail is a polling routine that adjusts its time to the next poll. Fine. Using inotify doesn't involve any polling and therefore a) it is "faster in clock time" and b) uses less Mips to monitor what you want.

        What is "best" is very subjective and depends upon the application. I am not judging "right or wrong". I am merely suggesting other ways, which is what I think that you asked for? I don't think that there is a single "right way". There are multiple ways. Whatever achieves the objective is "good". I guess this depends upon what the objective is and how you measure it?

Re^2: launch long running background program and watch final progress
by Anonymous Monk on Apr 22, 2016 at 20:46 UTC
    Hmm, your first question sounded like you were talking about shell job control (background and foreground process groups), but it seems you're just spawning a process and want to read its output. Looking at your code, I don't see any particular reason to use File::Tail... you could've just opened temp/stdout.txt as usual (with the open function) and read it line by line? Anyway, people normally use FIFOs for that, you can make one with POSIX::mkfifo, perlipc has an example.
      , I don't see any particular reason to use File::Tail
      oh, I see, the other process is constantly appending to the file and you don't want to get EOF.
        Yes, I think that EOF is the issue and why my 2nd idea won't work. But inotify will work. If file is modified, read it and see what happened. Don't continually poll for different content.