c-era has asked for the wisdom of the Perl Monks concerning the following question:

I’m teaching a co-worker of mine some perl. He has a script that currently goes through each file in a directory and processes the files. In order to process each file he has to execute several SQL statements to different servers. To speed things up he would like to process several files at a time (but he wants to be able to understand what the script does). Since fork doesn’t work I split the program into 2 parts. The first part reads each file in the directory and then launches the second script with the filename as the parameter, but in order to start the second program in the background I used eval (system calls are not run in the background). Here is a condensed version of my first program: 
# part1.pl
use strict;
my ($file, $string);
my @files = (“test1.txt”,”test2.txt”,”test3.txt”);
for $file (@files){
    $string = "open (A$file, \"perl part2.pl $file|\")";
    eval $string; # Error checking omitted for simplicity
}

[download]
More or less, I’m looking for a better way to solve this problem on a Win32 platform. Any thoughts and ideas are welcome. c-era

Replies are listed 'Best First'.
(jcwren) Re: Background processes in Win32
by jcwren (Prior) on Oct 23, 2000 at 17:14 UTC
    This would seem to be a fairly expensive operation, since each invocation requires a complete load of perl.exe, and the resulting compilation of the script. You will probably speed up the overall process, but with a heavier load cost to the system.

    I don't know what the expense of changing to Win32::Process would be, but it's probably less than using eval() to spawn the process.

    As much as I like Windows, this one paragraph makes for a lot of frusteration:
    One of the largest areas of difference is in the process model. UNIX has fork; Win32 does not. Depending on the use of fork and the code base, Win32 has two APIs that can be used: CreateProcess and CreateThread. A UNIX application that forks multiple copies of itself can be reworked in Win32 to have either multiple processes or a single process with multiple threads. If multiple processes are used, there are multiple methods of IPC that can be used to communicate between the processes (and perhaps to update the code and data of the new process to be like the parent, if the functionality that fork provides is needed).
    --Chris

    e-mail jcwren
[reply]
      That does work (overhead is not really in issue now, but it could be in the future). The problem is that if perl is not installed in the same spot on each workstation the program breaks. Anyone know of a way around this (other then installing perl in the same spot on each workstation)?

      c-era

[reply]
        There are a couple of solutions, of which the most elegant is to make sure that the directory containing perl.exe is placed in the system path.

        I imagine you know how this is done, but for others I'll iterate the process under WinNT 4.0. Start->Settings->Control Panel->System, select the Environment tab, click on System Path in the System Variables box, click in the value box, hit the end key, add ';d:\perl\xxx\bin', replacing that with the fully qualified path to the directory containing the Perl executable, click Set, then OK.

        If you're not an administrator on the system, you should be able to add it to the Path variable in the User Environments box. I've never been on a NT system where I'm not an administrator, so I'm not positive.

        A more hackish way is look at the Perl variable, $^X, although you're not guaranteed to get a path in that information. I believe you will under Windows, since argv[0] always contains a fully qualified path name, but you'd have to check that to be sure.

        --Chris

        e-mail jcwren
[reply]
        You can install perl on a shared file-server, set the PATH environment on the workstations to point at the \perl\bin dir there, and make sure that everybody has the same drive mapped to the same place. This may be a little easier than installing perl on each station.
        HTH
[reply]
(tye)Re: Background processes in Win32
by tye (Sage) on Oct 23, 2000 at 18:27 UTC 
     for $file (@files){
     $string = "open (A$file, \"perl part2.pl $file|\")";
     eval $string; # Error checking omitted for simplicity
 }

    [download]

    No good read to use eval for that. 

      use Symbol qw( gensym );
  for $file (@files){
      open( my $fh= gensym(), "perl part2.pl $file |" );
      # Error checking omitted for simplicity
  }

    [download]

    And since you don't seem to be doing anything with the pipe that you open, I'd suggest: 

      for $file (@files){
      system( 1, "perl part2.pl $file" );
      # Error checking omitted for simplicity
  }

    [download]

    But there are quite a few ways to start a process in the background under Win32. I'm not a big fan of Win32::Process which I think was poorly written, is usually overkill, and is very unportable to other operating systems. You can find many of them mentioned in the 3 or 4 other threads on this topic that have shown up in the last few weeks or even in Q&A. But I like the very portable (and probably still not officially documented) system(1,...).

            - tye (but my friends call me "Tye")
[reply]
[d/l]
[select]
      Thanks, the system(1,....) is great (I haven't seen it before). Now if I could just get fork to work ...
[reply]

Back to Seekers of Perl Wisdom