in reply to Re: Run a script in parallel mode
in thread Run a script in parallel mode

It is a simple command like the following:
system ("java -Xmx300m java_code/ALPHAtest -a tables/A.TRAINED -e tabl +es/E.TRAINED -c tables/conf.tat -f $infile.TO_SCAN > $infile.RESULT") +;

Replies are listed 'Best First'.
Re^3: Run a script in parallel mode
by BrowserUk (Patriarch) on May 26, 2015 at 13:28 UTC

    Then something simple like this might do:

    #! perl -slw use strict; my $protienFile = shift; my $size = -S protienFile; my $chunk = int( $size / 20 ); open IN, '<', $protienFile or die; my %procs; for my $n ( 1 .. 20 ) { open O, '>', "temp$n.in" or die $!; print O, scalar <I> while tell( O ) < ( $n * $chunk ); close O; my $pid; if( $pid = fork() ) { ++$procs{ $pid }; } elsif( defined $pid ) { exec "java -Xmx300m java_code/ALPHAtest -a tables/A.TRAINED -e + tables/E.TRAINED -c tables/conf.tat -f temp$n.in > temp$n.out"; } else { die "Fork failed"; } } while( keys %procs ) { my $pid = wait; delete $procs{ $pid }; } open O, '>', $protienFile . '.out' or die $!; for my $n ( 1 .. 20 ) { open I, '<', "temp$n.out" or die $!; print O, <i>; close I; unlink "temp$n.in", "temp$n.out"; } close O;

    Note:That's untested.

    You might get a little more clever and use the piped-open to run the commands and read the output back directly into the parent for merging, but handling multiple concurrent input streams without mixing up the results gets messy.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
    In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

      If the input data file is really big, it can be worth using named_pipes so you don't have to write copies of the data to the file system.

      You'd need a process to seek to the correct point in the file and then write blocks of data to the named pipe, and then the java program can open the named pipe as a regular file and just read from it. The details vary depending which operating system you're on, and it might be too much effort for your particular problem but it can be a useful technique.

      IIRC named pipes are a lot easier to work with on Linux/Unix than on Windows, but I haven't used them on Windows for some years - so things might have got better since then.

      Oh thanks a lot! I will try it out and see how it works, and, if this is not much to ask, I might bother you again with clarification questions :)