monkfan has asked for the wisdom of the Perl Monks concerning the following question:

Hi,
I usually run this piece code of mine under console shell as follows: 
$ perl mycode.pl filename > ~/MyPerl/result_ans/filename.out

[download]
Now, I want to run this command again but within a Perl code, using Shell module. Here is the snippet of how I do it: 
perl("mycode.pl filename > ~/MyPerl/result_ans/filename.out");

[download]
But this doesn't work. Why?
The full context of what I am trying to do is shown in the following complete script: 
use Shell;
use Proc::Queue size=>5, qw(run_back);

my @list = glob("*.fasta");


foreach my $list (@list) {
     run_back { run_code($list) } 
}

sub run_code {

    my $fname = shift;
    my $base = ( split( /\./, $fname ) )[0];
    perl("mycode.pl $base > ~/MyPerl/result_ans/$base.out");

    # Tried this too, but still doesn't work
    # system("perl mycode.pl $base > ~/MyPerl/result_ans/$base.out");

    return ;
}

1 while wait != -1;

[download]


What I am trying to do above is to run the script in paralel process across the globbed files.
Can anybody advice how can I go about this correctly?

Update: Parallel::ForkManager version of the above code also doesn't work. 
use Parallel::ForkManager;
use Shell;

my @list = glob("*.fasta");

my $pm = new Parallel::ForkManager(10);

foreach my $list (@list) {

    my $pid = $pm->start and next;
    my $base = ( split( /\./, $list) )[0];
    system("perl mycode.pl $base > ~/MyPerl/result_ans/$base.out");

    $pm->finish; # Terminates the child process

}

[download]

Regards,
Edward

Replies are listed 'Best First'.
Re: Running a Perl Code within a Perl Code Using "Shell Module"
by BrowserUk (Patriarch) on Nov 10, 2005 at 03:47 UTC

    You don't need any modules. 

    #! perl -slw
use strict;

## Count the lines in all the .pl file in the CWD
my $count = 0;
for my $file ( glob '*.pl' ) {

    ## The 1, causes the command to be run in the background
    system 1, "wc -l $file"; 
    $count++;
}

print "$count processes started";

    [download]

    Of course, you may want to control the number of concurrent processes, in which case P::FM or one of the others is probably a good idea.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
Re: Running a Perl Code within a Perl Code Using "Shell Module"
by Aristotle (Chancellor) on Nov 10, 2005 at 03:37 UTC 
    find -name '*.fasta' -maxdepth 1 -print0 \
| xargs -r -0 -P5 -n1 sh -c 'perl mycode.pl "$1" > ~/MyPerl/result_ans
+/"$1".out' ''

    [download]

    Whenever you find yourself thinking “I want to start a bunch of processes like this one with different parameters,” xargs probably has your back. When it involves files, find -print0 | xargs -0 is the standard tool.

    Or else you put parallelism into your main script itself – the feasibility of that varies. (In your case it would have to be adapted to create output logfiles per some sort of filename template.)

    Makeshifts last the longest.

[reply]
[d/l]
      Given this directory structure 
      ~/MyPerl
   |
   |__ ans # this stores all the .fasta files (input)
   |__ result_ans # to store the result
   |__ mycode.pl

      [download]
      Does this script does what it should: 
      find  ans/ -name '*.fasta' -maxdepth 1 -print0 \
| xargs -r -0 -P5 -n1 sh -c 'perl mycode.pl "$1" > ~/MyPerl/result_ans
+/"$1".out' ''

      [download]
      Cause it gives this message: 
      : /home/myname/MyPerl/result_ans/ans/hm14_ans.fasta.out: No such file 
+or directory

      [download]

      Regards,
      Edward
[reply]
[d/l]
[select]

        No, that can’t work. As you can see in the error message, the path component is included in the logfile’s path. Just execute this from within ~/MyPerl/ans and omit the path argument to find, that’s the easiest solution.

        Makeshifts last the longest.

[reply]
Re: Running a Perl Code within a Perl Code Using "Shell Module"
by salva (Canon) on Nov 10, 2005 at 11:13 UTC
    I can not see any error on your original script, maybe the problem is on the called script "mycode.pl". You should try to make your script work without any paralelism first: 
    my @list = glob("*.fasta");
foreach my $fname (@list) {
  my ($base) = split /\./, $fname;
  system "perl mycode.pl $base >~/MyPerl/result_ans/$base.out"
}

    [download]
    and later, once you have it working, add the Proc::Queue bits: 
    use Proc::Queue size=>5, qw(system_back);

my @list = glob("*.fasta");
foreach my $fname (@list) {
  my ($base) = split /\./, $fname;
  system_back "perl mycode.pl $base >~/MyPerl/result_ans/$base.out"
}
1 while wait != -1;

    [download]
    You can also activate Proc::Queue trace mode to see what's going on.
[reply]
[d/l]
[select]
Re: Running a Perl Code within a Perl Code Using "Shell Module"
by dragonchild (Archbishop) on Nov 10, 2005 at 03:29 UTC
[reply]

Back to Seekers of Perl Wisdom