Loop is not working

ozdemmer has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Loop is not working by haukex (Archbishop) on Oct 03, 2018 at 19:38 UTC
Welcome to Perl and PerlMonks, ozdemmer! There are quite a few things that should be improved in your script. First of all, you should start your script with the following, which is explained in Use strict and warnings and the Basic debugging checklist: `#!/usr/bin/perl use warnings; use strict; use diagnostics;` [download] When you do this, you will unfortunately see quite a few errors that you need to fix. For example, `cd ..` is not valid Perl (chdir is the right function), and you'll need to declare variables like `@files` and `$s` before using them. Also, you should be checking for errors in several more places, such as opendir (`opendir my $dh, '.' or die "opendir: $!";`) and chdir. This might seem a bit tedious at first, but in the long run, it will make for much better code, which is why these are important habits to get into. I think you might benefit from a read of perlintro. Among other things, you'll be introduced to some of the basic ways to process files. Tie::File is a good start, but note that it is quite inefficient - when you're processing files line-by-line, a `while` loop will be better. A few more notes: You do `opendir(DIR,".");`, but then don't do anything with `DIR`. You only need opendir if you'll be getting a list of files in that directory, but since you have a fixed list of files, it doesn't look like you need this. If you do want to list files, you could take a look at glob, but please note that it has several caveats - but if you use it with a fixed string as an argument (like `my @files = glob('');`), it should be fine. You're using backticks (`...`) to run an external command and get its output, but then are not doing anything with that output. You might want to use system instead, and you should always check for errors. The bare minimum I suggest is `die "exited with \$?=$?" if $?;`, see $?, and I wrote at length about using modules to do this much better here. (Update:* In this case, I might recommend IPC::System::Simple, which takes care of the error handling for you.) You're doing `foreach (@files)`, which will put each filename in the special variable $_. I would recommend that you use a named variable, such as `foreach my $file (@files)`, this will make for much easier to read code. Now, on to your issue: You say you want to work on multiple files, but I see that you are doing `chdir "/mnt/hgfs/ExpAutism/Scripts";` (a fixed directory) and then operating on a file `Level1Run1_spec.fsf` (also a fixed filename). I think your loop probably is running twice, but you're always operating on the same files. I don't see where the filenames you've specified in `@files` come into play? Thank you for posting your code and output. Could you please describe your input and expected behavior more? For example, I don't know if `SubjectASD202` and `SubjectASD203` are files or maybe directories, whether they are located in `/mnt/hgfs/ExpAutism/Scripts` or somewhere else, and so on. The best thing you can do, and the way you'll get help the quickest, is if you could post a Short, Self-Contained, Correct Example: something that we can just download and run and debug for ourselves.	[reply] [d/l] [select]
Re: Loop is not working by toolic (Bishop) on Oct 03, 2018 at 19:27 UTC
If your script hangs, I suspect it is because `/usr/local/fsl/bin/feat` hangs (whatever that is). Observations: You are discarding output from the backticks. For debugging purposes (to see if there is output): print `/usr/local/fsl/bin/feat Level1Run1_spec.fsf`; `cd ..` will not change your directory. You need chdir. The opendir does not really do anything in your code. Customarily, the use lines appear at the top of a script. `$s=$s+1` is typically written as `$s++` For other tips, refer to the Basic debugging checklist.	[reply] [d/l] [select]
Re^2: Loop is not working by ozdemmer (Novice) on Oct 04, 2018 at 01:48 UTC
Thank you so much for taking your time to comment. I used the suggestion by the user named poj. All the best.	[reply]
Re: Loop is not working by poj (Abbot) on Oct 03, 2018 at 20:17 UTC
Consider a simple read/write loop to customise your template #!/usr/bin/perl use strict; use warnings; # do second level statistical analysis run by run # go to the directory where the fsf-file for stats # and post-stats is stored chdir "/mnt/hgfs/ExpAutism/Scripts"; my $infile = 'Level1Run1.fsf'; my $outfile = 'Level1Run1_spec.fsf'; my $program = '/usr/local/fsl/bin/feat'; my @subjects = ("SubjectASD202","SubjectASD203"); foreach my $subject (@subjects){ open my $fh_IN, '<', $infile or die "Could not open $infile ; $!"; open my $fh_OUT, '>', $outfile or die "Could not open $outfile ; $!"; # replace wildcards with specific titles of subjects while (<$fh_IN>){ s/Subjectx/$subject/g; print $fh_OUT $_; } close $fh_IN; close $fh_OUT; system($program,$outfile) == 0 or die "$program $outfile failed: $?"; } print "\n +++ DONE +++ \n"; [download] poj	[reply] [d/l]
Re^2: Loop is not working by ozdemmer (Novice) on Oct 03, 2018 at 22:03 UTC
Dear poj, Thank you so much for your reply and precious time. I implemented your suggestion and here is the code: #!/usr/bin/perl use strict; use warnings; # do first level statistical analysis run by run # go to the directory where the fsf-file for stats # and post-stats is stored chdir "/mnt/hgfs/ExpAutism/Scripts"; my $infile = 'Level1Run1_test.fsf'; my $outfile = 'Level1Run1_test_spec.fsf'; my $program = '/usr/local/fsl/bin/feat'; my @subjects = ("SubjectASD203","SubjectTD106C"); foreach my $subject (@subjects){ open my $fh_IN, '<', $infile or die "Could not open $infile ; $!"; open my $fh_OUT, '>', $outfile or die "Could not open $outfile ; $!"; # replace wildcards with specific titles of subjects while (<$fh_IN>){ s/Subjectx/$subject/g; print $fh_OUT $_; } close $fh_IN; close $fh_OUT; system($program,$outfile) == 0 or die "$program $outfile failed: $?"; } print "\n +++ DONE +++ \n"; [download] The error message on the terminal: `C/usr/local/fsl/bin/feat Level1Run1_test_spec.fsf failed: 2 at Level1Run1_test.pl line 28.` The line 28 is `system($program,$outfile) == 0` I am confused again. The problem seems to be with the feat file but what does "2" mean there? Do you have any suggestions? PS1: The parent folder of each subject folder is `/mnt/hgfs/ExpAutism`and the parent folder of the fsf and pl files is `/mnt/hgfs/ExpAutism/Scripts` Thanks in advance. Best,	[reply] [d/l] [select]
Re^3: Loop is not working by Marshall (Canon) on Oct 04, 2018 at 01:14 UTC
In general, I would not do a chdir from the Perl script and prefer to use a path name. But if you do a chdir, you should check for success. Did `chdir "/mnt/hgfs/ExpAutism/Scripts";` work? -> check the result of all file system operations. In general, I've found that changing the running default directory causes more problems than it fixes. Just a thought.	[reply] [d/l]
Re^4: Loop is not working by haukex (Archbishop) on Oct 04, 2018 at 07:24 UTC
Re^3: Loop is not working by haukex (Archbishop) on Oct 04, 2018 at 07:15 UTC
`die "$program $outfile failed: $?";` ... `C/usr/local/fsl/bin/feat Level1Run1_test_spec.fsf failed: 2 at Level1Run1_test.pl` ... what does "2" mean there? See $? and system for the interpretation of values of `$?`. In this case, it means that the program exited due to signal nr. 2, which normally corresponds to `SIGINT`, which normally corresponds to hitting Ctrl-C on your keyboard, and which would also explain the `C` at the beginning of your output (on my system, it shows as `^C`). So it looks like you hit Ctrl-C while `/usr/local/fsl/bin/feat` was running. If so, why did you do that?	[reply] [d/l] [select]
Re^3: Loop is not working by NetWallah (Canon) on Oct 03, 2018 at 23:35 UTC
How did "C" get pre-pended in the output: `C/usr/local/fsl/bin/feat` [download] ? The assignment was : `my $program = '/usr/local/fsl/bin/feat';` [download] which should have worked. If you need the drive letter, use "C:". Memory fault -- brain fried	[reply] [d/l] [select]
Re^3: Loop is not working by stevieb (Canon) on Oct 03, 2018 at 23:16 UTC
This is a huge guess here, but I feel that this is due to a `#define ENOENT 2 /* No such file or directory */` that may have been encountered. You may be in a directory that you think you are but aren't, or possibly trying to work on a file from a location where it doesn't exist. Can you change: `system($program,$outfile) == 0 or die "$program $outfile failed: $?";` [download] ...to: `print -x $program . "\n"; open my $fh, '<', $outfile or die $!;` [download] ...within the exact same code and see if you get an error?	[reply] [d/l] [select]
Re: Loop is not working by jwkrahn (Abbot) on Oct 04, 2018 at 00:53 UTC
Another problem with your code: `for (@array) { #replace wildcards with specific titles of subjects s/Subjectx/$files[$s]/g; }` [download] You are using a fixed string for a regular expression which means that the first time through the loop the string "Subjectx" will be replaced so that the second time through the loop the string will not exist and nothing will be replaced.	[reply] [d/l]
Re^2: Loop is not working by ozdemmer (Novice) on Oct 04, 2018 at 01:42 UTC
Thanks to everyone for commenting on my question. I just added the following line `use diagnostics;` and reran the code. The loop worked successfully, and the result seems to be okay too. However, I received this long and confusing message in the terminal. What does this message mean exactly? sluser@localhost Scripts]$ perl Level1Run1_test.pl 1538605204768 addons.update-checker WARN onUpdateCheckComplet +e failed to parse update manifest: [Exception... "Update manifest is +missing a required addons property." nsresult: "0x80004005 (NS_ERROR +_FAILURE)" location: "JS frame :: resource://gre/modules/addons/Addo +nUpdateChecker.jsm :: getRequiredProperty :: line 473" data: no] Sta +ck trace: getRequiredProperty()@resource://gre/modules/addons/AddonUp +dateChecker.jsm:473 < parseJSONManifest()@resource://gre/modules/addo +ns/AddonUpdateChecker.jsm:483 < UpdateParser.prototype.onLoad/parser( +)@resource://gre/modules/addons/AddonUpdateChecker.jsm:655 < UpdatePa +rser.prototype.onLoad()@resource://gre/modules/addons/AddonUpdateChec +ker.jsm:675 < UpdateParser/<()@resource://gre/modules/addons/AddonUpd +ateChecker.jsm:593 1538605204784 addons.update-checker WARN onUpdateCheckComplet +e failed to parse update manifest: [Exception... "Update manifest is +missing a required addons property." nsresult: "0x80004005 (NS_ERROR +_FAILURE)" location: "JS frame :: resource://gre/modules/addons/Addo +nUpdateChecker.jsm :: getRequiredProperty :: line 473" data: no] Sta +ck trace: getRequiredProperty()@resource://gre/modules/addons/AddonUp +dateChecker.jsm:473 < parseJSONManifest()@resource://gre/modules/addo +ns/AddonUpdateChecker.jsm:483 < UpdateParser.prototype.onLoad/parser( +)@resource://gre/modules/addons/AddonUpdateChecker.jsm:655 < UpdatePa +rser.prototype.onLoad()@resource://gre/modules/addons/AddonUpdateChec +ker.jsm:675 < UpdateParser/<()@resource://gre/modules/addons/AddonUpd +ateChecker.jsm:593 1538605204810 addons.update-checker WARN onUpdateCheckComplet +e failed to parse update manifest: [Exception... "Update manifest is +missing a required addons property." nsresult: "0x80004005 (NS_ERROR +_FAILURE)" location: "JS frame :: resource://gre/modules/addons/Addo +nUpdateChecker.jsm :: getRequiredProperty :: line 473" data: no] Stack trace: getRequiredProperty()@resource://gre/modules/a +ddons/AddonUpdateChecker.jsm:473 < parseJSONManifest()@resource://gre +/modules/addons/AddonUpdateChecker.jsm:483 < UpdateParser.prototype.o +nLoad/parser()@resource://gre/modules/addons/AddonUpdateChecker.jsm:6 +55 < UpdateParser.prototype.onLoad()@resource://gre/modules/addons/Ad +donUpdateChecker.jsm:675 < UpdateParser/<()@resource://gre/modules/ad +dons/AddonUpdateChecker.jsm:593 1538605204818 addons.update-checker WARN onUpdateCheckComplet +e failed to parse update manifest: [Exception... "Update manifest is +missing a required addons property." nsresult: "0x80004005 (NS_ERROR +_FAILURE)" location: "JS frame :: resource://gre/modules/addons/Addo +nUpdateChecker.jsm :: getRequiredProperty :: line 473" data: no] Sta +ck trace: getRequiredProperty()@resource://gre/modules/addons/AddonUp +dateChecker.jsm:473 < parseJSONManifest()@resource://gre/modules/addo +ns/AddonUpdateChecker.jsm:483 < UpdateParser.prototype.onLoad/parser( +)@resource://gre/modules/addons/AddonUpdateChecker.jsm:655 < UpdatePa +rser.prototype.onLoad()@resource://gre/modules/addons/AddonUpdateChec +ker.jsm:675 < UpdateParser/<()@resource://gre/modules/addons/AddonUpd +ateChecker.jsm:593 1538605204826 addons.update-checker WARN onUpdateCheckComplet +e failed to parse update manifest: [Exception... "Update manifest is +missing a required addons property." nsresult: "0x80004005 (NS_ERROR +_FAILURE)" location: "JS frame :: resource://gre/modules/addons/Addo +nUpdateChecker.jsm :: getRequiredProperty :: line 473" data: no] Sta +ck trace: getRequiredProperty()@resource://gre/modules/addons/AddonUp +dateChecker.jsm:473 < parseJSONManifest()@resource://gre/modules/addo +ns/AddonUpdateChecker.jsm:483 < UpdateParser.prototype.onLoad/parser( +)@resource://gre/modules/addons/AddonUpdateChecker.jsm:655 < UpdatePa +rser.prototype.onLoad()@resource://gre/modules/addons/AddonUpdateChec +ker.jsm:675 < UpdateParser/<()@resource://gre/modules/addons/AddonUpd +ateChecker.jsm:593 + DONE +++ [download] Just in case you need to see the code I used again: #!/usr/bin/perl use strict; use warnings; use diagnostics; # do second level statistical analysis run by run # go to the directory where the fsf-file for stats # and post-stats is stored chdir "/mnt/hgfs/ExpAutism/Scripts"; my $infile = 'Level1Run1_test.fsf'; my $outfile = 'Level1Run1_test_spec.fsf'; my $program = '/usr/local/fsl/bin/feat'; my @subjects = ("SubjectASD203","SubjectTD106C"); foreach my $subject (@subjects){ open my $fh_IN, '<', $infile or die "Could not open $infile ; $!"; open my $fh_OUT, '>', $outfile or die "Could not open $outfile ; $!"; # replace wildcards with specific titles of subjects while (<$fh_IN>){ s/Subjectx/$subject/g; print $fh_OUT $_; } close $fh_IN; close $fh_OUT; system($program,$outfile) == 0 or die "$program $outfile failed: $?"; } print "\n +++ DONE +++ \n"; [download] Thanks a ton!	[reply] [d/l] [select]
Re^3: Loop is not working by 1nickt (Canon) on Oct 04, 2018 at 02:38 UTC
Hi, that message seems to refer to an issue with your Mozilla-based browser. I don't know how this comes into play in your program; I did look at the source of feat a little bit and saw that it outputs HTML, so maybe related? Anyway I wanted to point you to one of many ways to capture all the output from your external command, Capture::Tiny. You could use something like this to examine the output and perhaps see where the source of the message is: `use Capture::Tiny ':all'; my ($stdout, $stderr, $exit) = capture { system( $program, $outfile ); }; print sprintf('Stderr: >%s< Stdout: >%s<', $stderr, $stdout); if ( $exit != 0 ) { die sprintf('%s exited with status %s.', $program, $exit; }` [download] Hope this helps! The way forward always starts with a minimal test.	[reply] [d/l]
Re^3: Loop is not working by haukex (Archbishop) on Oct 04, 2018 at 07:39 UTC
I agree with 1nickt that this looks like error messages from a Mozilla browser. At the moment I don't see the version of `feat` that 1nickt found launching a browser though - could you point us to where you got `feat` and its accompanying tools? So in other words this isn't Perl outputting these messages, and I think you could probably just ignore them for now, as long as the rest of the code runs and the output looks good.	[reply] [d/l] [select]
Re^4: Loop is not working by ozdemmer (Novice) on Oct 05, 2018 at 16:52 UTC