Process each file one by one.

vinoth.ree has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks

I have a two files in a directory, I need to read that directory and need to process each file that directory contains.Both the file contains database queries.What I did say that I opened that directory and read the directory and saved all the files names into an array. Then I used foreach loop inside of that and used the system command to process those files,When those files get process at the same time it also update a log file says that query executed successfully. I need to process each file one by one only. When I check the log files both the log files get updated simultaneously, I thought that after a file completes another file will get process, but both the file get processing. how to process those files one by one.

Below is that foreach loop


foreach(@files{
system "perl download ../downloads/files_download/$_ 1>/tmp/$_.log 2>/
+dev/null &\n ";
}
[download]

Update:

Title updated.

Comment on Process each file one by one. Download Code

Replies are listed 'Best First'.
Re: Process a file one by one. by graff (Chancellor) on Sep 04, 2009 at 05:49 UTC
I gather from your "foreach" loop that "download" is the name of some perl script, and that "@files" has already been filled to hold the two file names that you are talking about. Well, if the two files are not very big, and if the "download" script is fairly simple, it might look like the two files are being processed "at the same time", because the whole operation on each file takes a very small fraction of a second. The foreach loop pretty much dictates that the two files are being done one at a time, in sequence. What evidence do you have that the two log files are being "updated simultaneously" (apart from the modification time on the files)? If you want to see a delay between the updates of the two files, try putting a sleep call inside the foreach loop, right before or right after the system call. Oh, and do you understand what the "&" is doing at the end of the command line you pass to system()? On a linux/unix box, that causes each command to be run "in the background"; the shell exits immediately, but the perl process that it runs keeps running until it finishes on its own. If you want to make sure the two iterations do not overlap in time, you have to get rid of the "&" at the end. (update: one more thing -- you don't need the "\n" at the end of the command-line string you pass to system.)	[reply]
Re: Process a file one by one. by ig (Vicar) on Sep 04, 2009 at 05:46 UTC
Since you haven't shown the value of `@files` it is difficult to be certain what your program fragment does. You should examine (print them or use the debugger) the values of `@files` to ensure they are what they should be and what you think they are. If they are not, you will have to correct the code that sets `@files`. If `@files` has reasonable values (i.e. simple file names, no wildcards, no shell metacharacters, etc.) then your program fragment looks like it will run your download script once for each file. What is it that makes you think that both log files are updated simultaneously? If it is that they have the same modification time, it may be that your program runs to completion within one second. If you want to ensure that the files are being processed at different times and that the log files will have different timestamps, you can add a delay to your loop: `foreach (@files) { print "processing \"$_\"\n"; system "perl download ../downloads/files_download/$_ 1>/tmp/$_.log + 2>/dev/null &\n "; sleep 5; }` [download] If that doesn't resolve your problem, then you should look at your "download" program.	[reply] [d/l] [select]
Re^2: Process a file one by one. by vinoth.ree (Monsignor) on Sep 04, 2009 at 06:43 UTC
I already checked the @files, its contains only the file names nothing else	[reply]
Re^3: Process a file one by one. by Anonymous Monk on Sep 04, 2009 at 07:06 UTC
You've said that before and you were mistaken, you need to prove it.	[reply]
Re^4: Process a file one by one. by vinoth.ree (Monsignor) on Sep 04, 2009 at 07:20 UTC
Re: Process each file one by one. by vitoco (Hermit) on Sep 04, 2009 at 13:30 UTC
All files are being processed simultaneously because you added the "`&`" at the end of the line. That says to the system shell not to wait for the command to complete, leaving it running in the background. Remove the "`&`" (and the "`\n`") and try again...	[reply] [d/l] [select]