Handling multiple output files simultaneously using arrays of filehandles and filenames

Cosmic37 has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Handling multiple output files simultaneously using arrays of filehandles and filenames by pryrt (Abbot) on Jan 21, 2021 at 16:35 UTC
The ">" should not be part of the filename; since you are using the three-argument version of open (good for you), the `>` is already specified in the open command, and doesn't need to be in the filename as well. Change `$sectorOutputFiles[$sector]='>'.$path.$filenamefirstpart.$sector.".txt";` to `$sectorOutputFiles[$sector]=$path.$filenamefirstpart.$sector.".txt";`, and it will work. (At least, it did for me when I declared `my $filenamefirstpart = '';` and fixed the filename.) Working SSCCE: `use strict; use warnings; my ($sector,$path); my @sectorOutputFiles; my @fh; $path="blah"; my $filenamefirstpart=''; for($sector=0;$sector<12;$sector++){ $sectorOutputFiles[$sector]=$path.$filenamefirstpart.$sector.".txt +"; print STDERR $sectorOutputFiles[$sector], "\n"; open($fh[$sector],'>',$sectorOutputFiles[$sector]) or die "Can't open sector filehandler for writing: $!\n"; }` [download]	[reply] [d/l] [select]
Re^2: Handling multiple output files simultaneously using arrays of filehandles and filenames by Cosmic37 (Acolyte) on Jan 21, 2021 at 20:19 UTC
Thank you for that great advice. That was a dunderhead mistake and has indeed arisen out of changing version of open arguments.	[reply]
Re: Handling multiple output files simultaneously using arrays of filehandles and filenames by talexb (Chancellor) on Jan 21, 2021 at 16:30 UTC
What does Perl say? Just looking at this code, I see you are missing `$!` in your error message -- and that would tell you what the problem is. You also don't need "\n" in the `die` message. It also looks like you have not defined `$filenamefirstpart`, so `strict` should report that's undefined. I highly recommend you put together a self-contained, working example before you post. We'll help you with stuff you don't understand, but you need to have a working example that compiles first. Alex / talexb / Toronto Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.	[reply] [d/l] [select]
Re^2: Handling multiple output files simultaneously using arrays of filehandles and filenames by Cosmic37 (Acolyte) on Jan 21, 2021 at 20:22 UTC
Thank you for that interesting advice. Please may I ask what is the basis upon which you presume that I need $! and no \n in the die output? Is this not a free choice for the programmer? Is there a particular weakness you are referring to?	[reply]
Re^3: Handling multiple output files simultaneously using arrays of filehandles and filenames by talexb (Chancellor) on Jan 21, 2021 at 21:50 UTC
The `$!` literally tells you what the error is. I have no idea why you would want to throw that information away by not displaying it. The "\n" in the output isn't necessary because `die` takes care of adding a newline automatically -- but I guess that's a matter of taste. Add one if you like. I never add newlines to `die` or `warn` statements. Alex / talexb / Toronto Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.	[reply] [d/l] [select]
Re^3: Handling multiple output files simultaneously using arrays of filehandles and filenames by Fletch (Bishop) on Jan 21, 2021 at 20:29 UTC
You need `$!` because otherwise you only know that there's been an error of some sort and not what specific error occurred. Just grobbling through the open(2) manual page on OS X there's 22 distinct error codes in 32 different cases which can be returned at the C level. The trailing newline is mostly an aesthetic choice though. The cake is a lie. The cake is a lie. The cake is a lie.	[reply] [d/l]
Re^4: Handling multiple output files simultaneously using arrays of filehandles and filenames by bliako (Monsignor) on Jan 21, 2021 at 20:57 UTC
Re^5: Handling multiple output files simultaneously using arrays of filehandles and filenames by Cosmic37 (Acolyte) on Jan 21, 2021 at 21:17 UTC
Some notes below your chosen depth have not been shown here
Re^4: Handling multiple output files simultaneously using arrays of filehandles and filenames by Cosmic37 (Acolyte) on Jan 21, 2021 at 21:04 UTC
Re: Handling multiple output files simultaneously using arrays of filehandles and filenames by AnomalousMonk (Archbishop) on Jan 21, 2021 at 18:27 UTC
The other part of this is that when you use a file handle that is an array or hash element, you must IIRC use the syntax `print { $fh[5] } "something \n";` (note the curly braces around the file handle that form a scope). I'd like to provide a doc link, but I can't offhand remember where this is discussed. I may be able to look this up later, but in the meantime perhaps another monk may fill the gap. :) Give a man a fish: `<%-{-{-{-<`	[reply] [d/l] [select]
Re^2: Handling multiple output files simultaneously using arrays of filehandles and filenames by pryrt (Abbot) on Jan 21, 2021 at 18:33 UTC
I'd like to provide a doc link print If you're storing handles in an array or hash, or in general whenever you're using any expression more complex than a bareword handle or a plain, unsubscripted scalar variable to retrieve it, you will have to use a block returning the filehandle value instead, in which case the LIST may not be omitted	[reply]
Re^3: Handling multiple output files simultaneously using arrays of filehandles and filenames by Cosmic37 (Acolyte) on Jan 21, 2021 at 20:28 UTC
Thanks for that reference. I had indeed read that document earlier today but suddenly felt drained of all energy when I read the phrase "bareword handle" which meant nothing to me. I must learn more. I must learn more. I must learn more. Thrash. Whip. I am a member of Opus Doh. :-D	[reply]
Re^2: Handling multiple output files simultaneously using arrays of filehandles and filenames by eyepopslikeamosquito (Archbishop) on Jan 22, 2021 at 01:09 UTC
A stylistic advice reference is Perl Best Practices, Chapter 10 (I/O), Item 136: Always put filehandles in braces within any print statement: It's easy to lose a lexical filehandle that's being used in the argument list of a print: `print $file $name, $rank, $serial_num, "\n";` [download] Putting braces around the filehandle helps it stand out clearly: `print {$file} $name, $rank, $serial_num, "\n";` [download] The braces also convey your intentions regarding that variable; namely that you really did mean it to be treated as a filehandle, and just didn't forget a comma. An alternative syntax using IO::Handle: `use IO::Handle; $file->print( $name, $rank, $serial_num, "\n" );` [download]	[reply] [d/l] [select]
Re^2: Handling multiple output files simultaneously using arrays of filehandles and filenames by Cosmic37 (Acolyte) on Jan 21, 2021 at 20:56 UTC
Thanks - you are right that I want to open these files in order that I will under some logical conditions write a data line to them. One of my data items per line is indeed a $sector value and I will breakd down the data by $sector, outputting each line of input to the relevant output file per sector. My thinking was to use the following but it looks like you are advising me to insert curly braces which I will try although I don't really understand why this would make a difference and why print would have a problem sending a string to a file handler without curly braces... `$mydata=<INPUT>; while(<INPUT>){ #Handle data file line by line splitting it at every new line \n my(@data)=split/\n/,$_; #Now lets go through the whole input data file line by line for($i=0;$i<@data;$i++){ $mydata=$data[$i]; if($mydata=~/^Blah$some_regex(\d+)$some_more_regex_perhaps/){ #Get the sector value which is listed in each matching line of data $DataSector=$1; #Output line of matching data to output file for the relevant sector print $fh[$dataSector] "$mydata\n"; } }` [download]	[reply] [d/l]
Re^3: Handling multiple output files simultaneously using arrays of filehandles and filenames by choroba (Cardinal) on Jan 21, 2021 at 21:22 UTC
> why print would have a problem sending a string to a file handler without curly braces You are conflating two levels: parsing of the source code and execution. When executed, print has no problem to send a string to a file handler, but there aren't any curlies anymore. The problem is the parser first needs to find out what and where to you want to print. And Perl parser is very simple and can never look ahead more than one token. Therefore, it can't check whether there's a comma after the `$fh[$dataSector]`, because it only sees `$fh` and the next token is the index. `map{substr$_->[0],$_->[1]\|\|0,1}[\\|\|{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^ARGV,3]`	[reply] [d/l] [select]
Re^3: Handling multiple output files simultaneously using arrays of filehandles and filenames by AnomalousMonk (Archbishop) on Jan 21, 2021 at 23:21 UTC
`... #Output line of matching data to output file for the relevant sector print $fh[$dataSector] "$mydata\n"; ...` [download] So the syntax you need to use is `print { $fh[$dataSector] } "$mydata\n";` (extra spacing around the curlies just for visibility/clarity). Give a man a fish: `<%-{-{-{-<`	[reply] [d/l] [select]
Re^4: Handling multiple output files simultaneously using arrays of filehandles and filenames by Cosmic37 (Acolyte) on Jan 22, 2021 at 14:01 UTC
Re^5: Handling multiple output files simultaneously using arrays of filehandles and filenames by AnomalousMonk (Archbishop) on Jan 22, 2021 at 19:49 UTC
Re: Handling multiple output files simultaneously using arrays of filehandles and filenames by betmatt (Scribe) on Jan 21, 2021 at 21:19 UTC
Hi, Nice to be called Monkees rather than Monks, :) There maybe are variables in your code that you haven't yet specified. I know that you probably have deleted them prior to posting, however it is better to indicate that. Hence the "Have you tested the code to get an error message" comment. As things stand it looks like you've written this out without testing the code, because of the absence of variable contents.	[reply]
Re^2: Handling multiple output files simultaneously using arrays of filehandles and filenames by xiaoyafeng (Deacon) on Jan 22, 2021 at 00:38 UTC
Sorry for asking a question out of the topic, As English is not my native language, I don't understand why Monkees are better than Monks, it sounds like monkeys? google told me Monkees is a popular band, But I still confused. Please enlighten me. ;) I am trying to improve my English skills, if you see a mistake please feel free to reply or /msg me a correction	[reply]
Re^3: Handling multiple output files simultaneously using arrays of filehandles and filenames by talexb (Chancellor) on Jan 22, 2021 at 03:21 UTC
The name 'Perlmonks' is a portmanteau word that combines 'Perl' with 'monks'; monks are religious followers who live together in a monastery. Thus, members of this website sometimes refer to each other as monks -- a description that is more metaphorical and less literal. In English, the ending 'y', 'ie' or 'ee' (all homonyms) are added to the end of nouns to make them cuter, and may also refer to a smaller version of the original. Thus, you might have a dog (perhaps full-grown), and also a doggy (perhaps a younger dog). They could also be described as a pup and a puppy (this isn't such a good example -- pup usually means a young dog in any case). Taking those two paragraphs as a starting point, from 'monk' you could get 'monkee', which might mean a younger monk, or a cute name for the regulars of this community. You're right that Google told you The Monkees were a pop band back in the 60's and 70's -- this is a typical American mis-spelling to catch the eye, and is also a play on the band that they were copying, The Beatles (see: beetles -> beatles, because they played music with a beat). Another example of an intentional mis-spelling is 'tonite', a simplification of 'tonight' that's also fewer letters, so can be made from larger letters on a sign. The English word night may be descended from the German word for the same thing, nacht, but nite is a homonym, so can be used in place of night. (OK, the British may squirm -- whatever.) Heh. Language is fun. Alex / talexb / Toronto Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.	[reply]
Re^4: Handling multiple output files simultaneously using arrays of filehandles and filenames by Fletch (Bishop) on Jan 22, 2021 at 16:22 UTC
Re^4: Handling multiple output files simultaneously using arrays of filehandles and filenames by xiaoyafeng (Deacon) on Jan 25, 2021 at 02:59 UTC
Re^3: Handling multiple output files simultaneously using arrays of filehandles and filenames by jcb (Parson) on Jan 22, 2021 at 02:24 UTC
I took it as an odd "cute" name for monks. Some of our questioners come up with unusual greetings like that.	[reply]
Re^4: Handling multiple output files simultaneously using arrays of filehandles and filenames by Cosmic37 (Acolyte) on Jan 22, 2021 at 13:56 UTC
Re^2: Handling multiple output files simultaneously using arrays of filehandles and filenames by Cosmic37 (Acolyte) on Jan 21, 2021 at 21:56 UTC
Hey hey we are the monkees... you are correct that I have deleted some code in order to remove some references that I wished to remain private; apologies if this made things less clear, I can see how that might be.	[reply]
Re^3: Handling multiple output files simultaneously using arrays of filehandles and filenames by AnomalousMonk (Archbishop) on Jan 21, 2021 at 23:45 UTC
The importance of providing a small, working example of code that exhibits your problem has already been mentioned here by pryrt (who even gave a small, working example :). Such an example (along with input data, expected and actual output, and warning/error messages) can greatly help us to help you. Again, please see Short, Self-Contained, Correct Example. And the process of composing an SSCCE can often lead you to understand the problem entirely on your own. Give a man a fish: `<%-{-{-{-<`	[reply] [d/l]


laziness, impatience, and hubris
	PerlMonks