script assistance please

perlnobie has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: script assistance please by hippo (Archbishop) on Feb 19, 2015 at 21:47 UTC
Some guidance, as requested: use strict use warnings indent your code to improve legibility ... and maybe some whitespace within the lines might help too comment your code liberally use the 3-argument form of open check your return values from open() use lexical filehandles don't assign to `@_` without good reason go through the (rest of the) Basic debugging checklist If you do all that, you'll be in a much better place to determine where you are going wrong.	[reply] [d/l]
Re^2: script assistance please by marinersk (Priest) on Feb 19, 2015 at 23:07 UTC
I'm out of votes for the day, but if I could, I would ++hippo for each of the following items from that checklist which were part of my troubleshooting steps: 1, 3, 4, 7, 10 I would further point out that things I had either intended to do, or would have done with my own work, include: 1, 2, 3, 4, 5, 6, 7, 9, 10 The only item missing is #8, and I don't use those merely out of habit -- but I do ensure every handle is unique globally, which is a less foolproof system, but due to my own rigorous adherence to my own standards, has never failed me in a few decades of using Perl. :-)	[reply]
Re^3: script assistance please by karlgoethebier (Abbot) on Feb 20, 2015 at 15:12 UTC
"..I don't use those merely out of habit...ensure every handle is unique globally...due to my own rigorous adherence...never failed me in a few decades..." The day will come...even after decades of luck. This applies also to the misuse of `@_`. Best regards, Karl ŤThe Crux of the Biscuit is the Apostropheť	[reply] [d/l]
Re^4: script assistance please by marinersk (Priest) on Feb 20, 2015 at 15:34 UTC
Re^5: script assistance please by karlgoethebier (Abbot) on Feb 20, 2015 at 17:37 UTC
Some notes below your chosen depth have not been shown here
Re: script assistance please by marinersk (Priest) on Feb 19, 2015 at 21:43 UTC
Made the following changes: Added `strict` to aid in debugging; Slight reformat for readability: `#!/usr/bin/perl use strict; $moyr="12/2014"; $,=" "; open(IN,"<COS_SGA_xref.txt"); while(<IN>) { chop; @_ = split(/ /); $csga{$_[0]}=$_[1]; } open(IN,"<cc.txt"); open(OUT,">new_cc.txt"); for($i=0;$i<9;$i++) {$_=<IN>;} ($mo,$yr)=@_ = split(/\//,$moyr); while(<IN>) { chop; @_ = split(/ /); $created=pop(@_); ($mocr,$day,$yrcr)=@_ = split(/\//,$created); if($yrcr.$mocr >= $yr.$mo) { $cc=$_[1]; $cc =~ y/C_//d; splice(@_,3,1); shift(@_); print OUT $created,@_,$csga{$cc}; print OUT "\n"; } } __END__` [download] Now for the errors: D:\PerlMonks>test1.pl Global symbol "$moyr" requires explicit package name at D:\PerlMonks\t +est1.pl line 5. Global symbol "%csga" requires explicit package name at D:\PerlMonks\t +est1.pl line 11. Global symbol "$i" requires explicit package name at D:\PerlMonks\test +1.pl line 15. Global symbol "$i" requires explicit package name at D:\PerlMonks\test +1.pl line 15. Global symbol "$i" requires explicit package name at D:\PerlMonks\test +1.pl line 15. Global symbol "$mo" requires explicit package name at D:\PerlMonks\tes +t1.pl line 16. Global symbol "$yr" requires explicit package name at D:\PerlMonks\tes +t1.pl line 16. Global symbol "$moyr" requires explicit package name at D:\PerlMonks\t +est1.pl line 16. Global symbol "$created" requires explicit package name at D:\PerlMonk +s\test1.pl line 20. Global symbol "$mocr" requires explicit package name at D:\PerlMonks\t +est1.pl line 21. Global symbol "$day" requires explicit package name at D:\PerlMonks\te +st1.pl line 21. Global symbol "$yrcr" requires explicit package name at D:\PerlMonks\t +est1.pl line 21. Global symbol "$created" requires explicit package name at D:\PerlMonk +s\test1.pl line 21. Global symbol "$yrcr" requires explicit package name at D:\PerlMonks\t +est1.pl line 22. Global symbol "$mocr" requires explicit package name at D:\PerlMonks\t +est1.pl line 22. Global symbol "$yr" requires explicit package name at D:\PerlMonks\tes +t1.pl line 22. Global symbol "$mo" requires explicit package name at D:\PerlMonks\tes +t1.pl line 22. Global symbol "$cc" requires explicit package name at D:\PerlMonks\tes +t1.pl line 23. Global symbol "$cc" requires explicit package name at D:\PerlMonks\tes +t1.pl line 24. Global symbol "$created" requires explicit package name at D:\PerlMonk +s\test1.pl line 27. Global symbol "%csga" requires explicit package name at D:\PerlMonks\t +est1.pl line 27. Global symbol "$cc" requires explicit package name at D:\PerlMonks\tes +t1.pl line 27. Execution of D:\PerlMonks\test1.pl aborted due to compilation errors. D:\PerlMonks> [download] I will start debugging this, but you should also. Holler if the above error messages don't make sense to you.	[reply] [d/l] [select]
Re^2: script assistance please by marinersk (Priest) on Feb 19, 2015 at 21:55 UTC
Made the following changes: Closed your files when you were done with them.; Declared your variables (required under `strict` and intensely recommended for a huge list of reasons) #!/usr/bin/perl use strict; my %csga = (); my $moyr="12/2014"; $,="\t"; open(IN,"<COS_SGA_xref.txt"); while(<IN>) { chop; @_ = split(/\t/); $csga{$_[0]}=$_[1]; } close IN; open(IN,"<cc.txt"); open(OUT,">new_cc.txt"); for(my $i=0;$i<9;$i++) {$_=<IN>;} my ($mo,$yr)=@_ = split(/\//,$moyr); while(<IN>) { chop; @_ = split(/\t/); my $created=pop(@_); my ($mocr,$day,$yrcr)=@_ = split(/\//,$created); if($yrcr.$mocr >= $yr.$mo) { my $cc=$_[1]; $cc =~ y/C_//d; splice(@_,3,1); shift(@_); print OUT $created,@_,$csga{$cc}; print OUT "\n"; } } close IN; close OUT; __END__ [download] And the results: `D:\PerlMonks>test2.pl D:\PerlMonks>` [download] It created an empty output file called `new_cc.txt`. I presume that's because I did not have the input file `COS_SGA_xref.txt`. I'm afraid additional debugging will be all guesswork unless you can provide sample input data. At a guess, however, your problem is one of two things: Since you didn't close `IN`, your attempt to open it again failed. If you would check for errors on `open`, you could catch such things. You only write output if this statement is true: `if($yrcr.$mocr >= $yr.$mo)`. Maybe it never is? A `print` statement right before the `if` could reveal the issue.	[reply] [d/l] [select]
Re^3: script assistance please by marinersk (Priest) on Feb 19, 2015 at 22:09 UTC
Made the following changes: Checked status of `open` statements and displayed errors where appropriate; Displayed suggested debugging statements. NOTE: These might mess with your use of `$_` and `@_`. I haven't used those in so long I've forgotten the rules. #!/usr/bin/perl use strict; my %csga = (); my $moyr="12/2014"; $,="\t"; if (!open(IN,"<COS_SGA_xref.txt")) { print "ERROR: Cannot open input file COS_SGA_xref.txt\n"; } else { while(<IN>) { # Using chop instead of chomp here makes me nervous - marine +rsk chop; @_ = split(/\t/); $csga{$_[0]}=$_[1]; } close IN; if (!open(IN,"<cc.txt")) { print "ERROR: Cannot open input file cc.txt\n"; } else { if (!open(OUT,">new_cc.txt")) { print "ERROR: Cannot open output file new_cc.txt\n"; } else { for(my $i=0;$i<9;$i++) {$_=<IN>;} my ($mo,$yr)=@_ = split(/\//,$moyr); while(<IN>) { # Use of chop instead of chomp here makes me nervo +us - marinersk chop; @_ = split(/\t/); my $created=pop(@_); my ($mocr,$day,$yrcr)=@_ = split(/\//,$created); print "DEBUG: -----------------------------\n"; print "DEBUG: \$yrcr.\$mocr = [$yrcr.$mocr]\n"; print "DEBUG: \$yr.\$mo = [$yr.$mo]\n"; if($yrcr.$mocr >= $yr.$mo) { my $cc=$_[1]; $cc =~ y/C_//d; splice(@_,3,1); shift(@_); print OUT $created,@_,$csga{$cc}; print OUT "\n"; } } # Cleanup close OUT; } # Cleanup close IN; } } __END__ [download] And the results: `D:\PerlMonks>test3.pl ERROR: Cannot open input file COS_SGA_xref.txt D:\PerlMonks>` [download] Any further debugging requires information from you. Good luck!	[reply] [d/l] [select]
Re^4: script assistance please by perlnobie (Initiate) on Feb 19, 2015 at 22:16 UTC
Re^5: script assistance please by marinersk (Priest) on Feb 19, 2015 at 22:58 UTC
Re^3: script assistance please by perlnobie (Initiate) on Feb 19, 2015 at 22:12 UTC
Marinersk, thanks for your reply. I hope this works out. Sample data COS_SGA_xref.txt 195003 COS 195010 COS 500512 COS 500511 COS 520536 COS 550551 COS 550625 COS 560119 COS cc.txt 01/31/2015 Dynamic List Display 1 Controlling Area C2 Date 01/01/1900 To 12/31/9999 Cost Center All Cost Centers Cost Ctr Description CoCd Profit Ctr Person Responsible Created on 6 PCS<>PCS MAIL CLOSED per B. Middleton 301 6000 Garrity, T. 05/24/2004 30 COST CENTER 000001 170 112 Mac Crawford 08/30/2004 50 150BS - Balance Sheet C/C-(No RT) 150 112 Mac Crawford 08/16/2004 60 COST CENTER 000001 160 112 Mac Crawford 08/16/2004 60 COST CENTER 000001 170 112 Mac Crawford 08/16/2004 70 COST CENTER 000001 170 112 Mac Crawford 08/16/2004 71 COST CENTER 000001 170 112 Mac Crawford 08/16/2004 74 COST CENTER 000001 170 112 Mac Crawford 08/16/2004 76 COST CENTER 000001 170 112 Mac Crawford 08/30/2004 77 COST CENTER 000001 170 112 Mac Crawford 08/30/2004 78 COST CENTER 000001 170 112 Mac Crawford 08/30/2004 80 State of NY 351 112 Bob Achettu 02/09/2006 80 State of NY 301 112 Bob Achettu 02/09/2006	[reply]
Re: script assistance please by karlgoethebier (Abbot) on Feb 20, 2015 at 14:45 UTC
Some more less or more useful little hints: Useful pragmas: `use strict; use warnings; use feature qw(say); use Data::Dump;` [download] Split (if you really need it): `my $moyr = "12/2014"; my ( $month, $year ) = split /\//, $moyr; say for ( $month, $year );` [download] Building the hash: `my %csga; while (<$in>) { my @record = split /\t/; chomp @record; $csga{ $record[0] } = $record[1]; } close $in; dd \%csga;` [download] Skipping headers when reading a file: `open( my $in, "<", q(cc.txt) ) \|\| die $!; while (<$in>) { next if $. <= 8; # print; } close $in;` [download] Best regards, Karl ŤThe Crux of the Biscuit is the Apostropheť	[reply] [d/l] [select]
Re^2: script assistance please by Laurent_R (Canon) on Feb 20, 2015 at 20:03 UTC
Hmm, for skipping a header, I usually prefer to do it before entering the main loop. For example: `open( my $in, "<", q(cc.txt) ) \|\| die $!; $useless_header_line = <$in> for 1..8; while (<$in>) { # print; }` [download] Two reasons for that. It is more self-documenting. If my file has 500 million lines (and, yes, I am commonly dealing with such file sizes), I do not like the idea to make a useless test so many times. The test is pretty fast and has probably relatively limited impact of performance, but, still, it is still against my sense of economy or ecology. Je suis Charlie.	[reply] [d/l]
Re^3: script assistance please by RonW (Parson) on Feb 20, 2015 at 21:14 UTC
When not using `<>` that makes sense. Though could also do it when using `<>` when also using `continue` `while (<>) { last unless (/^#/); } procline; while (<>) { procline; } continue { if (eof) { while (<>) { last unless (/^#/); } procline; } }` [download]	[reply] [d/l] [select]
Re^3: script assistance please by karlgoethebier (Abbot) on Feb 21, 2015 at 17:55 UTC
"...it is still against my sense of economy or ecology" Yes sure. But as i wrote: "...less or more useful little hints" IMHO it wasn't a bad idea to point the OP to $. "...The test is pretty fast and has probably relatively limited impact of performance" So let us compare: #!/usr/bin/env perl use strict; use warnings; use Benchmark qw ( :hireswallclock cmpthese timethese ); our $file = qq (huge.file); our $amount = 1_000_000; sub karl { our ( $file, $amount ); open( my $in, "<", $file ); while (<$in>) { next if $. <= $amount; 1; } close $in; } sub laurent { our ( $file, $amount ); open( my $in, "<", $file ); my $useless_header_line = <$in> for 1 .. $amount; while (<$in>) { 1; } close $in; } my $results = timethese( 10, { 'karlgoethebier' => 'karl', 'Laurent_R' => 'laurent', } ); cmpthese($results); __END__ karls-mac-mini:monks karl$ perl -e 'print qq(Lorem ipsum kizuaheli\n) +for 1..50_000_000;' > huge.file karl-mac-mini:monks karl$ ls -hl huge.file -rw-r--r-- 1 karl karl 1,0G 21 Feb 17:24 huge.file karls-mac-mini:monks karl$ wc -l huge.file 50000000 huge.file karls-mac-mini:monks karl$ ./1117372.pl Benchmark: timing 10 iterations of Laurent_R, karlgoethebier... Laurent_R: 65.9521 wallclock secs (63.43 usr + 2.34 sys = 65.77 C +PU) @ 0.15/s (n=10) karlgoethebier: 127.837 wallclock secs (124.90 usr + 2.55 sys = 1 +27.45 CPU) @ 0.08/s (n=10) s/iter karlgoethebier Laurent_R karlgoethebier 12.7 -- -48% Laurent_R 6.58 94% -- [download] "...relatively limited impact of performance" This is relative. It is like it is ;-) Thanks for your reply and best regards, Karl ŤThe Crux of the Biscuit is the Apostropheť	[reply] [d/l]