GertMT has asked for the wisdom of the Perl Monks concerning the following question:

I've been hesitating to post this question. It's a follow up re. my question of this morning Split pattern doesn't match last line of file. This time I'm less sure how to define the title.
I trying to write a script that summarizes data from text files. Right now I've got a problem to make calculations with data that's somewhere in the hash? I'm really puzzled.
Code produced so far is mentioned below. It all works well with the data except that I would like to print the result from the "C" and "D" values. I've tried but can't get to my data. From out of the foreach loop or can I? Any help or reference to how to work with hashes is appreciated.
Gert
#!/usr/bin/perl use strict; use vars qw !$file!; chdir '/test/' or die "Cannot change to /test: $!"; foreach $file ( readdir DH ) { } foreach $file ( glob "*.txt" ) { open FILE, '<', $file; my %saldi; while (<FILE>) { chomp; next unless $_; my @cellen = ( split /,/, )[ 3, 4 ]; next unless $cellen[0] && $cellen[1]; $saldi{ $cellen[0] } += $cellen[1]; } $file =~ m/^(\S+)\.txt/; print "$1\n"; foreach my $name ( keys %saldi ) { printf "\t$name\t" . "%16s", &big_money( $saldi{$name} ) . "\n +"; } close FILE; } closedir DH; sub big_money { #Learning Perl my $number = sprintf "%.2f", shift @_; #Add one comma each time though the do-nothing loop 1 while $number =~ s/^(-?\d+)(\d\d\d)/$1,$2/; #Put the dollar sign in the right place $number =~ s/^(-?)/$1/; $number; }
Data_1.txt
394,eur,2006,D,18.20 394,eur,2006,D,22 394,eur,2006,C,25
Data_2.txt
494,eur,2006,C,25 494,eur,2006,D,79

Replies are listed 'Best First'.
Re: Make calculation with values from hash
by johngg (Canon) on Jan 02, 2007 at 19:35 UTC
    I've had a look back at Split pattern doesn't match last line of file and I don't think your chomp; next unless $_; is going to cope if your data has lines containing just spaces (as implied in your OP), although it does cope with those that are just a newline. Given this data file

    1st line 2nd line, next line is just a newline 4th line, next line spaces and newline 6th and last line

    this script that uses your construct

    use strict; use warnings; my $inFile = q{spw592594.dat}; open my $inFH, q{<}, $inFile or die qq{open: $inFile: $!\n}; while (<$inFH>) { chomp; next unless $_; print qq{-->$_<--\n}; } close $inFH or die qq{close: $inFile: $!\n};

    produces

    -->1st line<-- -->2nd line, next line is just a newline<-- -->4th line, next line spaces and newline<-- --> <-- -->6th and last line<--

    and, as you can see, the line with spaces does not get rejected and the empty line does. A string of spaces is boolean true. Changing the script to

    ... while (<$inFH>) { chomp; # reject if 0 or more spaces anchored to start and end next if m{^\s*$}; print qq{-->$_<--\n}; } ...

    does reject correctly and produces

    -->1st line<-- -->2nd line, next line is just a newline<-- -->4th line, next line spaces and newline<-- -->6th and last line<--

    I hope this is of use.

    Cheers,

    JohnGG

      Thanks very much, this is certainly of use as I've had two file recently that had irregular patterns somewhere. Just to make sure that I really do understand it:
      next if m{^\s*$}; print qq{-->$_<--\n};
      the $_ stands for the next line that's received from the file!? Yes, must be. Gert
        Yes, you have that correctly. Put simply, if you read from a filehandle in a while loop without explicitly assigning what's read to a different variable, it goes into $_, i.e. while (<$inFH>) { # $_ gets each line in here } whereas while (my $line = <$inFH>) { # $line gets each line in here }.

        I think that only holds in a while loop though. If you are reading a single line outside of a loop you have to assign to a variable. If I do the following, thinking to print just the first line of the data file

        use strict; use warnings; my $inFile = q{spw592594.dat}; open my $inFH, q{<}, $inFile or die qq{open: $inFile: $!\n}; <$inFH>; chomp; print qq{-->$_<--\n}; close $inFH or die qq{close: $inFile: $!\n};

        I get

        Use of uninitialized value in scalar chomp at ./spw592594B line 11, <$ +inFH> line 1. Use of uninitialized value in concatenation (.) or string at ./spw5925 +94B line 12, <$inFH> line 1. --><--

        because the line is read then silently thrown away because I have not assigned it to any variable. I have to assign to $_ or some other variable to be able to use it, like this

        use strict; use warnings; my $inFile = q{spw592594.dat}; open my $inFH, q{<}, $inFile or die qq{open: $inFile: $!\n}; $_ = <$inFH>; chomp; print qq{-->$_<--\n}; close $inFH or die qq{close: $inFile: $!\n};

        which produces

        -->1st line<--

        I hope this makes it clear what is happening.

        Cheers,

        JohnGG

Re: Make calculation with values from hash
by wfsp (Abbot) on Jan 02, 2007 at 18:33 UTC
    Hi GertMT!

    I think you need to move the second foreach loop outside of the first foreach loop. You need to move my %saldi; to before the first foreach loop.

    I get (unformatted):

    C 50.00 D 119.20
    Is that something like what you're after?

    Let us know how you get on.

      testdata C 25.00 D 40.20 testdata2 D 79.00 C 25.00
      hi wfsp, thanks for your reply. Without making a change to the code the above is what I get as result. Which I'm pretty happy with. Next step though is to do something like: C -/-  Dfor all files (testdata..) and then take the total from this at the bottom of the report.
      My problem is how to get to substract 25 minus 40.20. How to refer to this data?
        My problem is how to get to substract 25 minus 40.20. How to refer to this data?

        If I'm understanding you correctly, that would be

        my $diff = $saldi{C} - $saldi{D};
        However, looking at the rest of the fine code you've produced already (no irony!), makes me think this can't really be your problem... :)

        So, let me guess, the problem is you want to access the values at a point in your program where they're no longer available, e.g. after having completed the foreach $file ( glob "*.txt" ) { ... } loop(?)

        In that case - if you want grand totals over all data files - you'll simply have to move the %saldi hash before the outer loop, as already suggested by wfsp -- as you have things now, %saldi will (a) get reinitialised for every new file (so the old values are lost), (b) fall out of scope when the loop is done (also meaning the values are lost).

        Otherwise - if you want to keep individual sums per file - you'd have to store those values away in a somewhat more complex data structure, e.g. a hash-of-hashes (HoH). Something like that:

        my %saldi; foreach $file ( glob "*.txt" ) { open FILE, '<', $file; my $data = {}; while (<FILE>) { chomp; next unless /\S/; my ($name, $price) = ( split /,/, )[ 3, 4 ]; next unless $name && $price; $data->{$name} += $price; } my ($datasetname) = $file =~ m/^(\S+)\.txt/; print "$datasetname\n"; foreach my $name ( keys %$data ) { printf "\t$name\t" . "%16s\n", big_money( $data->{$name} ); } # store it... $saldi{$datasetname} = $data; close FILE; } # now, you can access all data by specifying the appropriate # pairs of keys, e.g. printf "diff C-D: %.2f\n", $saldi{testdata}{C} - $saldi{testdata}{D}; # ...

        HTH :)

Re: Make calculation with values from hash
by GrandFather (Saint) on Jan 02, 2007 at 22:11 UTC

    It looks like some of your problem can be resolved by including the file name as a key:

    #!/usr/bin/perl use strict; my %saldi; my $file; while (<DATA>) { chomp; next unless $_; $file = $1, next if /(\w+\.txt)$/; # Required for sample code only my @cellen = ( split /,/, )[ 3, 4 ]; next unless $cellen[0] && $cellen[1]; $saldi{$file}{ $cellen[0] } += $cellen[1]; # file name as primary +key } my $dSum = 0; my $cSum = 0; my $deltaSum = 0; foreach my $filename ( keys %saldi ) { $cSum += $saldi{ $filename }{ C }; $dSum += $saldi{ $filename }{ D }; my $delta = $saldi{ $filename }{ D } - $saldi{ $filename }{ C }; $deltaSum += $delta; printf "$filename delta:\t%8s\n", big_money( $delta ); } printf "C sum: \t%16s\n", &big_money( $cSum ); printf "D sum: \t%16s\n", &big_money( $dSum ); printf "Delta sum:\t%16s\n", &big_money( $deltaSum ); sub big_money { #Learning Perl my $number = sprintf "%.2f", shift @_; #Add one comma each time though the do-nothing loop 1 while $number =~ s/^(-?\d+)(\d\d\d)/$1,$2/; #Put the dollar sign in the right place $number =~ s/^(-?)/$1/; $number; } __DATA__ Data_1.txt 394,eur,2006,D,18.20 394,eur,2006,D,22 394,eur,2006,C,25 Data_2.txt 494,eur,2006,C,25 494,eur,2006,D,79

    Prints:

    Data_1.txt delta: 15.20 Data_2.txt delta: 54.00 C sum: 50.00 D sum: 119.20 Delta sum: 69.20

    Note that the two files have been munged together into a DATA section to avoid reliance on external files for the sample code.


    DWIM is Perl's answer to Gödel
      Thanks to all who gave a hint into what direction I should work. All great examples given as well. Finally I've got something working which I'll post below. I'm still a bit puzzled why I need to put the closing bracket of the Foreach loop higher by using real datafiles
      } # real datafiles my ($basename) = ( $filename =~ m/^(\S+)\.txt/ ) or warn;
      as opposed to when I make use of the sample data
      printf " D + C: \t%15s%16s\n", &big_money( ( $saldi{$filename}{'"D"'} * -1 ) + ( $saldi{$filename}{'"C"'} + ) ), &big_money($deltaSum); } #sample
      but it works! Script below makes use of sample data. And only has a problem when irregular pattern appears to be somewhere in the middle of the file (which should never be the case).
      It thaugt me a great deal about hashes and how you need to refer to the data.
      thanks all, Gert
      #!/usr/bin/perl -w use strict; use diagnostics; use vars qw ! %saldi $filename !; #chdir # real code # '/Users.......' # real code # or die "Cannot open : $!"; #real cod +e #foreach my $filename ( glob "*.txt" ) { # real code # open FILE, '<', $filename or warn "Can't open FILE: $!";#real cod +e # while (<FILE>) { #real cod +e # chomp; #real cod +e # next unless $_; #real cod +e while (<DATA>) { # Required for sample code chomp; # Required for sample code next unless $_; # Required for sample code $filename = $1, next if /(\w+\.txt)$/; # Required for sample code my @fields = ( split /,/, )[ 0, 1, 2, 3, 4 ]; my ( $account, $currency, $year, $dofc, $amount ) = @fields; next unless $dofc && $amount; $saldi{$filename}{$dofc} += $amount; # file name as primary + key } my $dSum = 0; my $cSum = 0; my $deltaSum = 0; foreach $filename ( keys %saldi ) { $cSum += $saldi{$filename}{'"C"'}; $dSum += $saldi{$filename}{'"D"'} * -1; my $delta = ( $saldi{$filename}{'"D"'} * -1 ) + $saldi{$filename}{'"C"'} +; $deltaSum += $delta; #} real my ($basename) = ( $filename =~ m/^(\S+)\.txt/ ) or warn; printf "$basename\n"; printf "\tD sum: \t%16s", &big_money( $saldi{$filename}{'"D"'} * -1 ) . "\n"; printf "\tC sum: \t%16s", &big_money( $saldi{$filename}{'"C"'} ) . + "\n"; printf " D + C: \t%15s%16s\n", &big_money( ( $saldi{$filename}{'"D"'} * -1 ) + ( $saldi{$filename}{'"C"'} + ) ), &big_money($deltaSum); } #sample # close FILE or warn "Can't close FILE: $!"; # real code #} # real code sub big_money { #Learning Perl my $number = sprintf "%.2f", shift @_; #Add one comma each time though the do-nothing loop 1 while $number =~ s/^(-?\d+)(\d\d\d)/$1,$2/; #Put the dollar sign in the right place $number =~ s/^(-?)/$1/; $number; } __DATA__ Data_1.txt 394,eur,2006,"D",18.20 394,eur,2006,"D",22 394,eur,2006,"C",25 394,eur,2006,"C",16 Data_2.txt 494,eur,2006,"C",25 494,eur,2006,"D",79 494,eur,2006,"D",79 494,eur,2006,"D",79 494,eur,2006,"C",100 494,eur,2006,"C",1