buchi2 has asked for the wisdom of the Perl Monks concerning the following question:

Hello!

I have one datafile which contains
no, x, y, z, pressure1
and another which has
no, x, y, z, pressure2.

I have to write them to a result file building the difference, if the x/y/z coordinate from booth files are matching, which contains
no, x, y, z, pressure2-pressure1

The no should be from file1.
The problem now it, that the order of the coordinates in file1 and file2 is not the same.

file1: 0, 4.38336420e+00, -2.47429684e-01, 6.79128885e+00, 1.51447906e+05 1, 4.38336420e+00, -2.78662264e-01, 6.79128885e+00, 1.58727812e+05 2, 4.38336420e+00, -2.78662264e-01, 6.69878864e+00, 1.61132406e+05 3, 4.38336420e+00, -2.47429684e-01, 6.69878864e+00, 1.54500719e+05 file2: 0, 4.38336420e+00, -2.47429684e-01, 6.79128885e+00, 3.43969625e+05 1, 4.38336420e+00, -2.47429684e-01, 6.69878864e+00, 3.29992562e+05 2, 4.38336420e+00, -2.78662264e-01, 6.69878864e+00, 3.30176656e+05 3, 4.38336420e+00, -2.78662264e-01, 6.79128885e+00, 3.42821781e+05
(only the first lines, the files containing several thousand lines).

In my first try I compare every value with every to find matching x/y/z.

Is there a way to do it more efficient?
And is it possible, to trim the blanks during reading automatically away, wihout the trim sub?
#!/usr/bin/perl -w # Aufruf: test.pl file1_a file2_i fnout use strict; use warnings; no warnings qw/once/; my (@biga,@bigi); my (%all); my ($nr,$x,$y,$z,$p); my ($tmp); my ($fna,$fni,$fnout); my (@allnr); my ($i); my (@header); # Datei 1 (aussen) Datei einlesen { open (my $DAT,'<',$ARGV[0]) || do {print STDERR "***error*** during + opening $ARGV[0]: $!\n";die;}; (@biga) = <$DAT>; close $DAT; } chomp @biga; # Datei 2 (innen) Datei einlesen { open (my $DAT,'<',$ARGV[1]) || do {print STDERR "***error*** during + opening $ARGV[1]: $!\n";die;}; (@bigi) = <$DAT>; close $DAT; } chomp @bigi; # die Filenamen $fna = $ARGV[0]; $fni = $ARGV[1]; $fnout = $ARGV[2]; # in den Hash einsortieren foreach (@biga) { ($nr,$x,$y,$z,$p,undef) = split(/,/,$_); $nr = &trim($nr); $x = &trim($x); $y = &trim($y); $z = &trim($z); $p = &trim($p); $all{$nr}{'x'} = $x; $all{$nr}{'y'} = $y; $all{$nr}{'z'} = $z; $all{$nr}{'pa'} = $p; push (@allnr,$nr); } # innen dazusortieren foreach (@bigi) { ($nr,$x,$y,$z,$p,undef) = split(/,/,$_); $nr = &trim($nr); $x = &trim($x); $y = &trim($y); $z = &trim($z); $p = &trim($p); for $i (0 .. $#allnr) { if (($all{$i}{'x'} == $x) && ($all{$i}{'y'} == $y) && ($all{$i}{ +'z'} == $z)) { $all{$i}{'pi'} = $p; $all{$i}{'dp'} = $all{$i}{'pa'} - $all{$i}{'pi'}; next; } } } # Ausgabe schreiben { open (my $DAT,'>',$fnout) || do {print STDERR "***error*** during o +pening $fnout: $!\n";die;}; for $i (0 .. $#allnr) { print $DAT "$i, $all{$i}{'x'}, $all{$i}{'y'}, $all{$i}{'z'}, $all{$ +i}{'dp'}\n"; } close $DAT; } # ################################################ # TRIM A STRING # http://www.symablog.de/blog/perl-fuhrende-und-nachfolgende-leerzeich +en-aus-einem-string-entfernen-trim/ # ################################################ sub trim { my $str = $_[0]; $str =~ s/^\s+|\s+$//g; return $str; }
Regards, buchi

Replies are listed 'Best First'.
Re: quick way to add value of a hash with same x, y, z?
by QM (Parson) on Aug 10, 2017 at 10:32 UTC
    OK, I knocked this up quick, but it's untested.

    You didn't say what should happen if a coord triple is only in one file. And probably needs a lot of safety checks, if these are big files.

    #!/usr/bin/env perl # Store X,Y,Z coordinates from one file, along with a number and press +ure. # Find a matching coord in a second file, subtract pressure1 from pres +sure2. # Output with the number from the first file. # Sample data: # 0, 4.38336420e+00, -2.47429684e-01, 6.79128885e+00, 1.51447906e+05 # 1, 4.38336420e+00, -2.78662264e-01, 6.79128885e+00, 1.58727812e+05 # 2, 4.38336420e+00, -2.78662264e-01, 6.69878864e+00, 1.61132406e+05 # 3, 4.38336420e+00, -2.47429684e-01, 6.69878864e+00, 1.54500719e+05 # Assume there are 2 filenames on the command line, in the correct ord +er. use strict; use warnings; our $key_format = '%16.8f'; # Fixed decimals for use in keys our $out_format = '%e'; # scientific notation sub read_it { my $line = shift; chomp($line); my ($n, $x, $y, $z, $p) = split /,\s+/, $line; my $key = sprintf "$key_format,$key_format,$key_format", $x, $y, $ +z; return($n, $x, $y, $z, $key, $p); } our %press; # File 1 while (<>) { my ($n, $x, $y, $z, $key, $p) = read_it($_); $press{$key}{num} = $n; $press{$key}{press} = $p; } continue { # If this is the end of the first file, close it and escape the lo +op if (eof) { # "eof", not "eof()"! close ARGV; last; } } # File 2 while (<>) { my ($n, $x, $y, $z, $key, $p2) = read_it($_); if (exists($press{$key})) { my $pdiff = $p2 - $press{$key}{press}; my $n1 = $press{$key}{n}; print "%d, $out_format, $out_format, $out_format, $out_format\ +n", $n1, $x, $y, $z, $pdiff; } } exit;

    -QM
    --
    Quantum Mechanics: The dreams stuff is made of

Re: quick way to add value of a hash with same x, y, z?
by QM (Parson) on Aug 10, 2017 at 10:08 UTC
    Quick comments: Use a hash on the first file, such as:
    $hash{"$x,$y,$z"}{no} = $no; $hash{"$x,$y,$z"}{pressure} = $pressure;

    Then with the coords from the second file, search in the hash for matching coords.

    Problem: The coords have a lot of decimal places. These should be normalized before creating the hash entries, and before searching the hash keys. Decide how many significant digits are more than enough, and use sprintf with a fixed decimal ("f") descriptor.

    -QM
    --
    Quantum Mechanics: The dreams stuff is made of

Re: quick way to add value of a hash with same x, y, z?
by thanos1983 (Parson) on Aug 10, 2017 at 10:15 UTC

    Hello buchi2,

    Regarding the part of your question And is it possible, to trim the blanks during reading automatically away, wihout the trim sub?.

    Why you need to trim the string through a function and not onsite directly?

    Sample of code:

    #!/usr/bin/perl use strict; use warnings; use feature 'say'; sub trim { my ($str) = @_; $str =~ s/^\s+|\s+$//g; return $str; } my $test = ' sample '; say 'Before (trim):'; say $test; say trim($test); say 'Without (trim):'; say $test; $test =~ s/^\s+|\s+$//g; say $test; __END__ $ perl test.pl Before (trim): sample sample Without (trim): sample sample

    Just do it as in the function without call the function, but the only reason to use the function is to avoid retyping over and over and over again the same code.

    I do not have time now, to check the rest of the code but maybe later on I will find some time (hopefully).

    Hope this helps, BR.

    Seeking for Perl wisdom...on the process of learning...not there...yet!
      Hello!

      Thank you very much.

      >Just do it as in the function without call the function, but the only reason to use the function is to avoid retyping over and over and over again the same code.

      So I will keep the sub, I use it more times.

      Regards, buchi