in reply to Performing Mathematical Operation on Specific Column of text File
The first thing to recognize is that your program will need to see every line of the input before it begins writing the output, since it has to see all the frequencies before it can calculate their average so it can be applied to each line. So you'll need to either:
1. Loop through the input file twice, accumulating numbers on the first time through, then making the edits and printing the output on the second time through.
2. Loop through the input file once, but save each line in an array while you do the calculations, so that your second loop can go through the array instead of hitting the filesystem again.
I'd say the second solution will generally be the best unless the file is so large that putting it all in an array will cause memory problems. So let's do that. This loops through the file, adding the frequencies to an accumulator ($total) when a line matches the pattern, keeping track of how many ($howmany) frequencies it adds. Those two values will be divided to get the mean. It also saves each line in an array, along with a flag ($fixlater) to show whether that line is one containing a frequency. That way when I loop through the lines again, I don't have to split the ones that don't contain a frequency to change; the other lines can just be printed out.
This does literally what you said: subtracts the mean from each frequency. Since the mean is negative, subtracting it actually adds a positive, which may or may not be what you really want. If it's not, try to describe what you really want in more detail, and give us a couple examples of input and output frequencies.
#!/usr/bin/env perl use 5.010; use strict; use warnings; my @lines; my $total = 0; my $howmany = 0; my $output_separator = "\t"; # your choice while(<DATA>){ chomp; my($n, $f) = (split)[0,8]; my $fixlater; if ($n =~ /\A[A-Z]{2}\.[A-Z]{4}\Z/ and $f =~ /\A-\d\.\d{4}\Z/ ){ $total += $f; $howmany++; $fixlater = 1; } push @lines, [$fixlater,$_]; } my $mean = $total/$howmany; for (@lines){ if($_->[0]){ # fix this line my @f = (split ' ', $_->[1]); $f[8] = sprintf "%0.4f", $f[8] - $mean; say join $output_separator, @f; } else { say $_->[1]; } } __DATA__ MCCC processed: unknown event at: Tue, 14 Oct 2014 12:02:26 CST station, mccc delay, std, cc coeff, cc std, pol , t0_times + , delay_times ZJ.GRAW -0.7964 0.0051 0.9690 0.0139 0 GRAW.BHZ 301 +.1263 -1.8041 ZJ.KNYN -0.7065 0.0072 0.9760 0.0133 0 KNYN.BHZ 301. +3372 -1.9249 ZJ.LEON 0.9675 0.0072 0.9548 0.0292 0 LEON.BHZ 301. +2611 -0.1749 ZJ.RKST -0.2061 0.0114 0.9404 0.0383 0 RKST.BHZ 301. +3500 -1.4374 ZJ.SHRD 0.4382 0.0051 0.9542 0.0351 0 SHRD.BHZ 301. +7360 -1.1791 ZJ.SPLN 0.3033 0.0051 0.9785 0.0126 0 SPLN.BHZ 301. +0760 -0.6541 Mean_arrival_time: 300.1187 No weighting of equations. Window: 2.23 Inset: 1.17 Shift: 0.25 Variance: 0.00645 Coefficient: 0.96215 Sample rate: 40.000 Taper: 0.28 Phase: P PDE 2013 7 15 14 6 58.00 -60.867 -25.143 31.0 0.0 7.3
Aaron B.
Available for small or large Perl jobs and *nix system administration; see my home node.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^2: Performing Mathematical Operation on Specific Column of text File
by Bama_Perl (Acolyte) on May 15, 2015 at 19:52 UTC | |
by aaron_baugher (Curate) on May 16, 2015 at 02:07 UTC |