ybiC has asked for the wisdom of the Perl Monks concerning the following question:

I've written perl code that collects interface packet counts via SNMP.   So far, so good.   Now I'm asking for y'alls wisdom on getting the *real* data out of the following csv.

What I need to do goes something like this (in english):

Read the first and second lines
    deltaTime2 = epochSec2 - epochSec1
    Bps2 = (ifInOctets2 + ifOutOctets2)/deltaTime2
Read the second and third lines
    deltaTime3 = epochSec3 - epochSec2
    Bps3 = (ifInOctets3 + ifOutOctets3)/deltaTime3
Read the third and fourth lines
    deltaTime4 = epochSec4 - epochSec3
    Bps3 = (ifInOctets4 + ifOutOctets4)/deltaTime4
Lather, rinse, repeat...

In a past life, I'd have used Excel for the calculations {oh, the shame} but now I'd *really* like to do this in a Perlish manner.     Doing the simple arithmetic in Perl is easy.   I imagine the control code might involve split() and a LoL, but am quite unclear on where to start.   Any clueful monks care to point me in a good direction?
    cheers,
    Don
    striving toward Perl Adept
    (it's pronounced "why-bick")

Collected data csv: 

date,time,epochSec,device,ifIndex,ifInOctets,ifOutOctets,
2001-08-10,18:15:00,997485300,boston,141,1916238333,2743654663,
2001-08-10,18:18:12,997485492,boston,141,1917535505,2744657633,
2001-08-10,18:21:23,997485683,boston,141,1919320447,2747461877,
2001-08-10,18:24:34,997485874,boston,141,1921869420,2748722088,
2001-08-10,18:27:46,997486066,boston,141,1925884600,2764661750,
2001-08-10,18:30:57,997486257,boston,141,1929925399,2766849525,
...

[download]

Desired output csv: 

date,time,,device,ifIndex,Bps,
2001-08-10,18:15:00,boston,141,,
2001-08-10,18:18:12,boston,141,11649,
2001-08-10,18:21:23,boston,141,24969,
...

[download]

Replies are listed 'Best First'.
Re: control+data structures for multi-line math on csv
by tachyon (Chancellor) on Aug 12, 2001 at 03:06 UTC

    This should get you started. It doesn't give the results you want althought it seems to follow what you said you wanted. Must have misunderstood :-) I just read of the DATA filehandle so you will have to open your input and output files. We localise the change to $, to this block to save suprises. The last line data is in @last which we initialise to '' values to keep warnings happy 

    use strict;
use warnings;
{
   local $, = ',';      # print will print CSV with this set to ','
   my @last = (0) x 7;  # initialize array to 0's
   while(<DATA>) {
      chomp;
      next if /^\s*$/;           # skip blank lines
      my @data = split',';       # split out line into an array
      my $deltatime = $data[2]-$last[2];
      next unless $deltatime;    # protect against illegal div zero
      my $bps = ( $data[5] + $data[6] ) / $deltatime;
      print @data[0,1,3,4], $bps, "\n";  # this will be CSV
      @last = @data;             # remember last line content
   }
}
# Collected data csv: 
# date,time,epochSec,device,ifIndex,ifInOctets,ifOutOctets,
__DATA__
2001-08-10,18:15:00,997485300,boston,141,1916238333,2743654663,
2001-08-10,18:18:12,997485492,boston,141,1917535505,2744657633,
2001-08-10,18:21:23,997485683,boston,141,1919320447,2747461877,
2001-08-10,18:24:34,997485874,boston,141,1921869420,2748722088,
2001-08-10,18:27:46,997486066,boston,141,1925884600,2764661750,
2001-08-10,18:30:57,997486257,boston,141,1929925399,2766849525,

    [download]

    cheers

    tachyon

    s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

[reply]
[d/l]
Re: control+data structures for multi-line math on csv
by HamNRye (Monk) on Aug 12, 2001 at 02:59 UTC

    From what you look to be asking, is for a good control structure to push your data through a calculation subroutine?? The best way is to is to use a subroutine. 

    # Init vars
$prev_xists = "0";
@output = "";
@lines = get_lines();
@outlines = calc_lines(@lines)

sub get_lines {
     ...... #perl code for getting lines
}

sub calc_lines {
     for (@_) {

#Split out your data from the line
my ($date, $time, $epochSec, $dev, $ifIdx, $ifIn, $ifOut, $nul) = spli
+t ',';
if ($prev_xists) {
my $dTime = $old_epochSec - $epochSec;
my $bps = ($ifIn + $ifOut) / $dTime;
} else {
my $bps = "";
}
$old_epochSec = $epochSec;
$prev_xists = "1";
my $new_line = "$date,$time,$dev,$ifIdx,$bps";
push @output, $new_line ;
}
return @output;
}

    [download]

    This is not the most elegant solution, and there's some things I would change here and there, but this should get you started.

    After all that you should have your data in @outlines.
    ~Hammy

[reply]
[d/l]

Back to Seekers of Perl Wisdom