Avoiding Datecalc

Earindil has asked for the wisdom of the Perl Monks concerning the following question:

The Datecalc function in this loop slows my script down greatly. With it, it takes minutes to run and about 15 seconds without. I've tried to avoid using it by just adding 30 or 60 seconds to the $seconds variable but keep getting unusual results. Help? This takes a series of data and puts all the timestamps into 5 minute increments (and adds 3 hours)...

        my @data = split(/\,/,$Client{$client}{$cmd});
        my $time = $data[0];
        for ($i=3; $i<=$#data; $i++) {
                $seconds = &UnixDate("$time","%s");
                if ($cmd eq 'RTT') {
                        $time  = &DateCalc("$time","+ 1minute",\$err);
                        $value = sprintf("%0.2f", ($data[$i]/1000));
                }
                else {
                        $time  = &DateCalc("$time","+ 30seconds",\$err
+);
                        $value = sprintf("%0.2f", (($data[$i]*8)/1000)
+); # Convert to bits then / by 1000 for graphing
                }
                $seconds = (int($seconds/300))*300;
                $Counter{$seconds}++;
                $Tot{$seconds}+=$value;
                $Max{$seconds}=$value if ($value>$Max{$seconds});
                $Min{$seconds}=$value if (($value<$Min{$seconds})or(!$
+Min{$seconds}));
        }
[download]

This is what I was getting when I tried to avoid using Datecalc and instead used the $i variable and multiplied it by the # of seconds to skip... As you can see, it worked for a bit but then started drifting more and more...

04/28/05.07:15:00       0.03    0.03    0.03
04/28/05.07:20:00       0.03    0.03    0.03
04/28/05.07:25:00       0.03    0.03    0.03
04/28/05.07:30:00       0.03    0.03    0.03
04/28/05.07:35:00       0.03    0.03    0.03
04/28/05.07:40:00       0.03    0.03    0.03
04/28/05.07:50:00       0.03    0.03    0.03
04/28/05.08:00:00       0.05    0.05    0.05
04/28/05.08:10:00       0.03    0.03    0.03
04/28/05.08:20:00       0.03    0.03    0.03
04/28/05.08:30:00       0.03    0.03    0.03
04/28/05.08:45:00       0.03    0.03    0.03
04/28/05.09:00:00       0.03    0.03    0.03
04/28/05.09:15:00       0.03    0.03    0.03
04/28/05.09:30:00       0.03    0.03    0.03
04/28/05.09:45:00       0.03    0.03    0.03
04/28/05.10:05:00       0.03    0.03    0.03
04/28/05.10:25:00       0.03    0.03    0.03
04/28/05.10:45:00       0.03    0.03    0.03
.
.
.

This is all necessary because the original data looks like this. Time, interval, # of elements, then results

07:30:04,60,750,32058,41,98,41,41,98,43,41,73,98,18050,467,428,260,363
+,836,258,1499,247,489,1208,469,633,749,439,342,258,706,817,639
,2174,848,909,557,876,1633,1116,1090,665,1150,817,1592,791,1741,935,67
+5,2370,2039,2121,911,1955,1526,608,656,375,1488,529,376,268,10
82,1858,2036,1282,859,1356,1654,1620,2074,1546,488,2999,1564,1509,811,
+1427,2149,611,1548,803,2313,1823,1575,1733,1168,2511,1034,2090
,1491,2361,2663,2062,1455,957,3549,3097,2443,2680,2262,2152,2176,2645,
+3029,2078,1395,1961,801,1955,1395,1465,1030,976,1026,813,932,1
745,450,1172,1457,.....
[download]

Janitored by Arunbear - added code tags, to prevent distortion of site layout

Comment on Avoiding Datecalc Select or Download Code

Replies are listed 'Best First'.
Re: Avoiding Datecalc by tlm (Prior) on Apr 28, 2005 at 21:32 UTC
I don't quite understand what your code is doing. I suspect that what you are trying to do is pretty simple, but I'd need more details to give you more specific advice. Update: OK, I made an attempt to recode what you wrote (see below). It's not exactly the same, but it should give you an idea of how to do what you want to do. Also, as a wild guess, maybe the reason for the anomalous results you were getting is that you were inadvertently using floats as hash keys (in `%Counter`, etc.). The code you showed doesn't have this problem, because these hashkeys end up being all integers, but I wonder if that may have not been the case for the code that actually generated the output that you posted. Read more... (3 kB) the lowliest monk	[reply] [d/l]
Re: Avoiding Datecalc by 5mi11er (Deacon) on Apr 28, 2005 at 20:57 UTC
Well, I'm unclear on exactly what the rest of this is accomplishing, but, it is very likely when you were using the counter $i to figure out how far to "skip", you weren't converting that to/from seconds properly, or there were some other problems in the math in figuring out the resultant time. I'm unable to get my brain to parse the code you have posted. Could you strip out the "playing with data" portions of the code, and just leave in the relavant portions of the date calculations? And post how you were trying to calculate the new time with $i? That should get us on track to be able to help you better. -Scott	[reply]
Re: Avoiding Datecalc by Earindil (Beadle) on Apr 29, 2005 at 15:10 UTC
Thank you for the replies... This is what ended up fixing the issue and speeding the script up greatly; seconds instead of minutes. I was coverting the unix seconds into a five minute interval for graphing by overwriting the $seconds variable, using it and then adding the interval to it. Instead, I needed to retain the original value, so I needed to use a second variable for the five minute interval. $fivemin = (int($seconds/300))300; foreach $cmd ('WBU', 'RTT') { my ( $time, $interval, $n_elements, @data ) = split(/,/, $Clie +nt{$client}{$cmd}); my $seconds = &UnixDate("$time","%s"); foreach (@data) { if ($cmd eq 'RTT') { $interval = 60; ### interval not set correctl +y in dataset $value = sprintf("%0.2f", ($_/1000)); } else { $interval = 30; ### interval not set correctl +y in dataset $value = sprintf("%0.2f", (($_8)/1000)); # + Convert to bits then / by 1000 for graphing } $fivemin = (int($seconds/300))*300; $Counter{$fivemin}++; $Tot{$fivemin}+=$value; $Max{$fivemin}=$value if ($value>$Max{$fivemin}); $Min{$fivemin}=$value if (($value<$Min{$fivemin})or(!$ +Min{$fivemin})); $seconds+=$interval; } [download]	[reply] [d/l]