loop issue

chefchanyu has asked for the wisdom of the Perl Monks concerning the following question:

I try to fix this strange problem with Perl script. I tried 2 different type of loop they both doing same job but both got same strange result. The loop is checking each event make sure each day (86400 second for 1 day) has a data input if not exist will create itself with value 0. But the result first half were all going good, second half didn’t increase time with 86400 second instead with 3600 second. The issue always happen when I loading a large amount CDR (CDR which log files from network traffic).

sub RrdTimeLoop
{
    my $self = @_;

#    foreach my $eventName (sort keys %VAR::CDR_EVENT_COUNT)
#    {
#        print $eventName . "\n";
#        my $startTime = $VAR::CDR_FIRST_EVENT_TIME{$eventName};
#        my $endTime = $VAR::CDR_LAST_EVENT_TIME{$eventName};
#        print $startTime . "\n";
#        print $endTime . "\n";
#        
#        for (my $i = $startTime; $i < $endTime; $i = $i + 86400)
#        {
#            if($VAR::CDR_EVENT_COUNT{$eventName}{$i})
#            {
#                next;
#            }
#            else
#            {
#                
#                $VAR::CDR_EVENT_COUNT{$eventName}{$i} = 0;
#            }
#        }
#    }


foreach my $eventName (sort keys %VAR::CDR_EVENT_COUNT)
{
        print $eventName . "\n";
        my $startTime = $VAR::CDR_FIRST_EVENT_TIME{$eventName};
        my $endTime = $VAR::CDR_LAST_EVENT_TIME{$eventName};
        print $startTime . "\n";
        print $endTime . "\n";
        my $loopTime = $startTime;
        while ($loopTime < $endTime)
        {
            if ($VAR::CDR_EVENT_COUNT{$eventName}{$loopTime})
            {
                $loopTime = $loopTime + 86400;
            }
            else
            {
                $VAR::CDR_EVENT_COUNT{$eventName}{$loopTime} = 0;
                $loopTime = $loopTime + 86400;
            }
            
        }        
        
}



    return 1;
}
[download]

===============================================
CDR_TOTAL_901J902M905S906WAPENC906System932X:
1152712800:13063
1152799200:15380
1152885600:14867
1152972000:13009
1153058400:13958
1153144800:12740
1153231200:13670
1153317600:14504
1153404000:15835
1153490400:16723
1153576800:12946
1153663200:3425
1153749600:13786
1153836000:14842
1153922400:15662
1154008800:16623
1154095200:16439
1154181600:13298
1154268000:14772
1154354400:13811
1154440800:14447
1154527200:14818
1154613600:16036
1154700000:17369
1154786400:14217
1154872800:14774
1154959200:13703
1155045600:11091
1155132000:20775
1155218400:21350
1155304800:17332
1155391200:15304
1155477600:20050
1155564000:14044
1155650400:15466
1155736800:15228
1155823200:17039
1155909600:17334
1155996000:14747
1156082400:13846
1156168800:12145
1156255200:16038
1156341600:17133
1156428000:18627
1156514400:17642
1156600800:16155
1156687200:15882
1156773600:15419
1156860000:16304
1156946400:16968
1157032800:18795
1157119200:18025
1157205600:18084
1157292000:15204
1157378400:16383
1157464800:18042
1157551200:17261
1157637600:17203
1157724000:17220
1157810400:15859
1157896800:16099
1157983200:12820
1158069600:16116
1158156000:14816
1158242400:17131
1158328800:18768
1158415200:16738
1158501600:15524
1158588000:15763
1158674400:15979
1158760800:17012
1158847200:18542
1158933600:18606
1159020000:15407
1159106400:15264
1159192800:16871
1159279200:12338
1159365600:12044
1159452000:12384
1159538400:13173
1159624800:11149
1159711200:9564
1159797600:18737
1159884000:10039
1159970400:11128
1160056800:11687
1160143200:11663
1160229600:10283
1160316000:8808
1160402400:9675
1160488800:11577
1160575200:11560
1160661600:11950
1160748000:13208
1160834400:10888
1160920800:11170
1161007200:13173
1161093600:10904
1161180000:11921
1161266400:13416
1161352800:14055
1161439200:11579
1161525600:10508
1161612000:11038
1161698400:17644
1161784800:13199
1161871200:12495
1161957600:14027
1162044000:12132
1162126800:10963
1162130400:0
1162213200:12939
1162216800:0
1162299600:12849
1162303200:0
1162386000:13115
1162389600:0
1162472400:36835
1162476000:0
1162558800:16572
1162562400:0
1162645200:13960
1162648800:0
1162731600:20033
1162735200:0
1162818000:16053
1162821600:0
1162904400:36044
1162908000:0
1162990800:20430
1162994400:0
1163077200:16914
1163080800:0
1163163600:17179
1163167200:0
1163250000:16716
1163253600:0
1163336400:15366
1163340000:0
1163422800:15664
1163426400:0
1163509200:16109
1163512800:0
1163595600:17262
1163599200:0
1163682000:22097
1163685600:0
1163768400:18612
1163772000:0
1163854800:18173
1163858400:0
1163941200:16630
1163944800:0
1164027600:17290
1164031200:0
1164114000:18590
1164117600:0
1164200400:19905
1164204000:0
1164286800:19685
1164290400:0
1164373200:19673
1164376800:0
1164459600:17459
1164463200:0
1164546000:16934
1164549600:0
1164632400:24597
1164636000:0
1164718800:33439
1164722400:0
1164805200:18465
1164808800:0
1164891600:35423
1164895200:0
1164978000:19608
1164981600:0
1165064400:18890
1165068000:0
1165150800:31865
1165154400:0
1165237200:30261
1165240800:0
1165323600:39776
1165327200:0
1165410000:32191
1165413600:0
1165496400:23246
1165500000:0
1165582800:22949
1165586400:0
1165669200:21216
1165672800:0
1165755600:24016
1165759200:0
1165842000:35187
1165845600:0
1165928400:29515
1165932000:0
1166014800:39982
1166018400:0
1166101200:30533
1166104800:0
1166187600:22171
1166191200:0
1166274000:21785
1166277600:0
1166360400:30560
1166364000:0
1166446800:32155
1166450400:0
1166533200:20838
1166536800:0
1166619600:23556
1166623200:0
1166706000:23651
1166709600:0
1166792400:21862
1166796000:0
1166878800:24537
1166882400:0
1166965200:49693
1166968800:0
1167051600:25158
1167055200:0
1167138000:21605
1167141600:0
1167224400:25095
1167228000:0
1167310800:26650
1167314400:0
1167397200:24979
1167400800:0
[download]

Comment on loop issue Select or Download Code

Replies are listed 'Best First'.
Re: loop issue by Kenosis (Priest) on Aug 10, 2012 at 03:52 UTC
I don't know whether this will help, but notice the following transitional segment in the CDR data: `1161871200:12495 A 1161957600:14027 B 1162044000:12132 C 1162126800:10963 D <-3600 less than C 1162130400:0 E <-C + 86400 created and then set to 0 1162213200:12939 F 1162216800:0 G 1162299600:12849 H B = A + 86400 C = B + 86400 D = C + 82800 (3600 less) E = D + 3600 F = D + 86400` [download] Line D is 3600 seconds short of the pattern which your script implements, so it's creating entries from E on, based upon the assumption that entries are always 86400 seconds apart. Line D appears to be an outlier that's caused this problem. This would also account for the two different loops producing the same result. Given the above, perhaps this will work on your particular data set (untested, so please use with caution): `if ($VAR::CDR_EVENT_COUNT{$eventName}{$loopTime}) { $loopTime = $loopTime + 86400; } elsif($VAR::CDR_EVENT_COUNT{$eventName}{$loopTime-3600}) { $loopTime = $loopTime - 3600; # Adjust $loopTime for outlier value } else { $VAR::CDR_EVENT_COUNT{$eventName}{$loopTime} = 0; $loopTime = $loopTime + 86400; }` [download] This adds a conditional check for an entry only 82800 seconds beyond the previous entry, and then adjusts `$loopTime` down by 3600 seconds if found. (This downward adjustment will cause the outlier value to be looked at again, but it seems more readable to do this than to add 169200 (86400*2-3600) to `$loopTime` to skip over the outlier to the next entry.) Continuing to add 86400 to `$loopTime` within the loop should then work after the adjustment. Hope this helps!	[reply] [d/l] [select]
Re^2: loop issue by chefchanyu (Novice) on Aug 10, 2012 at 05:34 UTC
Thanks fix this issue :). after I check all those times, I did found a entry is 82800. it was 20061029 - 20061030. so why have this issue happen? CDR_TOTAL_901J902M905S906WAPENC906System932X: 1152799200 - 1152712800 = 86400 1152885600 - 1152799200 = 86400 1152972000 - 1152885600 = 86400 1153058400 - 1152972000 = 86400 1153144800 - 1153058400 = 86400 1153231200 - 1153144800 = 86400 1153317600 - 1153231200 = 86400 1153404000 - 1153317600 = 86400 1153490400 - 1153404000 = 86400 1153576800 - 1153490400 = 86400 1153663200 - 1153576800 = 86400 1153749600 - 1153663200 = 86400 1153836000 - 1153749600 = 86400 1153922400 - 1153836000 = 86400 1154008800 - 1153922400 = 86400 1154095200 - 1154008800 = 86400 1154181600 - 1154095200 = 86400 1154268000 - 1154181600 = 86400 1154354400 - 1154268000 = 86400 1154440800 - 1154354400 = 86400 1154527200 - 1154440800 = 86400 1154613600 - 1154527200 = 86400 1154700000 - 1154613600 = 86400 1154786400 - 1154700000 = 86400 1154872800 - 1154786400 = 86400 1154959200 - 1154872800 = 86400 1155045600 - 1154959200 = 86400 1155132000 - 1155045600 = 86400 1155218400 - 1155132000 = 86400 1155304800 - 1155218400 = 86400 1155391200 - 1155304800 = 86400 1155477600 - 1155391200 = 86400 1155564000 - 1155477600 = 86400 1155650400 - 1155564000 = 86400 1155736800 - 1155650400 = 86400 1155823200 - 1155736800 = 86400 1155909600 - 1155823200 = 86400 1155996000 - 1155909600 = 86400 1156082400 - 1155996000 = 86400 1156168800 - 1156082400 = 86400 1156255200 - 1156168800 = 86400 1156341600 - 1156255200 = 86400 1156428000 - 1156341600 = 86400 1156514400 - 1156428000 = 86400 1156600800 - 1156514400 = 86400 1156687200 - 1156600800 = 86400 1156773600 - 1156687200 = 86400 1156860000 - 1156773600 = 86400 1156946400 - 1156860000 = 86400 1157032800 - 1156946400 = 86400 1157119200 - 1157032800 = 86400 1157205600 - 1157119200 = 86400 1157292000 - 1157205600 = 86400 1157378400 - 1157292000 = 86400 1157464800 - 1157378400 = 86400 1157551200 - 1157464800 = 86400 1157637600 - 1157551200 = 86400 1157724000 - 1157637600 = 86400 1157810400 - 1157724000 = 86400 1157896800 - 1157810400 = 86400 1157983200 - 1157896800 = 86400 1158069600 - 1157983200 = 86400 1158156000 - 1158069600 = 86400 1158242400 - 1158156000 = 86400 1158328800 - 1158242400 = 86400 1158415200 - 1158328800 = 86400 1158501600 - 1158415200 = 86400 1158588000 - 1158501600 = 86400 1158674400 - 1158588000 = 86400 1158760800 - 1158674400 = 86400 1158847200 - 1158760800 = 86400 1158933600 - 1158847200 = 86400 1159020000 - 1158933600 = 86400 1159106400 - 1159020000 = 86400 1159192800 - 1159106400 = 86400 1159279200 - 1159192800 = 86400 1159365600 - 1159279200 = 86400 1159452000 - 1159365600 = 86400 1159538400 - 1159452000 = 86400 1159624800 - 1159538400 = 86400 1159711200 - 1159624800 = 86400 1159797600 - 1159711200 = 86400 1159884000 - 1159797600 = 86400 1159970400 - 1159884000 = 86400 1160056800 - 1159970400 = 86400 1160143200 - 1160056800 = 86400 1160229600 - 1160143200 = 86400 1160316000 - 1160229600 = 86400 1160402400 - 1160316000 = 86400 1160488800 - 1160402400 = 86400 1160575200 - 1160488800 = 86400 1160661600 - 1160575200 = 86400 1160748000 - 1160661600 = 86400 1160834400 - 1160748000 = 86400 1160920800 - 1160834400 = 86400 1161007200 - 1160920800 = 86400 1161093600 - 1161007200 = 86400 1161180000 - 1161093600 = 86400 1161266400 - 1161180000 = 86400 1161352800 - 1161266400 = 86400 1161439200 - 1161352800 = 86400 1161525600 - 1161439200 = 86400 1161612000 - 1161525600 = 86400 1161698400 - 1161612000 = 86400 1161784800 - 1161698400 = 86400 1161871200 - 1161784800 = 86400 1161957600 - 1161871200 = 86400 1162044000 - 1161957600 = 86400 --->1162126800 - 1162044000 = 82800 1162213200 - 1162126800 = 86400 1162299600 - 1162213200 = 86400 1162386000 - 1162299600 = 86400 1162472400 - 1162386000 = 86400 1162558800 - 1162472400 = 86400 1162645200 - 1162558800 = 86400 1162731600 - 1162645200 = 86400 1162818000 - 1162731600 = 86400 1162904400 - 1162818000 = 86400 1162990800 - 1162904400 = 86400 1163077200 - 1162990800 = 86400 1163163600 - 1163077200 = 86400 1163250000 - 1163163600 = 86400 1163336400 - 1163250000 = 86400 1163422800 - 1163336400 = 86400 1163509200 - 1163422800 = 86400 1163595600 - 1163509200 = 86400 1163682000 - 1163595600 = 86400 1163768400 - 1163682000 = 86400 1163854800 - 1163768400 = 86400 1163941200 - 1163854800 = 86400 1164027600 - 1163941200 = 86400 1164114000 - 1164027600 = 86400 1164200400 - 1164114000 = 86400 1164286800 - 1164200400 = 86400 1164373200 - 1164286800 = 86400 1164459600 - 1164373200 = 86400 1164546000 - 1164459600 = 86400 1164632400 - 1164546000 = 86400 1164718800 - 1164632400 = 86400 1164805200 - 1164718800 = 86400 1164891600 - 1164805200 = 86400 1164978000 - 1164891600 = 86400 1165064400 - 1164978000 = 86400 1165150800 - 1165064400 = 86400 1165237200 - 1165150800 = 86400 1165323600 - 1165237200 = 86400 1165410000 - 1165323600 = 86400 1165496400 - 1165410000 = 86400 1165582800 - 1165496400 = 86400 1165669200 - 1165582800 = 86400 1165755600 - 1165669200 = 86400 1165842000 - 1165755600 = 86400 1165928400 - 1165842000 = 86400 1166014800 - 1165928400 = 86400 1166101200 - 1166014800 = 86400 1166187600 - 1166101200 = 86400 1166274000 - 1166187600 = 86400 1166360400 - 1166274000 = 86400 1166446800 - 1166360400 = 86400 1166533200 - 1166446800 = 86400 1166619600 - 1166533200 = 86400 1166706000 - 1166619600 = 86400 1166792400 - 1166706000 = 86400 1166878800 - 1166792400 = 86400 1166965200 - 1166878800 = 86400 1167051600 - 1166965200 = 86400 1167138000 - 1167051600 = 86400 1167224400 - 1167138000 = 86400 1167310800 - 1167224400 = 86400 1167397200 - 1167310800 = 86400 [download]	[reply] [d/l]
Re^3: loop issue by Kenosis (Priest) on Aug 10, 2012 at 06:26 UTC
Am unsure why the issue happened, but am glad the script is now working for you...	[reply]