comment on

Thanks for sharing your knowledge shmem. I replaced the code you wrote (with some needed modifications) in my program. But Its really dead slow.

the code took:289 wallclock secs ( 6.43 usr 0.72 sys + 388.95 cusr 58.95 csys = 455.05 CPU)

This result is for my program (using grep & cut). But after modified in this 300 wallclock secs its runs the data only for 36 seconds. Then to complete the total 10 minutes data it will take nearly .... (Actually I dont know :-) ). My modified code is

my $greatest = 0;
my $total    = 0;
my @files    = glob "SMSCDR*$date$hour$minute*.log";

foreach my $min ($minute .. $minute+9)
{

        foreach my $sec (@seconds)
        {
#               my $SMPP_count = int ((split (/\s+/,`cut -d "|" -f 1,1
+0,13 SMSCDR*$date$hour$minute*.log |grep "Submit|GSM" |grep "$hour:$m
+in:$sec" |sort |uniq -c`)) [1]) + int ((split (/\s+/,`cut -d "|" -f 1
+,10,13 SMSCDR*$date$hour$minute*.log |grep "Submit|SMPP" |grep "$hour
+:$min:$sec" |sort |uniq -c`)) [1]);

                my $SMPP_count;
                my $stamp = "$hour:$min:$sec";
                foreach my $file (@files)
                {
                        open (FILE,"$file");
                        while(<FILE>)
                        {
                                chomp;
                                my @ary = (split /\s|\|/, $_) [3,21,24
+];
                                $SMPP_count++ if  $ary[0] eq $stamp an
+d $ary[1] eq "Submit" and $ary[2] =~ /(GSM|SMPP)/;
                        };
                }
                if ($SMPP_count > $greatest)
                {
                        $greatest = $SMPP_count;
                }
                $total = $total + $SMPP_count;
                print "$hour:$min:$sec","= $SMPP_count","\t",$total,$/
+;
        }
}
print $greatest,$/;

my $t1 = Benchmark->new;
my $td = timediff($t1, $t0);
print "the code took:",timestr($td),"\n";
[download]

Note: The result is same for both the programs. As you said the reading of files has to be faster than grep & cut. But here its not working like that. Where I am missing I don't understand.

Update:

Here when you use glob it will return three files. In that post_paid contains 8278 lines, prepaid contains 23072 lines, delivery_file contains 80097 lines. If you are calculating for first second it have to check "1,11,447" lines. Like that for 10 minutes (600 seconds) the program had to check "6,68,68,200" lines. Please show me a way to get rid of this

In reply to Re^4: using system command in regex by ravi45722
in thread using system command in regex by ravi45722

Title:
Use: `<p> text here (a paragraph) </p>`
and: `<code> code here </code>`
to format your post, it's "PerlMonks-approved HTML":

Posts are HTML formatted. Put `<p> </p>` tags around your paragraphs. Put `<code> </code>` tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

How do I compose an effective node title?

How do I post a question effectively?

Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:
a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

For: Use:

& `&`

< `<`

> `>`

[ `[`

] `]`

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.

	For:		Use:
	&		`&`
	<		`<`
	>		`>`
	[		`[`
	]		`]`