in reply to Parsing a lot of data. Very slow. Need suggestions

Well, off the top of my head, I'd say you could make all of your date calculations a whole lot simpler with Date::Calc.

Update: Here's the script, sans hand-rolled date processing:

#! /usr/bin/perl use strict ; use warnings ; $|++ ; use Date::Calc qw( Add_Delta_Days Day_of_Week Today ) ; my $days_back = shift || 120; my @start_ymd = Add_Delta_Days( Today, -$days_back ) ; my @end_ymd = Today ; my $start = sprintf( "%d-%02d-%02d", @start_ymd ) ; my $end = sprintf( "%d-%02d-%02d", @end_ymd ) ; print"$start\n$end\n\n"; system("parsecache -f nlclick -s $start -e $end > nlclick.cache"); system("parsecache -f nlimage -s $start -e $end > nlopen.cache"); for my $i ( 0 .. $days_back - 1 ) { my $wday = Day_of_Week ( @start_ymd ) ; if ( $wday > 0 && $wday < 6 ) { my $date = sprintf("%d%02d%02d", @start_ymd ); #print "$date\n"; $opens = `report pattern --regex='nl=$date' --filter=nlimage - +-sumonly < nlopen.cache`; if ( defined( $opens ) ) { $transfers = `report pattern --regex='nl=$date' --filter=s +tandard --sumonly +< nlclick.cache`; if ( !defined( $transfers ) ) { $transfers = 0 } printf("update msnNewsletter set open='%d', click='%d' whe +re dateCreated like '%d-%02d-%02d%%';\n", $opens, $transfers, @start_ +ymd ); } } } unlink "nlclick.cache"; unlink "nlopen.cache"; __END__

You're original handling of the system calls is rather odd, assigning to $_ and then capturing it all to a variable. The only thing that comes to mind is taint checking, which you're not doing here, so that section can also be simplified some.

Note that none of these necessarily have anything to do with the speed of your program, but may impact it's accuracy, and certainly improve it's maintainability.


_______________
DamnDirtyApe
Those who know that they are profound strive for clarity. Those who
would like to seem profound to the crowd strive for obscurity.
            --Friedrich Nietzsche