in reply to Re^3: Help me beat NodeJS
in thread Help me beat NodeJS
Got it. I went ahead and updated both MCE demonstrations to account for pattern matching. The more expensive regex (/".*?"|\S+/g) pattern is processed only if given line matches the initial string pattern. That will likely run faster.
Likewise, for Parallel::ForkManager.
#!/usr/local/bin/perl use strict; use warnings; use Parallel::ForkManager; my $pm = new Parallel::ForkManager(24); my $dir = '/data/logs/*.log.gz'; my @files = sort(glob "$dir"); my $pattern = "some_string"; $pm->set_waitpid_blocking_sleep(0); for my $file( @files ) { $pm->start and next; open( my $fh, "-|", "/bin/zcat", $file ) or die "open error: $!\n" +; while ( my $line = <$fh> ) { if ( $line =~ /$pattern/ ) { my @matches = $line =~ /".*?"|\S+/g; print "$matches[0],$matches[1],$matches[3],$matches[4]\n"; } } $pm->finish; } $pm->wait_all_children;
Regards, Mario
|
---|
Replies are listed 'Best First'. | |
---|---|
Re^5: Help me beat NodeJS
by RichardK (Parson) on Feb 13, 2016 at 22:54 UTC | |
by marioroy (Prior) on Feb 13, 2016 at 23:41 UTC | |
by rickyw59 (Novice) on Feb 14, 2016 at 04:10 UTC |