in reply to Re^3: Help me beat NodeJS
in thread Help me beat NodeJS

Got it. I went ahead and updated both MCE demonstrations to account for pattern matching. The more expensive regex (/".*?"|\S+/g) pattern is processed only if given line matches the initial string pattern. That will likely run faster.

Likewise, for Parallel::ForkManager.

#!/usr/local/bin/perl use strict; use warnings; use Parallel::ForkManager; my $pm = new Parallel::ForkManager(24); my $dir = '/data/logs/*.log.gz'; my @files = sort(glob "$dir"); my $pattern = "some_string"; $pm->set_waitpid_blocking_sleep(0); for my $file( @files ) { $pm->start and next; open( my $fh, "-|", "/bin/zcat", $file ) or die "open error: $!\n" +; while ( my $line = <$fh> ) { if ( $line =~ /$pattern/ ) { my @matches = $line =~ /".*?"|\S+/g; print "$matches[0],$matches[1],$matches[3],$matches[4]\n"; } } $pm->finish; } $pm->wait_all_children;

Regards, Mario

Replies are listed 'Best First'.
Re^5: Help me beat NodeJS
by RichardK (Parson) on Feb 13, 2016 at 22:54 UTC

    If you just looking for a plain string not a regex, then it should be quicker to use index

    while (my $line = <$fh> ) { if (index($line,$pattern) != -1 ) { ... } }

      yes, ++

        Very nice, thank you.