Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re^4: Help me beat NodeJS

by marioroy (Prior)
on Feb 13, 2016 at 22:10 UTC ( [id://1155184]=note: print w/replies, xml ) Need Help??


in reply to Re^3: Help me beat NodeJS
in thread Help me beat NodeJS

Got it. I went ahead and updated both MCE demonstrations to account for pattern matching. The more expensive regex (/".*?"|\S+/g) pattern is processed only if given line matches the initial string pattern. That will likely run faster.

Likewise, for Parallel::ForkManager.

#!/usr/local/bin/perl use strict; use warnings; use Parallel::ForkManager; my $pm = new Parallel::ForkManager(24); my $dir = '/data/logs/*.log.gz'; my @files = sort(glob "$dir"); my $pattern = "some_string"; $pm->set_waitpid_blocking_sleep(0); for my $file( @files ) { $pm->start and next; open( my $fh, "-|", "/bin/zcat", $file ) or die "open error: $!\n" +; while ( my $line = <$fh> ) { if ( $line =~ /$pattern/ ) { my @matches = $line =~ /".*?"|\S+/g; print "$matches[0],$matches[1],$matches[3],$matches[4]\n"; } } $pm->finish; } $pm->wait_all_children;

Regards, Mario

Replies are listed 'Best First'.
Re^5: Help me beat NodeJS
by RichardK (Parson) on Feb 13, 2016 at 22:54 UTC

    If you just looking for a plain string not a regex, then it should be quicker to use index

    while (my $line = <$fh> ) { if (index($line,$pattern) != -1 ) { ... } }

      yes, ++

        Very nice, thank you.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1155184]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chanting in the Monastery: (2)
As of 2024-04-26 03:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found