Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Re: Faster and more efficient way to read a file vertically

by karlgoethebier (Abbot)
on Nov 04, 2017 at 11:14 UTC ( [id://1202744] : note . print w/replies, xml ) Need Help??


in reply to Faster and more efficient way to read a file vertically

Stolen, cannibalized and slightly adopted from this older thread: Threads From Hell #2: How To Search A Very Huge File [SOLVED]:

#!/usr/bin/env perl # http://www.perlmonks.org/?node_id=1202693 # $Id: loop.pl,v 1.2 2017/11/04 11:02:41 karl Exp karl $ use strict; use warnings; use MCE::Loop; use Time::HiRes qw( time ); use feature qw(say); my $file = q(data.txt); MCE::Loop::init( { max_workers => 4, use_slurpio => 1 } ); my $start = time; my @result = mce_loop_f { my $slurp_ref = $_[1]; my @column; open my $fh, '<', $slurp_ref; binmode $fh, ':raw'; while (<$fh>) { push @column, substr( $_, 10, 1 ) } close $fh; MCE->gather(@column); # sleep 2; } $file; say join( '', @result ); printf "Took %.3f seconds\n", time - $start; __END__

Thanks to marioroy.

See also MCE.

Update: To avoid the call to binmode please see Encoding horridness revisited: What's going on here? [SOLVED].

Regards, Karl

«The Crux of the Biscuit is the Apostrophe»

perl -MCrypt::CBC -E 'say Crypt::CBC->new(-key=>'kgb',-cipher=>"Blowfish")->decrypt_hex($ENV{KARL});'Help