File is remote, but I think everything I do is remote since I have to log in to the server
The question is whether the file is local or remote to the processor running the program; not the human who initiates it.
You could try this, but it is doubtful if it will be any faster when using multiple threads than just one unless the file is on a local, fast, SSD:
#! perl -slw use strict; use threads; sub worker { my( $filename, $target, $start, $end ) = @_; open my $fh, '<', $filename or die $!; seek $fh, $start, 0; <$fh> if $start > 0; ## discard first partial line my $count = 0; 1+index( <$fh>, $target ) and ++$count while tell( $fh ) < $end; return $count; } our $T //= 4; my( $filename, $target ) = @ARGV; my $fsize = -s $filename; my $chunksize = int( $fsize / $T ); my @chunks = map{ $_ * $chunksize } 0 .. $T-1; push @chunks, $fsize; my @threads = map{ threads->new( \&worker, $filename, $target, $chunks[ $_ ], $chunks +[ $_+1 ] ) } 0 .. $T-1; my $total = 0; $total += $_->join for @threads; print "Found $total '$target' lines";
Usage:
thisScript.pl -T=n theFile.txt "the string"
Note:The count is printed to stdout. Redirect it if you need it in a file.
In reply to Re^3: How to split file for threading?
by BrowserUk
in thread How to split file for threading?
by diyaz
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |