in reply to Multi threading a

What exactly do you want to do in the threads? There's no exhaustive computation to parallelise. Reading from a single file in multiple threads tends to be slower than reading from it in a single thread (not sure about SSD).

map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

Replies are listed 'Best First'.
Re^2: Multi threading a.. huge file -- MCE::Grep
by Discipulus (Canon) on Sep 03, 2021 at 07:51 UTC
    Hello wiser choroba,

    > Reading from a single file in multiple threads tends to be slower

    Are you sure? Is not what MCE::Grep is for?

    ## File path, glob ref, IO::All::{ File, Pipe, STDIO } obj, or scalar 
+ref
## Workers read directly and not involve the manager process
my @e = mce_grep_f { /pattern/ } "/path/to/file"; # efficient

    [download]

    Also: MCE::Grep#PARSING-HUGE-FILES

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
[reply]
[d/l]
      Oh, so in each thread, you read a large chunk from the file and then process it line by line in memory! This might work if you're sure your pattern can't be split between two chunks.

      map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
[reply]
[d/l]
[reply]

In Section Seekers of Perl Wisdom