> Sadly, the documentation for fast_io seems to be non-existent and I always struggle to figure out how to use it.
Same here. This took a team. I've been grepping for clues inside the fast_io benchmark and examples folders. To break 0.5 seconds, I updated the allinone2 demonstration with fast_io memory mapping.
# https://perlmonks.org/?node_id=11152186 $ ./rtoa-pgatram-allinone2 t1.txt t1.txt t1.txt t1.txt | cksum do_it_all time : 0.515 secs fast_io scan, line_get do_it_all time : 0.490 secs fast_io memory mapping 737201628 75552000
Memory mapping update:
// Read an input file of Roman Numerals and do it all static void do_it_all( std::string_view fname // in: file name containing a list of + Roman Numerals ) { try { #if 1 // Load entire file to memory through memory mapping. using file_loader_type = fast_io::native_file_loader; file_loader_type loader(fname, fast_io::open_mode::in | fast_io: +:open_mode::follow); // Loop through contiguous container of the file. for (char const *first{loader.data()}, *last{loader.data()+loade +r.size()}; first!=last; ) { auto start_ptr{first}; first = fast_io::find_lf(first, last); + auto end_ptr{first}; if (start_ptr == end_ptr) continue; int dec = roman_to_dec(std::string_view(start_ptr, end_ptr - +start_ptr)); fast_io::io::println(dec); ++first; } #else fast_io::filebuf_file fbf(fname, fast_io::open_mode::in | fast_i +o::open_mode::follow); for (std::string line; fast_io::io::scan<true>(fbf, fast_io::mnp +::line_get(line)); ) { fast_io::io::println(roman_to_dec(line)); } #endif } catch (fast_io::error e) { fast_io::io::perrln("Error opening '", fname, "' : ", e); }; }
It requires running Perl on Clear Linux to keep up :)
# https://perlmonks.org/?node_id=11152186 $ time ./rtoa-pgatram-allinone2 t1.txt t1.txt t1.txt t1.txt | cksum do_it_all time : 0.490 secs fast_io memory mapping 737201628 75552000 real 0m0.492s user 0m0.471s sys 0m0.037s # https://perlmonks.org/?node_id=11152168 max_workers => 32 $ time perl rtoa-pgatram-mce.pl t1.txt t1.txt t1.txt t1.txt | cksum rtoa pgatram start time 0.480 secs 737201628 75552000 real 0m0.504s user 0m13.887s sys 0m0.231s # https://perlmonks.org/?node_id=11152168 max_workers => 64 $ time perl rtoa-pgatram-mce.pl t1.txt t1.txt t1.txt t1.txt | cksum rtoa pgatram start time 0.425 secs 737201628 75552000 real 0m0.449s user 0m21.884s sys 0m0.592s
Perl MCE is mind-boggling to me. Because, the UNIX time includes all of it; the time to launch Perl, load modules, spawn workers, notify workers per each input file / run, and finally reap workers. The overhead is 0.075 and 0.093 seconds for 32 and 64 workers, respectively. The overhead also includes MCE::Relay where workers must wait their turn orderly by chunk_id value, behind the scene.
I commented out a few lines of code to measure the overhead time running MCE including MCE::Relay.
user_func => sub { my ( $mce, $slurp_ref, $chunk_id, $output ) = ( @_, '' ); # open my $fh, '<', $slurp_ref; # while ( <$fh> ) { # chomp; # my $n = 0; # $n += $_ - $n % $_ * 2 for @rtoa[ unpack 'c*', $_ ]; # $output .= "$n\n"; # } # close $fh; # output orderly MCE::relay { # print $output; }; }
How much time does Perl MCE chunking four input files including MCE::Relay take, factoring out compute and output? Notice how doubling up on workers only adds less than 2 hundreds of a second to the UNIX real time. But notice the user and sys times. Chunking an input file and involving relay occur simultaneously behind the scene. The relay block runs serially.
# max_workers => 32 $ time perl rtoa-pgatram-mce.pl t1.txt t1.txt t1.txt t1.txt rtoa pgatram start time 0.052 secs real 0m0.075s user 0m0.122s sys 0m0.170s # max_workers => 64 $ time perl rtoa-pgatram-mce.pl t1.txt t1.txt t1.txt t1.txt rtoa pgatram start time 0.070 secs real 0m0.093s user 0m0.195s sys 0m0.441s
In reply to Re^9: Risque Romantic Rosetta Roman Race - All in One
by marioroy
in thread Risque Romantic Rosetta Roman Race
by eyepopslikeamosquito
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |