in reply to Re^4: Write large array to file, very slow
in thread Write large array to file, very slow
"Twice as fast" seems like a lot for in memory operations when there are also disk accesses.
Yes, I thought so too. Looks like my data set was so large it ate into swap. :-)
Re-running with a smaller data set still shows quite a decent speed up, however. Here's my bench and results:
#!/usr/bin/env perl use strict; use warnings; use Benchmark 'cmpthese'; my $size = 50_000_000; my @big = (rand () x $size); cmpthese (10, { 'interp' => 'interp ()', 'Eily' => 'eily ()', 'OFS' => 'ofs ()', }); exit; sub interp { open FH, '>', 'mergedlogs.txt' or die "can't open mergedlogs.txt: +$!"; local $| = 0; foreach (@big) { print FH "$_\n"; } close FH; } sub eily { my $output_file = "mergedlogs.txt"; open my $output_fh, ">", $output_file or die "Can't open $output_f +ile: $!"; local $| = 0; local $\ = "\n"; foreach (@big) { print $output_fh $_; } close $output_fh; } sub ofs { my $output_file = "mergedlogs.txt"; open my $output_fh, ">", $output_file or die "Can't open $output_f +ile: $!"; local $| = 0; local $\ = "\n"; local $, = "\n"; print $output_fh @big; close $output_fh; }
s/iter interp Eily OFS interp 1.83 -- -35% -35% Eily 1.20 53% -- -1% OFS 1.19 54% 1% --
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^6: Write large array to file, very slow
by Eily (Monsignor) on Aug 21, 2018 at 08:29 UTC | |
by hippo (Archbishop) on Aug 21, 2018 at 08:52 UTC |