Re^5: looping efficiency

I’m going to repeat the excellent advice you have already heard. :P

Dealing with the FS and IO is drastically slower than all of these approaches. So you should not, arguably, be choosing by speed or cleverness but for legibility and future maintenance. Choosing by speed will not be noticed unless you are processing billions of files daily. I adore and use features like string incrementing in Perl. I probably wouldn’t do it at work, maybe, because it’s pretty idiomatic. That said, it’s also the fastest–

            Rate      glob    unpack     split  n_nested   sprintf inc
+rement
glob      78.0/s        --      -23%      -23%      -85%      -87%    
+  -90%
unpack     101/s       29%        --       -0%      -80%      -84%    
+  -87%
split      101/s       30%        0%        --      -80%      -84%    
+  -87%
n_nested   513/s      558%      409%      407%        --      -18%    
+  -36%
sprintf    623/s      698%      517%      515%       21%        --    
+  -22%
increment  797/s      922%      690%      687%       55%       28%    
+    --
[download]

Comment on Re^5: looping efficiency Download Code

Replies are listed 'Best First'.
Re^6: looping efficiency by Anonymous Monk on Dec 30, 2020 at 13:30 UTC
Thanks again. Obviously, writing all of the data (often exceedding 25 GB) to an old-fashioned hard disk represents the bottleneck for this application. I knew what my loops did (satisfies legibility and future maintenance), and from my past (though relatively limited) Perl experience knew that they represented a reasonably efficient method of accomplishing this task, too. I never even would have asked about this topic if someone (a 14-year-old, no less) didn't complain about my methods. :) That said, I do want to avoid learning poor techniques, and didn't know that Perl could do string incrementing, so I have gained a fair amount of knowledge from this thread.	[reply]

Replies are listed 'Best First'.

Re^6: looping efficiency
by Anonymous Monk on Dec 30, 2020 at 13:30 UTC

Thanks again. Obviously, writing all of the data (often exceedding 25 GB) to an old-fashioned hard disk represents the bottleneck for this application. I knew what my loops did (satisfies legibility and future maintenance), and from my past (though relatively limited) Perl experience knew that they represented a reasonably efficient method of accomplishing this task, too. I never even would have asked about this topic if someone (a 14-year-old, no less) didn't complain about my methods. :) That said, I do want to avoid learning poor techniques, and didn't know that Perl could do string incrementing, so I have gained a fair amount of knowledge from this thread.

[reply]