Re: Is foreach split Optimized?

The question seems to be, "does perl split the entire string before beginning the for loop?" We can answer that by looking at how much memory it is using.

use strict;
use warnings;
my $n = 10_000_000;
system "ps -orss $$";
my $x = 'foo' x $n;
system "ps -orss $$";
for my $y (split /f/, $x) {
   system "ps -orss $$";
   last;
}
[download]

The answer seems to be, "yes."

   RSS
  3712
   RSS
 33028
   RSS
674900
[download]

Comment on Re: Is foreach split Optimized? Select or Download Code

Replies are listed 'Best First'.
Re^2: Is foreach split Optimized? by haukex (Archbishop) on Jul 11, 2017 at 10:28 UTC
Thank you, this is probably the best way of looking at it! I took your idea and expanded on it, below. It seems that the answer to my original question is No, there isn't any magic going on with `for (split ...)`, it's just that while it may be a memory hog, split is still pretty fast - that's what threw me off initially. #!/usr/bin/env perl use warnings; use 5.014; my $cnt = 5000000; system($^X,'-sE',<<'_END_','--',"-cnt=$cnt"); say `ps -orss $$`=~s/\s+/ /gr; # RSS 4340 $x = "abc\n" x $cnt; say `ps -orss $$`=~s/\s+/ /gr; # RSS 23936 @y = ("abc") x $cnt; say `ps -orss $$`=~s/\s+/ /gr; # RSS 455604 _END_ say "---"; system($^X,'-sE',<<'_END_','--',"-cnt=$cnt"); say `ps -orss $$`=~s/\s+/ /gr; # RSS 4544 $x = "abc\n" x $cnt; say `ps -orss $$`=~s/\s+/ /gr; # RSS 23972 @y = split /\n/, $x; say `ps -orss $$`=~s/\s+/ /gr; # RSS 423544 _END_ say "---"; system($^X,'-sE',<<'_END_','--',"-cnt=$cnt"); say `ps -orss $$`=~s/\s+/ /gr; # RSS 4540 $x = "abc\n" x $cnt; say `ps -orss $$`=~s/\s+/ /gr; # RSS 23964 for (split /\n/, $x) { say `ps -orss $$`=~s/\s+/ /gr; # RSS 457512 last } _END_ say "---"; system($^X,'-sE',<<'_END_','--',"-cnt=$cnt"); say `ps -orss $$`=~s/\s+/ /gr; # RSS 4336 $x = "abc\n" x $cnt; say `ps -orss $$`=~s/\s+/ /gr; # RSS 23932 open my $fh, '<', \$x or die $!; while (<$fh>) { say `ps -orss $$`=~s/\s+/ /gr; # RSS 24000 last } _END_ [download]	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^2: Is foreach split Optimized?
by haukex (Archbishop) on Jul 11, 2017 at 10:28 UTC

Thank you, this is probably the best way of looking at it! I took your idea and expanded on it, below. It seems that the answer to my original question is No, there isn't any magic going on with for (split ...), it's just that while it may be a memory hog, split is still pretty fast - that's what threw me off initially.

#!/usr/bin/env perl
use warnings;
use 5.014;

my $cnt = 5000000;
system($^X,'-sE',<<'_END_','--',"-cnt=$cnt");
    say `ps -orss $$`=~s/\s+/ /gr;  # RSS 4340
    $x = "abc\n" x $cnt;
    say `ps -orss $$`=~s/\s+/ /gr;  # RSS 23936
    @y = ("abc") x $cnt;
    say `ps -orss $$`=~s/\s+/ /gr;  # RSS 455604
_END_
say "---";
system($^X,'-sE',<<'_END_','--',"-cnt=$cnt");
    say `ps -orss $$`=~s/\s+/ /gr;  # RSS 4544
    $x = "abc\n" x $cnt;
    say `ps -orss $$`=~s/\s+/ /gr;  # RSS 23972
    @y = split /\n/, $x;
    say `ps -orss $$`=~s/\s+/ /gr;  # RSS 423544
_END_
say "---";
system($^X,'-sE',<<'_END_','--',"-cnt=$cnt");
    say `ps -orss $$`=~s/\s+/ /gr;  # RSS 4540
    $x = "abc\n" x $cnt;
    say `ps -orss $$`=~s/\s+/ /gr;  # RSS 23964
    for (split /\n/, $x) {
    say `ps -orss $$`=~s/\s+/ /gr;  # RSS 457512
    last }
_END_
say "---";
system($^X,'-sE',<<'_END_','--',"-cnt=$cnt");
    say `ps -orss $$`=~s/\s+/ /gr;  # RSS 4336
    $x = "abc\n" x $cnt;
    say `ps -orss $$`=~s/\s+/ /gr;  # RSS 23932
    open my $fh, '<', \$x or die $!;
    while (<$fh>) {
    say `ps -orss $$`=~s/\s+/ /gr;  # RSS 24000
    last }
_END_
[download]

[reply]
[d/l]
[select]