Re^2: grep return the index instead of the element content, and its speed

Hi ikegami,

Thank you your quick reply.

Question for your update: why the for loop version use much less memory?

Also, will the use of direct access the array index make the grep slower?

(* the speed is much important than the memory as I need to repeat this action with thousand times, and the string in the array for searching usually has hundred of words...)

Comment on Re^2: grep return the index instead of the element content, and its speed

Replies are listed 'Best First'.
Re^3: grep return the index instead of the element content, and its speed by ikegami (Patriarch) on Feb 28, 2008 at 10:30 UTC
why the for loop version use much less memory? `for (x..y)` loop and `for (@array)` loop (but no other kind of `for (LIST)` loops) are optimized to iterate over the list without flattening it. `$n = 10_000_000; print ": "; <>; # 2MB $f = 1; for (0..$n-1) { if ($f) { $f = 0; print ": "; <>; } } # 2MB` [download] `$n = 10_000_000; print ": "; <>; # 2MB push @a, --$n while $n; print ": "; <>; # 240MB $f = 1; for (@a) { if ($f) { $f = 0; print ": "; <>; } } # 240MB` [download] `grep { ... } LIST` flattens the list, so `grep { ... } @array` loads the entire array* on the stack and `grep { ... } 0..$#array;` create a list in memory as big as the array. `$n = 10_000_000; print ": "; <>; # 2MB $f = 1; grep { if ($f) { $f = 0; print ": "; <>; } 0 } 0..$n-1; # 240MB <--` [download] `$n = 10_000_000; print ": "; <>; # 2MB push @a, --$n while $n; print ": "; <>; # 240MB $f = 1; grep { if ($f) { $f = 0; print ": "; <>; } 0 } @a; # 280MB <--` [download] * - The amount of memory taken is proportional to the number of elements. It doesn't matter how much memory each of those element takes. Also, will the use of direct access the array index make the grep slower? I don't completely understand what you said, but the answer is clear: Write a Benchmark test to find out. Updated: Added code snippets.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^3: grep return the index instead of the element content, and its speed
by ikegami (Patriarch) on Feb 28, 2008 at 10:30 UTC

why the for loop version use much less memory?

for (x..y) loop and for (@array) loop (but no other kind of for (LIST) loops) are optimized to iterate over the list without flattening it.

$n = 10_000_000;
print ": ";  <>;                                       # 2MB
$f = 1;
for (0..$n-1) { if ($f) { $f = 0; print ": "; <>; } }  # 2MB
[download]

$n = 10_000_000;
print ": "; <>;                                   #   2MB
push @a, --$n while $n;
print ": ";  <>;                                  # 240MB
$f = 1;
for (@a) { if ($f) { $f = 0; print ": "; <>; } }  # 240MB
[download]

grep { ... } LIST flattens the list, so grep { ... } @array loads the entire array* on the stack and grep { ... } 0..$#array; create a list in memory as big as the array.

$n = 10_000_000;
print ": ";  <>;                                         #   2MB
$f = 1;
grep { if ($f) { $f = 0; print ": "; <>; } 0 } 0..$n-1;  # 240MB  <--
[download]

$n = 10_000_000;
print ": "; <>;                                     #   2MB
push @a, --$n while $n;
print ": "; <>;                                     # 240MB
$f = 1;
grep { if ($f) { $f = 0; print ": "; <>; } 0 } @a;  # 280MB  <--
[download]

* - The amount of memory taken is proportional to the number of elements. It doesn't matter how much memory each of those element takes.

Also, will the use of direct access the array index make the grep slower?

I don't completely understand what you said, but the answer is clear: Write a Benchmark test to find out.

Updated: Added code snippets.

[reply]
[d/l]
[select]