Re: Iterating through Two Arrays. Is there a better use of memory?

If memory really is a problem (and I doubt it from the numbers you gave), you can could use

for (0..$#line2) {
   my $i = $line2[$_];
   for (0..$#line1) {
      my $j = $line1[$_];
      print $i if if $i eq $j;
   }
}
[download]

If memory isn't a problem, you can use a much faster solution:

my %lookup;
++$lookup{$_} for @line2;

for (@line1) {
   print if $lookup{$_};
}
[download]

Finally, you might want to get rid of duplicate messages.

my %lookup;
$lookup{$_} = -1 for @line2;

for (@line1) {
   print if !++$lookup{$_};
}
[download]

Comment on Re: Iterating through Two Arrays. Is there a better use of memory? Select or Download Code

Replies are listed 'Best First'.
Re^2: Iterating through Two Arrays. Is there a better use of memory? by Jeri (Scribe) on Oct 13, 2011 at 17:47 UTC
Again, many thanks. Any yes, everyone is correct. The numbers I gave won't effect my memory now, but I'm working on a preliminary model. Which means, I will probably run into some memory issues later. I know for the actual model I am dealing with at least two files that are 3.6GiB and 5.9GiB. Others could potentially be larger. I will heed and take everyone's advice!thanks! Jeri	[reply]
Re^3: Iterating through Two Arrays. Is there a better use of memory? by ikegami (Patriarch) on Oct 13, 2011 at 17:56 UTC
I know for the actual model I am dealing with at least two files that are 3.6GiB and 5.9GiB. Totally irrelevant to your question. The sum of the number of elements in `@line1` and `@line2` is what matters. You said the product of their size is going to be 10,000, so the sum of their sizes is going to be between 200 and 10,001, rather small numbers. If you can fit 10GiB into memory, don't fret about using less than 100KB to loop through the data.	[reply] [d/l] [select]