Stepping through a hash while modifiying its content

Gorgarian has asked for the wisdom of the Perl Monks concerning the following question:

Hi all in the monastry -- hope you are not all sworn to silence during the festive season.

I have a small challenge, being fairly new to perl. I wish to step through a hash with

  while ( ($key, $value) = each %hash ){
  }
[download]

but then modify the hash by deleting key,value pairs from the hash after some processing if they meet a certain condition, with the while loop in progress.

I have this nagging worry that modifiying the contents of the hash on the fly will stuff up the orderly progression of the while loop and that some records may be missed.

Can anyone with a deeper knowledge of perl advise on this?

A workaround would be to establish a companion array

  @array = keys %hash;
  foreach $key (@array){
   # Access the hash, and do the deed including any hash
   # key, value deletions.
  }
[download]

but this doubles the memory requirements, and my files are huge. I would like to avoid this if it is not necessary.

Looking forward to hearing from you. My first post.

Comment on Stepping through a hash while modifiying its content Select or Download Code

Replies are listed 'Best First'.
Re: Stepping through a hash while modifiying its content by Anonymous Monk on Dec 28, 2012 at 04:58 UTC
each If you add or delete a hash's elements while iterating over it, entries may be skipped or duplicated--so don't do that perlfaq4#What happens if I add or remove keys from a hash while iterating over it? (contributed by brian d foy) The easy answer is "Don't do that!" If you iterate through the hash with each(), you can delete the key most recently returned without worrying about it. If you delete or add other keys, the iterator may skip or double up on them since perl may rearrange the hash table. See the entry for each() in perlfunc.	[reply]
Re^2: Stepping through a hash while modifiying its content by Gorgarian (Initiate) on Dec 28, 2012 at 12:13 UTC
OK, thank you for confirming my suspicions, and pointing out the documentation that covers this. I can probably get away with only deleting the most recently returned entry, and not the second entry I reference,though with some considerable increase in runtime. Have a good new year.	[reply]
Re: Stepping through a hash while modifiying its content by Anonymous Monk on Dec 28, 2012 at 05:02 UTC
# Access the hash, and do the deed including any hashkey, value deletions You should replace that comment with some code :) You could delay deletion :) `{ my @goners; while( my( $k, $v ) = each %hash ){ push @goners, $k; } delete @hash{ @goners }; undef @goners; }` [download]	[reply] [d/l]
Re^2: Stepping through a hash while modifiying its content by Gorgarian (Initiate) on Dec 28, 2012 at 12:19 UTC
That is a good suggestion too -- I could run the code in blocks, saving up the deletions, and doing a clean-up after each 1M iterations, then restarting the while loop. I will see how much longer it takes to run after restricting myself to deleting only deleting the most recent iteration reference, and if it is a problem, move to your idea as a Plan B. Thanks for your help. Learning ... :)	[reply]
Re: Stepping through a hash while modifiying its content by karlgoethebier (Abbot) on Dec 28, 2012 at 14:43 UTC
Perhaps you can do it with `List::Pairwise` instead? `use List::Pairwise qw(grepp); # grep hash subset my %subset1 = grepp {$a =~ /^ba/} %hash; my %subset2 = grepp {$b < 5} %hash;` [download] See List::Pairwise "...and my files are huge" I don't know how your files look. Perhaps you can provide an example... I'm just guessing. Regards, Karl ŤThe Crux of the Biscuit is the Apostropheť	[reply] [d/l] [select]