next if defined $val; will skip any keys from %hash that we have previously assigned a value to.

$hash{$key} = $known{$key};next; will assign any values we learned from the last run and then move on to the next record.

$heap->insert($key); will only insert records into the heap for keys that we have not assigned a value to (either in a previous run or this run). Update: According to the documentation, max_count => $at_once will throw out items from the heap beyond that point. If that doesn't work as advertised, that may be the source of the additional memory.