in reply to Self-Populating Tree Data Structure

Just concatenate your three fields together and use it as a hash key. The de-dup becomes a one-pass, O(1) lookup process:

my %lookup; open IN, '<', ... open OUT, '>', ... while( <IN> ) { my( $file, $line, $rule ) = m[...(...)...(...)...(...)] or warn("Bad data: '$_' line: $.\n") and next; if( exists $lookup{ join $;, $file, $line, $rule } ) { ## Duplicate, discard next; } else { ## New, record. $lookup{ join $;, $file, $line, $rule } = 1; print OUT; ## output $_ } } close IN; close OUT;

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."