in reply to Finding duplicates in a text file

use strict; use warnings; my @lines1 = ( 'The, slow, lazy, brown and green dog', qq~I now want it to ignore special characters such as (), ",", ', +" etc.~, qq~I have code which finds duplicates in 2 text files~, ); my @lines2 = ( 'The slow lazy brown and green dog', qq~I now want it to ignore special characters such as (), ",", ', +" etc.~, qq~I have code which finds duplicates in 2 files and spits into 3 +files~, ); my %where; @lines1 = map {(my $clean = $_) =~ s/[^\s\w]//g; [$_, $clean]} @lines1 +; @lines2 = map {(my $clean = $_) =~ s/[^\s\w]//g; [$_, $clean]} @lines2 +; $where{$_->[1]} = ["1", $_->[0]] for @lines1; $where{$_->[1]} = [($where{$_->[1]}[0] .= "2"), $_->[0]] for @lines2; for (sort keys %where) { my $where = $where{$_}->[0]; my $what = $where{$_}->[1]; if ($where =~ /12/) { print "12: $what\n"; } elsif ($where =~ /1/) { print "1: $what\n"; } else { print "2: $what\n"; } }

Prints:

2: I have code which finds duplicates in 2 files and spits into 3 file +s 1: I have code which finds duplicates in 2 text files 12: I now want it to ignore special characters such as (), ",", ', " e +tc. 12: The slow lazy brown and green dog

is close, but only shows one variant of the dog line. It's really a question of how you actually want to handle that situation.


DWIM is Perl's answer to Gödel