http://qs1969.pair.com?node_id=1166649

bms9nmh has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I have the following perl script which compares words in the 2nd field of two csv files, and if 5 or more words match then it prints both line together as a positive match.
#!/bin/perl + + my @csv2 = (); + open CSV2, "<csv2" or die; + @csv2=<CSV2>; + close CSV2; + + my %csv2hash = (); + for (@csv2) { + chomp; + my ($title) = $_ =~ /^.+?,\s*([^,]+?),/; #/ match the title + $csv2hash{$_} = $title; + } + + open CSV1, "<csv1" or die; + while (<CSV1>) { + chomp; + my ($title) = $_ =~ /^.+?,\s*([^,]+?),/; #/ match the title + my %words; + $words{$_}++ for split /\s+/, $title; #/ get words + ## Collect unique words + my @titlewords = keys(%words); + my @new; #add exception words which shouldn +'t be matched foreach my $t (@titlewords){ + push(@new, $t) if $t !~ /^(rare|vol|volume|issue|double|magazi +ne|mag)$/i; } + @titlewords = @new; + my $desired = 5; + my $matched = 0; + foreach my $csv2 (keys %csv2hash) { + my $count = 0; + my $value = $csv2hash{$csv2}; + foreach my $word (@titlewords) { + my @matches = ( $value=~/\b$word\b/ig ); + my $numIncsv2 = scalar(@matches); + @matches = ( $title=~/\b$word\b/ig ); + my $numIncsv1 = scalar(@matches); + ++$count if $value =~ /\b$word\b/i; + if ($count >= $desired || ($numIncsv1 >= $desired && $numI +ncsv2 >= $desired)) { $count = $desired+1; + last; + } + } + if ($count >= $desired) { + print "$csv2\n"; + ++$matched; + } + } + print "$_\n\n" if $matched; + } + close CSV1;
I would now like to add extra functionality so that- if both the lines contain a year, in the format 1989, then the years in each csv have to match for it to be considered a positive match. However, if only one of the lines contains a year, then I would like the usual 5 matching words rule to apply and the year becomes irrelevant. Examples just to clarify. In this example field 2 contains 5 matching words but the two years (1973 + 2013) are different so this would be discounted as a match:
2523021356, RARE TV RADIO TIMES MAGAZINE DOCTOR WHO 1973 THE THREE 3 +DOCTORS DR JON PERTWEE, http://www.example.co.uk, 12 12278788, TV & SATELLITE WEEK 11 MAY 2013 GILLIAN ANDERSON DOCTOR WHO +NOT RADIO TIMES , http://www.example.co.uk, 12
In this example the years are the same AND there are 5 or more matching words so this would be a positive match:
2523021356, RARE TV RADIO TIMES MAGAZINE DOCTOR WHO 1973 THE THREE 3 +DOCTORS DR JON PERTWEE, http://www.example.co.uk, 12 12278788, TV & SATELLITE WEEK 11 MAY 1973 GILLIAN ANDERSON DOCTOR WHO +NOT RADIO TIMES , http://www.example.co.uk, 12
In this example, only one of the titles contain a year (1973), but there are also 5 or more matching words, I would like this to be considered a positive match:
2523021356, RARE TV RADIO TIMES MAGAZINE DOCTOR WHO 1973 THE THREE 3 +DOCTORS DR JON PERTWEE, http://www.example.co.uk, 12 12278788, TV & SATELLITE WEEK 11 MAY GILLIAN ANDERSON DOCTOR WHO NOT R +ADIO TIMES , http://www.example.co.uk, 12
In this example, none of the titles contains a year, but 5 or more words match so this would be a positive match:
2523021356, RARE TV RADIO TIMES MAGAZINE DOCTOR WHO THE THREE 3 DOCTO +RS DR JON PERTWEE, http://www.example.co.uk, 12 12278788, TV & SATELLITE WEEK 11 MAY GILLIAN ANDERSON DOCTOR WHO NOT R +ADIO TIMES , http://www.example.co.uk, 12
How can I add this functionality to the script without making wholesale changes?