I've seen a simple approach to a similar problem. It went something like:

1. Convert each string to an array of words.
2. Delete "noise" words ("a", "the", "to", etc.) from each array.
3. Convert each remaining word into a "canonical" form (e.g., all lower-case, apostrophe's deleted.)
4. Compute a score based on the Hamming Distance between the source array is to the target array. (Hamming distance is a measure of how close two strings are to each other. In one form^*, it considers transpositions and deletions.)

Try the simple approach first: convert all words to lower-case, drop noise words, then see how close the two arrays are in terms of common elements in a common order.

----
^*I did a quick google search. Most descriptions talk in terms of Hamming's original definition, which was in terms of bit flips. I've seen this applied to strings of arbitrary data somewhere -- perhaps in an Algorithms text. The string comparison that limewire uses to guess at whether a search string matches an MP3 title might also be worth checking, though it's in Java.

-Blake

$longest  = soundex of longer string
$shortest = soundex of shorter string

if ($longest =~ /$shortest/) {
  ...
}
[download]

cLive ;-)

A Super Search came back with This very similar thread, except searching a whole database for such sorta-matches. Sure enough, the consensus winner was This XOR solution from japhy, although there were many other good sugestions.

drinkd

In the node you cite, the goal was to ensure that two strings are "differnet enough". In this node, the goal is to determine if two nodes are "similar enough"

That's an important distinction.

japhy's solution considers two strings different if their lengths are significantly different. I'm fairly certain that's not what Cody wants. If one quote is a complete subset, or a complete superset of another, with only an few subtle spelling mistakes, it should still be considered "similar enough". I would definitely go with String::Approx and set the 'I' and 'D' modifiers very high to allow a large number of Inserts and Deletes (and of course: the 'i' modifier for case insensativeity). You'll have to really play with the "approximateness percentage" to ensure that it allows enough subtle word spelling differences, but doesn't complain that every quote that uses the word "the" is the same.

And of course: I wouldn't recomend this to flat out reject any quote, just to flag it as a potential duplicate.

Here's how it hands the "gonna" -> "going to" change.

use strict;
use Algorithm::Diff qw(traverse_sequences);

my @quote1 = split(' ', <<QUOTE);
I walk. I talk. I shop, I sneeze. 
I'm gonna be a fireman when the floods roll back.
There's trees in the desert since you moved out. 
And I don't sleep on a bed of bones.
QUOTE

my @quote2 = split(' ', <<QUOTE);
I'm going to be a fireman when the floods roll back.
There's trees in the desert since you moved out.
QUOTE

my $match = 0;
traverse_sequences(\@quote1, \@quote2,
                    { MATCH => sub { $match++ } }
                  );

my $smaller = @quote1 < @quote2 ? @quote1 : @quote2;
my $percent_match = int(($match / $smaller) * 100);

print "$match words matched ($percent_match%)\n";
__END__
18 words matched (90%)
[download]

A few minutes perusing the Algorithm::Diff POD might give you some additional ideas.