Hello TravelAddict,

I’ve been thinking on and off about this problem, and it occurred to me that you can leverage a module like String::Diff or Algorithm::Diff if you map the words in your sentences onto character tokens and then apply one of the standard string diff methods to the resulting token strings. Here is a proof of concept:

#! perl
use strict;
use warnings;
use Data::Dump;
use String::Diff 'diff';

my @pairs =
(
    [
        'This apple is colored red.',
        'This apple is coloured red.',
    ],
    [
        'I need to buy a round trip ticket.',
        'I need to buy a return ticket.',
    ],
    [
        'Jack rode the elevator to the top floor.',
        'Jack took the lift to the top floor.',
    ],
);

my %diffs;

for my $pair (@pairs)
{
    my (@sent, %ids, @seq);
    my  $id = 33;
    my ($seq0, $seq1);

    for my $i (0, 1)
    {
        $sent[$i] = $pair->[$i] =~ s/'s?\b//gr;              # Remove 
+possessives
        $sent[$i]               =~ s/[-[\].,;:'"(){}<>]//g;  # Remove 
+all other punctuation

        for (split /\s+/, $sent[$i])
        {
            $ids{$_} //= $id++;
            $seq[$i]  .= chr($ids{$_});
        }
    }

    my  %lookup     = reverse %ids;
    my ($old, $new) = diff($seq[0], $seq[1]);
    my  @old        = $old =~ /\[(.+?)\]/g;
    my  @new        = $new =~ /\{(.+?)\}/g;

    while (@old && @new)
    {
        my $o = join(' ', map { $lookup{ord $_} } split(//, shift @old
+));
        my $n = join(' ', map { $lookup{ord $_} } split(//, shift @new
+));

        $diffs{$o} = $n;
    }
}

dd \%diffs;

[download]

Output:

 1:42 >perl 1269_SoPW.pl
{
  "colored"    => "coloured",
  "elevator"   => "lift",
  "rode"       => "took",
  "round trip" => "return",
}

 1:42 >

[download]

Hope that helps,

Athanasius <°(((>< contra mundum Iustus alius egestas vitae, eros Piratica,

In reply to Re: Comparing strings to extract the different words/expressions by Athanasius
in thread Comparing strings to extract the different words/expressions by TravelAddict

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.