elef has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to do a somewhat tricky search and replace in a tab delimited txt, and I'm getting odd results.

I want to check line by line if the contents of the first cell are repeated in the second, and if so, delete the contents of the first cell. Otherwise, the line should be reprinted with no changes.

The script I came up with does what I want it to, but in many lines, the last character of the first cell gets deleted. I don't see how this could be caused by an error in my regex, as it should either leave the whole line unchanged or leave the first cell empty; it's not supposed to mess around within the first cell. The strange thing is, I also tried sed and it produces similar errors. (BTW, I'm using Activeperl 5.10.0 on Win XP.)

Here's my script:

#!/usr/bin/perl use strict; use warnings; open(IN, "<:encoding(UTF-8)", "in.txt") or die "Can't open input file: + $!"; open(OUT, ">:encoding(UTF-8)", "out.txt") or die "Can't open output fi +le: $!"; while (<IN>) { s/([^\t]*)\t([^\t]*)\1([^\t]*)\t/\t$2$1$3\t/; print OUT $_; } close IN; close OUT; print "Replacements done"; <STDIN>;
And here's a snippet of my input file (note: my tabs seem to be converted to 4 spaces by the forum software)
abandon/to abandon/to abandon/to (a child) abandon/to (a road) abandon/to (claim) abandon/to (motion) abandon/to (one's appeal) abandon/to (privilege on document) abandon/to abandon rented premises/to abandon/to steal or abandon/to (personal property) abandon/to vacate or abandon/to (possession or occupation of a lan +d) abandoned/to be abandoned/to be abandoned/to be appeal that is abandoned abandoned/to be application that is abandoned abandoned/to be motion that is abandoned abandoned motor vehicle site abandoned motor vehicle site abandoned orchard abandoned orchard abandonment abandonment abandonment (appeal) abandonment (land) abandonment (mines or minerals)
And the output I'm getting (see lines 3,5,6,7,13 etc. for the errors):
abandon/to abandon/to (a child) abandon/t (a road) abandon/to (claim) abandon/t (motion) abandon/t (one's appeal) abandon/t (privilege on document) abandon abandon rented premises/to steal or abandon/to (personal property) vacate or abandon/to (possession or occupation of a land) abandoned/to be abandoned/to b appeal that is abandoned abandoned/to b application that is abandoned abandoned/to b motion that is abandoned abandoned motor vehicle site abandoned orchard abandonment abandonment (appeal) abandonment (land) abandonment (mines or minerals)
Thanks for any help!
Update: solved.
As AnomalousMonk pointed out, I should have made sure the first matched substring starts at the beginning of the line, i.e. s/^([^\t]*)\t([^\t]*)\1([^\t]*)\t/\t$2$1$3\t/;

Replies are listed 'Best First'.
Re: Regex search and replace appears to be broken
by AnomalousMonk (Archbishop) on Aug 17, 2010 at 11:26 UTC

    Perhaps seeeing exactly what is being matched and captured will help (/x regex modifier used to help my poor eyes):

    >perl -wMstrict -le "my $s = qq{abandon/to\t(a road)\t}; $s =~ s{ ([^\t]*) \t ([^\t]*) \1 ([^\t]*) \t }{\t$2$1$3\t}xms; print qq{1 '$1' 2 '$2' 3 '$3'}; print qq{'$s'}; " 1 'o' 2 '(a r' 3 'ad)' 'abandon/t (a road) '

    Update: This may help (note anchor at start of string): If the entire content of the first cell is repeated anywhere in the second cell, delete the entire content of the first cell:

    >perl -wMstrict -le "for my $s (qq{abandon/to\t(a road)\t}, qq{abandon/to\txxx abandon/to yyy \t}, ) { print qq{'$s'}; (my $t = $s) =~ s{ \A ([^\t]+) (\t [^\t]* \1 [^\t]* \t) }{$2}xms; print qq{1 '$1' 2 '$2'} if defined($1) and defined($2); print qq{'$t' \n}; } " 'abandon/to (a road) ' 'abandon/to (a road) ' 'abandon/to xxx abandon/to yyy ' 1 'abandon/to' 2 ' xxx abandon/to yyy ' ' xxx abandon/to yyy '

    And this, capturing less, may be slightly better:
        s{ \A ([^\t]+) (?= \t [^\t]* \1 [^\t]* \t) }{}xms;

      You're absolutely right about the anchoring, I completely overlooked that.
      On some lines, my regex was matching the last letter or the last couple of letters of the first cell instead of the whole cell. s/^([^\t]*)\t... instead of s/([^\t]*)\t... fixes the problem.
Re: Regex search and replace appears to be broken
by roboticus (Chancellor) on Aug 17, 2010 at 11:29 UTC

    elef:

    Keep in mind that no characters is a valid string.

    The first line is converting correctly: $1 = "abandon/to", $2="", and $3="". Then when you do the substitution, you get "\tabandon/to\t".

    ...roboticus