nbhatia has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to translate a large java file - approximately 2000 lines of code. I need to replace all calls to a legacy API to a newer API. Essentially, I need to match patterns spanning multiple lines and transform them into new patterns. Anything that does not match these patterns should pass through without any change. For example: 
Original file
=============
public Product getProduct(Node node) {
    // --- bunch of logic here ---
    if (nodeName.equals("ItemNum")) {
        product.setItemNumber(
            new Integer(StringUtils.parseInt(nodeValue, 0)));
    }
    if (nodeName.equals("AvgPrice")) {
        product.setAveragePrice(
            new Double(StringUtils.parseDouble(nodeValue, 0.0)));
    }
}

Converted file
==============
public Product getProduct(Node node) {
    // --- bunch of logic here ---
    if (msg.isSetField(ItemNum.FIELD)) {
        product.setItemNumber(msg.getInt(ItemNum.FIELD));
    }
    if (msg.isSetField(AvgPrice.FIELD)) {
        product.setAveragePrice(msg.getDouble(AvgPrice.FIELD));
    }
}

[download]
I just picked up Perl yesterday (thanks to perlintro) and trying to figure out the best approach to do this. My first attempt uses regular expressions one line at a time – if it matches /nodeName.equals/, then it tries to match the following two lines, a) to pick up the method name for the “product” call and b) to make sure that the type is Integer or Double (there can be other types with different patterns). Here’s my Perl code: 
while (<STDIN>) {
    if ($_ =~ /nodeName.equals/) {
        my $line1 = $_;
        my $line2 = <STDIN>;
        my $line3 = <STDIN>;
        if ($line3 =~ /new Integer/) {
            chomp($line1);
            $_ = "$line1$line2";
            s/nodeName.equals\("(.*)"\).*product\.(.*)\(/msg.isSetFiel
+d($1.FIELD)) {\n        product.$2(msg.getInt($1.FIELD))\;/;
            print $_;
        }
        elsif ($line3 =~ /new Double/) {
            chomp($line1);
            $_ = "$line1$line2";
            s/nodeName.equals\("(.*)"\).*product\.(.*)\(/msg.isSetFiel
+d($1.FIELD)) {\n        product.$2(msg.getDouble($1.FIELD))\;/;
            print $_;
        }
        else {
            print $line1;
            print $line2;
            print $line3;
        }
    }
    else {
        print $_;
    }
}

[download]
Is there a better way?
1) Can I somehow match and convert the entire four line block with one substitution?
2) Is regex the best tool to do this, or should I be looking at something else?

Thanks for your help.

Replies are listed 'Best First'.
Re: How to match and transform a multiline pattern?
by BrowserUk (Patriarch) on Apr 20, 2007 at 18:09 UTC

    Update: Corrected and simplified.

    Based upon the single example you've provided this will need modifications, but it might give you a starting point. The basic assumption it makes are that functions in your java code always end with a single '}' in the first column of an otherwise blank line. It uses that to read the file one function at a time and performs the modifications.

    As coded, the regex is pretty sensetive to variations in the formating of the source code. This can be loosened as appropriate, but increases the risk of false substitutions. It attempts to retain the original indentation, but it may not work in all situations. 

    #! perl -slw
use strict;

$/ = "\n}\n";

while( <DATA> ) {
    s[
        (
            nodeName.equals \( \s* " ( [^"]+ ) " ( .+? )  #"
           new \s+ ( Integer | Double ) \(  .+? \)\)\);
        )
   ][msg.isSetField(${2}.FIELD${3}(msg.get$4(${2}.FIELD));]sgx;
   print;
}

__DATA__
public Product getProduct(Node node) {
    // --- bunch of logic here ---
    if (nodeName.equals("ItemNum")) {
        product.setItemNumber(
            new Integer(StringUtils.parseInt(nodeValue, 0)));
    }
    if (nodeName.equals("AvgPrice")) {
        product.setAveragePrice(
            new Double(StringUtils.parseDouble(nodeValue, 0.0)));
    }
}
public Product getProduct(Node node) {
    // --- bunch of logic here ---
    if (nodeName.equals("ItemNum")) {
        product.setItemNumber(
            new Integer(StringUtils.parseInt(nodeValue, 0)));
    }
    if (nodeName.equals("AvgPrice")) {
        product.setAveragePrice(
            new Double(StringUtils.parseDouble(nodeValue, 0.0)));
    }
}

public Product getProduct(Node node) {
    // --- bunch of logic here ---
    if (nodeName.equals("ItemNum")) {
        product.setItemNumber(
            new Integer(StringUtils.parseInt(nodeValue, 0)));
    }
    if (nodeName.equals("AvgPrice")) {
        product.setAveragePrice(
            new Double(StringUtils.parseDouble(nodeValue, 0.0)));
    }
}

    [download]

    Outputs:

    C:\test>junk1
public Product getProduct(Node node) {
    // --- bunch of logic here ---
    if ( msg.isSetField(ItemNum.FIELD)) {
        product.setItemNumber(
            (msg.getInteger(ItemNum.FIELD));
    }
    if ( msg.isSetField(AvgPrice.FIELD)) {
        product.setAveragePrice(
            (msg.getDouble(AvgPrice.FIELD));
    }
}

public Product getProduct(Node node) {
    // --- bunch of logic here ---
    if ( msg.isSetField(ItemNum.FIELD)) {
        product.setItemNumber(
            (msg.getInteger(ItemNum.FIELD));
    }
    if ( msg.isSetField(AvgPrice.FIELD)) {
        product.setAveragePrice(
            (msg.getDouble(AvgPrice.FIELD));
    }
}


public Product getProduct(Node node) {
    // --- bunch of logic here ---
    if ( msg.isSetField(ItemNum.FIELD)) {
        product.setItemNumber(
            (msg.getInteger(ItemNum.FIELD));
    }
    if ( msg.isSetField(AvgPrice.FIELD)) {
        product.setAveragePrice(
            (msg.getDouble(AvgPrice.FIELD));
    }
}

    [download]

    Which is just 3 repetitions of the one example.

    I'm not sure I would bother to code a filter for a single 2000 line sourcefile. I could almost certainly do that much more quickly in my editor with a macro or two--or even manually--but its fun to try :)

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]
      Thanks. This works great! I must say that I do not understand your regular expression yet, I only know the s/.../.../ variety. Haven't seen the "s[][]" variety yet. I guess I need to read up on regex now :-).

      Thanks again.
[reply]
        I only know the s/.../.../ variety. Haven't seen the "s[][]" variety yet.

        They are the same, they just use different delimiters. You can choose which character(s) you use as a delimiter. If you use bracketing delimiters (eg. () or {} or [] or <>) then it can simplify the use of somethings. See the docs for the full skinny.

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

        See perlretut and perlre for a start. If you are new to Perl then have a good browse around the Tutorials section here too.

        A small nomenclature note: the "regular expression" is the bit that does the matching. In a substitution it is the first expression. In a match it is the only expression. In a substitution the other expression is called the substitution expression.

        For complicated regular expressions it is often helpful to use the /x switch which allows you to use white space and comments in your expression so that it is easier to see the different parts and to provide comments for the parts of the expression.

        DWIM is Perl's answer to Gödel
[reply]
Re: How to match and transform a multiline pattern?
by apl (Monsignor) on Apr 20, 2007 at 17:05 UTC
    In your regex you could specify \n to indicate a newline. (Play with this; your file might contain \r or a control-character as a line separator.)

    So, for example, you could have  if /.+FIRST.+\n.+SECOND.+\n.+THIRD.+\n.+FOURTH/

    which would only match a string spanning four lines, the first containing FIRST, the second containing SECOND, etc.
    This is my first attempt at helping out, so I apologize in advance. 8-)
[reply]
[d/l]
      I think I know what you are saying, but my mental block is with how to get the four lines in the first place. Since these lines are being read one at a time from a file, how would I know when to read more than one? Are you suggesting that I read the whole file in to a single string and do substitutions on the whole thing?
[reply]
        Two thousand lines of text is not a great deal.
        If you did treat it all as a single string, you could do a single $a =~ s/p/r/g;, changing all occurrences of the pattern p to r in buffer a.
[reply]
[d/l]

Back to Seekers of Perl Wisdom