shemp has asked for the wisdom of the Perl Monks concerning the following question:

I have a lot of code that is used to cleanup, correct, & verify mailing addresses from various data sources. (I post a lot about this). We have been designing the next-generation address parser for work, and are trying to figure out the best way to incrememtally test the new code and replace the old code. We'll want to run the old version in parallel with the new version and compare the results. The comparisons would be written to a report that would help us tweak the new code. A simple way to do this would be something like: 
sub parallel_tester {
   # assume neither parser modifies @_
   my $old_result = old_parser(@_);
   my $new_result = new_parser(@_);

   compare_n_report($old_result, $new_result, @_);
}

[download]
But, at some point the new parser will want to take over from the old parser. If all the functions / classes / methods in the new parser are coded with different names, we'll have to go through that code and modify all the names of the functions and the calling of them.

So, we're trying to figure out the best way to deal with this code crossover. The main idea we have is to play some sort of games with package names so that all that needs to be done is change what is required / used etc.

I know that this is a quite vague question, and we haven't experimented much, but i see us going down a lot of very wrong paths before figuring out a good way to handle this upgrade. And unfortunately, this isnt a matter of replacing a function with a new version and testing the logic of that function, because the function could behave exactly as we designed it to, but because of the overall problem, the whole thing needs to be tested as a unit, and there will actually be very little replacing of one part of the old code with a new version, the whole project will be re-written.

Replies are listed 'Best First'.
Re: Clean code transition - how?
by dragonchild (Archbishop) on May 03, 2005 at 18:13 UTC
    It all depends on one factor - can you shoehorn your new API into your old one? If you can, then augment the calls to the old parser's functions with simultaneous calls to the new parser's functions. Then, when you're happy, remove the old parser code and let the old parser functions map to the new parser's functions.

    Then, you do two more things:

    1. New development goes against the new parser
    2. Maintenance of old development includes changing calls from the old parser to the new parser.

    Eventually, you remove all references to the old parser and then you remove it. Usually takes between 6-24 months.

    Now, if you can't shoehorn the calls ... well, good luck.

    The Perfect is the Enemy of the Good.

[reply]
Re: Clean code transition - how?
by scmason (Monk) on May 03, 2005 at 18:46 UTC
    My best advise would be to develop the systems separately (in different namespaces/machines whatever). Run unit tests on the new code (as we always should), and also compare the complete output of the both when ran over the same inputs to make sure that the results of the new are at least as good as the old (or why else would you make the new?).

    When you are sure that the new system is complete and good, then replace the old system in its entirety.

    To be more precise:

    1. Make a copy of the old program and call it the new program.
    2. Rewrite all of your classes/modules.
    3. Test
    4. Modify
    5. Replace the old program with the new program.
    6. Sit back and have a cigar knowing that you have done well.

[reply]
Re: Clean code transition - how?
by Fletch (Bishop) on May 03, 2005 at 18:37 UTC

    Might try to make a version of the old API which croaks when any of the old routines are called.

    And get a copy of Refactoring by Fowler if you don't already have one.

[reply]
[d/l]
Re: Clean code transition - how?
by mstone (Deacon) on May 03, 2005 at 21:00 UTC

    Consider building another layer of abstraction into your current code.

    As I understand your question, you don't want to get stuck searching-and-replacing function names in the code that uses these two libraries. By and large, that means both your brain and your survival instinct are in good working order. ;-)

    A set of wrapper functions will let you modify your client code once, then switch between the two different libraries with ease. If your client code presently looks like this: 

        {...}

    $some_value = old_parser_function_X (@args);
    
    {...}

    [download]

    Swap in a generic function name, and then create a new function which calls the original one: 

        -- client code --
    {...}
    
    $some_value = generic_function_X (@args);
    
    {...}
    
    -- wrapper library --
    
    sub generic_function_X {
        return (old_parser_function_X (@_));
    }

    [download]

    The payoff is that you can now write another wrapper that calls functions from your new parser library: 

        -- new wrapper --
    
    sub generic_function_X {
        {assemble a result by calling:
            new_parser_function_A (@some_args)
            new_parser_function_D (@some_other_args)
            new_parser_function_N (@still_more_args)
        }
        return ($result);
    }

    [download]

    and it will work as a drop-in replacement for the old wrapper, even though the two parser libraries have completely different APIs.

    Writing a test suite that compares the two wrapper libraries then becomes a one-banana problem.

[reply]
[d/l]
[select]
Re: Clean code transition - how?
by graff (Chancellor) on May 04, 2005 at 02:28 UTC
    I'm not speaking from experience here, so "add salt to suit your taste"...

    Based on your vague description, it sounds like the plan is for the new-generation parser to have the same API as the old one, so there is an existing code base using the old parser that (theoretically) shouldn't change at all as you change the parser.

    If the old parser modules are in some specific directory, and the new replacement modules are in some other directory, you could try doing two runs over a given range of input data, and just twiddle $ENV{PERL5LIB} in order to control which set of modules is loaded. (And of course, you'll want the outputs stored separately as well.)

    If the old modules are in the same directory as the code base that uses them (along with other modules that aren't being updated, which means you can't take "." out of @INC), you should be able to relocate the old modules to some other directory, and again twiddle $ENV{PERL5LIB} to run your contrasting tests.

    (It's not something I've tried myself, but it seems plausible...)

[reply]

Back to Seekers of Perl Wisdom