CPAN module versioning/backwards compatibility

bulk88 has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to fix XS CPAN mod that is atleast a decade old. I read Planning for Backwards Compatibility. The C docs I refer to are the docs on the C API that this CPAN mod wraps. They are not written by the person who wrote the CPAN mod. When I contact the current maintainer of the CPAN the maintainer mentioned they dont have a copy of the C library (its closed source) and said to keep backwards compatibility, and the original author who also doesn't have a copy of the C library anymore didn't mention backwards compatibility and suggested that trying to sync the Perl side of the module to the C docs is not necessary in his opinion. The module API has various problems such

misspelled keys in returned hashrefs
"$arrayref = $obj->parse('field1,field2,field3');" the actual array ref is ['field1', 'field2,field3' ], it should be ['field1', 'field2', 'field3'] according to the POD.
The option codes which exist in C docs and in the POD, are given as parameters to methods whose effects are swapped with other option codes, POD and C docs say "3" for very high, "2" for high, "1" for low, "0" for very low. The actual behavior is "3" for low, "2" for high, "1" for very high, "0" requires next @_ parameter to be a object from a different class of the module or the perl interp crashes in the XS code, 1 2 and 3 dont have a 2nd @_ parameter, the very low setting not reachable actually from Perl
errors numbers are always returned as signed, they are unsigned in the C constant header file and unsigned in the module's PM perl const sub definitions for those constants
docs say to pass a hashref with struct like data, actual parameter is a packed string of the C struct data

None of these are show stopper bugs from using the module, and I'm pretty sure everyone who used the module over the last 10 years has placed workarounds in their code for these quirks. I did atleast. Now how to fix this mess. None of the fixes are new features or new method calls, but they break backwards compatibility with all existing code. How does someone go about fixing a CPAN library like this?

Currently I wrote a VERSION and import() sub to deal with the case of "use Local::Foo 'importme';", "use Local::Foo 3.05 'importme' ;" and "use 3.05 Local::Foo '3.05' 'importme';" with calls Exporter and UNIVERSAL with "shift->SUPER::VERSION(@_);" and "shift->SUPER::import(@_);". In the VERSION and import subs I wrote, a const sub is evaled, something like this

    if($_[0] >= 3.05){
        eval "sub USE_NEWAPI () { 1 }";
    }
    else{
        eval "sub USE_NEWAPI () { 0 }";
    }
[download]

In the methods for this module, the USE_NEWAPI is called and the perl optimizer constant folds away conditional branches to either behavior of the old API or the new API (checked with B::Deparse). The module's test files were updated to reflect old api and new api behavior and make sure old way of doing things still works. Some games with require and another PM file are used to create a 2nd compiling context for the new or old api affected methods to allow the constant folder to work. The only problem with this design is, what if the CPAN downloader, in his script does

use Local::Foo::Parser;
use Local::Foo 3.08;
[download]

internally Local::Foo::Parser does

use Local::Foo;
[download]

The module Local::Foo::Parser is a separate CPAN library, and there are about half a dozen CPAN libraries that use Local::Foo, bugs and all. I think what will happen is, the CPAN download will not get the new API, because Local::Foo was "use"d with the "old" buggy api in a different mod. If the order of the CPAN downloader's use statements are reversed, Local::Foo::Parser will develop a ton of bugs because of the API changes that Local::API suddenly has.

So I ask, how is this supposed to be fixed?

caller checks to see if its main:: or not in Local::Foo?

a localized "$Local::Foo::newapi = 1;" (does local work this way)?

Who cares about Local::Foo::Parser, I'm not bug fixing it, and they should keep up with module they use internally! its their fault not threatening to do a CPAN unauthorized release 7 years ago when their RT bug report and their patch for the bug were filed and then after not getting a response in a couple days they decided to put work arounds in Local::Foo::Parser that have been there 7 years and counting.

Or forking the Local::Foo class is the only way to fix this mess? create a PM file in Local::Foo's tarball called "Local::Foo::______" and @ISA inherit the methods that aren't buggy or dependent on newapi or oldapi behavior from Local::Foo. Maybe put buggy methods in a PM file called Local::Foo::OldMethods.pm and require them on demand if someone calls Local::Foo::new()?

Where does the POD for Local::Foo::______ go? in a new POD document in Local::Foo::______ or add sections "With ::______ methodname does XXXXXX" in the section of Local::Foo where methodname is documented? What should _______ be? if its the word "new" it wont be "new" 10 years from now, so thats dumb. Plus "$obj = new Local::Foo::new("parameter 1");" looks funny.

Or screw backwards compatibility, fix the existing method calls and you a cpan downloader should never upgrade a CPAN mod unless you've performed due diligence by read the POD and changes file on it first and then tested your existing code and and indirectly other existing CPAN modules you use (which may or may not use the mod your using internally). The old tarball is still on cpan.org for anyone who wants it.

Change the existing buggy methods to a croak("MethodName has been removed as of version 3.05 of Local::Foo") and make a sub called MethodNameV3?

I'd like to keep the discussion abstract rather than mention the specific CPAN library and then get answers specific to that CPAN library that don't apply to anything else in the perl world.

Comment on CPAN module versioning/backwards compatibility Select or Download Code

Replies are listed 'Best First'.
Re: CPAN module versioning/backwards compatibility by moritz (Cardinal) on May 12, 2012 at 19:02 UTC
Don't go checking caller, it's a sure way to make this stuff even harder to maintain, and will eventually lead to code that lies to itself because it wants to trick the caller check that you now consider installing. Forking the module and redistributing it under a different name seems like the cleanest solution to me. Perl 6 - the future is here, just unevenly distributed	[reply]
Re^2: CPAN module versioning/backwards compatibility by sauoq (Abbot) on May 12, 2012 at 20:01 UTC
Forking the module and redistributing it under a different name seems like the cleanest solution to me. I agree 100%. `-sauoq "My two cents aren't worth a dime.";`	[reply]
Re: CPAN module versioning/backwards compatibility by tobyink (Canon) on May 12, 2012 at 18:53 UTC
How many users does this module have? If it's a small number, then you can start a mailing list and co-ordinate as a group how you're going to proceed. Fix it in stages. Announce that in the next release Foo and Bar are going to be fixed; share code samples with each other to cope with the changes. If it's got a large, established user base, then I'd say keep the API stable. Fix any problems which result in crashes or security issues. Fix the documentation to match how the module really behaves. Use big, bold text where it involves misspellings or other oddities. And then create a new, sane module with a different name. Deprecate the old module in its favour. `perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'`	[reply]