Planning for Backwards Compatibility

While reading for the Nth time "The interface could be better but we're stuck with it now", I began thinking about ways to provide better backwards and forward compatibility. I have some ideas but nothing ground-breaking and I'd like to know about any other techniques out there.

A few approaches to changes in API or Interfaces are:

Don't. Stick with the sub-optimal past
Do. Let the new version croak, the old versions still available on CPAN
Deprecation cycle, see perlmodstyle:

Backwards compatibility
Modules which are "stable" should not break backwards compatibility without at least a long transition phase and a major change in version number.
Opt-in back-compat. Require users to ask for a deprecated feature (eg. TT2's V1DOLLAR)
Fork. Budd-off into a new namespace, leaving the old API behind
Provide a custom VERSION sub, "use Mod 1.1", perldoc -f use
use version; the module.

If you plan to evolve an API or want to release a non-binding alpha version, you could provide an alternate name that you document as stable, while the main-line can change. For example:

    use Some::Module::alpha; # supported as is
    use Some::Module::alpha::warn; # as is, opt-in to future warnings
    use Some::Module; # may evolve without warning
[download]

Initially, "alpha" would basically be an proxy for the main module. Later it might be a backward-compatibility wrapper for the new interface. If the backward compatibility becomes impractical or restrictive then the alpha interface could become it's own package. The ::warn name extension (or "compat_warn => 1") could be used to opt-in to future compatibility change warnings.

I'm just brain-storming here and am interested in what others think.

What have other people tried?
What works and, importantly, what doesn't?
Are there already PM nodes addressing this?
How does Perl 6's module versioning system address these issues?
(The naming of v6-alpha inspired me in part)

Brad

Comment on Planning for Backwards Compatibility Download Code

Replies are listed 'Best First'.
Re: Planning for Backwards Compatibility by LTjake (Prior) on Aug 20, 2007 at 12:19 UTC
Ignoring backwards compatibility isn't really an option, IMO. I like this post by Alias on that particular subject. One way to add new features but ensure back-compat is to have a comprehensive test suite. We've tried very hard with the Catalyst project to have a sufficient test suite so as to make sure that any piece that gets changed doesn't affect the expected behavior. Sometimes you can't help but break back-compat. In cases like that, you could probably do that under a major revision increment (1.x -> 2.x). That way people might be able to understand the api breakage. I wish CPAN worked a little differently. Currently if you release 2.x of module, then 1.x is basically dead. This doesn't give your users the option to keep their app stable with the 1.x releases while getting security/bug fixes. Either they switch to 2.x or they're left in the lurch. You can still release a new 1.x version, but it just means that people have to install it manually. Ah well. Update: Just thought that i might also mention dev releases. In order to smooth over the transition, you could release a series of 1.99_0X dev releases where your bleeding-edge users can try out the new api and prepare for the upcoming 2.x release. -- "Go up to the next female stranger you see and tell her that her "body is a wonderland." My hypothesis is that she’ll be too busy laughing at you to even bother slapping you." (src)	[reply]
Re^2: Planning for Backwards Compatibility by bennymack (Pilgrim) on Aug 20, 2007 at 16:40 UTC
From Aristotle's post: In the next few days, Sachmet and I are going to be announcing a new, huge project. Something that it would be nice to see as one of next year's Big Things in the Perl community. And it's going to take quite a number of people to get it done, from coders to expert sysadmins to normal people that just want to lend a hand. To be honest, if it comes off anything close to it's potential I will be majorly impressed. It should make PPI look like a cute little toy in comparison. What ever happened to this big project? Anyone know?	[reply]
Re^3: Planning for Backwards Compatibility by bsb (Priest) on Aug 21, 2007 at 00:26 UTC
I think that's PITA PS. Alias != Aristotle	[reply]
Re: Planning for Backwards Compatibility by eyepopslikeamosquito (Archbishop) on Aug 20, 2007 at 12:16 UTC
Apart from offering a couple of references: Perl 6 S11 (see Versioning section) Discussion of Perl 6 versioning on perl6.language Python's back to the __future__ I don't feel qualified to comment on this difficult topic ... though I'm keenly interested in it. I'm sure @Larry drew lessons from the CPAN, Python's __future__ module, and other versioning approaches in designing the new Perl 6 versioning features. I'd be interested to learn which versioning prior art most influenced their design.	[reply]
Re: Planning for Backwards Compatibility by lima1 (Curate) on Aug 20, 2007 at 12:15 UTC
A compatibility layer (either in the same package or in a new namespace) can soon become very ugly, for example when you introduce new data structures. When such an compatibility layer would be absolutely necessary, my vote is: don't do the API changes unless the refactoring is not much more than method renamings. Another option is a long transition phase with developer releases on CPAN. When you are happy with the new API, increment the major version number and release.	[reply]
Re: Planning for Backwards Compatibility by bsb (Priest) on Aug 26, 2007 at 23:53 UTC
Backwards compatibility in Makefiles (from The Art of Unix Programming) Why the tab in column 1? Yacc was new, Lex was brand new. I hadn't tried either, so I figured this would be a good excuse to learn. After getting myself snarled up with my first stab at Lex, I just did something simple with the pattern newline-tab. It worked, it stayed. And then a few weeks later I had a user population of about a dozen, most of them friends, and I didn't want to screw up my embedded base. The rest, sadly, is history. -- Stuart Feldman	[reply]
Re: Planning for Backwards Compatibility by snowhare (Friar) on Aug 24, 2007 at 01:08 UTC
You left out another approach: Keep the default as is but add a 'opt in' method for the new API. This has happened to me a couple of times with CGI::Minimal. The first time was when I needed to support 'mixed' GET/POST semantics (processing GET style CGI parameters and POST style CGI parameters in a single POST request). Which is wierd and non-RFC, but required to integrate with some specific code I didn't have control over. Changing the default would likely break code for someone (probably me ;) ), so I added a static method, `CGI::Minimal::allow_hybrid_post_get(1);`, that let the programmer request the new semantics. A few days ago someone requested a change to CGI::Minimal's behavior when running under ModPerl2. By default, they ran their server with %ENV hash filling with the CGI environment variables disabled (which give a small performance boost) while CGI::Minimal, by default, forces filling %ENV (which is the case I personally need). Changing the default would definitely break existing code. But I could see the utility of his request (although my benchmarking puts the performance pop only in the single digit percentages). So I added a 'use' flag (:no_subprocess_env) to request the API behavior he wanted.	[reply] [d/l]