Hello monks,

Just some random thoughts about some things I've encountered recently due to some poor judgments on my part, and some thoughts about the concept of data-hiding.

At work I have a bunch of Perl scripts that access some central data. The data is a hideous mess and I'm unable to clean it at its source. Because I found myself re-parsing the same data over and over, I threw it into a module to attempt to make it easier to access; the module parses the data, organizes it and outputs it as needed. This is a good place to use OOP, I think: provide a cleaner interface to dirty data.

My problem is that the module's interface (the way it stores data internally, the way it's initialized, the way it sends data back to the user) is functional, but far from optimal. What I THOUGHT I'd need to do with the data is different than what I actually need to, for one thing.

Lesson learned: YAGNI! I first heard that term in the chatterbox a few days ago; if only I'd heard it 3 months sooner. Don't make guesses at what you'll need someday, because you're probably going to be wrong. Instead design something with separate, clean pieces that you can later put together to create anything you need.

Now, I know enough about OOP that I'm not directly accessing the module's guts in my scripts; I have accessor methods. But my accessor methods aren't the best. For example instead of returning a hash of hashes of hashes of hashes and letting the user deal with it, what I really should have been doing is returning an array of specifically what the user asked for. Doing that way would be good because 1) it's less confusing for the user, 2) the module can initialize/fetch parts of the data at a time as needed, instead of all at once, 3) it forces me to store the data in a cleaner way in the module, 4) it lets me change the backend without breaking the frontend. etc. etc. I wasn't doing quite enough parsing and data-simplification at the module level; using my module was easier than using the original raw data, but it wasn't as easy as it could have been. The bad thing is that as a result, my scripts are tied too tightly to the original modules' interface (or the structure of the data the module outputs).

Lesson learned: don't puke any more data at the people who are using your modules than they specifically ask for and require.

So the best solution is to AVOID interface changes. But barring that, assuming a change is required or greatly desired, the quesiton is : How do you handle interface changes when a lot of people are still using your old interface?

There are various ways I see of correcting this problem:

I'm unsure which option is the best. The answer is probably "it depends". (Isn't that the answer to all questions? (Obvious answer: It depends on the question.)) But do you have any other thoughts or experiences?

  • Comment on How to avoid (or handle) module interface changes

Replies are listed 'Best First'.
Re: How to avoid (or handle) module interface changes
by eyepopslikeamosquito (Archbishop) on Oct 21, 2005 at 22:00 UTC

    Some general advice on interface design:

    • Start by imagining the best possible interface, unconstrained by the practicalities of implementation; though you may need to temper this ideal based on the practicalities of implementation, start by imagining your perfect interface.
    • As pointed out by TheDamian in PBP, you must "play test" your interface for a while to find a good, usable, practical one. Writing tests first helps here. "Play test" it from several different perspectives: newbie user perspective, expert user perspective, and so on.
    • Deriving an interface from real use cases and YAGNI helps keep the interface small and simple (and avoid over-engineering).
    • Be patient. You are most unlikely to design the perfect interface the first time. After you have used the interface in anger for a few weeks, re-think it again.

    See also Ingy's "Swiss Army Light Sabre" - or, "how do you design your APIs?".

Re: How to avoid (or handle) module interface changes
by dragonchild (Archbishop) on Oct 21, 2005 at 23:26 UTC
    Since the new version is mostly likely to have a superset of the old version's functionality and better decomposed to boot, you will usually want to provide a compat layer for the new one that exposes the old one's API. A few caveats:
    • That API has to be perfect, including any bugs.
    • You have to be willing to force people to switch if they're asking for a new feature that's already in the new version. Don't backport to the compat layer.
    As always, be completely upfront. One important thing is to release a documentation update to the old version saying that this is obsolete and new development should target the new version. Then, point them to the compat layer and walk away.

    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
Re: How to avoid (or handle) module interface changes
by rir (Vicar) on Oct 21, 2005 at 21:29 UTC
    You left out the option of your new code living in the same namespace: to extend and wrap and deprecate stuff in one module.

    Then you have all your code proximate and can keep your users and push them toward the new code. In your case it seems the pushiness may be tolerable since users will be attracted to the better interface functions. It also leaves you in a good position to refactor and reuse.

    Coming up with new names for functionality can be a pain.

    Wrappers are a simple way to extend and abstact existing code. I think your point regarding wrappers is valid but I feel that you are viewing them too negatively in this situation.

    Be well,
    rir

Re: How to avoid (or handle) module interface changes
by 5mi11er (Deacon) on Oct 21, 2005 at 19:22 UTC
    From your description of your particular problem, it sounds to me like a creating an entirely new module will be your best option.

    To eventually be able to replace the errant module, you might want to entertain a sub-name space that can be, at first, optionally "used/required", then later as the old interface is depricated, becomes the default. This is somewhat like the idea of wrappers, but possibly using inheritance to 'fix' the broken interface with a new version, but leaving the old one accessible as is for now...

    -Scott

Re: How to avoid (or handle) module interface changes
by swampyankee (Parson) on Oct 22, 2005 at 18:04 UTC

    I'm one of those rather peculiar programmers who would finds excuses to write library functions 8-), so I can understand why you write a module.

    I tend to like your 3d option (complete re-write, keep the old interface for the existing code, and make the new interface available for new code) the most attactive from a technical point of view, and your second option most from an overall point of view. I really do not like either of the last two options, which (in my opinion) are really different ways of doing the same thing.


    Added "attractive" so the sentence actually makes sense.

    emc

Re: How to avoid (or handle) module interface changes
by sfink (Deacon) on Oct 26, 2005 at 18:21 UTC
    Require all your users to rewrite their scripts using the new interface. Theoretically the best and most correct option, might be best in some cases, but not likely unless time has no value.
    I don't think you're giving this option enough credit. The issue is not whether time has value; it's whether you'll spend more time providing wrappers and backwards-compatibility shims than rewriting all code that uses the interface.

    I understand that it may be your users' time versus your time, but surely those aren't complete noninterchangeable? If you volunteer to go through all current users' code and update them to the new interface, then will they really have that big of an issue with it changing? And in the process, you'll learn for certain whether the new API really handles actual users' needs.

    Admittedly, this only works with a very limited user base (and almost certainly only in a closed-source situation), but I'm guessing that's the situation you're in anyway.

    In short: never pass up an opportunity to avoid the backwards-compatibility abyss. If you're going to write wrappers or adapters or whatever, be very very sure that you really need to and aren't just bending over backwards for artificial reasons. You'll be stuck with it soon enough; postpone the inevitable as long as you possibly can.

    Also, take a look at ingy's only for requesting specific versions of a module.


    I work for Reactrix Systems, and am willing to admit it.