Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things

Ingy's "Swiss Army Light Sabre" - or, "how do you design your APIs?"

by kal (Hermit)
on Mar 23, 2004 at 10:16 UTC ( #338941=perlmeditation: print w/replies, xml ) Need Help??

This meditation is my reaction to reading Brian Ingerson's article on his new IO::All module, which (in brief) is an API for doing IO.

Now, IO::All is interesting for a number of reasons, but I'm interested only in the design of the API in this node. If you read the article, or download the module and play with it, it's my opinion that you'll find this is a pretty good API. Some parts of it I'm less fanboy of, but overall I think it's great - it hits you as, "why didn't Perl have this from the start?!".

How does one go about designing this kind of thing? In fact, do you design it at all? Ingy put it down to XP practices, at least in part. And there is definitely some "knowing when to break the rules" going on - such as exporting, by default, a function to the caller. This kind of thing is usually frowned upon, but is... well, Spiffy.

There is, of course, no real answer to the "how to I get my API right from the start?". It's likely even Ingy didn't get it right on his first attempt, maybe not even his second. And, of course, the light sabre is the tool of the Jedi, which probably impies something. But, what works for people? Let the sands of time erode the rough edges? Decimate the functionality with every major release, to instill fear into the remaining functions? Keep rewriting it until you like it? I guess this is a fixed-chord problem, but your thoughts are most welcome.

  • Comment on Ingy's "Swiss Army Light Sabre" - or, "how do you design your APIs?"

Replies are listed 'Best First'.
Re: Ingy's "Swiss Army Light Sabre" - or, "how do you design your APIs?"
by flyingmoose (Priest) on Mar 23, 2004 at 13:48 UTC
    IO:All is controversial to me, as I have alluded to here.

    I like API's to be simple, straight forward, and they need to involve as little 'theory of operations' as possible. In my question above, it turns out IO:All blocks the user from doing something nonsensical, yet, the API intrinsically implies such a thing is possible. (Not to be picking on IO::All, I'm just using it as an example). Another API discourse -- Many API's, for example Linux PAM have a rather bad 'theory of operations', meaning you have to call a lot of functions in a certain order to get things right. I like functions to be one-shot-one-kill and do what they say, in the right context. Required domain knowledge should be internal to the API methods, not be forced upon the caller. Java methods are typically the anthithesis of this, requiring tons of methods across various classes to get any functional unit of work done. Brevity and simplicity in the interface is lacking. IO::All does a good job here (most API's don't), and it even does checking to block the crazy implications of routing a directory to a socket, there is another issue.... normality and obviousness.

    If you didn't have the documentation, and only the method names, you should know exactly how it works and it should work as expected. You also shouldn't be able to blow your leg off, much less shoot yourself in the foot.

    As for breaking API's, I've seen this done to me millions of times and it annoys me to no end. Not in Perl, but C++. I guess a good way to start in Perl it to have most functions take named parameters (hashes) and not to change method signatures. But that takes discipline and up-front design -- not everyone is into that. With C, have your functions take structures and add to the end of them to preserve binary compatibility -- but even this can be hard. It's design-up-front. Not always easy to see the future.

    But in all, it's an artform, and stringency that removes fun and coolness isn't always worth it. IO::All has some cool ideas, but it walks a fine line between cool and too cool. But is there such a thing as too cool? Only for production work.

      I'm not sure whether I agree with your "theory of operations" statement; although of course that would depend on whether we draw the "as possible" line in the same place. For me, a module should give you a good idea of how it is operating. With IO, for example, being able to tell the difference between buffered and unbuffered IO is sometimes crucial. I'm not sure you can ever get away from that kind of thing completely, or if you would ever want to (i.e., what is the opportunity cost?)

      The thing about method names sounds like an orthogonality argument, which is probably quite right. You could probably make it orthogonal, but for me Perlishness doesn't require orthogonality (although it's always good). And, the documentation does tell you about < and > and when and how they work (maybe it has been changed since you looked).

      I suspect I wouldn't use the overloaded operators. But, you never know. I like the shellness of them. Perhaps some of this is a case of "too cool" though. Hmm, I'm not sure.

        FYI -- my comments about 'theory of operations' and the need to have a heavy and strict order of method calls (see PAM code in C, the Windows API, Java socket API's, whatever) is annoying and could be cleaned up -- but that's not a comment about IO::All, but rather API design in general.

        You lost me when you speak of 'an orthogonality argument', what are you proposing should be othogonal to what? Again, the commentary of IO::All is just interlaced into my post as reference, I think the '>' operators are indeed (1) succinct but are not (2) obvious enough. My biggest issue with API design is that API's are not clean and succint enough, and IO::All does a good job in this case. But it's sketchy when it comes to expected behavior, IMHO -- there are obviously tradeoffs to consider between the two.

Re: Ingy's "Swiss Army Light Sabre" - or, "how do you design your APIs?"
by EvdB (Deacon) on Mar 23, 2004 at 12:51 UTC
    My favorite test for an API is "Can I remember how to use this after a week and in a hurry without the docs?".

    If I can remember the method names and the arguments then the API works for me. If I need the docs open all the time then the API is wrong. Obviously some things are more complicated than others but this is a good start.

    --tidiness is the memory loss of environmental mnemonics

Re: Ingy's "Swiss Army Light Sabre" - or, "how do you design your APIs?"
by simon.proctor (Vicar) on Mar 23, 2004 at 10:59 UTC
    I generally find that after I have designed and built my API the only true test is when I come to use it. An API, for me, tidies away all the niggly things and makes my *front end* code very simple. If my API doesn't allow me to quickly build applications and write clear and concise code I need to rethink the API. After all, if I can't use it who can?

    Of course I am in a team of one so I have only myself to blame ;).
Re: Ingy's "Swiss Army Light Sabre" - or, "how do you design your APIs?"
by stvn (Monsignor) on Mar 23, 2004 at 16:01 UTC

    There are already some really excellent comments here. I agree very much with flyingmoose that an ideal would be that if you only knew the method names, you would still understand the API/class. Personally, I think this is something every API designer should strive for, but few ever actually reach. I also agree with dragonchild that an obsessive drive for simplicity and domain experience are two characteristic likely to be found in all good API desingers. (Although on occasion, a fresh perspective is good, not that this is mutually exclusive with domain experience, but sometimes good abstraction requires a distancing of ones self from the details of the domain.)

    To these points I would like to add consistency. To me, this is an equally important element of a good API and one I strive to instill in every one I design. Consistent handling in method/function/variable naming conventions as well as in argument names and argument ordering (something I think is key to perl since it lacks named parameters). An obsessive drive to organize (a place for everything and everything in its place) is also something I think most good API/library designers have, a disorganized API can kill other positive aspects very quickly.

    If you are instested in further study of API design, I would like to recommend this book. Reusable Software - The Base Object-Oriented Compontent Libraries by Bertrand Meyer, it is heavy on Eiffel (whose base libraries it is talking about), but really really insightful and smart. I believe that Eiffel actually begun not as a language, but as an attempt to "understand the fundemental structures and paradigms of software development". This book is basically about that process. It is one of the only books I have found that actually discusses the principles of good library construction in detail, including class names, proper use of inheritance, ideal class size, etc. etc. I actually keep it on my "within-arms-reach" shelf, next to the O'Reilly Perl catalog and my favorite Addison Wesley titles (Refactoring, Design Patterns, Pragmatic Programmers, etc).

Re: Ingy's "Swiss Army Light Sabre" - or, "how do you design your APIs?"
by dragonchild (Archbishop) on Mar 23, 2004 at 13:56 UTC
    To me, a good API is something that maps a thought-space onto a problem-space without significant overlap or underlap. It's a translation between the way people think about something and the way a computer needs to be told about it.

    As for how you come up with it? I don't think there's a formula for the next good API. I do think we can discuss characteristics of good APIs, though. In fact, this is exactly what the Perl6 language discussion is all about - a good API. And, from following those discussions, I think the most important thing an API designer can have is extensive experience in the problem-space. That and an obsessive-compulsive drive for simplicity. (Sorry, TimToady. *grins*)

    We are the carpenters and bricklayers of the Information Age.

    Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose

Re: Ingy's "Swiss Army Light Sabre" - or, "how do you design your APIs?"
by tilly (Archbishop) on Mar 24, 2004 at 03:57 UTC
    One unobvious connection between XP practices and good APIs.

    You quickly learn a lot about how much an API makes sense (or doesn't) when you try to use it. When you try to write a test suite for the API that you are developing, you are using it right there, and get immediate feedback on how that API is suboptimal. Before you have developed it, and when it is still cheap to change the API.

      One unobvious connection between XP practices and good APIs

      Another is the bottom-up-ness and simple design that comes out of You Ain't Going To Need It and Do The Simplest Thing That Could Possibly Work.

      You get an API that's generated from actual use-cases rather than one that was possibly over-designed to handle cases that are never needed - producing a smaller and simpler API.

Re: Ingy's "Swiss Army Light Sabre" - or, "how do you design your APIs?"
by jonadab (Parson) on Mar 23, 2004 at 21:50 UTC

    To be perfectly honest, my first thought when I saw IO::All is that it is a solution in search of a problem. Maybe I'm just missing something, but it seems rather pointless to me, a needless exercise in wheel-reinventing on a grandscale with no benefit. Maybe this is because I'm comfortable with the way Perl does IO already, for the most part, or maybe I just don't think the same way as the guy who did IO::All, but in any case I just don't see the value.

    Regarding API design, I agree with EvdB's comment, "Can I remember how to use this after a week and in a hurry without the docs?". My favourite example of a module that gets this pretty-much right, IMO, is DateTime, which I can almost use in my sleep. (There are some modules that are even easier to remember how to use, such as Data::Dumper, but what DateTime does is sufficiently complex that one might expect a more difficult API.

    ;$;=sub{$/};@;=map{my($a,$b)=($_,$;);$;=sub{$a.$b->()}} split//,".rekcah lreP rehtona tsuJ";$\=$;[-1]->();print
      wheel-reinventing on a grandscale

      I find it hard to call it wheel reinventing when he actually uses the wheels you're claiming he's reinventing. As I understand it, IO::All doesn't actually do the IO itself, it just hooks everything together with an, erm, interesting API. To me, it seems like something that might at least be worth playing with.

      Update: it appears my comment about hooking existing modules together was right. According to the very perldocs linked above, "The IO::All object is a proxy for IO::File, IO::Dir, IO::Socket, IO::String, Tie::File and File::ReadBackwards."

      Update: I have another comment, on the following:

      Maybe this is because I'm comfortable with the way Perl does IO already

      I don't see how being comfortable with something means it can't be improved. I'm not saying IO::All is necessarily an improvement on Perl's current IO, since I haven't used it, but I think this reason for discounting it might be bogus. Here's my obligatory, gratuitous analogy: I'm sure there are a lot of people who are comfortable slicing and dicing text with C, but that doesn't mean Perl's abundance of text manipulation functions are "a needless exercise in wheel-reinventing" or "a solution in search of a problem."

        but I think this reason for discounting it might be bogus.
        I don't think its bogus. It's a personal value judgement. If jonadb doesn't feel IO::All is an improvement, why would he switch (certainly would be a waste of his time)? As for your analogy, i'm sure to that C programmer Perl was "a needless exercise in wheel-reinventing" which is why that C programmer never learned perl. Doesn't stop us from enjoying :)(btw, I didn't think much of IO::All except that it's got bugs {see rt})

        MJD says "you can't just make shit up and expect the computer to know what you mean, retardo!"
        I run a Win32 PPM repository for perl 5.6.x and 5.8.x -- I take requests (README).
        ** The third rule of perl club is a statement of fact: pod is sexy.

Re: "how do you design your APIs?"
by bsb (Priest) on Mar 23, 2004 at 22:54 UTC
    Often you want two APIs, a comprehensive and flexible one, and an idiomatic DWIM wrapper Blah::Simple.

    One way to reality check your design decisions is to compare your API to another API from a different but comparable domain.

    Another test (that I need) is to return to your design after some months. "What was I thinking?!" is often the result when you lose the blinkers of your original problem.

    I don't like API designs that second guess you, nor those that strictly follow a non-Perlish design (usually found in library or system wrappers)

Re: Ingy's "Swiss Army Light Sabre" - or, "how do you design your APIs?"
by bart (Canon) on Mar 24, 2004 at 23:25 UTC
    My main objection to this module is that it's a catch-all for everything. If I do
    my $foo = io('some/thing');
    then, what is it? What do I expect it to be, a file, a directory? It can be either. It can even be something else entirely, I don't know.

    Yet, later on, if it's a directory, I can do $foo->next, if it's a file, I can do $foo->slurp, but not vice versa.

    This is the exact opposite of where strong typing wants to go: bugs aren't visible at compile time, it won't even complain at run time before you've already gone deep into the code: at the time of the next/slurp call.

    I don't like this. If you want a dir, say so. If you want a file, idem ditto. The module shouldn't have to guess. Because the chances are, it guesses wrong.

    No, I don't think one constructor io() for everything is a proper API.

    Ideally, I'd like an io::dir() call to produce a directory thingy, and a io::file() call for a file. These look nice, but, as those a fully qualified names, thus you can't subclass them. io_dir() or io_file() don't look even half as nice.

      Exactly my thoughts. Thanks for putting into words and a simple example what I couldn't seem to, obvious as it may be in retrospect.

      Makeshifts last the longest.

      What about IO::All, instead of being a facade, instead is a factory?

      We are the carpenters and bricklayers of the Information Age.

      Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose

      My main objection:

      my $foo = io('some/thing');

      Yes it would be better to trap such a mistake ASAP. Would this be OK?

      my $foo = io(-filename => 'some/thing');

      You could even add some kind of policy enforcement aspect that will trap all dubious instantiations on runtime.

      I am not worried about a developer calling $foo->slurp when the intention was to use $foo as a directory, for the same reason that I am not worried about it being used as a Person object. It is just a coincidence that file and directory features are in the same physical package.

      BTW: The Java class is also a file or a directory, and it implements some methods that are directory specific, like listFiles(). So it is a similar situation, despite the static typing. Of course that is a poor example, and anybody with a brain would use a util lib, e.g. the apache commons.

      It is leaky abstraction, but it looks like a pretty cool one to me! The freedom from having to check return values is a justification for this by itself.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://338941]
Approved by grinder
Front-paged by broquaint
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (5)
As of 2023-03-30 13:53 GMT
Find Nodes?
    Voting Booth?
    Which type of climate do you prefer to live in?

    Results (74 votes). Check out past polls.