in reply to Reinventing the spaceship

I don't think you should feel the need to apologise. I've just read the thread 3 times and see nothing much for you to apologise for:)

From my perspective, CPAN is a great idea, and much of what is on CPAN is great too, but some of it is not-so-great, and some is not good at all.

If there is one single critisism that I would level at CPAN--and I know I will take a hit for saying this out loud--it is a lack of coordinated design.

This manifests itself in several ways:

Having nailed my colors to the mast on things CPAN related, it is tempting to try and propose solutions to the critisisms I have outlined. I will refrain from doing so as I am aware that my relative newness to the Perl community, and my total lack of contributions to CPAN will make any proposals I do have--which are many--come from a foundless base.

What I would like to do is see if there is anyway that I can contribute to a resolution to the issues I have identified and the first step is to enquire if there is sufficient support for starting a movement to address those issues.

Do enough people within the community consider some or all of the above to be problems?

Is there any support for moves to 'fix' those problems?

Is this a suitable forum for raising the issues and coordinating efforts to do so?

Often the biggest issues in attempting to resolve issues of this type are

  1. Finding enough people with enough time to contribute to persuing the tasks.
  2. Coordinating the efforts in a way that allows wide participation.
  3. Embuing the efforts with sufficient authority to make the exercise worthwhile.

Personally, I can find time to contribute to the process, and I have had some experience that might contribute to the coordination and development of some solutions. The area where I can not contribute is that of authority. My lack of standing withing the community means that I could never lead such moves, but if there is any will to address the issues, I would be only to willing to contribute in any and every way I can.


Examine what is said, not who speaks.
1) When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
2) The only way of discovering the limits of the possible is to venture a little way past them into the impossible
3) Any sufficiently advanced technology is indistinguishable from magic.
Arthur C. Clarke.

Replies are listed 'Best First'.
Re^2: Reinventing the spaceship (no technical solutions)
by Aristotle (Chancellor) on Apr 09, 2003 at 11:14 UTC
    and I know I will take a hit for saying this out loud
    Why do you feel the need to mention that? Being a rebel doesn't gain you anything, being coherent and cogent will. I briefly considered refusing my upvote you otherwise earned, due to this remark.
    This segregation would have several benefits:
    I don't see these anywhere near as clear-cut. It would certainly not be easier to install a module tree than a single-file module. Maintenance could be easier or it could become harder. Documentation can certainly suffer; I've seen a lot of badly written modularized documentation where you constantly have to crossreference two or three other PODs (and look for the bits in question in those as well, of course) just to find out simple facts. I've also seen huge, very well-written monolithic PODs, though rarely.

    .. the glue code required spends time undoing stuff that one module did, in order to supply the intermediate data to the second module which promptly redoes it.

    ...

    There are often inconsistancies within the interface of a single module.

    ...

    .. duplicated functionality as small parts of several modules.

    Interface design is what you're complaining about here. Few people are good at interface design. Most interfaces out there, the overwhelming majority of them in fact, suck. Badly.

    I say this not as a master of interface design, but as someone who is aware of his own inadequacy. I usually go through at least three iterations of responsibility distribution and interface redesign before I am even close to satisfied with my own code. And I rarely think my results are more than just about usable.

    I think most people don't even go into that much effort.

    It's like the difference between a top novel writer's books and a pupil's homework essay.

    Many modules suffer from bad documentation.

    Definitely. Most of them draw a rough and often incomplete sketch of the interface and leave it at that. Rarely do you see any discussion of how the parts fit together, and the joyous occasions there's actually useful, working and helpful example code to look are very few and far between. Ovid's HTML::TokeParser::Simple is one example of well done POD - although I do think the sheer volume of the examples makes the documentation harder to navigate, I've never had any problem looking anything up and finding it quickly. Of course, the self-explanatory interface plays a large part in this too.

    All of these boil down to interface design on different levels (documentation is just another kind of abstraction), and aren't really CPAN specific at all. By and large, these problems tend to be just as bad outside the Perl world. We're just in the lucky position to have a well run centralized resource with lots of decent code in it whose concentration makes these issue that much more striking.

    Yes, I agree. CPAN is full of half thought out (to put it mildly) stuff. However, I don't think there's any single and certainly no technical solution to these problems. They are not trivial to solve at all. I'm still glad the CPAN exists. The best we can do about it, I believe, is to be aware of the problems, and make an effort to address them in our own work. And of course, to provide feedback - if you have some gripe with a module, go over the source, meditate about it, and produce a suggestion or even a patch (but don't forget to explain your reasoning). Free software thrives on its users' contributions.

    Makeshifts last the longest.

Re: Re: Reinventing the spaceship
by revdiablo (Prior) on Apr 09, 2003 at 07:52 UTC

    Wow. This is an extremely long Note, but I'd like to make a probably-too-general reply, if I may. I agree with many of the problems of CPAN you pointed out, though I'm not entirely sure your major premise (that CPAN needs coordinated, overarching design) is necessarily inferred by those problems. Perhaps some gentle nudging by well-respected members of the Perl community would suffice. Or perhaps all it needs is a small amount of hacking on the Perl community itself. I'm not convinced that the anarchy-like Open Source method of development (which, to me, CPAN is a perfect example of, and ESR likes to call a Bazaar) doesn't have an answer to the problems you cited. In any case, I doubt there's going to be any strong policy put in place on CPAN -- that could be considered overarching design -- anyway. :)

Re: Re: Reinventing the spaceship
by perrin (Chancellor) on Apr 09, 2003 at 21:25 UTC
    One effort to deal with this was the P5EE project. Some people wanted to do it by defining a set of APIs that are "P5EE" and then implement those, by writing new modules and retrofitting existing ones. This is sort of how J2EE works, with a central authority defining the APIs.

    I prefer an alternative approach, which is to identify a set of high-quality modules which provide a complete toolkit for building most applications. It's very hard to do this in an "official" capacity, because people do have ego tied up in their CPAN modules and choosing one over another can get tricky. It would be easier as a sort of "branded" Perl distribution, although I don't know if P5EE is really the best name for it.

      I had never heard of P5EE, but after a quick google I see that it started out with similar ideas and intentions to those that have been building in my mind for a while. I also see from a few of the snippets I read that the original intent of "one really great way" fairly rapidly bacame a "collection of posssibly great ways" with the hope that one would become a clear leader.

      I also saw that several groups that had considerable investment in one particular module or set of modules that were invited to participate by bringing their modules in line with the philosophies of P5EE pretty much rejected the notion outright.

      Basically, I'd summarise what I have been reading as several groups decided to form teams which diluted the talents and efforts, and others simply didn't want to play, which is a shame. The biggest problem--other than the aweful name:)--would seem to be the blue-sky starting point of the project. A large number of well-thought-through and desirable subprojects and goals, but probably too much and too wide-spread to be successfully tackled as a voluntary, cooperative effort.

      I wouldn't mind betting that some considerable efforts have already been done to define and integrate a rationalised subset of the total that is CPAN by various large-scale users of perl already. If only it were possible to obtain the high-level criteria from a few of the more successful of these, it probably wouldn't be too hard to find common threads and strategies within them that could be used as the basis of a rationalised, integrated subset of CPAN that was based upon proven, real requirements, rather than idealised, blue-sky desires.

      Thanks for the pointer to P5EE. I will spend a bit more time reading the google-droppings and see if the project is still active and if there is somewhere there I could expend some of my excess time and energies.

      It seems fairly clear from the relative dearth of response here that I am barking up the wrong tree (again).

      Or maybe I am simply the wrong person to be raising the matter?


      Examine what is said, not who speaks.
      1) When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
      2) The only way of discovering the limits of the possible is to venture a little way past them into the impossible
      3) Any sufficiently advanced technology is indistinguishable from magic.
      Arthur C. Clarke.
        It's just a very difficult thing to do, because you are talking about imposing a lot of structure and some management on people who basically gave us all some free code out of the goodness of their hearts. I remember Michael Schwern made an attempt at this too, with his "Kwalitee" and "CPANTS" ideas. I think he gave up on it, but I don't know the whole story. You might want to ask him.
Re: Re: Reinventing the spaceship
by zby (Vicar) on Apr 09, 2003 at 12:08 UTC
    The 'suitable forum' might be the Reviews section on PM.