Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Upgrading CPAN - Yes We Can

by jdrago_999 (Hermit)
on Jun 03, 2009 at 18:19 UTC ( [id://768105]=perlmeditation: print w/replies, xml ) Need Help??

Monks,

After using GitHub, Google Code and other online coding collaboration tools, CPAN (once a shining jewel in my view) is starting to feel a bit 1990's-ish.

There are some nice tools - bugtracking, annotations, dependency generation, reviews, discussion forums, etc. However, they are not truly integrated into a single interface and -- because of the distributed, mirrored architecture of search.cpan.org -- probably never will be.

Even though CPAN is 100 miles above and beyond what you will find for other languages, we cannot leave well enough alone. I say it's high time we -- as a community -- or someone -- as an individual -- create a replacement for search.cpan.org.

Surely we cannot wait until after Perl6 is ready to begin this. It must happen now.

A shortlist of functionality I would expect (at a minimum would include:

  • Listing of Modules/Distributions/Authors (like what we already have)
  • Module activity levels - like on GitHub
  • Author activity levels - like on GitHub
  • The ability to "follow" modules - so you will know when they are updated.
  • Discussion forums that:
    • Are faster.
    • More modern.
    • Include the features that thousands of other forums should Just Have.
  • Source control - either via SVN, Darcs, Git or what-have-you.
  • Forking/Branching/Merging - like on GitHub. Abandoned modules should not be allowed to languish forever. This process could replace the current "Take over a distribution" workflow in PAUSE.

It would also be great to have a kind of Twitter-like thing that you could watch to see things happen in real time. Downloads, test failures, reviews, ratings, forks, etc. We all know that there is a great deal of CPAN activity, but because it's spread out we don't get visibility into what is happening As It Happens.

Yes - I know that some of these features simply Would Not Work under the current architecture. A new architecture will be needed.

Ladies and Gentlemen, we can rebuild it. We have the technology. We have the capability to build something better than the current search.cpan.org.

Replies are listed 'Best First'.
Re: Upgrading CPAN - Yes We Can
by mirod (Canon) on Jun 03, 2009 at 18:50 UTC

    Improvements on CPAN are always welcome. You just have to remember its basic principle: make life easy for authors. In other words, the barrier to entry should be as low as it can be.

    So the basis of CPAN, the mirrored repository of plain tarballs, IMHO needs to stay the same. Some authors don't like RT, or web forums and don't even use source control. That's fine. Their contributions are no less important than those of "good citizens".

    That said, there is certainly room for improvement, and in fact in the last few years, a lot of work has been done _above_ the current CPAN: RT, CPAN Forum (is it missing features?), Anno CPAN,, CPAN Testers, CPANTS... and recently alias's Top 100(s), amongst others.

    Now it's true that all those sites do not have a common look and feel, but it looks like people are taking notice and doing something about it.

    What's missing there from your list are mostly GitHub specific features, but nothing prevents you, or any other CPAN author, to use it. It looks like they do actually, as a search on Perl shows 1592 repositories. And if you don't like GitHub (I don't), you can use an other repository (launchpad seems nice enough).

    What I want to say here, is that instead of planning of completely redoing the CPAN, maybe working incrementally on what you don't like (improve CPAN Forums maybe?) might be a more practical solution. You won't be alone in doing this.

Re: Upgrading CPAN - Yes We Can
by xdg (Monsignor) on Jun 03, 2009 at 20:02 UTC

    Are you talking about upgrading CPAN or search.cpan.org? They aren't the same thing.

    CPAN is the network of mirrored sites that contain distribution files, indices and so on. search.cpan.org is just one particular portal CPAN and the ecosystem of websites that people have built for bugs, forums, annotations, test reports, etc.

    So what part of the ecosystem do you think needs fixing?

    Of things you mentioned:

    * Module activity levels - like on GitHub * Author activity levels - like on GitHub

    Do you mean repository commit activity or release activity?

    If the former, some META.yml files in distributions are now listing repositories (these show up on search.cpan.org if they exist). Of course, some of those URLs are to actual repositories and others are to repository "dashboards". If you can figure out the difference, go for it.

    If the latter, there are already several "recent upload" feeds. Just start tracking one and create a site to report aggregate details.

    * The ability to "follow" modules - so you will know when they are upd +ated.

    Same answer as I just gave -- except now create the user registration and tracking preferences to go with it.

    * Discussion forums that: o Are faster o More modern o Include the features that thousands of other forums should Just Ha +ve.

    Do you realize that the forum software that drives http://cpanforum.com is the CPAN::Forum module on CPAN? There's even a page of notes for developers including the subversion repository. So if you think it needs features/fixes, I'm sure Gabor is open to discussing your patches.

    * Source control - either via SVN, Darcs, Git or what-have-you.

    This will never work, since CPAN has thousands of authors who may prefer different source code systems. And why should the Perl community rebuild what already exists on googlecode, sourceforge, github, etc.? That's not a good use of anyone's time.

    * Forking/Branching/Merging - like on GitHub. Abandoned modules should + not be allowed to languish forever. This process could replace the c +urrent "Take over a distribution" workflow in PAUSE.

    You're mixing several issues here. Fork/branch/merge is about evolution of a code tree. "Take over a distribution" workflow is about ownership -- more specifically, permission over changes to the index for a particular namespace. Nothing stops you from uploading your own Foo::Bar module if one already exists, but it won't be indexed for CPAN/CPANPLUS clients unless you have permissions for that namespace.

    A new site or a github-like "fork" button isn't going to change the permissions process. A willing author can give permissions in under a minute on PAUSE. And there is a process -- manual and subject to administrator oversight -- to do the same in the case where an original author can't be contacted. And I don't think we should ever take human judgment out of the loop in the latter case.

    So, ultimately, if you think "CPAN" can be upgraded, then start writing code -- no one is stopping you.

    -xdg

    Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re: Upgrading CPAN - Yes We Can
by JavaFan (Canon) on Jun 03, 2009 at 20:43 UTC
    After using GitHub, Google Code and other online coding collaboration tools, CPAN (once a shining jewel in my view) is starting to feel a bit 1990's-ish.
    CPAN is not an online coding collaboration tool. CPAN was, and will always be, a distributed distribution mechanism. Nothing more, nothing less.

    CPAN is also not a community effort. It was created by a few who 'just fucking did it'. IMO, CPAN was great, still is great, and will be great in the future. It's simplicity and unpretentiousness are the keys to its success. That allows the many mirrors to exists - the simplicity means virtually no (CPAN-specific) maintenance.

    Ladies and Gentlemen, we can rebuild it. We have the technology. We have the capability to build something better than the current search.cpan.org
    That reads as "I want you to do it".

    If you want something better than CPAN, start building. Words are cheap. In the open source world, you're judged by your actions, not your speeches.

      CPAN is not an online coding collaboration tool. CPAN was, and will always be, a distributed distribution mechanism. Nothing more, nothing less.

      I wouldn't say "always" there. Or at least "always" + "only" - I might have misunderstood the new whizbang features in Perl6's modules but this part right here:

      The syntax of a versioned module or class declaration has multiple parts in which the non-identifier parts are specified in adverbial pair notation without intervening spaces. Internally these are stored in a canonical string form which you should ignore. You may write the various parts in any order, except that the bare identifer must come first. The required parts for library insertion are the short name of the class/module, its version number, and a URI identifying the author (or authorizing authority, so we call it "auth" to be intentionally ambiguous).
      For example:

      class Dog:ver<1.2.1>:auth<cpan:JRANDOM>; class Dog:ver<1.2.1>:auth<http://www.some.com/~jrandom>; class Dog:ver<1.2.1>:auth<mailto:jrandom@some.com>;

      Since these are somewhat unweildy to look at, we allow a shorthand in which a bare subscripty adverb interprets its elements according to their form:

      class Dog:<1.2.1 cpan:JRANDOM>

      So instead of just

      use DBI;

      I could have

      use DBI:ver<4.2>:auth<http://www.devstack.com.com/cpan/DBI>;

      and get a completely different set of code than the "canonical" version (who knows how we decide on that).

      Just Perl6's package loader tells me that something significant is about to happen to CPAN - and someone (I'll do it if I can get the time to) will need to accommodate the new features Perl6 will provide. I figure we could extend those features to Perl5 by adding coderefs to @INC. Something like:

      use NiftyPerl6PackageNames; use DBI:ver<4.2>:auth<http://www.devstack.com.com/cpan/DBI>;

      ..............................

      CPAN is also not a community effort.

      If you say so...

      It was created by a few who 'just fucking did it'. IMO, CPAN was great, still is great, and will be great in the future.

      I have a strong feeling that a single, comprehensive archive of instantly-installable, free and open-source code will not only belong to Perl much longer. Other platforms could really benefit from something similar, and before long I'm sure CPAN will be viewed as a "predecessor to XYZ" (where XYZ is the Great New Thing for something else, like Ruby, .Net or Python).

      I also believe that if (for example) someone like Microsoft were to create a "CPAN" for .Net, it would have all the features I've mentioned above, and more. Ultimately it would make our beloved CPAN look like a toy from the 1990's.

      It's simplicity and unpretentiousness are the keys to its success. That allows the many mirrors to exists - the simplicity means virtually no (CPAN-specific) maintenance.

      Uh-huh. Simplicity is good. So is a lack of pretense.

      You and I and every programmer in the world knows that the first time you write a system, you know you could always do it better the second time around. The thing is that every system becomes a prototype for its replacement. Even Perl5 will one day succumb to Perl6. It is inevitable.

      Along similar lines, the CPAN and all of its satellites will one day either become or become eclipsed by another CPAN-esque service that is simply Way Better.

      Saying that what we have is Good Enough, or slinging the good old "Well then crank some code out, dude" just won't cut it. I have a few modules on CPAN and am always adding more. I do crank out code when I can, and when I see a need that's not yet fulfilled - like this one.

      Creating Yet Another CPAN Alternative is, while within my realm of skill, not what I would aim for. Instead, I would like to open the discussion for what the Perl community and user base will need for the next 20 years with Perl6.

      If that discussion has already taken place, I will probably it 2 minutes after typing this response - and read it. And perhaps see why I'm barking up the wrong tree. Or maybe I'll post a link to that discussion within this thread.

      Wish me luck ;)

      You know what pissed me off about cpan.. Is when the lil portraits of the author's popped up :-)

      I used to think the place looked so damn pro without any sillyness.. until.. that one thing came along..

      I had fantasies of starting a campaign to get authors to associate blank squares for the pics .. hahaha...

      Well.. at least I now know what Mark Stosberg looks like.

    A reply falls below the community's threshold of quality. You may see it by logging in.
Re: Upgrading CPAN - Yes We Can
by Anonymous Monk on Jun 03, 2009 at 18:59 UTC
    Problem with all these github/googlecode... want to become center of your universe -- one stop shop, their way. We already know people have preferences, so you end up with so much duplication ... they all want mindshare (and accounts).
Re: Upgrading CPAN - Yes We Can
by ambrus (Abbot) on Jun 04, 2009 at 15:15 UTC

    If you're collecting features you'd want, could you add alternate (disjunctive) module dependencies, where a module can depend on different combinations of other modules (and module versions)? The current dependency system is a bit restrictive, apt and other package managers have had disjunctions for ages. Also, maybe there could be recommended and suggested dependencies, not just required ones.

      could you add alternate (disjunctive) module dependencies, where a module can depend on different combinations of other modules (and module versions)?

      This is Perl. Of course we can :P

      Actually I'm sure that it's possible simply by extending the grammar or perhaps even with something like this:

      use any(qw( ModuleA ModuleB ModuleC ));

        No, I mean dependencies the CPAN installer handles, so that when a module is installed it installs all the dependencies as well. (There are two varieties of these: dependencies that are only needed for building and that need to be permanently installed, but that is irrelevant now.)

        The difficult part with these is that you don't want to make everyone update their CPAN package, so the only way this could go is that the info file of new modules can contain both the old style dependencies (which is just a plain conjunction of versioned modules) and new style ones. However, as ExtUtils::MakeMaker generates the info file from Makefile.PL, it could also write a sensible default for the old style if you give a dependency expression that has disjunctions in it.

Re: Upgrading CPAN - Yes We Can
by Anonymous Monk on Jun 03, 2009 at 18:51 UTC
Re: Upgrading CPAN - Yes We Can
by Anonymous Monk on Jun 03, 2009 at 19:41 UTC
    I use the RSS feed to follow modules.
Re: Upgrading CPAN - Yes We Can
by stonecolddevin (Parson) on Jun 06, 2009 at 07:32 UTC

    ltjake is working on this. Might do you well to check it out.

    meh.
Re: Upgrading CPAN - Yes We Can
by Plankton (Vicar) on Jun 20, 2009 at 05:51 UTC
    Why does it have to be either or (CPAN vs New and Improved CPAN)? If you got something that is better than CPAN, just go ahead and do it. If its better then it will survive if not this is as far as it will go.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://768105]
Approved by zwon
Front-paged by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others meditating upon the Monastery: (9)
As of 2024-04-19 07:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found