zby has asked for the wisdom of the Perl Monks concerning the following question:

The initial plan from Larry Wall was:
Perl 5 was my rewrite of Perl. I want Perl 6 to be the community's rewrite of Perl and of the community.
How does that stand against the reality and 8 years of development? I think I need to add that I don't want to start a flamewar here - but I am just curious. It seems like a natural evolution from the simple structure of Larry authoritarian rule to more complex and (at least partially) democratic organisation. It clearly was much more difficult than anyone expected - and thus it can be an intersting datapoint for Open Source theory (to contrast it with: Open Source Projects Manage Themselves? Dream On.).
  • Comment on How much is Perl6 the community rewrite of Perl?

Replies are listed 'Best First'.
Re: How much is Perl6 the community rewrite of Perl?
by rurban (Scribe) on Jan 01, 2009 at 15:58 UTC
    Well perl6 development is organized differently. I personally wouldn't call it a "community's rewrite".

    p5p is also a community as parrot-dev is, but parrot/perl6 hackers more are sponsored than p5p,
    and the process is more "business orientated", than in p5p.
    At least it's faster and more practical in perl6 land.

    git e.g. is decentral and more community oriented than svn.
    Monthly parrot/perl6 releases is more practical than the chaotic p5p release process. chromatic recently summarized the differences on p5p.

    But we have completely different projects here.
    perl5 dev. is mostly bugfixing and discussion,
    perl6 is only about adding the missing features.
    perl6 has a tight and organized roadmap, with stable and bound ressources,
    the perl5 roadmap is "flowing", as the ressources come and go.

        Yes I was more thinking about the design - but actually you cannot entirely separate that from the development, there is no clear cut.
Re: How much is Perl6 the community rewrite of Perl?
by particle (Vicar) on Jan 02, 2009 at 17:06 UTC

    over my years of development on parrot, pugs, and rakudo, i've become much less active in the perl 5 community. for example, i used to frequent perlmonks, but time no longer allows it. instead, i'm now quite active in the perl 6 community. historically, i'm not part of @Larry (the nickname for the perl 6 design team), and although i've been invited to attend the weekly design meeting, it was as a parrot and rakudo developer, not as a designer.

    while developing rakudo, i felt the compiler was growing to a point where it'd be useful for folks to do day-to-day programming tasks, and have a little fun with the language revision we've been waiting so long to get. so, i started adding command-line options like -e, -h, -c and -v. rakudo became much friendlier to use, since i could check the syntax of a perl 6 script i wrote, and find what revision of rakudo was failing to parse or run a particular script so i could submit a proper bug report.

    then two things happened: it bothered me that there was no spec for the command-line so i couldn't really implement anything more than what i'd done already, and ian hague donated a pile of money to tpf for perl 6 development. i decided to apply for a grant to define and implement the perl 6 command-line syntax, and my grant was accepted. now, i'm in the process of designing a working draft of Synopsis 19, describing the command-line syntax, and soon i'll be writing tests and implementing it in rakudo. all this is being done in a public repository (pugs) with a liberal commit policy, and is being discussed on open mailing-lists and irc channels. i've gotten quite good feedback, a few nice ideas, some excellent questions, and some corrections this way. i wouldn't succeed at this task if not for the support of these helpful folks, too numerous to mention.

    i became part of the perl 5 community the first time i wandered online for help (probably sometime late in 1996). i've never submitted a patch to perl5-porters, so although i'm quite adept in using perl 5, i don't come from the perl 5 illuminati. changing my focus to perl 6 development has given me a great education at little cost (just spare time), and i'm now able to give back by designing a crucial feature of perl 6 that countless people will use every day for years to come. if my story isn't a shining example of a "community rewrite" of perl 6, i don't know what is.

    ~Particle *accelerates*

      i'm in the process of designing a working draft of Synopsis 19, describing the command-line syntax,

      I don't suppose there's any chance that you'll add a perl5 -s style option for the simple passthrough of simple command line options to the script is there?


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
Re: How much is Perl6 the community rewrite of Perl?
by zentara (Cardinal) on Jan 01, 2009 at 16:36 UTC
    I'm probably one of the least qualified monks to talk about Perl6 development, but I would say the following: If a dictatorial(junta) run development process has taken 8+ years, I would shudder to think how long a "community driven rewrite" would have taken. The most likely scenario would have been 5 years of planning, and 3 years of arguing, resulting in people quitting the project, and leaving it dead.

    A good wise king is still the most effective government...but to find one, we have to wait for the Second Coming of Jesus. (Has anyone seen Larry Wall lately ? Is he sporting a halo? ) :-)


    I'm not really a human, but I play one on earth Remember How Lucky You Are
      A good wise king is still the most effective government...but to find one, we have to wait for the Second Coming of Jesus.

      Democracy or autocracy, this is a question. But history proved a committe seems be better than a wise king. ;)



      I am trying to improve my English skills, if you see a mistake please feel free to reply or /msg me a correction

        But history proved a committe seems be better than a wise king. ;)

        And what history might that be? What I mean is, although everyone extols how much democracy is better than king, when a big problem arises, the leader almost always assumes "temporary dictatorial control", to tackle the problem. Then there is also the problem of defining what "wise" is. A king may not be wise, and that may be where the committee is needed ....to oust the bad king. But then again...there is the dreaded "tryanny of the majority" that Alexis deTocqueville wrote about, as the worst aspect of democracy. Just ask any black in America about democratic rule in the early 20th century..... it's called class repression.


        I'm not really a human, but I play one on earth Remember How Lucky You Are
Re: How much is Perl6 the community rewrite of Perl?
by JavaFan (Canon) on Jan 01, 2009 at 16:32 UTC
    <joke type = 'half'>
    8 years of development?
    How much more prove of 'community rewrite' do you need? Without "the community" (there wasn't even a large one even if he had wanted to include them) it took Larry less than 8 years to write perl 1, 2, 3, 4 and 5.
    </joke>
      Without "the community" ... it took Larry less than 8 years to write perl 1, 2, 3, 4 and 5.

      How long should it take to discover the need for, invent, and prototype a hybrid top-down, bottom-up parser which features lexically scoped modifications and runs efficiently on multiple implementations?

      If you can answer that question, you've solved the software project management problem.

      This is calling for a much longer discussion - but for now a short answer - perhaps it was worth it. In our culture there is the worship of the lone inventor - but in fact this is a myth - all of the great inventions required assembling of ideas from many people (see The Myths of Innovation). So maybe it was worth it to break out from the single God King way of development? Of course there is also the point that if Larry did not so ostensibly step off from the reign - then would that be be really different? After all Larry was never shy to admit that he borrows ideas from other people. And now even after that declaration - isn't Larry still the main force behind the development? Actually - this is a part of my original question.
Re: How much is Perl6 the community rewrite of Perl?
by elmex (Friar) on Jan 02, 2009 at 09:08 UTC

    Well, I think, "rewriting" the language to be completely different from Perl 6 and also wanting to "change" the community, sounds like a very unrealistic dream.

    I followed the development of Perl 6 as a mostly inactive lurker since 2002 (edit: A lurker who has been subscribed to perl6-all mailing list, reading regularly and even discussing back then).

    It didn't look to me like Perl 5 was that much dictated by Larry, at least not in the last 6-8 years. But it's indeed true that the process of "Perl 6" was a quite democratic one, starting with the RFCs it was based on direct community input from the start.

    Perl 5 evolved over years of development, being iteratively improved from version to version, always having a working implementation which actually did stuff in a production environment. That formed a community of people who mostly itch their own scratch, which is a very good motivational factor to invest time into the development of Perl 5.

    Perl 6 didn't evolve, they threw away most of Perl 5, maybe except the sigil syntax, also things were promised that never were realized or could actually be realized, which formed a community of dreamers, following the dream of the ultimate programming language. And currently it looks like those dreamers weren't able to deliver _anything_ at least remotely stable, except a very fuzzy specification and a test suite.

    Status is, as far as I followed it the last years and recently had a rough look at the parrot code: parrot got some basic subsystems that qualify it as an experimental VM: A JIT, a very simplistic GC (remember: the incremental and generation collectors, which actually float around in it's code, don't work) and a lot of intermediate languages leveled on top of each other.

    "Perl 6" is a language specification which at least iterated _once_ over the 8 years, through the development of the now abandoned PUGS. But it still got no stable release yet, after 8 years. At least the work is now flowing into something thats probably going to be the final Perl 6 compiler, called, I think, Rakudo.

    I maybe sound a bit cynic, I originally was very fond of the idea of a _dynamic_ VM and a new version of Perl 5, which got some more syntactic sugar and concepts. But I felt stalled, with Perl 6 being developed and hyped all over the internet. Coding Perl 5 felt like coding in an obsolete language. Also others who I asked felt the same. Promises of the money founded Perl 6 community weren't and couldn't be delivered after 2, 3, 4, 5, 6, 7 and finally 8 years of money founded development.

    A year (or maybe 2, I guess everyone had a different "timeout") ago many finally realized that there won't be a replacement of Perl 5 anytime soon, and the Perl 5 community moved on, leaving "Perl 6" behind, and actually improved the language they all loved and knew, which even works in production environments.

    Someone of the Perl 6 community is probably coming soon and will jump all over my statements. So remember that this is just one view, and mostly from the outside, of the state of affairs.

      Perl 6 didn't evolve, they threw away most of Perl 5....

      Untrue. The apocalypses have always been deltas against Perl 5.

      Promises of the money founded Perl 6 community weren't and couldn't be delivered after 2, 3, 4, 5, 6, 7 and finally 8 years of money founded development.

      How much money? Do you know? (I'll tell you this -- all of the money that's gone into Parrot, Pugs, Perl 6, Rakudo, SMOP, Elf, and whatever else in the past eight years couldn't buy you one year of the time spent on the JVM or the CLR. You might be able to pay for the coffee breaks of all of the developers of either system.)

      So remember that this is just one view, and mostly from the outside, of the state of affairs.

      That amuses me. It's not as if we're some secret cabal who meets in shadowy, dark rooms to cackle and drink brandy and smoke big cigars and plot the future. There are eight freakin' years of email you could dig through to find out what's happened. I've published design minutes from weekly Perl 6 meetings for years. There's no super-secret initiation ceremony into the hidden mysteries of Perl 6. All you have to do to become part of the secret shadowy cabal of volunteer community members developing Perl 6 is ask a question on IRC or a mailing list, submit a patch, write a test, report test results, point out a typo in the documentation, or a dozen other very small tiny things. I'm not sure how we could make that easier, other than kidnapping you and tying your hands to the keyboard.

      I just don't get why it's easier to speculate about things from a perspective you admit is flawed, especially when it's so simple to get a better perspective.

        I just don't get why it's easier to speculate about things

        As you say, there are "eight freakin' years of email" to "dig through." It's so much easier to speculate!

        I've semi-casually tried to get an idea of "what's going on with Perl 6" a few times. Perhaps my attempts have been unreasonably casual, but they've not been very satisfying. Getting a good perspective is simple the same way reading the Camel from cover to cover is simple, which is to say it's easier to understand how to do it than it is to take the time to do it.

        1. I said I followed the development as a lurker. As a lurker subscribed to perl6-all mailing list, reading mostly regularly. I know pretty well what happened the last 6 years.

        2. I also didn't say or admit that my perspective is flawed. I only said what my perspective is.

        3. Why should I spent one minute on coding for Perl 6 when I can actually do new stuff and solve my problems with an already good language which I don't have to bugfix (Perl 5).

      The question if the rewrite is an interesting subject, given that it is still not finished and there are reasonable doubts if it will ever be finished, is another discussion. Personally I became quite confident in the Perl6 development after the publishing of The Parrot Roadmap. I am waiting for one for Rakudo now.

        Ok, then more close to the original subject: Originally it was a "community rewrite", but recently Larry got more and more involved with actual "Perl 6" development, but is still not the driving force. _If_ "Perl 6" is ever finished the current way it is developed, I would call it a rewrite, which was driven by a "community".

        I would say it's not "the community rewrite" but "a community rewrite".

        The question is also which community we are talking about. It certainly is not the Perl 5 community (largely consisting of people active on p5p and all the authors who upload their modules to CPAN). Of course there are some people who are involved in both communities, like for instance Nicolas Clark or chromatic.

        As far as I perceived it: The bulk of the people who originally formed the "Perl 6" community, writing and discussing the RFCs, attracted people who were interested in experimenting with programming languages. After some years of breeding out new ideas and no real progress on the parrot/perl6 implementation front the bulk of Perl 5 people left mostly.

        So we got the community of active developers who want to get things done in realtime on the Perl 5 side, and another community of language experimentalists on the Perl 6 side.

      I too have been eagerly waiting for Perl6 for a long time and now it has receded to be just a vague dream. I like the idea of Perl6 a lot and I hope that it will see the light of day. Preferably sooner than later, but only in a form that is usable, like Perl5.

      --
      seek $her, $from, $everywhere if exists $true{love};
Re: How much is Perl6 the community rewrite of Perl?
by moritz (Cardinal) on Jan 04, 2009 at 16:50 UTC
    How does that stand against the reality

    The reality taught Larry and us that designing a language isn't a task for which a community is really suited all that well. Everybody wants to get their pet project in, but that way you don't get a uniform, smooth and coherent language.

    So in the end it was decided that Larry designed most of the core language, which many ideas from the community, and some specs written by other people (for example S05), and unified under Larry's maintainership.

    As the various implementations proceed, more people hack on the Synospis again, because he just doesn't have enough time to care about every detail, so others chime in, collect thoughts and publish drafts. (For example Daniel Ruoso has been working on the draft of S07, Laziness and Iterators, and particle on S19, Command Line Interface).

    Still the comment about the community rewrite isn't all wrong, because all implementations are community projects.

    Update: These days I'm collecting thoughts for a draft of S24, on the topic of a built-in testing system. So if I'm considering myself "community", then I'm another example for community-driven design. What drives me is that in my work with the test suite I noted that different things are assumed about the testing functions in different places, so there should a be a place where such things are clarified. And since nobody else does it, I do. But I'm digressing a bit here... follow p6l in the next few weeks if you're curious.

Re: How much is Perl6 the community rewrite of Perl?
by mr_mischief (Monsignor) on Jan 05, 2009 at 15:55 UTC
    Don't imagine that because Larry plays a big part in the community that it isn't a community effort. There is no hard delineation between one man's work and a community's work unless that one man does it all on his own.

    There are parts of language design that don't work very well as a pure democracy. Some things are done the best by a single mind. Some are done best by a small, tightly-bound committee. Some are suitable and even helped by wide and varied points of view. Those groups of tasks are best done by the part of the community to which they are best suited.

    Things like RFCs, testing, asking good questions, making suggestions about possible oversights or killer features, and generally providing multiple points of view are great tasks for a community at large. Implementing the well-specified designs and setting up the standard test suites are good jobs for more dedicated community members.

    Things like hashing out the details of the design and implementing the core in an internally consistent way are good jobs for a smaller group within the community. Providing a coherent overall design with the flavor of Perl is a good job for Larry, although he was initially hoping some other people could pull it off.

    I've never yet seen a community project in which everyone put in perfectly equal effort and won perfectly equal results for the group with those efforts. Some people are more suited to certain work. Some people put in more effort. Some people are just in a better situation at the right moments to make a significant contribution. This is true in raising funds for a charity, selling cars in a dealership, or developing software. Some people will work harder. Some people will do more. Those two are strongly correlated, but one is not always a proportionate reflection of the other.

    To present Larry's work and the community's work as some sort of dichotomy I think misses the point. Larry is himself a part of the community, albeit a very central and important part. He even takes the time when he has a chance to talk to people about Perl 6 (and many other topics) who've had little or nothing to do with its development.

    Larry could have written something from scratch by himself in his little free time and called it Perl 6. He has more than earned the right to call his language designs Perl, after all. Yet he has a group of people working closely with him in public view. There is a larger group working a little less closely, and an even larger group working around the fringes. I've never seen a functional community over the size of about five people in which every member contributed to every decision and bit of work. Why should Perl be any different?

      Hmm - OK. Maybe a better question would be - is there something new to learn about scaling Open Source projects from the Perl6 development effort? I understand that you say - that not much - it is just like any other Open Source project.
        Perhaps the most important lesson is the one that's mentioned several times in the thread already. Some tasks scale better to larger groups of people than others.

        As programmers, this should be somewhat intuitive. Certain tasks scale better to multiple threads or multiple processes than others. Loosely coupled problems are easier to do in parallel. Problems that are tightly coupled due to interdependency often require so much communication and coordination overhead as to be infeasible or inadvisable in broadly scaled implementations.

        Everything that is difficult about interdependency, communication, and coordination among some otherwise simple subroutines is compounded when you replace those bits with people. Coordination and communication between people is not so simple and constrained, and therefore the costs rise even faster.

        If you really want to be part of the core Perl 6 team, then all you have to do is earn your keep among them. This is much like what Eric S. Raymond calls "Meritocracy" -- everybody is given an equal chance democratically, but their contribution toward the goal is what gives them their final influence. Your share of the action is what you make of it.

        In fact, I'd say that if things were not this way, nothing would get done at all. Why hold back the progress being made by those willing and able to do the work waiting for input from people not willing and able to provide it?

Re: How much is Perl6 the community rewrite of Perl?
by xiaoyafeng (Deacon) on Jan 02, 2009 at 10:16 UTC

    I bet this post will attract many perl6 hackers.

    Moritz, Where are you?! :-)



    I am trying to improve my English skills, if you see a mistake please feel free to reply or /msg me a correction

      Hollidays, with very little time with internet access ;-).

      Still I wrote reply below, albeit a bit late.