I think the essential issue is one of culture. Perl has descended from the "Unix Traditions", which come mostly from University culture. Even the choice of a representative way to present itself to others "Perl Monks," gives this away. Monks take a vow to abandon the outside world, and retreat to an inner world. In the monastery they work on interesting problems and avoid the world of Bologna, or to use a popular spelling "Baloney". In a twist of irony, the word "Bologna" is a popular college town in Italy. But I digress.

I am not criticizing the monastery, but these ideas are relevant to my exposition.

Some of the Unix Traditions (according to me): "Doing it right," "Minimalism", "Teaching/Learning/Culture embedded in University forms (professor-student, focus on research, etc.)", "Evolution and Competition", "Openness".

One of the things accepted in University is the idea that not everything is obvious, and the onus is on the pupil to persevere to attain knowledge. This is in direct contradiction to "market driven consumer culture," which tries to make things as oriented as possible toward the consumer so they will give us money. The web started out initially as a University exercise, and took a little while until it exploded into the all encompassing thing that it is today. At this point, consumer demand has overshadowed the original culture that brought it to life. Google is not so much famous for page-rank (a good idea, but is it really that good? matrix math? c'mon). Google is famous for creating a huge, consumer driven, money machine. Of course there are all the Googley things like Google maps, and things are free, and they have a "unique culture" (more marketing...think Xerox Parc, IBM, MIT, etc. etc.). The real thing they did better than anyone else is to capture the consumer demand. If you think about it, Google search is really just a box that people type consumer demands into.

I mention the web because Perl really took off with it's use in CGI. Before that it was just an interesting way to not have to use sed, awk, grep as the bits and pieces. It was the Unix Borg Unit, which gobbled up all the evolutionary DNA. It was the Pathetically Eclectic Rubbish Lister. Anyone who had the Unix history would immediately realize it's value. But then the web came along, and Perl gained huge popularity through it's use in the web. It harnessed, I would argue accidentally, the huge consumer demand that was now creating 100K jobs (a lot of money at the time) for anyone who could write a for-loop. Oh, you also had to have a nose ring, at least in San Francisco, that was mandatory: for loop (check), nose ring (check) OK you're hired. I have nothing against nose rings. In fact they are an essential part of street cred in various technical enclaves in SF.

The web technologies, despite being developed in the hallowed halls of CERN, were always designed to gain mass adoption. Look at how simple HTML is. It made a generation of people who could use tags call themselves "HTML coders". The audacity. Look at how simple HTTP is (I refer to HTTP 1.0, and not the far more complex 1.1), GET, POST, URLs. Totally brilliantly simple. So simple, that our nose-ringed-for-loop-equipped web master can use it to make 100K. Do I fault him? Please. This is why the web is important - because everyone can use it to enrich himself and others. The barrier to entry is very low. I would argue that the web is complicated, but many of the things that people interact with hide that complexity (and that is where the A.C. Clarke comment on magic comes from - the hiding of complexity, not the display).

We should not confuse "easy to learn" with "easy to use". Nor, should we believe that they are mutually exclusive. Legos, for example are easy to learn, but they also provide incredibly expressive power for those willing to risk the carpal strain. Making something easy to learn, does not imply you have to dumb it down. However, this is a strategy which works. You can often dumb something down and make it more popular. However, there are different ways to make things easy to learn. Training wheels don't ruin a bike. Well, not quite true, they do mar its appearance as one lists to one side...however, one eventually take thems off. In the same way Microsoft created the "Wizard", a thing that would parametrically generate a working application, which you would then fill in the blanks. This made C++ accessible for the newbies. It was the training wheels. Eventually people took the wheels off, and yes would laugh at the newbies listing to one side. But the underlying Visual C++ "bicycle" was pretty good technology for the day. At the end of the day they were successful in capturing the market because they focused on the training wheels that gave the average programmer, a good "out-of-the-box" experience.

The reason PHP has taken the spotlight away from Perl is that it was designed to make generating web pages really fast and easy. The creators concentrated on this. It was a goal. In the same way that Microsoft concentrated on the "legions of average programmers", PHP focused on providing this simple, out-of-the box experience. It can be argued that this simplicity "costs you down the line", but this is the sound of one hand clapping (i.e. nothing). Very few people are going to take the time to rewrite a system in a completely different language, just to learn a few things. Again, this is a University culture idea => research, the quest and attainment of knowledge.

Competing with PHP, if this is, in fact, something that is desirable, could be done by concentrating on the "out-of-the-box" perl for the web experience. Increasingly, people are consuming things in LAMP stacks. Also note the P used to be Perl but now it is increasingly PHP (there is the newly minted acronym LAMPP). Perl, mod_perl and all the relevant Web toolkits (I'm thinking Mason is the thing to push) should be in every single LAMP distribution.

Another out-of-the-box issue, from my experience: I recently got a new Mac laptop and could not use CPAN. The problem was that this was the new Intel architecture (64 bit). The solution was, after searching for a long time, to install xcode and set ARCHFLAGS. How the heck is someone supposed to know to search for ARCHFLAGS?.

The more interesting meta question is: why was it so hard for me to find the answer to this? It surely must affect thousands of people. I think the answer has to be "culture". I looked around and found all the little details, got it working and moved on. If the culture had the goal of widespread adoption, there would be a single website that had the steps to fix this, it would come up at the top of the search page, and would have been discovered quickly after the new macs came out. Alarm bells would have been ringing.

In closing, I think Perl+Mason _may be_ far superior to PHP. I'm going to give it a shot in my pursuit for knowledge. I certainly enjoy writing Perl far more than PHP. I'm currently "performing the experiment" to see what the objective differences are.

I'd argue that it's important for Perl to mount a comeback in the web space against PHP and things like Python. I'd also argue that the comeback strategy would require realizing that the goals of making things easy for newbies to adopt, and having a superior flexible technology are not contradictory.

I've got an analogy. Maybe it's twisted. Unix is to Ubuntu as Perl is to X. Solve for X. Perhaps X = Perl, mod_perl, Mason, tutorials, community outreach, LAMP(P) stack ubiquity, and a good out-of-the-box experience. Remember, the training wheels do eventually come off (and you get to laugh at people who list to one side as they ride).