This kind of dynamic has been going on since antiquity. Scripting will always have a niche, and 'enterprise-scale' development will too. Rarely it is based strictly on nuts-and-bolts practicality. Let's not forget: human beings are always subject to political pressure, the desire to feel 'knowledgable' and 'prestigious', and the desire to make things difficult for potential competitors through barriers to entry (turf preservation).

You see it in every realm of human activity, which includes programming and language design.

For people with a 'nuts-and-bolts' inclination, this "political" aspect can be a bit frustrating and bewildering. Nevertheless some PHBs realize that if they don't use Oraclefoo or Javawhiz for high-profile projects, there is a certain level of 'prestige loss' (and perhaps other issues) that cannot be quantified in mere nuts and bolts.

In some industry segments, this is not a major factor; but for universities, large corporations, utilities, financial institutions and other realms where prestige and reputation is the name of the game, you can just bet you are more likely to be Oracalized than MySQLated.

'm not suggesting that there aren't real applications that need the protections that layered code and team-oriented development using design patterns and Application Servers et al provide

Not that you can't do those applications in Ruby or Perl or Lisp or whatever too...

I'm more and more of the opinion that most language choice is made for cultural/social reasons rather than technical reasons.

That's very true. There are numerous modules on CPAN which support such efforts. Having started to learn the Java pantheon in order to understand my guys' production, I would say that the programming model espoused in J2EE development takes an extreme view, where _everything_ is completely defined through enforcement of edict. The goal is to prevent anything ever from being a loose end. A laudable goal, but a royal pain in the @ss in the way of actually getting anything done.

It does have a benefit for organizations which prefer to have programmers be lowest common denominator interchangeable parts, because it raises the bar of what one can turn in at the end of a shift to at least be something which the compiler will accept. This reassures managers -- who usually are LCD interchangeable parts themselves (to the org, anyway) -- that progress is being made. It doesn't in any way ensure that the _logic_ is complete or correct, but it does expose that level for discussion, which is a good thing.

On the political/cultural/social front, I heartily agree with both of you, adrianh and dimar. Indeed, one of my reasons for letting my guys proceed was the marketing value of that little cup of coffee.

I have to admit that I allowed them to go ahead without really researching it in depth. As I do so now, I see some advantages to the systematic approach espoused / enforced by J2EE, but also lots of disadvantages in the daily execution. I especially see that it's even more critical to understand the layering of the tools you use and their effects on execution. Simply put, there are lots more of them. Java is only one piece, and even in just the core language you have to be aware of the execution costs of things like passing around copies of objects as opposed to references, sorting of fancy collections, etc. Add to that key extensions like Hibernate, Jakarta, and JBoss, and you have an entire system that can be an order of magnitude more complex than Embperl/Apache/MySQL. Is it better? Well, it sure ain't faster, but, yes, it will be more scalable. Our first generation had lots of custom programming to support the PHP/MySQL main site for database and state replication, so that our customers would not be barfed like SalesForce.com is doing as we speak. Such protective mechanisms are much more cleanly created in the new system (although I see many J2EE sites that still don't bother; oh, well. Painful Learning Experiences coming up!) and we will be able to serve more customers faster in the future.

Did we need to do all this? In our case, I think the answer is 'yes and no'. Certainly the scripted version is working very well to serve our present customers. Certainly, the Java version is taking longer to deploy than updating the scripted version would have. Our particular case is a specialized SaaS app that has customers who are nearly all Fortune 500-caliber, and they do depend on us to be up and running 24x7. I think there are very few other applications which do demand this level of service, though, and especially for smaller companies looking to break in, the scripted web service model is entirely adequate and much less costly. Just as we have built our reputation on a scripted system, I believe it is the best way for other businesses to start out and build their way to success. It is even more appropriate for those businesses for which web services are not their core product.

Don Wilde
"There's more than one level to any answer."

Add to that key extensions like Hibernate, Jakarta, and JBoss, and you have an entire system that can be an order of magnitude more complex than Embperl/Apache/MySQL. Is it better? Well, it sure ain't faster, but, yes, it will be more scalable.

Well - I'd have to disagree with that :-) I've had more problems with scaling Java systems than I have with Perl ones. Not that this was due to any deficit in Java per se, just that J2EE led them to modularise the systems in a way that could only scale by buying bigger hardware, rather than more hardware.

It's just as simple to write a scalable and reliable Perl application (or Ruby, or Python, or Lisp, or whatever) as it is to write a Java one. Most of the expertise in producing something reliable and scalable sits in the developers head rather than in the language.

I agree with most of your points. I don't have enough experience with large management structures to say anything concrete about it, but it does sound plausible that most decisions in those environments aren't primarily based on technical considerations.

One thing that puzzles me though, is that you say that J2EE is a lot slower, yet more scalable than the perl/apache/mysql solution. I don't understand that at all. Surely the slowness implies that you need more boxes to reach the same level of service?

I find making mod_perl apps scalable a breeze: the standard Apache reverse-proxy setup combined with a replicated/clustered mysql setup means that:

1. I can have any number of frontend webservers
2. Those reverse-proxy to any number of mod_perl application servers
3. Those communicate with any number of database servers
That means there is no single point of failure, and increasing scalability simply means adding another box to one of the tiers. Since mod_perl apparently outperforms J2EE, that means you need less machines for the middle tier.

For me, that says a LAMP setup can be both cheaper to start with, quicker to develop, and easier to scale.

It's entirely possible to set up this 3-tier model on a single machine, and I generally do so for my clients if they anticipate growth. Moving it to a multi-machine setup is a piece of cake, and doesn't require specialised perl skills. In my opinion, this means a LAMP setup is both extremely effective for small or starting businesses, but equally (cost)effective for large businesses.

Unless I miss something blindingly obvious, the only thing in favor of J2EE would be cheaper developers; but you'd need more of them (which means you need more managers), and they have to work harder, with more complex tools, to reach the same level of service.

It seems like you consider Java not to be a scripting language. I don't understand why. As far as i can tell java meets all the requirements of a scripting language.

And it seems to me that there is no case at all to say that scripting is going to go away, ever. The fact is that scripting languages solve a whole bunch of problems of traditional software development and because of this they will never go away.

What criteria do i use to say something is a scripting language? The following:

1. Does not compile to native machine code but instead runs on a virtual machine of some sort.
2. Does not require (one might even say does not allow) the user to manage memory.

I don't understand why.

I understand why. Because it's verbose. Very verbose. If nothing else, this alone keeps it from being seen as a "scripting" language by nearly everybody. If you think, "This task is so simple it shouldn't take me more than 5 minutes to write a script to do it", you're not going to reach for Java. (Of course, there's always the hammer/nail syndrome; but those whose only hammer is Java probably don't believe in "scripting" anyway.)

Does not compile to native machine code but instead runs on a virtual machine of some sort.

I don't think so. There's little to stop Java or Perl compilers from targeting real machines; and a Java or Perl "real machine" could be constructed. In the real world :-), compilers for C targeting virtual machines have been around a long time. So this really can't be a discriminator.

Essentially, whether a language is a "scripting" language or not is really in the mind of the programmer. It's about ease of use; about how quickly one can go from problem to algorithm to solution. Whether the machine running that solution is real or virtual rarely if ever enters into that thought process.

perlfaq1: Is it a Perl program or a Perl script?

Most of the above is a regurgitation of a discussion that occurred in the chatterbox.

We're building the house of the future together.

See also John Ousterhout's comparison of scripting vs. system programming languages. It's somewhat misguided in calling Perl a scripting language and saying

Scripting languages aren't intended for writing applications from scratch; they are intended primarily for plugging together components.

In my mind, to be a script,
1. The source code must be ready-to-run. That is, you hand your source file directly to the interpreter and your program is running. Any language could meet this criterion with an "interpreter" that did the compile and run, but C and Java typically don't meet it.
2. The language cannot require blocks. A script should be able to run top to bottom without any further structure than lines.
I had thought there would be more requirements, but I think that's it. You can expect that a scripting language will have some form of eval, but that isn't a defining characteristic.

I also like this brief explanation of the difference.

---

Caution: Contents may have been coded under pressure.

good point, demerphq. Perhaps 'scripting' is not the appropriate delimiter. Let's try 'lightweight'?

Let me make one thing perfectly clear (jowls wobble a la RMN!), I'm all for scripting / lightweight web development in 95% of all cases.

Don Wilde
"There's more than one level to any answer."