in reply to Time to write a "serious" http server in Perl?

No.

Perl is an easy language to write a prototype in, but is a horrible language for writing a serious webserver. If you try to do it you'll be forced down one of a few basic approaches, and all are bad ideas in Perl:

  1. Simple single-threaded: This is what most of those *::Simple and *::Lite servers do. This approach can't serve two requests at once.
  2. Forking: A simple CRUD application will have to connect to the database on each request. When you're under volume, most databases will not survive. This approach is available in some of the pure Perl webservers.
  3. Pre-fork: This is how Apache used to work by default, and it can still be made to do so. (Smart mod_perl shops tend to use this approach combined with a reverse proxy.) The idea is that you start out with a number of children, and one of them serves the request. So connections get reused across requests. Unfortunately Perl processes use too much memory to have large numbers of them. This is the primary reason why serious mod_perl sites use a reverse proxy configuration. You just can't afford to tie up a ton of memory in a process whose job is to dribble bytes at whatever rate the client's dialup can accept it.
  4. Threading: My opinion is that Perl threading has all of the disadvantages of threading and none of the advantages. Others don't agree with me, but still even its advocates would not try to use it to scale to a busy website.
  5. Asynchronous programming: This is a promising approach until you look at how much of the infrastructure people use Perl for is not asynchronous. For example database access is done with DBI, which is synchronous. So your first long-running query takes your website down. Not good.
Those are your basic options. None of them would be a good choice to handle serious volume. Oh, you might be able to make them work, but at what cost in hardware? Why bother when Apache does the same thing with fewer resources?
  • Comment on Re: Time to write a "serious" http server in Perl?

Replies are listed 'Best First'.
Re^2: Time to write a "serious" http server in Perl?
by betterworld (Curate) on Aug 11, 2008 at 10:58 UTC
    Pre-fork: [...] Unfortunately Perl processes use too much memory to have large numbers of them.

    Could you explain why this is a problem? At least on the operating systems that I have worked with, forked processes continue to use their parent's memory pages until they need to write to them (aka "copy-on-write"). So if the parent (master) process is designed to load all the big modules and stuff, the children should produce little overhead in memory.

[reply]
[d/l]
      People do this. Unfortunately due to Perl's reference counting, in normal use you wind up writing to data that you're just accessing. So while you can preload lots of stuff, and it starts off shared, it doesn't stay shared.

      Again, this is not a theoretical problem. It is a real problem that people at high-volume mod_perl sites have been dealing with for years. And the solution is standard as well.

[reply]
Re^2: Time to write a "serious" http server in Perl?
by John M. Dlugosz (Monsignor) on Aug 11, 2008 at 05:02 UTC
    What is a reverse proxy?
[reply]
      Usually proxies handle outgoing requests from a network. A reverse proxy handles incoming requests to a network.

      In this context, a reverse proxy would read the request from a client, ask another Apache httpd instance for the data, then return that data to the client.

      This way, the mod_perl server can finish the request quickly and doesn't bog down a process (with a presumably large chunk of RAM) with downloading the response bytes to the client (on a slow connection).

[reply]

In Section Meditations