in reply to Re: Time to write a "serious" http server in Perl?
in thread Time to write a "serious" http server in Perl?

First, writing a serious web server is not trivial in any language. It's a complex network server that requires intense attention to security (witness the number of IIS exploits). Not something to be taken lightly.
Absolutely - however, it would appear that a large number of those exploits are the result of IIS being too tightly-coupled with the Windows OS, and written in C/C++.

Were IIS written in managed C#, at least some of those problems may not have been introduced. Of course, you could argue that "fixing" such problems with .Net just gives you another problem :)
Second, I think you're underestimating the power and flexibility of Apache2. Let's look at some of your shopping list:

...lots of ++excellent++ points...
Having used Apache httpd since 1999 I have learned the ropes (enough to get my job done). However, I have always felt that Apache httpd configuration and deployment has always been something of an esoteric pile of "this is just the way it's done" dogma.

Why does one need to understand reverse proxies, ports, networking, system resource balancing, etc, etc, etc - just to set up 2 VirtualHosts running completely independent of one another, so that you could stop/start one and not the other? If I don't have to manage memory and garbage-collection within Perl, why do I have to be a network- and system-administration- wizard to set up something fairly trivial like that in Apache httpd? It's not as if running more than one VirtualHost on a single machine is an uncommon thing that only expert-level users would do.

Running a proxy in front of mod_perl has long been the recommended configuration.
And yet I have never seen an instance of Apache httpd installed (or set up by default) with mod_perl behind a proxy. Not since Redhat 5, not on Ubuntu 8.04, not on Fedora 9, CentOS or RHEL 5. Not with mod_perl 1.3x or with mod_perl 2.x. Not ever, not even once.

Apache httpd has been around for a long time, and will probably be around a another long time or two. If it works for you, then keep doing it. It doesn't work for me anymore, especially with Microsoft's recent donations (and the implied (or my inferred)) influence over what happens with all of the Apache Software Foundation's projects. For all I know the money could be Microsoft's way of saying "Sit. Stay. Good boy."

Back to my original question - Are Perl's networking libraries are capable of serving millions of requests per day, reliably and without leaking memory or causing other problems? Is it possible? Is it advisable?

Thanks for sharing your well-seasoned opinion!
  • Comment on Re^2: Time to write a "serious" http server in Perl?

Replies are listed 'Best First'.
Re^3: Time to write a "serious" http server in Perl?
by perrin (Chancellor) on Aug 10, 2008 at 20:08 UTC
    Were IIS written in managed C#, at least some of those problems may not have been introduced.

    True, but the close integration to the OS is the only way they were able to achieve the performance numbers they have and add features like session storage. If you try to write an IIS application in a language that Microsoft doesn't support, I doubt you'll be able to take advantage of features like that. They made a trade-off.

    Why does one need to understand reverse proxies, ports, networking, system resource balancing, etc, etc, etc - just to set up 2 VirtualHosts running completely independent of one another, so that you could stop/start one and not the other?

    It's another trade-off. Rather than limit you to doing what the original developers thought of, Apache provides a very flexible solution which can be used in a variety of ways. That requires users to know something about the problem domain. Hiding those details would limit what you can do with the server.

    And yet I have never seen an instance of Apache httpd installed (or set up by default) with mod_perl behind a proxy. Not since Redhat 5, not on Ubuntu 8.04, not on Fedora 9, CentOS or RHEL 5. Not with mod_perl 1.3x or with mod_perl 2.x. Not ever, not even once.

    You're talking about default packages shipped by OS vendors? No one running a serious site on mod_perl would use those. I've never seen a mod_perl site getting real traffic that runs without a proxy or uses vendor binaries.

    It doesn't work for me anymore, especially with Microsoft's recent donations (and the implied (or my inferred)) influence over what happens with all of the Apache Software Foundation's projects. For all I know the money could be Microsoft's way of saying "Sit. Stay. Good boy."

    Ok, this is just FUD and it's offensive. Although I don't work on the httpd project, I am a member of the ASF (through work on mod_perl) and I can assure you that I don't have any plans to change my behavior due to Microsoft's contribution.

    If your problem with Apache2 is that you don't like the way you configure it, try another server. There are literally hundreds of open source web servers out there, including some serious ones in Perl. There's no reason to start from scratch. Maybe you'll like lighttpd or AxKit2 or perlbal or one of the many others.

      "Ok, this is just FUD and it's offensive. Although I don't work on the httpd project, I am a member of the ASF (through work on mod_perl) and I can assure you that I don't have any plans to change my behavior due to Microsoft's contribution."

      This is not the way the world works. The way that it goes is that when you least expect it, Microsoft will start requesting some small little changes, and without quite saying it, there'll be a general feeling that the MS money may go away if you're not willing to Be Reasonable on these small, little things.

      Inside the Foundation, you'll then have a split between the people who see no problem with Being Reasonable, and the people who think you should stand on principle.

      Even if the Reasonable people really are just being reasonable, and even if this compromise is no big deal, and even if there are no further compromises down the road, Microsoft will still have managed to cause a rift in the ranks.

      The Apache foundation's reputation has already been slightly damaged by taking the money. Are you surprised that people are wondering what it means? Duh.

        You simply don't know what you're talking about. The code is all under a good license and no corporation can force their will on the developers. If they could, Sun and IBM would have done so a long time ago.
      You're talking about default packages shipped by OS vendors? No one running a serious site on mod_perl would use those. I've never seen a mod_perl site getting real traffic that runs without a proxy or uses vendor binaries.
      Yes I am talking about the defaults. How many "serious" sites run IIS without requiring the web dev team to "compile from source" and apply special proxy setups? Plenty, I'm sure. For that matter, why shouldn't someone trust that the vendor-supported version binary be suitable for a "serious" site? Should we call RHEL "Redhat Non-Serious Linux?" or "Linux for Dummies?"

      If the vendor says (or even implies) than their product can perform some task, it should be capable of doing just that. I am aware of the disclaimer portions of EULA's that will say something to the effect that "this software is not guaranteed to perform anything or do whatever you thought it should." However, wouldn't it be a bit like purchasing a new automobile, then having to take it apart and put it back together before you can do any "serious" driving? If the manufacturer says it goes the speed limit and has a 5-star safety rating, it better measure up in the real world.

      That said, I usually compile Apache httpd (and libapreq and mod_perl) from source, downloaded from a mirror linked to from the Apache httpd, libapreq and mod_perl websites. The main reason is that finding a compatible libapreq2 RPM or DEB is nigh impossible. Just as simple to install from source.

      As for proxies being the recommended configuration for mod_perl (yes I've read this in several places) - the config file that ships with the source doesn't come with a proxy setup either. Never has from what I can tell. I've read "Writing Apache Modules with Perl and C" and "Practical mod_perl" - proxies barely warranted a mention.

      Ok, this is just FUD and it's offensive.
      Yes it is. Fear, Uncertainty and Doubt. I have all three at this point. And I'll have it regarding any group willing to take money off of Microsoft's hands.

      Although I don't work on the httpd project, I am a member of the ASF (through work on mod_perl) and I can assure you that I don't have any plans to change my behavior due to Microsoft's contribution.
      I commend your principle (and your long track record of holding it as well). It's not the individual developers like yourself that I wonder about. It's the addiction to the money hand-outs that worry me. Organizations bloat after receiving money. Only Hunger can drive groups like the ASF forward. Necessity is the mother of invention, not pandering to multi-national mega-corps for more donations.
        Red Hat, and other vendors, ship a general-use product that is suitable for basic work in many areas. The httpd they ship is compiled in a way that makes it friendly to RPM installs of .so packages, not to high-performance or to any specific use. They can't ship a proxy configuration because it wouldn't be the right thing for people using CGI or Java servlets and they have to please everyone.

        For a dedicated server, you will nearly always need to compile your own httpd. This is true of Perl too -- the one Red Hat ships has threads compiled, which slows it down significantly for people who don't use threads. A simple recompile of Perl with all defaults will give you a significant speed boost.

        Regarding your feelings about the ASF, I can't reason with paranoia, but I can tell you that the ASF includes many cantankerous old-timers and has had large contributions in the past from various other companies without compromising the quality of the projects I pay attention to.

Re^3: Time to write a "serious" http server in Perl?
by sasdrtx (Friar) on Aug 10, 2008 at 20:00 UTC
    If Apache doesn't "work" for you, what does? If Microsoft influence is a negative, why would you prefer IIS? As for influence by donation, I think you're confusing volunteer programmers with professional politicians. The former aren't generally so easily corrupted.

    sas
      If Apache doesn't "work" for you, what does?
      Nothing out there looks Just Perfect. Hence my desire to make something new.

      If Microsoft influence is a negative, why would you prefer IIS?
      I don't prefer IIS - the problems I have with Apache httpd one Nothing compared to my laundry list of problems with IIS. Given the two to choose from, I will take Apache httpd any day of the week.

      As for influence by donation, I think you're confusing volunteer programmers with professional politicians. The former aren't generally so easily corrupted.
      It's not the giving of money. It's the addiction to that money, and the kind of things organizations tend to do when they want to continue receiving that money in the future. Ask any individual (i.e. the venerable Perrin) and of course they would say something like "I could never be swayed by money." Ask a group and the response is not always as well-principled.

        Re "desire to make something new": That's great! Google proved that even though the search market was saturated, there was still room for a better way. However, having outed myself as an Apache fanboy, I really have a hard time believing that if there's a need it hasn't addressed, you'd be better off writing a module for it rather than starting from scratch.

        Re "addiction to that money": I don't have enough experience with volunteer open-source projects in general, or the Apache Foundation in particular, but here's my perception. While they are eager to get more money, more money doesn't go to anybody's benefit; it goes toward unavoidable expenses. More to the point, the programmers are not paid, so why would they code towards the benefit of donors?


        sas
Re^3: Time to write a "serious" http server in Perl?
by mr_mischief (Monsignor) on Aug 11, 2008 at 16:25 UTC
    All you need to understand to have two completely different Apache instances serving two completely separate web sites are any one of: a reverse proxy, binding to a single IP address, virtual servers, mod_rewrite, port forwarding, or having two servers on your network. If your server administrator doesn't understand at least one of those technologies, you need to fire your server administrator. Not all of those are ideal, but any one would get the job done.
Re^3: Time to write a "serious" http server in Perl?
by ides (Deacon) on Aug 11, 2008 at 15:56 UTC

    The reason you don't see Apache setup with a proxy out front by a distro, by default, is very simple. Most people don't need it, it would be confusing to them if they ran across it, and those of us who do need it know it's trivially simple to setup.

    Apache isn't setup "by default" to do lots of the things I use it for regularly, but it's flexibility allows me to setup pretty much whatever I want, however I want, very easily.

    Frank Wiles <frank@revsys.com>
    www.revsys.com