in reply to Re^2: Should I just print my own HTTP headers?
in thread Should I just print my own HTTP headers?

You make it sound like you're going to load those lines of code with your bare hands in an ice storm. Trust me, this is not the bottleneck you're looking for! I've profiled a lot of slow web-apps in my day and I've never once seen a CGI.pm method in the top 10.

-sam

  • Comment on Re^3: Should I just print my own HTTP headers?

Replies are listed 'Best First'.
Re^4: Should I just print my own HTTP headers?
by Cap'n Steve (Friar) on Apr 06, 2007 at 09:08 UTC
    And uphill both ways! But according to this test, printing one header by hand is 11331% more efficient than using CGI to do it. Did I miss something?
    #!/usr/bin/perl use warnings; use strict; use Benchmark ':all'; open OUT, '> nul' or die $!; cmpthese(100000, { 'cgi' => 'use CGI ":cgi";my $cgi = new CGI;print OUT $cgi->header( +"text/html");', 'plain' => 'print OUT "Content-type: text/html\n\n";' });
      It's actually even worse. You're actually loading CGI only once. If you'd reload CGI every time in the loop, you'd get far worse figures.

      You could do that by deleting 'CGI.pm' from %INC. You'd then probably get a lot of warnings on redefined subs, so you should rather do something like

      delete $INC{'CGI.pm'}; no warnings 'redefine'; # or 'redefined'? Gah, I hate that! require CGI; my $cgi = CGI->new; print OUT $cgi->header;

      Also, CGI->new parses the form parameters, you should take that into account. It's something you probably want to do anyway.

        Thanks for the tips. Forcing it to search for CGI.pm every time only makes the print statement 12471% more efficient, but when not using the object-oriented interface, the print statement is 3551% more efficient.
      But according to this test, printing one header by hand is 11331% more efficient than using CGI to do it. Did I miss something?

      Two things:

      • No useful program will ever print out 100,000 HTTP headers on purpose and exit. That'd be like me benchmarking various ways to roll a cigarette--completely useless because I don't smoke.
      • chromatic's second rule of performance optimization: You're doing IO! Microbenchmarks don't matter!

      Latency and throughput are much, much more important performance characteristics in such an application, and it's highly unlikely you'll optimize those to the degree that microoptimizations to save milliseconds in program startup time will have any benefit.

      The only applicable scenario for this type of optimization is where you're running the program on your own machine and can't run it in a persistent process... but I can't imagine any use for that.

        No useful program will ever print out 100,000 HTTP headers on purpose and exit. That'd be like me benchmarking various ways to roll a cigarette--completely useless because I don't smoke.
        Obviously not, but it's it's quite likely that the program that prints 1 header will be run 100,000 times. I kind of wish this thread would just die since it keeps costing me experience points, but let me sum up like this: Is "it doesn't take all that long to load" really a good argument to advocate the use of a module? I'm sure CPAN is full of quick loading modules, but why would I use them if I don't need to?
          A reply falls below the community's threshold of quality. You may see it by logging in.

        Question: Why continue to advocate the use of a module that carries the weight of all that extraneous functionality, when it seems that everybody who has any expertise in perl/cgi applications adamantly decries the use of so much of it in favour of templating? Especially when there is a drop-in, cut-down replacement derived from it, that does away with that weight?


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

      it's not about the microsecond or so it takes to import CGI that matters (because you're going to be using CGI.pm for other things in your CGI script anyway)

      Also, using print "some-header: value" is more fragile when exposed to maintenence programmer tampering

      Also, is the functional interface to CGI.pm any quicker if you race it too?

      @_=qw; ask f00li5h to appear and remain for a moment of pretend better than a lifetime;;s;;@_[map hex,split'',B204316D8C2A4516DE];;y/05/os/&print;
      And uphill both ways! But according to this test, printing one header by hand is 11331% more efficient than using CGI to do it. Did I miss something?

      Yes, you missed something. Two things in fact - "bottleneck" and "profiled". I didn't say I'd benchmarked lots of individual lines of web-apps - I said I'd profiled lots of web-apps! When you profile you find out where the important things to tune are - the really slow things that actually matter. Things like database access, file access, search algorithms, etc. NOT things like CGI.pm!

      Benchmarking is a useful tool but it's nearly useless as a way to figure out what's worth spending time optimizing. If you find yourself wanting to disagree then I suggest you stop programming in Perl immediately - C is so much faster! Just try benchmarking:

        for (0 .. 100000) { $a++; }

      Versus:

        for (int x = 0; x <= 100000; x++) { a++ }

      -sam

        When you profile you find out where the important things to tune are - the really slow things that actually matter.
        But what if you've already tuned the important things? We're talking about a script that probably won't exceed 30 lines so it's not exactly difficult to maintain.
        Benchmarking is a useful tool but it's nearly useless as a way to figure out what's worth spending time optimizing.
        Yes, but it is useful in that it tells you how to optimize what you've already decided to optimize. I'd love to write it in C, but 1) I don't know C. and 2) I don't know of any webhost that allows compiled programs without a dedicated server.
      Well, on the next day, you find out that you need to set up a cookie so you add that Set-Cookie line. Next day, you know you need some logic to determine whether to send PDF, a compressed file, a music or video file, or even some mime type of your very own creation. So, you struggle a bit to fit in the code. Then you're thinking..., Hey, do I have to send those bizard mime type files and let the browser displays at its will, or do I have to let the browser knows that the file is indeed for download? How do I state that again? Then after some reading you find out about Content-Disposition.

      Oh, a few days later, you are asked to add a feature so your app is able to send the content in different languages based on some condition (the contents are ready, you are only asked to deliver them). So, you fixed those lines again and again. And I bet you will surely be very careful about those double EOLs.

      Now, tell me, after you implement this with your own, and benchmark it again againsts the CGI.pm version of the same requirement. How much would you gain from the processing time advantage, would it be worth the implementation time?

      Simple benchmark won't qualify for the real problem.

        And then the next day I decide to automatically generate all of that, so I should use Image::Magick and PDF just to be safe, right?

        I'll just have to deal with my future multiple personalities when they start developing. Until then, I'd rather deal with the actual code instead of the theoretical.