As far as I can tell, they just pointed Siege at a couple of the urls in the app, with two sets of configured durations and concurrency levels. It would be helpful to have more details on it though, I agree.

It'd also make the task slightly easier if they had supplied the full source code to the Ruby app. The OnLAMP article only describes the steps required to build it. For those of you who are too lazy1 to RTFA, here's the short description:

We'll create an online collaborative cookbook for holding and sharing everyone's favorite recipes. We want our cookbook to:

As for the Perl frameworks you mention, I think the toughest will be Jifty. CGI::Application and Catalyst can be configured to use the same rendering engine and database abstraction layer, but I don't believe this is true for Jifty. I suspect that will make it rather difficult to create a meaningful benchmark.

That being said, I don't expect wildly varying outcomes. What would interest me more is developer speed2. The most important reason to choose a framework is not for its performance, but for how fast you can get work done, and how much code you need to write yourself to accomplish your goals. In my opinion, if an application has performance problems, it's either because of obvious design flaws, or a lack of hardware resources. The former should be easily fixed by any reasonably experienced developer, and the latter is a simple matter of upgrading your machines3. The real issue is how fast you can add new features, and how much work it takes to refactor your code.

I just realised while writing this up: you may not be after a comparison between these three Perl frameworks, but their performance relative to the three in the original article. In that case we'll need similar hardware and the same software versions for a fair comparison.

Update: The ruby source code for the example is available from page 2 of the OnLAMP article, right below this short description. I should learn to read...

1 Just kidding :)
2Hat tip to dragonchild for pointing this out in #cgiapp.
3Obviously this is a simplification

In reply to Re: an interesting web frameworks benchmark by rhesa
in thread an interesting web frameworks benchmark by perrin

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.