Re: an interesting web frameworks benchmark

As far as I can tell, they just pointed Siege at a couple of the urls in the app, with two sets of configured durations and concurrency levels. It would be helpful to have more details on it though, I agree.

It'd also make the task slightly easier if they had supplied the full source code to the Ruby app. The OnLAMP article only describes the steps required to build it. For those of you who are too lazy¹ to RTFA, here's the short description:

We'll create an online collaborative cookbook for holding and sharing everyone's favorite recipes. We want our cookbook to:

Display a list of all recipes.
Create new recipes and edit existing recipes.
Assign a recipe to a category (like "dessert" or "soup").

As for the Perl frameworks you mention, I think the toughest will be Jifty. CGI::Application and Catalyst can be configured to use the same rendering engine and database abstraction layer, but I don't believe this is true for Jifty. I suspect that will make it rather difficult to create a meaningful benchmark.

That being said, I don't expect wildly varying outcomes. What would interest me more is developer speed². The most important reason to choose a framework is not for its performance, but for how fast you can get work done, and how much code you need to write yourself to accomplish your goals. In my opinion, if an application has performance problems, it's either because of obvious design flaws, or a lack of hardware resources. The former should be easily fixed by any reasonably experienced developer, and the latter is a simple matter of upgrading your machines³. The real issue is how fast you can add new features, and how much work it takes to refactor your code.

I just realised while writing this up: you may not be after a comparison between these three Perl frameworks, but their performance relative to the three in the original article. In that case we'll need similar hardware and the same software versions for a fair comparison.

Update: The ruby source code for the example is available from page 2 of the OnLAMP article, right below this short description. I should learn to read...

¹ Just kidding :)
²Hat tip to dragonchild for pointing this out in #cgiapp.
³Obviously this is a simplification

Comment on Re: an interesting web frameworks benchmark

Replies are listed 'Best First'.
Re^2: an interesting web frameworks benchmark by perrin (Chancellor) on Jul 17, 2006 at 00:33 UTC
That's right, I don't really care which of the Perl frameworks is fastest, but I'd like to see that they are keeping up with other languages in terms of speed. I'd expect Perl to do very well in this type of comparison, since the Perl 5 implementation has been tuned a great deal over the years. It's not important to use the same hardware as the original test, as long as one can recreate at least one of the apps tested in that one as a reference.	[reply]

Replies are listed 'Best First'.

Re^2: an interesting web frameworks benchmark
by perrin (Chancellor) on Jul 17, 2006 at 00:33 UTC

It's not important to use the same hardware as the original test, as long as one can recreate at least one of the apps tested in that one as a reference.

[reply]