in reply to an interesting web frameworks benchmark

Over the past week, I've been trying to setup a similar benchmark between the Ruby app and a CGI::Application version. I haven't been able to setup mod_ruby at all, so I've limited myself to comparing the plain cgi versions, and the mod_perl version. I'm afraid this doesn't do justice to Ruby, but since my wife and I are expecting our first child in the coming weeks, I haven't been as eager to persue this. I'd appreciate it if someone else could lend me a hand with the ruby setup.

My CGI::Application version uses the following main components:

I made it into a framework-like application by using the following plugins and modules:

If anyone is interested, I will put my code online somewhere.

Even though the numbers are not as informative as I'd like them to be -- due to my lack of Ruby skills -- I still want to show you the benchmark results I got sofar:

Summary - 1 minute test with 15 concurrent users
  Ruby/CGIPerl/CGIPerl/mod_perl
Transactions 70 212 1685
Transaction rate 1.17 t/s3.53 t/s28.11 t/s
Longest transaction13.53 s 10.40 s 0.19 s

Summary - 3 minute test with 50 concurrent users
  Ruby/CGIPerl/CGIPerl/mod_perl
Transactions 159 607 14245
Transaction rate 0.88 t/s3.37 t/s79.23 t/s
Longest transaction88.64 s 22.76 s 5.24 s 

Machine config:
         1 Xeon 3.0GHz
         2GB RAM

OS:      Debian Sarge amd64 
Software: Apache/2.0.54 (Debian GNU/Linux)
         mod_ruby/1.2.4
         Ruby/1.8.2(2005-04-11)
         mod_perl/1.999.21
         Perl/v5.8.4
         Siege 2.61
         MySQL 4.1.11

====
siege -t 1M -c 15
====

==
ruby cgi
==
** siege 2.61
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege...      done.
Transactions:                     70 hits
Availability:                 100.00 %
Elapsed time:                  59.96 secs
Data transferred:              97770 bytes
Response time:                 11.36 secs
Transaction rate:               1.17 trans/sec
Throughput:                  1630.59 bytes/sec
Concurrency:                   13.27
Successful transactions:          70
Failed transactions:               0
Longest transaction:           13.53
Shortest transaction:           8.30

==
perl cgi
==
** siege 2.61
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege...      done.
Transactions:                    212 hits
Availability:                 100.00 %
Elapsed time:                  60.05 secs
Data transferred:             118853 bytes
Response time:                  3.56 secs
Transaction rate:               3.53 trans/sec
Throughput:                  1979.23 bytes/sec
Concurrency:                   12.56
Successful transactions:         212
Failed transactions:               0
Longest transaction:           10.40
Shortest transaction:           0.91

==
mod_perl
==
** siege 2.61
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege...      done.
Transactions:                   1685 hits
Availability:                 100.00 %
Elapsed time:                  59.95 secs
Data transferred:             920626 bytes
Response time:                  0.02 secs
Transaction rate:              28.11 trans/sec
Throughput:                 15356.56 bytes/sec
Concurrency:                    0.60
Successful transactions:        1685
Failed transactions:               0
Longest transaction:            0.19
Shortest transaction:           0.01


====
siege -t 3M -c 50
====

==
ruby cgi
==
** siege 2.61
** Preparing 50 concurrent users for battle.
The server is now under siege...
Lifting the server siege..      done.
Transactions:                    159 hits
Availability:                 100.00 %
Elapsed time:                 180.47 secs
Data transferred:             243663 bytes
Response time:                 48.77 secs
Transaction rate:               0.88 trans/sec
Throughput:                  1350.16 bytes/sec
Concurrency:                   42.97
Successful transactions:         159
Failed transactions:               0
Longest transaction:           88.64
Shortest transaction:          29.70

==
perl cgi
==
** siege 2.61
** Preparing 50 concurrent users for battle.
The server is now under siege...
Lifting the server siege..      done.
Transactions:                    607 hits
Availability:                 100.00 %
Elapsed time:                 179.98 secs
Data transferred:             333548 bytes
Response time:                 13.61 secs
Transaction rate:               3.37 trans/sec
Throughput:                  1853.25 bytes/sec
Concurrency:                   45.89
Successful transactions:         607
Failed transactions:               0
Longest transaction:           22.76
Shortest transaction:           6.25

==
mod_perl
==
** siege 2.61
** Preparing 50 concurrent users for battle.
The server is now under siege...
Lifting the server siege...      done.
Transactions:                  14245 hits
Availability:                 100.00 %
Elapsed time:                 179.80 secs
Data transferred:            7772678 bytes
Response time:                  0.12 secs
Transaction rate:              79.23 trans/sec
Throughput:                 43229.58 bytes/sec
Concurrency:                    9.64
Successful transactions:       14245
\Failed transactions:              0
Longest transaction:            5.24
Shortest transaction:           0.01

[download]

In Section Meditations