Using network calls to load test

sdyates has asked for the wisdom of the Perl Monks concerning the following question:

Folks, once again I bow before the monks seeking direction. I want to load test an image server, but I do not want to use simple GET requests. I want to do the following:

1) send a request and determine its success
2) determine how long the request took to full fill
3) send multiple requests without waiting the first responce
4) send requests only after the previous request has been returned or fullfilled.

Now to do this, I do not think Get requests will suffice... I will need to be able to read the TCP headers and such.

My problem is that I am not sure which perl modules will work best for me. I want something that will do the job, but is easy enough to work with and not require a CIE--I am only a ccna--lol

Your guidance is much appreciated

Comment on Using network calls to load test

Replies are listed 'Best First'.
Re: Using network calls to load test by tachyon (Chancellor) on Feb 26, 2003 at 16:54 UTC
To use LWP::UserAgent to get a webpage (or part therof), complete with parsed header you would do: `use Data::Dumper; use LWP::UserAgent; my $gif = 'http://some.domain.com/path/to/some.gif'; my $ua = new LWP::UserAgent; my $req = new HTTP::Request GET => $gif; my $res = $ua->request($req); print Dumper $res;` [download] You will see the data hash you get back ie $res->{_content} , $res->{_rc} is the return code ie 200 OK or 503 overloaded, etc. I have included Data::Dumper so you can see the data structure, not because you need it. Now if you want to time this you would use Time::HiRes `use Time::HiRes 'time'; my $begin = time(); # do stuff, lets waste some time $cnt++ for 1..1000000; my $end = time(); my $time = ($end - $begin) * 1000; print "Took $time milliseconds\n" ;` [download] So if you want to time stuff just add parts 1 and 2. Now if you want to do load testing either use this in combination with LWP::Parallel::UserAgent or just fork off X number of kids running this code yourself. `my $kids = 100; for ( 1.. $kids ) { my $pid = fork(); next if $pid; # add code for child to process here }` [download] This will fork you 100 kids so just stick the code above in, write the data from each kid to a logfile and Bob's your uncle. cheers tachyon s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print	[reply] [d/l] [select]
Re: Using network calls to load test by joe++ (Friar) on Feb 26, 2003 at 16:32 UTC
Hmm, did you look into LWP::Parallel::UserAgent already? Setting appropriate callback handlers for multiple requests seems feasible to me... Admitted, maybe even less than $0.02 ;-) -- Cheers, Joe	[reply]
Re: Using network calls to load test by cees (Curate) on Feb 26, 2003 at 16:40 UTC
Apache comes with a good benchmarking package called ab. It should work with any web server. I would look at this and other benchmarking solutions before re-inventing the wheel.	[reply]
Re: Using network calls to load test by Abigail-II (Bishop) on Feb 26, 2003 at 16:42 UTC
What make you think you need to see the TCP headers? You can't deduce how long it took to full fill the request from the TCP headers. send a request and determine its success That's a GET request. determine how long the request took to full fil That requires a clock of some sorts. send multiple requests without waiting the first responce That's several GET requests, in parallel. send requests only after the previous request has been returned or fullfilled. That's several GET requests, in series. So, what makes you think that GET requests won't do? Abigail	[reply]
Re: Using network calls to load test by adrianh (Chancellor) on Feb 26, 2003 at 16:42 UTC
Take a look at POE, in particular POE::Component::Client::HTTP. From the POD of the latter: POE::Component::Client::HTTP is an HTTP user-agent for POE. It lets other sessions run while HTTP transactions are being processed, and it lets several HTTP transactions be processed in parallel. I've not used it personally, but it would seem to allow all you need. LWP::Parallel might also be of use.	[reply]