naxxtor has asked for the wisdom of the Perl Monks concerning the following question:

Hello everyone!

I am a network field technician by day, and one of the sites I work on has a strange problem. The external Internet connection is about as stable as a tightrope walker on a tightrope in a dingy in a very violent storm. Quite a skilled one mind, because the connection doesn't actually go down, it just gets very slow every 3 seconds or so.

Enough of the back story. I'm trying to write a utility that will monitor the speed of a download over it's progress. I'm not interested in averages, which are comparatively simple to put together (just time a socket connection till it's finished, say) and can be done with tools like iperf. An average will just iron out the weirdness that I'm trying to detect.

At first I thought I could do something along these lines:
my $sock = new IO::Socket::INET->new( # open up a socket to the + testing server PeerAddr => 'vm1.naxxtor.com', PeerPort => '80', Proto => 'tcp', Type => SOCK_STREAM ) or die("Can't open socket $!"); print "Sending request\n"; print $sock $request; # send the request for th +e test file print "Starting transfer\n"; { local $| = 1; my $chunk_start = 0; # initialise some variabl +es my $chunk_time = 0; local $/ = \10240; # read in chunks 10240 b +ytes long $chunk_start = [gettimeofday]; # set initial timer while (<$sock>) { # loop through each chun +k $chunk_time = tv_interval($chunk_start); # calculate the time +taken to download this chunk. if ($chunk_time != 0) { push(@speeds,$chunk_time); } $chunk_start = [gettimeofday]; } }

Unfortunately this doesn't work - as sockets are BUFFERED, so if it recieves the data quicker than perl can get through the loop then you end up with a zero time for that chunk. It just goes plain screwy.

I have a horrible feeling I'm going to have to write this in C/C++ :(

What would probably work is to put a timestamp into a list for each chunk of 1024 bytes recieved, then process it later on. I can then do some windowing to get a reasonable estimation of speed at a particular point, and throw it out to a file which I can gnuplot.

I'm not sure. Any advice will be very well recieved :-)

Replies are listed 'Best First'.
Re: Network Reliability Testing, problems with buffered sockets?
by MonkE (Hermit) on Mar 21, 2007 at 14:22 UTC
    Have you tried TIME::Elapse. It reports elapsed time on the order of microseconds, and as such may be able to help you. There is also a Time::Hires module as well on CPAN that exports a gettimeofday() function.
      Cheers for the response.

      Yep, I should have included that in the snippet, I'm using Time::Hires exporting gettimeofday to time the chunks :) Time::Elapse looks promising though!

Re: Network Reliability Testing, problems with buffered sockets?
by 5mi11er (Deacon) on Mar 21, 2007 at 15:39 UTC
    While not a perl solution, how about using a linux box and using tcpdump? Or a real packet analyzer? There are free ones available on the net.

    As a complete stab in the dark, I have run across something like this ages ago, turned out to be an email message that kept getting bounced back and forth between two email servers, each adding a "he's on vacation" message to the end. We found it about 3 days after it had started, and it was gigantic.

    Good luck

    -Scott

      Oddly enough I've come across something similar before, except it was with a spam filter labelling something as spam, then the server rejecting the unknown header and sending it back to the spam filter, which labelled it as spam and returned it again.

      I did think about using packet sniffing/logging to analyse the flow, but unfortunately it wouldn't give me enough information, and I'd need to work out how to track the connection.

      Thanks for the idea :)

        I think maybe you're in tunnel vision mode with the problem. It sounds like you're assuming there's something wrong with a single application's packet flows. I'm betting there's a better chance that there is other traffic that is using enough bandwidth to keep the one application from responding well. A sniffer would allow you to see all the traffic, not just the one application you're having issues with.

        -Scott

Re: Network Reliability Testing, problems with buffered sockets?
by bsdz (Friar) on Mar 21, 2007 at 14:53 UTC
    You may also want to set some socket options, i.e. turn off Nagle's algorithm. See setsockopt and TCP_NODELAY. You can also set the socket buffer size to zero using SO_RCVBUF and SO_SNDBUF options. Setting SO_SNDBUF to 0 on Windows bypasses the OS's TCP/IP stack buffer. Perhaps this problem is more suited to C++? Let us know how you get on.

      Yeah, that looks interesting. I couldn't find a lot of documentation on setting socket options. I'll look into it :)