Software to Observe HTTP Transactions? (HTTP Sniffer?)

princepawn has asked for the wisdom of the Perl Monks concerning the following question:

I need to observe what my browser is sending to my server under two different circumstances. I am betting that, even when the return content type is application/x-zip-compressed that IE only makes byte-range requests when the URL it is requesting does not have a query string.

I have observed this phenomenologically. I had the following simple webpage:


<form action=file.zip>
<input type=submit value="Download Now!">
[download]

and IE would resume the download if I cancelled in the middle.

However the following webpage:


<form action=download.cgi?filename=file.zip>
<input type=submit value="Download Now!">
[download]

did not result in IE making partial requests for the zip file even though the CGI program is returning the same MIME type as above and the file does download successfully. But for some reason, a partial request is not supported here.

I hypothesize that IE is not making a partial request but would like to ensure this by getting a printout of what HTTP is being sent to my webserver.

So my question is

How can I get an exact printout of everything that was in IE's request.

i was nearly successful with HTTPi A small webserver written entirely in Perl, but it only serves executables and HTML files.

I think I have seen various Perl HTTP sniffers around, but I couldn't find any in a websearch just now.

Comment on Software to Observe HTTP Transactions? (HTTP Sniffer?) Select or Download Code

Replies are listed 'Best First'.
Re: Software to Observe HTTP Transactions? (HTTP Sniffer?) by Corion (Patriarch) on Apr 03, 2001 at 20:44 UTC
The easiest way is just to use a transparent proxy server (written in Perl), which dumps out all traffic going through it. Of course, this will not work if the browser behaviour differs wildly from the no-proxy behaviour. httptech had a small proxy in RE: Benevolent Ad Filter, and in that thread there is also mentioned FilterProxy. It shouldn't be too hard to glue together some frankensteinian abomination with Net::Daemon, Net::Server and LWP either, but that means your platform needs fork() support I guess.	[reply]
Re: Re: Software to Observe HTTP Transactions? (HTTP Sniffer?) by merlyn (Sage) on Apr 03, 2001 at 20:50 UTC
Don't forget that I've written a few of those as well. In particular, look at columns 11 and 34 in my WT columns. -- Randal L. Schwartz, Perl hacker	[reply]
(Ovid - packet sniffer) Re: Software to Observe HTTP Transactions? (HTTP Sniffer?) by Ovid (Cardinal) on Apr 03, 2001 at 20:44 UTC
You can find a packet sniffer on Lincoln Stein's home page. Cheers, Ovid Join the Perlmonks Setiathome Group or just click on the the link and check out our stats.	[reply]
Re: Software to Observe HTTP Transactions? (HTTP Sniffer?) by lachoy (Parson) on Apr 03, 2001 at 21:31 UTC
There's a nice little GTK utility (in C) for this called wsnitch which I've used for quite some time with no problems. It's helped me identify a lot of errors/assumptions. Chris M-x auto-bs-mode	[reply]
Re: Software to Observe HTTP Transactions? (HTTP Sniffer?) by strredwolf (Chaplain) on Apr 03, 2001 at 23:49 UTC
I've written one into my web proxy. Look at my homenode for wsproxy. -- $Stalag99{"URL"}="http://stalag99.keenspace.com";	[reply]