Re^2: Extract chunked/gzip data from pcap file

I have the PerlIO::gzip module installed, and I've tried using it. However, there's more to the puzzle than just using that module.

When I look at these packets in Wireshark, a packet can be viewed in three different forms: (1) full frame, with the binary data intact and still encoded; (2) "De-chunked entity body", which exhibits the majority of the binary data intact -- but which has removed the "chunked encapsulation" (for lack of a better term); and (3) "Uncompressed entity body", which shows no binary data, just the decoded HTML text.

This leads me to believe that there is some intermediate step that is required to remove the "chunked encapsulation", before being able to hand the "clean" compressed data to PerlIO::gzip as a stream for decompression.

Comment on Re^2: Extract chunked/gzip data from pcap file

Replies are listed 'Best First'.
Re^3: Extract chunked/gzip data from pcap file by Corion (Patriarch) on Dec 28, 2009 at 14:28 UTC
Most likely, you will need to reassemble the TCP frames into the complete TCP message, then parse the HTTP response from that, and then decode the payload of the HTTP response. This is something I've done with Sniffer::HTTP, which gices you a HTTP::Response object for each completed request. You can then query the `->decoded_content` method of HTTP::Response to get the uncompressed data out. Note that I have at least one report of Sniffer::HTTP having a memory leak, so be sure to test the memory requirements before rolling it out on a large scale. Unfortunately, Net::Pcap doesn't currently build for me on Win32, so I can't conveniently replicate the environment to actually test things.	[reply] [d/l]
Re^4: Extract chunked/gzip data from pcap file by oakb (Scribe) on Jan 04, 2010 at 16:19 UTC
I can't get Sniffer::HTTP to install on Win32 ActivePerl. From the sound of it, I might be better off not installing it anyway. Of course, I have HTTP::Response as part of my default installation. Is there any way to create an HTTP::Response object without Sniffer::HTTP?	[reply]
Re^5: Extract chunked/gzip data from pcap file by Corion (Patriarch) on Jan 04, 2010 at 16:22 UTC
Likely by doing what Sniffer::HTTP does. Or by looking at HTTP::Response, and then using its methods and the data you glean from the packet stream to fill in the object. Which is what Sniffer::HTTP does.	[reply]