It depends. The code in your test script assumes that the input consists of 5 completely distinct gzip data streams. So they will each contain the gzip header, the compressed payload and gzip trailer data. If that is what is actually happening with the WWW::Mechanize application, and the gzip data streams aren't that big, then your approach should be fine.

I'm not convinced that is what is happening in the real application though. The snippet of code below, from earlier, along with the observation that uncompressing $collected resulted in more of the real uncompresed payload data suggests that this is a single gzip data stream

            $collected .= $data;
            gunzip \$collected, \$out;
            print $out, $/;
[download]

If that is the case then IO::Uncompress::Gunzip will only work if you are prepared to read the entire compressed data stream and uncompress the lot in one go. If we are dealing with a potentially infinite compressed data stream, that isn't going to work.

The code I posted that uses Compress::Zlib will uncompresses the data as it gets it, one chunk at a time.

I see, thanks. Can you explain, when it's possible that the output of "gunzip" is valid (but partial, truncated) uncompressed data plus obviously binary, still compressed "tail", as in Re^4: Uncompress streaming gzip on the fly in LWP::UserAgent/WWW::Mechanize. I couldn't get such result regardless of "Transparent" and all other parameters -- always uncompressed partial data only, instead.

IO::Uncompress::Gunzip expects to be given valid and complete gzip data stream. If that doesn't happen, it fails.

I'll use the compressed data in $gzipped below to illustrate.

    use IO::Compress::Gzip qw(gzip);
    use IO::Uncompress::Gunzip qw(gunzip $GunzipError);

    my $data = 'I include only the bare bones because I tried somethin
+g';

    # Create some compressed data
    my $gzipped ;
    gzip \$data => \$gzipped ;
[download]

Lets start with the valid part. If I create data corruption in the compressed data stream, bad things happen

    my $corrupt = $gzipped;

    # Overwrite part of the compressed data with junk
    substr($corrupt, 10, 3, "BAD") ;

    gunzip \$corrupt => \$uncompressed
      or print "Cannot gunzip: $GunzipError\n";
[download]

That will output

    Cannot gunzip: Inflation Error: data error
[download]

If you get that, there is no point in continuing.

Next is a truncated data stream (which is what this ticket is all about).

    # truncate the compressed data
    my $truncated = substr($gzipped, 0, 10);

    gunzip \$truncated => \$uncompressed
      or print "Cannot gunzip: $GunzipError\n";
[download]

that will output this

    Cannot gunzip: unexpected end of file
[download]

In this instance, you can try to get more data, append to the input buffer ($truncated in this case) and uncompress the whole thing again. The only semi-valid use for this technique is when you are certain that you will eventually get a complete gzip data stream. That does not seem to be the case in this instance.