in reply to Re^2: uncompress gzip data in a callback
in thread uncompress gzip data in a callback

Try this 
use LWP::UserAgent;
use Compress::Raw::Zlib;

sub mkGunz
{
    my $gun = new Compress::Raw::Zlib::Inflate(
                WindowBits => WANT_GZIP);

    return 
        sub 
        {
            my $out;
            my $status = $gun->inflate($_[0], $out);

            if ($status == Z_OK || $status == Z_STREAM_END)
            {
                print $out;
            }
            else
            {
                die $status;
            }
        } ;
}


my $ua = LWP::UserAgent->new;
my $URL = 'http://whatever/';


my $res = $ua->request(HTTP::Request->new(GET => $URL), mkGunz());

[download]

Replies are listed 'Best First'.
Re^4: uncompress gzip data in a callback (Updated)
by BrowserUk (Patriarch) on May 04, 2010 at 16:16 UTC

    Update: Closed: Dumb user error! See below.

    This does not seem to work. And I wonder how could it? Given that you pass $_[0] to inflate() each time, and the callback gets called with lots of little bits, none of which is decodable stand alone?

[reply]
      It can accept lots of little bits of compressed data because that is what I designed Compress::Raw::Inflate to do. In the code I posted the $gun object that the mkGunz closure creates will maintain the context of the overall compressed data stream - that is the key part that allows it to be fed small snippets of compressed data a bit at a time.

      The code worked fine when I tried it before posting. How did you compress the data used in your origin server?

[reply]

        Update: Added WindowBits(see below) and it now works fine! (Why is that not the default?)

        Dodging all the issues of LWP configuration, Accept-Encodings etc. I tried this which is equivalent: 

        C:\test>gzip junk.htm

C:\test>type junk38.pl
#! perl -slw
use strict;
use Compress::Raw::Zlib;

my $o = new Compress::Raw::Zlib::Inflate(
######### I forgot this. With it, it works perfectly.#################
+#######
    WindowBits => WANT_GZIP_OR_ZLIB,  
);

binmode STDIN;

while( read( STDIN, my $in, int( rand 1024 ) ) ) {
    my $st = $o->inflate( $in, my $out );

    die "$st" unless $st == Z_OK;

    print $out;
}

C:\test>perl junk38.pl < junk.htm.gz
data error at junk38.pl line 12.

        [download]

        And I always get 'data error'?

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]

In Section Seekers of Perl Wisdom