Re^2: uncompress gzip data in a callback

Actually you can - the DEFLATE algorithm encodes data as a stream of blocks which can be decoded sequentially, as zcat does. Gzip wraps a DEFLATE stream with a header and footer but since the footer is for integrity checking it is not useful until after the whole stream has been decompressed.

If you're specifically talking about PKZIP you're correct, but that's because PKZIP is an archive format containing multiple files each compressed with DEFLATE, and thus is not the same thing.

Comment on Re^2: uncompress gzip data in a callback

Replies are listed 'Best First'.
Re^3: uncompress gzip data in a callback by BrowserUk (Patriarch) on May 04, 2010 at 13:39 UTC
The problem is, the `:content_cb` callback gets called with odd sized chunks of data as they are received, and if you pass those to inflate(), you just get ' data error'. I've tried accumulating the odd sized chunks as they arrive, and passing the buffer to inflate() as they accumulate, but it has never actually decoded anything until the entire request has completed. And if you have to accumulate the entire content before decoding it, you might as well skip the callback and let LWP decode it for you. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. RIP an inspiration; A true Folk's Guy	[reply] [d/l]
Re^4: uncompress gzip data in a callback by pmqs (Friar) on May 04, 2010 at 16:03 UTC
That sounds like you aren't maintaining the context of the compressed data stream between invocations of inflate. The code I posted earlier today shows how to handle that.	[reply]
Re^3: uncompress gzip data in a callback by BrowserUk (Patriarch) on May 04, 2010 at 18:47 UTC
You're right! Once you match the WindowBits on the inflator, to the compressor used by the server, and don't try to do the buffering yourself, Compress::Raw::Zlib works perfectly. Many thanks. I've wanted this for ages.	[reply]