geoffleach has asked for the wisdom of the Perl Monks concerning the following question:

I'm using LWP::UserAgent to GET a large file. The server is returning it in pieces. I've retrieved the first piece. A subsequent GET merely retrieves the first piece. Where do I find the next?

Just to be clear. I can discover that the first GET is short by comparing content_length with length(content). Do I need to use a different URI for the next GET? If so, where in the response do I find that puppy?

Replies are listed 'Best First'.
Re: Where to GET the pieces
by Athanasius (Archbishop) on Feb 18, 2016 at 06:53 UTC

    Hello geoffleach,

    I think you need to provide more information. What is the web page you’re trying to retrieve? And what exactly is the command you issue via LWP::UserAgent to send the GET request?

    In the meantime, lwpcook#LARGE-DOCUMENTS has a code snippet showing how to use LWP::UserAgent to process a large document in chunks, as it arrives. That may be what you’re looking for.

    Hope that helps,

    Athanasius <°(((>< contra mundum Iustus alius egestas vitae, eros Piratica,

[reply]
[d/l]
      Thanks. That is exactly what I needed.

      For anyone who might need this in future, the secret is to use the content callback ($content_cb) in the call to $ua->request().
[reply]
Re: Where to GET the pieces
by Anonymous Monk on Feb 18, 2016 at 07:40 UTC

    The server is returning it in pieces. ...

    What does that mean?

[reply]
[reply]
        Thanks for correcting my technical verbiage :-)
[reply]

Back to Seekers of Perl Wisdom