Fellow Monks,

A strange problem has arisen today. Most strange. I have the following code:

my $res = $ua->get( $url, ':content_file' => $filename );

being used under LWP::UserAgent. The returned content, stored in the file $filename, seems to contain the HTTP headers as well as the content - I was expecting just the content. So, for example, if $url references a JPG on the Internet, I don't get a readable JPG back - I seem to get a full HTTP response.

The code, under Windows, works fine. Ran on a Linux box, broken. Now that is odd!

Same issue if getstore(...) is used from LWP::Simple instead of the full UserAgent... Same issue if I store the Response in a variable and use/print it. No matter what I do, I get these unwanted HTTP header lines at the top of the file/response which means the file returned by LWP is actually useless with manual modification...

Opening the resulting file in an editor I seem to see a ^M at the end of the lines - certainly in the unwanted header-looking stuff. Could this be something significant? If so, why is LWP putting it there? Don't worry about this bit anymore ;)

This is a brand new build of a Fedora 14 desktop. Windows was using an older build of ActiveState.

The docs say that I should ONLY get the content into $filename - so why are those HTTP headers there too!.... :(

**UPDATE** A quick review of my code shows I am calling the "get" or "getstore" from within a thread. Outside of threading, files are retrieved as expected. In a threaded environment, the data stored contains these unwanted HTTP headers. Question still remains though......why?

fx, Infinity is Colourless

In reply to LWP UserAgent and Simple keeping headers in content by fx

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.