bageler has asked for the wisdom of the Perl Monks concerning the following question:

I am requesting a page with LWP::UserAgent. If I use a normal browser, such as safari or firefox or mozilla, my response includes several cookies. However, if I properly set a cookie jar with LWP::UserAgent and all, there are NO cookies in the response from the remote server. Any ideas why this might happen, and how I could rectify the situation? Thanks!
  • Comment on LWP::UserAgent and no cookies in my response?

Replies are listed 'Best First'.
Re: LWP::UserAgent and no cookies in my response?
by tlm (Prior) on Jun 09, 2005 at 02:23 UTC

    One thing I find useful when comparing the behavior of my browser to the behavior of a LWP::UserAgent object is to configure the object so that it mimics a real browser, because some servers will respond differently depending on the type of browser they think they are dealing with, and can be particularly unhelpful if they think they're dealing with a robot. (Yes: anti-mecha discrimination is rampant.) Try something like: 

    $ua->agent( 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/2
+0050223 Firefox/1.0.1' );

    [download]
    See the agent parameter in the docs for LWP::UserAgent. (BTW, the code given above works as-is even if you are using LWP::Simple, as long as you import the package variable $ua from LWP::Simple.)

    To determine what string to use, for Firefox I use the very handy Live HTTP headers.

    the lowliest monk

[reply]
[d/l]
      yeah, been there done that. Still no worky. I'm up to forging the cookies now.
[reply]
Re: LWP::UserAgent and no cookies in my response?
by inman (Curate) on Jun 09, 2005 at 15:59 UTC
    Where are you looking for the cookies? Only persistent cookies will be stored in a cookie jar. Have you checked the http header to see if they have been returned by the cookie?

    Can you post the code that you have tried witha sample data so that we can give it a go!

[reply]
      looking for Set-Cookie in the response headers. I see them in my safari cookie jar, and via ethereal when using safari, however not when using lwp. I was in the process of artificially forging all the headers that safari sends when I had one drink too many and had to shut the computer ;-)
[reply]

Back to Seekers of Perl Wisdom