Site won't give cookie with LWP::UserAgent

casimo has asked for the wisdom of the Perl Monks concerning the following question:

For the life of my I can't get a cookie from the following code/site using LWP::UserAgent and HTTP::Cookies. It's frustrating because I get cookies at this url in Firefox and I get cookies with this code elsewhere.

Any ideas?

(sorry for breaking the URL up but didn't want to post it directly)

use LWP::UserAgent;
use HTTP::Cookies;

$letter1 = "d";                                                       
+                               
$letter2 = "r";
$letter3 = "f";

$ua = LWP::UserAgent->new;
$ua->cookie_jar(HTTP::Cookies->new(file => "lwpcookies.txt", autosave 
+=> 1, ignore_discard => 1));                                         
+                                             
$req = HTTP::Request->new( GET => "http://www.$letter1$letter2$letter3
+.com" );
print $req->as_string, "\n";
$response = $ua->request($req);
print $response->as_string;
[download]

Comment on Site won't give cookie with LWP::UserAgent - HTTP::Cookies Download Code

Replies are listed 'Best First'.
Re: Site won't give cookie with LWP::UserAgent - HTTP::Cookies by ikegami (Patriarch) on Jan 23, 2011 at 03:49 UTC
I get cookies at this url in Firefox No, no cookies with Firefox either. GET / HTTP/1.1 Host: www.drf.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.1 +3) Gecko/20101203 Firefox/3.6.13 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0. +8 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,;q=0.7 Keep-Alive: 115 Connection: keep-alive X-Behavioral-Ad-Opt-Out: 1 X-Do-Not-Track: 1 HTTP/1.1 200 OK Date: Sun, 23 Jan 2011 03:47:06 GMT Server: Apache Last-Modified: Sun, 23 Jan 2011 03:28:59 GMT Accept-Ranges: bytes ntCoent-Length: 84954 Expires: Sun, 19 Nov 1978 05:00:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre- +check=0 Connection: close Content-Type: text/html; charset=utf-8 Content-Encoding: gzip Transfer-Encoding: chunked [download] Perhaps one of the images on that page returns a cookie, but not that page. Update*: No, it's not the images either. It's only when I allowed scripts that I got a cookie. So it's either set via some JavaScript or via Flash.	[reply] [d/l]
Re^2: Site won't give cookie with LWP::UserAgent - HTTP::Cookies by casimo (Sexton) on Jan 23, 2011 at 16:28 UTC
Thanks for all the help guys. Every reply was accurate and helpful. Ultimately my issue was not about cookies but headers. I appreciate all the replies.	[reply]
Re: Site won't give cookie with LWP::UserAgent - HTTP::Cookies by PeterPeiGuo (Hermit) on Jan 23, 2011 at 02:21 UTC
As for those cookies that you saw in your browser, they didn't come with this particular URL. Your browser sent out multiple requests not one. Peter (Guo) Pei	[reply]
Re: Site won't give cookie with LWP::UserAgent - HTTP::Cookies by Anonymous Monk on Jan 23, 2011 at 02:22 UTC
use WWW::Mechanize;, compare the headers firefox sends to those Mechanize sends, and adjust your mechanize code to mimic firefox	[reply]

Peter (Guo) Pei