in reply to LWP::UserAgent, redirection does not work

LWP::UserAgent does not interpret META tags like your browser does. Also, it does not follow 30x-style redirects unless you tell it to.

You don't show us what result code the first ->get() returns. Maybe it's 200, maybe it's 301 or 302. You need to react to those accordingly - either by parsing the returned HTML page for META tags, or by telling LWP::UserAgent to follow the redirect.

Also, if you're automating a website, WWW::Mechanize acts more like a browser than LWP::UserAgent does.

Replies are listed 'Best First'.
Re^2: LWP::UserAgent, redirection does not work
by ikegami (Patriarch) on Oct 08, 2007 at 17:23 UTC

    Also, it does not follow 30x-style redirects unless you tell it to

    It's true that it won't parse the HTML response to look for and interpret META elements, but it does handle 3xx reponses automatically (for GET and HEAD queries by default). 

    use LWP::UserAgent qw( );

my $response = LWP::UserAgent->new()->get('http://www.w3.org/html');
for (;;) {
   my $uri    = $response->request()->uri();
   my $status = $response->status_line();
   print("$uri => $status\n");

   $response = $response->previous();
   last if !defined($response);

   print("as a result of\n");
}

    [download]
    http://www.w3.org/html/ => 200 OK
as a result of
http://www.w3.org/html => 301 Moved Permanently

    [download]
[reply]
[d/l]
[select]
Re^2: LWP::UserAgent, redirection does not work
by dibucho (Initiate) on Oct 14, 2007 at 13:19 UTC
    Thanks for your reply, I just checked the result code of the first get(), it's 200. But something is strange with the redirection link,it seems like an infinitive redirection cycle, because the site redirects on itself. But if I use a browser like Firefox or IE the redirection works fine. Any idea? bye dibucho
[reply]

      As always when automating things and finding different behaviour between other programs, you need to find out where exactly the differences lie. My first guess would be that you're not sending the expected Referer (sic) header. But only a network trace like the HTTP Live Headers or Wireshark produce can tell for sure.

      Maybe WWW::Mechanize already works for you as it mimics browser behaviour far closer than LWP::UserAgent does of itself.

[reply]
[d/l]
        I traced the headers with a firefox plugin for the requested page: 
        GET /livelink_en/livelink/fetch/2000/customview.html?func=llworkspace 
+HTTP/1.1

Host: tsi-myworkroom.telekom.de

User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/
+20070725 Firefox/2.0.0.6

Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9
+,text/plain;q=0.8,image/png,*/*;q=0.5

Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip,deflate

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Keep-Alive: 300

Connection: keep-alive

Cookie: LLCookie=vDF63XqtCC4%3D; LLTZCookie=0; SMSESSION=caPsWnnX9ytaP
+TWAYPBJ+2xhOj0oFmloJFw+cXCJr3g9IB1OIYg7HTAEWpgkmEVgxQEreXnHGNO7Abczo7
+ssm+cVc2rrlcZcGWnbdt9q48X3r2xW6pV64fOyyfz1hWT4Z9vS7Tsmmae+hObhK7QL+QE
+5iYCbQmqeOn3PMlM08v5x9wWhB5a0QnArS5nMz0aAJek74dMM5+l5Mh2MQfEGjWNZdYTl
+2N656AnddQVskgWKkHcj+q3pXAO6kA1HBgz786quiy3Lb3xkRiyQMD754Oa7kr+EBRO6e
+xG0QPzGV5GGcDZmpbGMzz3GTxDsmKm6ZJMysCm9CiowVFkPQFf5RAdhkNgDFWxJGa6E8s
+VjW0/IjL6o8mlnXO2mOC7AGeVp5QnYE28ibKLlyLJoopEFDR1G8TKUNOrL6FtXooYleTw
+Fj7oZgdR1vguCQi677j7BckZsMQXQJWKWZ4kw30kk5t/w2Ff8LkKyU2LXDQgktjPso2iD
+E370bT3lHk4c0EOtx8gKvRrChwvvPElmI5Kz6NjBMo6nnzw0Hzu/lL3h190BeLwqFQ2YK
+qdYNz+xc9SWxAvY7n5BVxy6wqRCtaWlGS7fXe4l7iECMzKUff8/Q61j2IYSQ38Xt1DgIY
+hyvWYBTTCCxAkFS9D+CHFJNGI9p0X88+9QU0GZ1gyAx/oyKp6fszPH0M2HMUlm3IO6Ctn
+JbypG9mAhZIFa8G5o6FbNFnwUO6LqqyJBu/clxNRFkrV2mfn4DXZN5eFJnF8Uf8PYD7cz
+0jwc3YliaiHIfPWC5ppzctoZTth+V/OH+rYOBsYXPVroPNPpkYxf7gJAoGMPD26DbR6VK
+v1nohNIXYY0wVqa9gqqsRwaFkkMxlXG9oCxDs1YeA3IQgAZbo8xE5ISSotVqK0X+u59HP
+g8k3sqPVv2ZiJH8bm8l/UPa5xfdIpG+K2GLClipnKh6gAhZDCZbW6se2IE4eGSwSHmnEa
+dewjoopX4RH7okoSFbSkZBPK81sJwrKVIwwhZ+vjfPGOaDbaWVd4IMN0=; JSESSIONID
+=6809B0E13E9DEBF6265377F798EAD315; AtlasURLCookie=https%3A%2F%2Ftsi-m
+yworkroom.telekom.de%2Flivelink_en%2Flivelink%2Ffetch%2F2000%2Fcustom
+view.html%3Ffunc%3Dllworkspace

Cache-Control: max-age=0

        [download]
        nothing strange so far.. But I noticed something in the response header which is given back to the request of my perl script. 
        Connection: close
Date: Thu, 18 Oct 2007 10:44:48 GMT
Via: 1.1 G8NTQV01.bmbg01.telekom.de (Alteon iSD-SSL/3.1.6.2)
Server: Sun-ONE-Web-Server/6.1
Content-Type: text/html
Expires: -1
Client-Date: Thu, 18 Oct 2007 10:44:49 GMT
Client-Peer: 164.29.91.244:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /C=DE/O=T-Systems Enterprise Services GmbH/OU=
+Trust Center Deutsche Telekom/CN=Deutsche Telekom CA 5
Client-SSL-Cert-Subject: /O=T-Systems CDS GmbH/OU=CS Unit 7/L=Bamberg/
+ST=Bayern/C=DE/CN=tsi-myworkroom.telekom.de
Client-SSL-Cipher: DES-CBC3-SHA
Client-SSL-Warning: Peer certificate not verified
Link: </livelink_ensupport/livelink.css>; rel="stylesheet"; type="text
+/css"
Refresh: 0; URL=/livelink_en/livelink/fetch/2000/customview.html?func=
+llworkspace
Set-Cookie: LLInProgress=; Expires=Sun, 01 Jan 1995 08:00:00 GMT; path
+=/livelink_en/
Set-Cookie: LLCookie=vDF63XqtCC4%3D; path=/livelink_en/
Set-Cookie: LLTZCookie=0; path=/livelink_en/
Title: Livelink - Redirection

        [download]
        It says "Client-SSL-Warning: Peer certificate not verified", could this be the reason? And what should I do next? thanks! dibucho
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom