Site that I'm parsing with perl script recently moved to human-readable urls. I'm trying to get redirects from "id" requests to current addresses. For example, when I'm going to "http://www.giftman.ru/show.php?id=294" in browser, site server redirects it to "http://www.giftman.ru/catalog/amulets/"

But when I'm trying to get this moved location in my script, I don't get 301 answer, it returns "200 OK" for some reason.

Script:

use LWP::UserAgent; use HTTP::Cookies; use HTTP::Headers; $ua = LWP::UserAgent->new; $hh = HTTP::Headers->new( User-Agent => 'Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20100101 +Firefox/21.0', Accept => 'text/html,application/xhtml+xml,application/xml;q=0.9,*/* +;q=0.8', Accept-Language => 'en-us,en;q=0.7,ru;q=0.3', Accept-Encoding => 'gzip, deflate', Connection => 'keep-alive', ); $ua->default_headers( $hh ); $cookie_jar = HTTP::Cookies->new( ); $ua->cookie_jar($cookie_jar); @rename = ( 294 , 9806 , 9807 , ); for $ren (@rename) { $res = $ua->get("http://www.giftman.ru/show.php?id=$ren"); print $res->header('Location')."\n"; }

I used http sniffer to see what's going on with browser, and there's nothing special, really:

GET /show.php?id=294 HTTP/1.1 Host: www.giftman.ru User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20100101 Firef +ox/21.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0. +8 Accept-Language: en-us,en;q=0.7,ru;q=0.3 Accept-Encoding: gzip, deflate Cookie: __utma=83753984.1287093182.1370328704.1371539232.1371576574.7; + __utmz=83753984.1370328704.1.1.utmcsr=(direct)|utmccn=(direct)|utmcm +d=(none); __utmb=83753984.10.10.1371576574; _ym_visorc=w; PHPSESSID=4 +p0ql1mitskhkbg3os47v1hc11; __utmc=83753984 Connection: keep-alive

By accident I stumbled on this: if I use $ua->get("http://www.giftman.ru/show.php?id=$ren 0"), i.e. space and some symbols after URL string, I'm getting completely different response, and there it is, "301 moved" and new location.

I can't understand what's happening.


In reply to Different answers for script and browser (LWP) by Sly_G

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.