How do I programatically follow redirects?

bwarn has asked for the wisdom of the Perl Monks concerning the following question:

I'm checking to see if a site exists (I'm using LWP::UserAgent), is up, etc. before directing a user to a site. Unfortunately, if the destination site itself redirects users with a query (i.e., uses ? in the URL), my program bombs.

Does anyone have any suggestions how to successfully follow redirects?

$ua = new LWP::UserAgent;
$request = new HTTP::Request('HEAD', "$URLVAL{URL}");
($response) = $ua->request($request);
if ($response->is_success) {
  print "Location:$URLVAL{URL}\n\n"; # quits if destination site redir
+ects using '?' [or encoded equivalent] in its redirection script
}
else {
  # send user to another cgi that displays some 'failed URL' text.
}
[download]

Originally posted as a Categorized Question.

Comment on How do I programatically follow redirects? Download Code

Replies are listed 'Best First'.
Re: How do I programatically follow redirects? by gunder (Novice) on Apr 04, 2003 at 09:37 UTC
I would maybe try a different approach by accessing site information via TCP sockets? Using (use Socket) you cn verify an alive site by grabing some info from the page if you like... hell I have never tried this, but it should work right? Let me know if it does... There is a wrapped up IO class called IO::Socket::INET that should for websites... try `$client = IO::Socket::INET->new("PeerAddr => "the.host.com", PeerPort +=> 80, Type => SOCK_STREAM, Timeout => (How ever long you think it sh +ould wait for a reply)) or die $@;` [download] Then (in case you want to get some data from the connected socket) `bind(SOCKET, $client) or die "bind: $!";` [download] if you get this far without error, then the site should be up and you can read from that socket and should be able to pick out bytes of the HTML markup... I think... Read more on sockets to get ideas on how to read via `$client->recv($data_read, $flags) or die...`	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re: How do I programatically follow redirects?
by gunder (Novice) on Apr 04, 2003 at 09:37 UTC

There is a wrapped up IO class called IO::Socket::INET that should for websites...

try

$client = IO::Socket::INET->new("PeerAddr => "the.host.com", PeerPort 
+=> 80, Type => SOCK_STREAM, Timeout => (How ever long you think it sh
+ould wait for a reply)) or die $@;
[download]

bind(SOCKET, $client) or die "bind: $!";
[download]

Read more on sockets to get ideas on how to read via $client->recv($data_read, $flags) or die...

[reply]
[d/l]
[select]