Proper way to get redirected url from LWP::UserAgent response?

cormanaz has asked for the wisdom of the Perl Monks concerning the following question:

Hi all. I am trying to expand some links found in twitter. They are all t.co links. I have been able to do it with those with a cheat:

use LWP::UserAgent;
our $ua = LWP::UserAgent->new;
$ua->max_redirect(10);
my $twitter = ' https://t.co/NqACDPYdD9';
my $tiny = 'https://tinyurl.com/69rmnakp';
expand($twitter);
sub expand {
    my ($short) = @_;
     my $response = $ua->get($short);
    if ($response->is_success) {
            my $long = my $long = my $long = $response->header('refres
+h');
            $long =~ s/0\;URL\=//;
            return $long;
    }
    else {
        say $response->status_line;
    }    
}
[download]

But there is no $response->{_headers}->{refresh}; element for the tinyurl link. In fact, browsing through $response with my debugger I can't find anything that looks like the expanded link. I know it's a valid link because my browser expands it.

What is the proper way to get the penultimate redirected-to link from the $response object?

Comment on Proper way to get redirected url from LWP::UserAgent response? Select or Download Code

Replies are listed 'Best First'.
Re: Proper way to get redirected url from LWP::UserAgent response? by cormanaz (Deacon) on Jul 28, 2021 at 23:59 UTC
I think I worked this out. I found in some WWW::Mechanize docs that the proper way to query the redirect is using `$response->redirects()`. It seems that t.co does not return this header (anyone know why?) so you have to get it with the refresh parameter. But all the other shorteners (at least a slew of them I tested) do. So this code should work for most purposes. use strict; use feature ':5.10'; use LWP::UserAgent; our $ua = LWP::UserAgent->new; $ua->max_redirect(10); my @links = qw( https://tinyurl.com/69rmnakp https://t.co/NqACDPYdD9 https://bit.ly/3eikm4z http://apne.ws/HGpbLTW ); foreach my $link (@links) { say "$link > ".expand($link); } sub expand { my ($short) = @_; my $long; my $response = $ua->get($short); if ($response->is_success) { my @redirects = $response->redirects(); if (@redirects) { $long = $redirects[0]->header('location'); } elsif ($response->header('refresh')) { $long = $response->header('refresh'); $long =~ s/0\;URL\=//; } return $long; } else { return $short; } } [download]	[reply] [d/l] [select]
Re^2: Proper way to get redirected url from LWP::UserAgent response? by Corion (Patriarch) on Jul 29, 2021 at 06:06 UTC
This is mostly because HTTP::Response / HTTP::Message only concern themselves with HTTP redirect responses (`3xx` codes). A redirect using the `refresh` header could be an interesting addition, but it is distinct as it is (usually) served with a `2xx` code and not with a `3xx` code.	[reply] [d/l] [select]


Clear questions and runnable code get the best and fastest answer
	PerlMonks