jayto has asked for the wisdom of the Perl Monks concerning the following question:

So, I have a new problem. I am trying to validate large amounts of links which I have scraped. These links are supposed to lead to mp3 files.
My problem is that sometimes I get links like these:
http://writeups.info/new-age/na09/Relaxing%20Songs%20Music%20-%20Wind%20Beneath%20My%20Wings.mp3

This is a real valid page I guess but it is not a link to an mp3 file so LWP UA doesnt timeout. What can I do to verify that links such as that ^^, are actual links to mp3 files?

Replies are listed 'Best First'.
Re: Link validation problem
by tobyink (Canon) on Jul 17, 2012 at 19:04 UTC 
    my $response = LWP::UserAgent::
  -> new
  -> head("http://writeups.info/new-age/na09/Relaxing%20Songs%20Music%
+20-%20Wind%20Beneath%20My%20Wings.mp3");
if ($response->is_success and $response->content_type =~ m{^(?:audio|a
+pplication)/(?:x-)?(?:mp3|mpeg)$})
{
    say "It's probably an MP3!";
}

    [download]
    perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'
[reply]
[d/l]

      Thanks that worked!

[reply]
Re: Link validation problem
by frozenwithjoy (Priest) on Jul 17, 2012 at 18:49 UTC
    It seems that the problem with that link is that it redirects you. You could disable redirects (or set max redirects to 0). See  max_redirect and requests_redirectable in LWP::UserAgent and section 3.4.4 here.
[reply]
[d/l]
[select]
Re: Link validation problem
by onelesd (Pilgrim) on Jul 17, 2012 at 19:01 UTC
    You'll probably want to set your User-Agent: header to something browser-ish. "Firefox" is usually a safe choice.
[reply]

Back to Seekers of Perl Wisdom