perlquestion
jimhenry
<p>I'm writing a podcatcher and I ran into a problem where one particular podcast (whose RSS feed downloads fine in Firefox or wget) was failing to download when I used the default HTTP headers I used with all the other podcasts I've tested with. It would give a "406 Not Aceptable" error, which research suggested was caused by bad Accept headers. I ran wget -d to see what headers it was using and copied them into my script (or rather the config file for my script), and got the same error with this podcast. Below, find a simplified version of the code that reproduces the problem.</p>
<code>
#! /usr/bin/perl
use strict;
use warnings;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->agent('Mozilla/5.0');
$ua->show_progress( 1 );
my %headers = ( 'Accept' => '*/*',
'Accept-Encoding' => 'identity',
'Connection' => 'Keep-Alive',
# 'Host' => 'feed.podbean.com' );
'Host' => 'makeoursmarvel.com' );
#my $url = 'https://feed.podbean.com/betheserpent/feed.xml';
my $url = 'http://makeoursmarvel.com/feed/podcast/';
my $response = $ua->get( $url, %headers );
if ( $response->is_success ) {
print "Okay!\n";
} else {
my $status = $response->status_line;
print "failed to download $url: $status\n";
}
</code>
<p>If I use the commented out lines instead for the Host and $url value (checking a different podcast's RSS feed), everything works fine. I also tried using the default Firefox Accept: header, based on https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation/List_of_default_Accept_values, and got the same 406 error. The same podcast also give me a 406 error when I try to get an individual mp3 file.</p>
<p>Any ideas how to narrow the problem down further, if not fix it?</p>
<p>I'm using Perl v5.26.1 and LWP::UserAgent version 6.31.</p>