Hi all,

I have on very many occasions found help on from you Monks, but this is the first time I am stumped and can't find anything similar to help. I am trying to download files using File::Fetch. The program I was using had worked without issue several times in the past, including up to the day before yesterday. Now, when I try to download a file I get error messages like: Fetch failed! HTTP response: 400 Bad Request [400 Bad Request]  at ...

I am wondering if it is my setup or possibly the web server (this is the SEC's XBRL RSS feed). I have tried this code on two different machines (home and work) and I still get the error most of the time (not all the time). The weird thing is that the file does indeed download (I have yet to check if the file is 100% accurate). This happens to both the RSS feeds as well as other documents from the same server and the links seem to work fine in a browser. File::Fetch seems to work OK with other sites, so I am guessing it's the SEC's server, but I'd like to be sure that is the case and, if possible, know why the bad response is being given. I was hoping someone could add light onto my situation. Here is the relevant code that produces the error.

use warnings;
use strict;
use File::Fetch;

unlink '/tmp/xbrltest/xbrlrss-2015-09.xml';
unlink '/tmp/xbrltest/xbrlrss-2015-10.xml';
unlink '/tmp/xbrltest/xbrlrss-2015-11.xml';

if (-e '/tmp/xbrltest/xbrlrss-2015-09.xml') {
    print "File 09 Exists\n";
} else {print "Need to Download 09\n";}

if (-e '/tmp/xbrltest/xbrlrss-2015-10.xml') {
    print "File 10 Exists\n";
} else {print "Need to Download 10\n";}

if (-e '/tmp/xbrltest/xbrlrss-2015-11.xml') {
    print "File 11 Exists\n";
} else {print "Need to Download 11\n";}





{
    print "Fetching File with URL in double quotes\n";
    my $url="http://www.sec.gov/Archives/edgar/monthly/xbrlrss-2015-09
+.xml";
    my $ff  = File::Fetch->new(uri=>$url) or die("Something went wrong
+ in fetching file: $!");
    my $where = $ff->fetch(to=>'/tmp/xbrltest/') or die $ff->error;
}

{
    print "Fetching File with URL in single quotes\n";
    my $url   = 'http://www.sec.gov/Archives/edgar/monthly/xbrlrss-201
+5-10.xml';
    my $ff    = File::Fetch->new(uri=>$url) or die("Something went wro
+ng in Fetching RSS Feed: $!");
    my $where = $ff->fetch(to=>'/tmp/xbrltest/') or die $ff->error;
}

{
    print "Fetching File with URL built from parts\n";
    my $year  = '2015';
    my $month = '11';
    my $site = 'http://www.sec.gov/Archives/edgar/monthly/';
    my $rssFile = "xbrlrss-" . $year . "-" . $month . ".xml";

    my $url   = $site . $rssFile;
    my $ff    = File::Fetch->new(uri=>$url) or die("Something went wro
+ng in Fetching RSS Feed: $!");
    my $where = $ff->fetch(to => '/tmp/xbrltest/') or die $ff->error;
}


if (-e '/tmp/xbrltest/xbrlrss-2015-09.xml') {
    print "Download 09 Successful\n";
} else {print "Download 09 Failed\n";}

if (-e '/tmp/xbrltest/xbrlrss-2015-10.xml') {
    print "Download 10 Successful\n";
} else {print "Download 10 Failed\n";}

if (-e '/tmp/xbrltest/xbrlrss-2015-11.xml') {
    print "Download 11 Successful\n";
} else {print "Download 11 Failed\n";}


print "Done!\n"

[download]

Thanks,

Joe

In reply to HTTP response: 400 Bad Request by JoeJohnston

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.