I try to download filings from SEC website using the code provided on the website: http://www.wrds.us/index.php/tutorial/view/26 I received an error "No such file or directory at FILE.pl line 45, <dlthis> line 6868." I have a text file containing url links to the forms that I require. There are 6867 url and one header. So in total there are 6868 lines in the txt file. I am new to Perl. Any help is much appreciated. 

#!/usr/bin/perl
use LWP;
use HTTP::Request;
sub get_http
{
    my $url = shift;
    my $request = HTTP::Request->new(GET => $url);
    my $response = $ua->request($request);
    if (!$response->is_success)
    {
        print STDERR "GET '%s' failed: %s\n",
            $url, $response->status_line;
        return undef;
    }
    return $response->content;
}
# user agent object for handling HTTP requests
my $ua = LWP::UserAgent->new;
 
# if you only want a portion of the filing, un-comment the next line
#$ua->max_size(50000);  # 50k byte limit
 
######################### write dir , use "\\" and not "\", for exampl
+e: "C:\\temp"
$write_dir = "C:\\temp";
######################### write dir
 
######################### filename with urls (put in same directory as
+ script)
open dlthis, "c_10K_list.txt" or die $!;
######################### filename with urls (put in same directory as
+ script)
 
######################### log
open LOG , ">download_log.txt" or die $!;
######################### log
 
my @file = <dlthis>;
 
foreach $line (@file) {
 #CIK, filename, blank is not used (included because it will capture t
+he newline)
 ($CIK, $get_file, $blank) = split (",", $line);
 $get_file = "http://www.sec.gov/Archives/" . $get_file;
 $_ = $get_file;
  
 if ( /([0-9|-]+).txt/ ) {
   $filename = $write_dir . "/" . $CIK . ".txt";
   open OUT, ">$filename" or die $!;
   print "file $CIK \n";
 
   my $request = HTTP::Request->new(GET => $get_file);
   my $response =$ua->get($get_file );
   $p = $response->content;
   if ($p) {
    print OUT $p;
    close OUT;
   } else {
    #error logging
    print LOG "error in $filename - $CIK \n" ;
   }
 } 
}
close LOG;

[download]

In reply to No such file or directory error by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.