in reply to Re^2: Combining Ffile:fetch with MySQL
in thread Combining Ffile:fetch with MySQL

my $dbh = DBI->connect($dsn, $user, $password)
  or die "Couldn't connect to database: " . DBI->errstr;

[download]

Here you test that the connection has succeeded and bail out if it hasn't. This is good. However, on the subsequent 4 $dbh->do statements you make no such tests. How can you tell if one or more of these have failed?

my $SQL = "select url,document_id FROM $tablename where left(publish_d
+ate,6) ='$date'";
my $query = $dbh->prepare($SQL) or die "prepare: ".$dbh->errstr;
$query-> execute() or die "execute: ".$dbh->errstr;

[download]

No reason at all not to use a placeholder here:

my $SQL = "select url,document_id FROM $tablename where left(publish_d
+ate,6) = ?";
my $query = $dbh->prepare($SQL) or die "prepare: ".$dbh->errstr;
$query-> execute($date) or die "execute: ".$dbh->errstr;

[download]
it runs with no errors, it just doesn't download

Does it print the URL correctly each time through the loop or not?

🦛

Replies are listed 'Best First'.
Re^4: Combining Ffile:fetch with MySQL
by afoken (Chancellor) on Jul 24, 2022 at 10:13 UTC
    However, on the subsequent 4 $dbh->do statements you make no such tests.

    ... and with a little bit of RTFM, none of those tests is needed. Change

    my $dbh = DBI->connect($dsn, $user, $password)
  or die "Couldn't connect to database: " . DBI->errstr;

    [download]

    to

    my $dbh = DBI->connect($dsn, $user, $password, { RaiseError => 1, Prin
+tError => 0 })
  or die "Couldn't connect to database: " . DBI->errstr;

    [download]

    and DBI will automatically check for errors, no manual checks needed.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
[reply]
[d/l]
[select]
Re^4: Combining Ffile:fetch with MySQL
by justin423 (Scribe) on Jul 24, 2022 at 15:28 UTC
    you are right. it doesn't print the url. it just exits after printing url: But the table has the correct url in the field url.

    so this is the code where there is an issue.

     
while ($url = $query->fetchrow_array()) {
   print "url: $row->{url}\n";
   # logic to get unique filename for output
     my $outfile = $path.$document_id;
   print $outfile;
   my $ff = File::Fetch->new( uri => url );
   $ff->fetch(to => $outfile) or die "failed to fetch '$url' to '$outf
+ile'.";

    [download]
[reply]
[d/l]
      ok. I am getting somewhere. here is the code: 
       while (my $ref = $query->fetchrow_hashref()) {
   print "url: $ref->{url}\n";

   my $ff = File::Fetch->new(uri=> '$ref' );
   my $where = $ff->fetch(to => '/data'| \$scalar) ;

      [download]

      when I run it with the where commented out, it prints the correct URL's, but errors out on each one with hostname required when fetching from ''

      running it with the $where uncommented, I get the same error, but it drops out on the first line of data with a can't call method Fetch on an undefined value.

[reply]
[d/l]
        my $ff = File::Fetch->new(uri=> '$ref' );

        You have potentially 2 errors here. Firstly you are enclosing $ref in single quotes which prevent interpolation. Just lose the quotes entirely. Secondly, my understanding is that you need to pass $ref->{url} as the argument to new() rather than just $ref. Try those fixes and see if you get any further.

        🦛

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom