use DBI;
use threads;
my $dbh = DBI->connect(...);
my $sth = $dbh->prepare("SELECT ...");
$sth->execute();
$thr1 = threads->new(\&some_sub, $sth);
$thr2 = threads->new(\&some_sub, $sth);
$sth->finish();
$dbh->disconnect();
$thr1->join; $thr2-join;
sub some_sub {
my ($db_source) = @_;
while (my $ref = $sth->fetchrow_hashref()) {
... do some processing there ...
}
}
The reason for this is to create a web scrapper which runs several threads each using different proxy/network interface and each thread sleeps some time between requests to avoid ban.
I was thinking about Threads::Queue object first, but think it would be inefficient in terms of memory usage if database returns large set (because threads pause between requests a lot of data will be in memory for long time waiting for one of threads to dequeue them); In reply to Sharing DBI between threads by bagent
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |