Hello Monks, and thank you for feedbacks.
Luckily, I managed to solve my performance issues, I had to add a couple of VMW parameters, I also updated guest OS and open-vm-tools to last unstable release.
Now scraper is doing well, not as fast as phyisical, but about 50% that is acceptable for me.
I have a question: I managed to see many error coming from this piece of code..
sub wait_until_xpath(){
my $xpath=shift;
my $retries = 90;
while ($retries-- and ! $mech->is_visible( xpath => $xpath )) {
print "wax waited..\n";
usleep(50*$msec);
};
warn "wait_until_xpath(): timeout at retry #$retries" if 0 > $retr
+ies;
}
I now changed it inside an eval block, and it magically fixed it..It works but I don t know why! :). Please comment
sub wait_until_xpath(){
eval{
my $xpath=shift;
my $retries = 90;
while ($retries-- and ! $mech->is_visible( xpath => $xpath )) {
print "wax waited..\n";
usleep(50*$msec);
};
warn "wait_until_xpath(): timeout at retry #$retries" if 0 > $retr
+ies;
}
}
|