Re^5: Occasional Read Timeout with Mech

Yes, timeouts happen. But five times in a row, at two different times a day, and only on the same days, seems like a very specific problem that shouldn't yet be discarded as "timeouts happen."

And maybe not at first it's not as convenient. But it gets a ton of code out of the way that might be the problem. And right now, there's a problem that the OP doesn't completely understand. And if the OP wants something a little higher level, then go with something like LWP. But Mechanize is very over-featured for the task and we don't clearly know yet if it's Mechanize or the socket.

The point of going lower level is to eliminate possibilities of what's causing the problem. Then, with everything out of the way, if the problem is still there, you have a clearer shot at what it might be. The OP's task as I understand it (grabbing a web page), is basic enough for this to be a worthwhile step in debugging, imo.

Comment on Re^5: Occasional Read Timeout with Mech

Replies are listed 'Best First'.
Re^6: Occasional Read Timeout with Mech by Anonymous Monk on Dec 20, 2014 at 23:02 UTC
Yes, timeouts happen. But five times in a row, at two different times a day, and only on the same days, seems like a very specific problem that shouldn't yet be discarded as "timeouts happen." I've had that happen, connection reset from DSL provider, takes a minute or so to restablish DSL connection, then it works ... problem likely has nothing to do with server code or client code, its just internets then go with something like LWP. But Mechanize is very over-featured for the task and we don't clearly know yet if it's Mechanize or the socket. Mechanize is just very very thin layer on top of LWP The point of going lower level is to eliminate possibilities of what's causing the problem... Sure, I can agree with that :) but there is no need to write more code, just turn on debugging Re^5: HTTP error response code 500 using LWP::UserAgent on one site, but not on any other says use IO::Socket::SSL qw(debug3); Also, http://search.cpan.org/perldoc/Crypt::SSLeay#SSL_diagnostics_and_Debugging lists $ENV{HTTPS_DEBUG} = 1;	[reply]

Replies are listed 'Best First'.

Re^6: Occasional Read Timeout with Mech
by Anonymous Monk on Dec 20, 2014 at 23:02 UTC

Yes, timeouts happen. But five times in a row, at two different times a day, and only on the same days, seems like a very specific problem that shouldn't yet be discarded as "timeouts happen."

I've had that happen, connection reset from DSL provider, takes a minute or so to restablish DSL connection, then it works ... problem likely has nothing to do with server code or client code, its just internets

then go with something like LWP. But Mechanize is very over-featured for the task and we don't clearly know yet if it's Mechanize or the socket.

Mechanize is just very very thin layer on top of LWP

The point of going lower level is to eliminate possibilities of what's causing the problem...

Sure, I can agree with that :) but there is no need to write more code, just turn on debugging

Re^5: HTTP error response code 500 using LWP::UserAgent on one site, but not on any other says

use IO::Socket::SSL qw(debug3);

Also, http://search.cpan.org/perldoc/Crypt::SSLeay#SSL_diagnostics_and_Debugging lists $ENV{HTTPS_DEBUG} = 1;

[reply]