Re^3: Occasional Read Timeout with Mech

If you're just making a simple get request and reading a web page, I'd go with IO::Socket::SSL. In general, I prefer to do things with as simple and basic of modules as possible. This way, when you do have bugs like you're having, there's far less code to sift through to find the source, and you also have less that can get in the way as bugs.

Give that a shot and report back. If you're still having problems, report back with more details on exactly what you're doing in terms of the URL you're grabbing (or one very similar) and how the rest is set up. I'll then try and recreate it on my end to see if we can unwind the issue.

Comment on Re^3: Occasional Read Timeout with Mech

Replies are listed 'Best First'.
Re^4: Occasional Read Timeout with Mech by noxxi (Pilgrim) on Dec 20, 2014 at 23:19 UTC
"...If you're just making a simple get request and reading a web page, I'd go with IO::Socket::SSL..." - I would advice against this. Getting a HTTP request correctly and especially parsing the response is more complex than it seems from looking at some examples, at least of you want to do it correctly. Especially chunked mode (length not known up-front), content-encoding (compression) and persistent connections (keep-alive) regularly cause problems. Also, LWP::UserAgent takes care of proxies, cookies etc.	[reply]
Re^4: Occasional Read Timeout with Mech by Anonymous Monk on Dec 20, 2014 at 20:32 UTC
um, WWW::Mechanize uses IO::Socket::SSL underneath ... going lower level like IO::Socket::SSL isn't very convenient .... timeouts happen	[reply]
Re^5: Occasional Read Timeout with Mech by benwills (Sexton) on Dec 20, 2014 at 20:49 UTC
Yes, timeouts happen. But five times in a row, at two different times a day, and only on the same days, seems like a very specific problem that shouldn't yet be discarded as "timeouts happen." And maybe not at first it's not as convenient. But it gets a ton of code out of the way that might be the problem. And right now, there's a problem that the OP doesn't completely understand. And if the OP wants something a little higher level, then go with something like LWP. But Mechanize is very over-featured for the task and we don't clearly know yet if it's Mechanize or the socket. The point of going lower level is to eliminate possibilities of what's causing the problem. Then, with everything out of the way, if the problem is still there, you have a clearer shot at what it might be. The OP's task as I understand it (grabbing a web page), is basic enough for this to be a worthwhile step in debugging, imo.	[reply]
Re^6: Occasional Read Timeout with Mech by Anonymous Monk on Dec 20, 2014 at 23:02 UTC
Yes, timeouts happen. But five times in a row, at two different times a day, and only on the same days, seems like a very specific problem that shouldn't yet be discarded as "timeouts happen." I've had that happen, connection reset from DSL provider, takes a minute or so to restablish DSL connection, then it works ... problem likely has nothing to do with server code or client code, its just internets then go with something like LWP. But Mechanize is very over-featured for the task and we don't clearly know yet if it's Mechanize or the socket. Mechanize is just very very thin layer on top of LWP The point of going lower level is to eliminate possibilities of what's causing the problem... Sure, I can agree with that :) but there is no need to write more code, just turn on debugging Re^5: HTTP error response code 500 using LWP::UserAgent on one site, but not on any other says use IO::Socket::SSL qw(debug3); Also, http://search.cpan.org/perldoc/Crypt::SSLeay#SSL_diagnostics_and_Debugging lists $ENV{HTTPS_DEBUG} = 1;	[reply]