I have a LWP type program using Mechanize that has been running every hour for the past 7 years without problems. All of a sudden, I get an error report that I determined resulted because of a corrupted DB - a duplicate value appeared in a column where all the values should be unique. The code is specifically designed to prevent this. However, I theorized that if 2 instances of the program where running, they could "fight" and cause this problem. But I didn't see how that could possibly happen, until I saw this in the log file: 
2022-08-25 04:21:18|Good record fetched here
2022-08-25 04:21:23|Error: Retry Attempt 1 of 3
2022-08-25 05:45:16|Error: Retry Attempt 2 of 3
2022-08-25 05:45:53|Next good page fetched here

[download]
Retry #1 took about 90 minutes!! WOW!

Here is what the ancient code does: 

my $success=0;
my $tries=0;
while (! $success and $tries++ < 3) 
{
    eval { $m2->get($fullurl); };
    if (! $@) {
        $success = 1;
    }
    else {
        print STDERR "".cur_gmt()."Error: Retry Attempt $tries of 3\n"
+;
        print LOG    "".cur_gmt()."Error: Retry Attempt $tries of 3\n"
+;
        sleep (2);
    }
}

die "aborted Web Site Error: $!" unless $success; #ultimate failure!! 
+PROGRAM ABORT !!!!

[download]
The retry ultimately succeeds but the long wait time has the effect of pushing the run time into the next hour's timeslot.

This is a normal HTTP (not HTTPS) Url. Over the years, there could be that a lot of things that have changed at the website's end. I have no idea. The default timeout for Mechanize is supposed to be about 3 minutes - it really doesn't matter to me as long as it is not measured in hours! I don't know how often this super long request problem happens. A retry historically happens about every 2-3K requests with this particular site and a couple seconds later, all is well. I have no idea what is actually causing the hang.

Thoughts and ideas are welcome.

In reply to WWW::Mechanize timeout problem by Marshall

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.