I've been working on a project now for a couple of months. This project involved polling a number of websites, and getting specific information from that page.

From the beginning of the coding, I was intent on using LWP to get the pages, but I ran into a bit of an issue. Though LWP does have a lot of functionality, I wansn't using most of it, and I ended up wasting a lot of CPU cycles waiting for LWP overhead.

So, I changed over to using HTTP::Lite. Speed improved greatly, cutting processing and waiting times almost in half. This is a good thing, since the project required the polling of multiple pages to occur simultaneously. It's an on-demand world and I have an on-demand-er client.

HTTP::Lite lacks one small bit of functionality that I needed, though. A way to check for a timeout. What to do? Do I muddle through without it, and have some occasions with 15-20 second waits for a page? Or do I go back to using the slower, but more reliable LWP?

Answer: Neither.

The timeout factor for page loading is something that I was already considering in my code, and was contained in a variable. Since I could pass that to LWP, why not use it in another way for HTTP::Lite.

Here is the HTTP::Lite with a timeout : 

sub HTTP_Request {
    (undef, $timeout, $url) = @_;
    $http = new HTTP::Lite;
    $st = time();
    until ((time()-$st > $timeout) ||
           ($req = $http->request($search))) { sleep(.1); }
    if ($req ne "200") {
        if ($req eq "") { return "Error: Timeout"; }
        else { return "Error: $http->status_message(); }
    } else { return $http->body(); }
}

[download]
Usual caveats apply here, such as declaring variables and 'use' statements. Bonus here is that with adding the sleep(.1), the code acutally runs faster.

In reply to Making Timeout for Yourself by mcogan1966

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.