IB2017 has asked for the wisdom of the Perl Monks concerning the following question:

Hello

I use the following to retrieve from server the version number of the latest available update of my software.

my $ua = LWP::UserAgent->new; my $req = HTTP::Request->new(GET => $url); my $res = $ua->request($req); $ua->timeout(0.1);

This works fine execpt if there is a connection problem. For example, the server has been down for the whole weekend now (!!!). In such a case, my GUI application - while running this peace of code - freezes for ca. 15 seconds. This is not acceptable. What I want is that if there is no response from the server, no matter the reasons, the application just skip this check. I tried to replace LWP::UserAgent with LWPx::ParanoidAgent, but this did not lead to any improvement. Any other idea?

Replies are listed 'Best First'.
Re: LWP::UserAgent timeout
by bliako (Abbot) on Dec 17, 2018 at 13:36 UTC

    1. LWP::UserAgent's default timeout is 180 seconds.

    2. You set the timeout of your UserAgent AFTER you made the request. Obviously, it has no effect for the previous request ... (maybe use LWP::UserAgent::Pscychic for such cases? - just a joke :) )

    3. Your GUI app is bound to freeze while conducting the server for long or short time. HTTP::Async can help you here, it will spawn a request in the background and when you are ready, you check if it has borne any fruits, like so (mostly borrowed from the manpage):

    use strict; use warnings; use HTTP::Async; use HTTP::Request; my $url = ...; my $async = HTTP::Async->new; $async->add(HTTP::Request->new(GET => $url)); while ( $async->not_empty ) { if ( my $response = $async->next_response ) { # we have a reply from server or timeout # deal with $response } else { # still waiting for a response from server, # so do something else } }

    bw, bliako

      I'd just add that, in either case, given that you are dealing with a GUI, it's usually a best practice to use a separate thread for the UI events processing and another for the processing:

      1. LWP::UserAgent::timeout() it's a lower bound: "The requests is aborted if no activity on the connection to the server is observed for "timeout" seconds.This means that the time it takes for the complete transaction and the request() method to actually return might be longer."
      2. Depending on what you do when you # deal with $response there are good chances you'll end freezing again your UI

        Multi-threads, i.e. separating UI and other stuff, make sure sense. However, it frightens me turning my application in a multi-threads one. My GUI is Tk and Tk is said to be "not thread safe". Zero experience on my side.

      Oh thanks. Anticipating the timeout declaration to:

      my $ua = LWP::UserAgent->new; $ua->timeout(0.1); my $req = HTTP::Request->new(GET => $url); my $res = $ua->request($req);

      makes it work. However, the idea to make it async is for sure a better idea. I'll give it a try.

Re: LWP::UserAgent timeout
by talexb (Chancellor) on Dec 17, 2018 at 14:35 UTC

    I use LWP::UserAgent::Determined to retry my API calls when they fail. The standard retry intervals for this module are 1, 3 and 15 seconds -- that wasn't enough for my situation, so I made mine 10, 20, 30, 60, 60, 60, 90, and 90 seconds. My preference is to "Get it done .. eventually." rather than "Get it done as quickly as possible." Your situation may be different.

    Alex / talexb / Toronto

    Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

      Thank you for your comment. In my particular case, I simply prefer to "go on" without retrying if there is a connection problem. It is just the app automatically check if an update is available. If this check cannot be done, it does not matter. It will retry at a later stage (typically when the app is started again). What I do not want it is freezing the app while trying and trying again (as I do not use - unfortunately - any non-blocking mechanisms).

Re: LWP::UserAgent timeout
by markong (Pilgrim) on Dec 17, 2018 at 13:20 UTC

    Are you sure you can pass a fractional number to timeout() ?

    This LWP::UserAgent::Paranoid has a similar method which accepts fractional values, but their semantics are different: LWP::UserAgent checks that there has been no socket activity for timeout seconds.
    You could always double check with wireshark that is effectively so.

    Otherwise you can always fork() another process for the HTTP session(s) and avoid freezes (doesn't your GUI framework provides already something similar?)

Re: LWP::UserAgent timeout
by perlancar (Hermit) on Dec 19, 2018 at 11:23 UTC