stevieb has asked for the wisdom of the Perl Monks concerning the following question:

I'm beginning the process of writing unit tests for Tesla::API, and want to test my timeout retry routine. By default, if calling Tesla's API time's out (ie. 500), it'll retry the call two more times.

This appears to work, but I want to test it. I can't mock code() or is_success() etc, because prior to running the code posted below, there are authentication routines that run that need to return proper codes and other pertinent information.

Essentially, how do I mock a mechanize timeout?

    my $url = URI->new(URL_API . $uri);

    my $header = ['Content-Type' => 'application/json; charset=UTF-8']
+;

    if ($auth) {
        my $token_string = "Bearer " . $self->_access_token;
        push @$header, 'Authorization' => $token_string;
    }

    my $request = HTTP::Request->new(
        $type,
        $url,
        $header,
        JSON->new->allow_nonref->encode($api_params)
    );

    my $response;

    for (1 .. API_TIMEOUT_RETRIES) {
        # If a timeout (ie. code 500) occurs, repeat the API call

        $response = $self->mech->request($request);

        if ($response->is_success) {
            my $response_data = _decode($response->decoded_content)->{
+response};

            $self->_cache(
                endpoint => $endpoint_name,
                id       => $id,
                data     => $response_data
            );

            return $response_data;
        }
        elsif ($response->code == 500) {
            next;
        }
    }

[download]

Thanks!

-stevieb

Replies are listed 'Best First'.
Re: Forcing WWW::Mechanize to timeout
by Corion (Patriarch) on Apr 12, 2022 at 19:17 UTC

    Do you really need to run through the authentication gauntlet just to verify that your retry logic applies?

    I would set up the test so that the test program does not have to do any of the authentication dance and you simply check that the HTTP timeout occurs.

    I find that structuring my code to be modular in the way that I can test functionality in separate, I recognize the relevant parts and can move these into testable subroutines.

[reply]
Re: Forcing WWW::Mechanize to timeout
by Marshall (Canon) on Apr 12, 2022 at 23:32 UTC
    Hi Stevieb!

    As a suggestion, your code that mentions "500" could be simplified. 

    for (1 .. API_ERRORS) {
    # If a timeout (ie. code 500) occurs, or
    # some kind of other error, repeat the API call

    $response = $self->mech->request($request);

    if ($response->is_success) {
        # sucessful processing  code here...
        return $response_data;
    }
    # all non-success code cause pause, then re-try
    # elsif ($response->code == 500) {
    #   next;
    #} 
    sleep(1);
}

    [download]
    Some sites will give a 404 or whatever and then a repeat of the same request will succeed. I don't see the need to differentiate between 500 and some other kind of error, at least not in the code that you show. For some sites, I have found that a brief pause "sleep (1)" helps. Sometimes in very, very rare situations, mechanize just gets "stuck" and no amount of retries will fix it.

    Update: I looked back at some code from ~8 years ago:
    I like your code better, the question should be how to simulate anything other than success rather than just a 500 error code. 

    my $m2= WWW::Mechanize->new(); # New Mechanize Object for detailed res
+ults
                               # may help save memory?? TBD... 
                               # update (it did)

my $success=0;
my $tries=0;
while (! $success and $tries++ < 10) 
{
    eval { $m2->get($fullurl); };
    if (! $@) {
        $success = 1;
    }
    else {
        print STDERR "Error: Retry Attempt $tries of 10\n";
        print LOG    "Error: Retry Attempt $tries of 10\n";
        sleep (3);
    }
}

die "aborted Web Site Error: $!" unless $success; #ultimate failure!! 
+PROGRAM ABORT !!!!

    [download]
    This probably is not typical, but in my app, a website error happens about once per 2,000 requests. One retry is almost always sufficient. I have never seen a successful attempt on the 3rd retry. "ultimate failure" happens way less than once in a million website accesses, but it does happen. I have no explanation for that. This is a "chron" job and it will run again once per hour and pick up where it left off. Anyway there is more than just "time out" or "success". Other error codes can and do come back and more likely than not, a simple retry will fix them.
[reply]
[d/l]
[select]
Re: Forcing WWW::Mechanize to timeout
by Discipulus (Canon) on Apr 13, 2022 at 07:12 UTC
    Hello stevieb,

    perhaps I miss the point and I'm saying the obvious, but.. you get back a HTTP::Response object so why in your test you dont just craft a desired response? You can reuse some part of the original, succesful response to build up your mocked one if it is the case.

    Sorry if I miss the point :)

    L*

    PS my kawasaki never timeout!

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
[reply]
[d/l]
Re: Forcing WWW::Mechanize to timeout
by Anonymous Monk on Apr 13, 2022 at 07:02 UTC
[reply]

Back to Seekers of Perl Wisdom