jdtoronto has asked for the wisdom of the Perl Monks concerning the following question:

Esteemed monks,

I am sitting here on my laptop miles away from my code. I have an issue to deal with and maybe someone here can throw somne light on some code or at least an alogrithm.

My client application checks a SOAP server periodically for data to be downloaded, at present each client checks every two minutes or so, and with 1600 of them in the field the server sometimes gets a pretty big hit. Especially so because many of the clients rarely, if ever, have data to collect. In the TCP/IP suite, or maybe in some link layer protocols, an exponential or random backoff or retry mechanism is used to improve channel availability and fairness.

I htink I need to introduce a mechanism like this so that clients not getting data will rapidly back off and only query the server infrequently. As soon as a client sees data they should then start using a much lower numbered slot and request data more often.

I haven't found anything by way of code so far, does anyone know of an example, a module, or even a good reference where I could find an algorithm?

jdtoronto

Replies are listed 'Best First'.
Re: Help with exponential backoff
by jasonk (Parson) on Oct 11, 2006 at 15:39 UTC

    Something like this?

    my $counter = 1;
while( 1 ) {
    if ( data_is_waiting() ) {
        my $data = read_data();
        do_stuff( $data );
        $counter = 1;
    } else {
        sleep( 2 * $counter );
        $counter++ if $counter < 60;
    }
}

    [download]
    We're not surrounded, we're in a target-rich environment!
[reply]
[d/l]
      Thanks jasonk, that will do it.

      I was also pondering the old method of randomly selecting a random 'sleep' between 1 and 2**n, but I don't think that is necessary.

      By the way, exponentiation operator is ** not *.

      jdtoronto

[reply]
Re: Help with exponential backoff
by Fletch (Bishop) on Oct 11, 2006 at 15:52 UTC

    Aside from backoffs you could make groups of clients stagger their requests. Split them up so that half the clients make requests on even minutes (0,2,4,...) and the other half on odd minutes (1,3,5,...) (maybe based on the last octet of their IP address, or something similar).

[reply]
      Not so silly either.

      The application is Perl/Tk, so by using the Tk 'after' timer event I hope I get a random distribution.

      jdtoronto

[reply]
Re: Help with exponential backoff
by Madams (Pilgrim) on Oct 13, 2006 at 18:04 UTC

    Also take a look at Higher Order Perl on MjD's site.

    A useful concept is Fibonacci Numbers. Fibonacci numbers get larger the farther into the sequence you get: 1,1,2,3,5,8,13...

    If you choose the "next" fibonnaci number in sequence as your new "time out" (or whatever) the hits should start to spread out a little more.

    "All too often people confuse their being able to think with their actually having done so. A more pernicious mistake does not exist."

    --Moraven Tollo in Michael A. Stackpole's A Secret Atlas

    My Unitarian Jihad Name is: Sibling Pepper Spray of Loving Kindness. Get yours.

[reply]

Back to Seekers of Perl Wisdom