Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

hi

I am new to perl and I have a program which goes though a while loop reading in data from file and then makes a request to an external program based on the input data. The usage restrictions of the server are that i don't make more than 3 requests per second. The simplest thing I can think of is to simply wait for half a second using the sleep function. Is there anything else/better I could do?

thanks a lot
  • Comment on do something no more than 3 times a second

Replies are listed 'Best First'.
Re: do something no more than 3 times a second
by roboticus (Chancellor) on Nov 26, 2010 at 12:48 UTC

    In loops with multiple things going on, I typically compute the earliest time to re-execute the thing:

    my ($quit, $next_print, $next_foo) = (0, 0, 0); while (!$quit) { <<<do stuff>>> my $cur_time = time; if (time > $next_print) { print "...\n"; $next_print = time+5; # wait 5s for next print } if (time > $next_foo) { foo(...); $next_foo = time+7; # wait 7s for next foo() } <<<do more stuff>>> }

    This way, print runs no more often than once per five seconds, irrespective of the speed of the stuff you're doing in your loop. A variation of this might be useful to you:

    my ($quit, $next_req, $cur_time, $cnt) = (0,0,0,0); while (!$quit) { <<<do stuff>>> if ($cur_time<time) { # A new second has arrived, permit 3 more requests $cnt=3; $cur_time=time; } if ($cnt) { --$cnt; request(...); } <<<do more stuff>>> }

    ...roboticus

Re: do something no more than 3 times a second
by JavaFan (Canon) on Nov 26, 2010 at 13:57 UTC
    Well, you cannot sleep for half a second using the standard sleep function; you'd need the sleep from Time::HiRes. I would only sleep if there's an actual need to sleep. Perhaps something like:
    sub do_request { use Time::HiRes qw[sleep time]; use 5.010; state $sleep = [0, 0, 0]; my $url = shift; while ($$sleep[0] + 1 > time) { # Loop, as sleep() may get interrupted. sleep time - $$sleep[0] + 1; } $sleep = [@$sleep [1 .. $#$sleep], time]; ... fetch from server ... }

      you could use select as described in perlfunc

      You can effect a sleep of 250 milliseconds this way: select(undef, undef, undef, 0.25);
        Sure, but select() isn't the standard sleep either. And you still need to figure out what the fourth argument should be - select() isn't going to tell you that.
Re: do something no more than 3 times a second
by rovf (Priest) on Nov 26, 2010 at 15:11 UTC
    If you don't mind doing a *busy* loop (which might or might not a bad idea, depending on application), you could take the requirements literally: Before starting the external program, you remember the value of time(). If you found that during the last three executions, you already had recorded the same time value, you just busily loop until the time value increases.

    Of course I can imagine circumstances where you will be shot doing this programming style, but this is something you can best decide yourself.

    -- 
    Ronald Fischer <ynnor@mm.st>

      This can be done without a busy loop.

      use Time::HiRes qw( time sleep ); use constant { TIMES_PER_PERIOD => 3, # Integer PERIOD => 1.0, # Fractional seconds }; my @times = (0) x TIMES_PER_PERIOD; while (my $r = get_request()) { my $wait = ($times[0] + PERIOD) - time(); if ($wait > 0) { sleep($wait); redo; # In case sleep got interrupted. } do_request($r); shift(@times); push(@times, time()); }

      (My placement of do_request prevents more than three request from being served per second. Move do_request after the push to prevent more than three requests from being started per second. roboticus and JavaFan used the latter interpretation.)

      It's not that useful here. Since the requests are coming from a file, they are coming in as fast as they can be serviced, so waiting is going to be necessary.

        This can be done without a busy loop.
        Of course, it can, if you use a HiRes timer. I've suggested the busy loop as an alternative (since the timer was already suggested by JavaFan anyway), in case the OP doesn't want to use the timer. Though doing a busy loop is quite often a bad idea, there *are* cases where it makes more sense than using this kind of timer. I thought that if the OP sees these alternatives, s/he can choose what works best in the current situation.

        -- 
        Ronald Fischer <ynnor@mm.st>
Re: do something no more than 3 times a second
by ambrus (Abbot) on Nov 29, 2010 at 11:11 UTC

    I have some such code in the source of cbstream, but there it's completely unnecessary so I can donate it to you. (Technical details: I'm using it to limit the amount of text sent to the irc server so it doesn't kick the client, but that limit is not a constant, and instead I should be sending a PING command in every 1024 bytes and wait till the PONG that shows the server has processed the data).

    Search for TimedRateLimiter in the cbstream source code. The way the implementation works is that it stores two numbers: a count that tells how many more requests you could send (this could be fractional), and the timestamp of when you last updated that first number. When you try to send a request, you first update the first number by increasing it by 3 s-1 times the time that has passed since the timestamp and updating the timestamp, then clamp the first number from above. Then, you check if the first number is at least 1: if it is, you can send a request now, you only need to decrease the number by 1; if it's less than 1, you have to sleep for the approperiate amount.