in reply to Re: making a loop script with a remote URL call faster
in thread making a loop script with a remote URL call faster

I need to burn the CPU with an infinite loop. I'm looking to avoid the 0.2 second time it takes Net::Curl to respond with the pricing from the API it's calling. Every 60 seconds I have a 0.2 second delay. I'm hoping to avoid that by forking off that Net::Curl call or perhaps making that Net::Curl call part of a another script that my infinite loop can call via localhost. . Sleep would do the opposite of what I want to accomplish.
  • Comment on Re^2: making a loop script with a remote URL call faster

Replies are listed 'Best First'.
Re^3: making a loop script with a remote URL call faster
by LanX (Saint) on Jan 15, 2022 at 13:58 UTC
    I have problems to understand your code. Your indentation is misleading, most probably because you mix tab with whitespace.

    Your infinite loop will execute "continue on with my perl code" many times with old $pricing till it's finally updated again after 60 sec. Is this really what you want? Or should it rather only be executed again if $pricing was updated?

    Anyway

    > I'm looking to avoid the 0.2 second time it takes Net::Curl to respond with the pricing from the API it's calling.

    you should update the $time just after the if-condition, before calling Net::Curl.

    Like this you won't include the lag

    my pricing = 0; my $time = 0; while (1) { if (time() >= ($time + 60)) { # updates the pricin +g ... $time = time(); # start interval # ... Net::Curl to remote URL... request here. $pricing = [from net::curl]; } # ... continue on with my perl code # even with old pric +ing }

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery

Re^3: making a loop script with a remote URL call faster
by cavac (Prior) on Jan 16, 2022 at 20:22 UTC

    More exact timing can be achieved by properly calculating the interval. This is a rough sketch, taken from memory

    #/usr/bin/env perl use strict; use warnings; # We need sub-second precision here :-) use Time::HiRes wq(time sleep); my $lastrun = 0; while(1) { # Time consuming stuff here my $now = time; # Calculate the time we need to sleep. First we calculate the dif +ference between now and the last run. # That's how long the last run took. Now, calculate how many seco +nds remaining in the current minute. # If the answer is negative, one of two things happened: Either i +t's our first run, or the last run took # longer than a minute my $sleeptime = 60 - ($now - $lastrun); if($sleeptime > 0) { sleep($sleeptime); } $lastrun = $now; }

    If you want the code to run at a specific second in every minute, you could also do that. Again, this is untested and from memory:

    #/usr/bin/env perl use strict; use warnings; # We need sub-second precision here :-) use Time::HiRes wq(time sleep); my $activesecond = 42; # Run whenever the seconds are "42" while(1) { my $now = time; # The built-in modulus function converts to integer, # which would introduce jitter of up to nearly a second. # So, out with the traditional: # my $cursecond = $now % 60; # ...and in with the more manual version: my $cursecond = $now - (60.0 * int($now / 60.0)); if($cursecond != $activesecond) { # Need to wait my $sleeptime = $activesecond - $cursecond; if($sleeptime < 0) { # Handle rollover $sleeptime += 60.0; } sleep($sleeptime); } # Time consuming stuff here }

    Hope that helps a bit.

    Edit: If, for some strange reason you need to use International Atomic Time, you need to take the UTC/TAI offset into account. This is currently 37 seconds, but you will need to consult the IERS Bulletin C twice a year to check for leap second announcements. But in essence, all you need to do is add the appropriate offset when assigning $now:

    my $taioffset = 37; ... my $now = $time + $taioffset;

    perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'

      I just only realized that the OP mixes timed and non-timed code. This only works if jitter is not considered a major problem.

      If the timing needs to be quite exact, my usual approach is to run two different process and use some form of interprocess communication. One process fetches the data from the remote server (however long that takes), parses it and then sends the relevant information to the other process that does cyclic processing.

      There are many ways to do this, depending on the data size and the operating system. TCP or UDP work OK, Unix Domain Sockets are quite a bit faster though. There are also pipes and stuff.

      Often it's quite a lot easier to use an existing messaging solution, though. Again, there are many solutions, depending on your requirements. Personally, i of course will shamelessly plug my own, which is Net::Clacks. There are some examples that should you get started. If not, i'm a regular here. And there is even a small (slightly out-of-date) mini-tutorial here on PM: Interprocess messaging with Net::Clacks. The Upgrade Guide inluded in more recent versions of Net::Clacks also contains quite a lot of information not documented otherwise.

      perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'
        I have problems following, and it really depends what the OP really wants (this has a strong stench of an XY problem)

        For one possible interpretation:

        use Time::HiRes (sleep time); my $next_time = time(); while (1) { next if time() < $next_time; $next_time += 60; # no accumulated lag my $price = fetch(); do_it($price); my $took = time()- ($next_time-60); handle_edge_case() if $took > 60; sleep 59 - $took; }

        (totally untested, there are most probably dragons...)

        The idea is that you only skip at max 59 seconds (or even a bit more) with sleep to let the loop catch the "exact time".

        It still needs to handle the edge-case that fetch() and do_it() $took longer than 60 secs tho.

        But how exactly really depends on the problem to solve...

        So what am I missing justifying two communicating processes???

        edit

        The logic might be clearer if I calculated $remain = $next_time - time(); instead of $took ... left as task for the interested reader :)

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery