in reply to Re^2: Schizophrenic var
in thread Schizophrenic var

I used "dualvar" also, in MCE::Shared::Cache. When max_age is specified during construction of the cache, the dualvar is used internally to store the expiration time with the key name in the keys array.

# update expiration $keys->[ $off ] = ( $exptime >= 0 ) ? dualvar( time + $exptime, $_[1] ) : dualvar( -1, $_[1] );

Basically, the "dualvar" capability in Perl allows the "max_age" feature without increasing the memory consumption of the cache or impacting performance.

Replies are listed 'Best First'.
Re^4: Schizophrenic var (semipredicate problem)
by eyepopslikeamosquito (Archbishop) on Jul 01, 2023 at 09:43 UTC

    Interesting!

    In computer programming, a semipredicate problem occurs when a subroutine intended to return a useful value can fail, but the signalling of failure uses an otherwise valid return value. The problem is that the caller of the subroutine cannot tell what the result means in this case.

    The division operation yields a real number, but fails when the divisor is zero. If we were to write a function that performs division, we might choose to return 0 on this invalid input. However, if the dividend is 0, the result is 0 too. This means there is no number we can return to uniquely signal attempted division by zero, since all real numbers are in the range of division.

    -- from Semipredicate problem (wikipedia)

    Vaguely related is C++-17's std::variant and Haskell's Type Inference system, with types like Maybe Real and Either String Char.

    So perhaps we can say that Perl's dualvar helps solve the Semipredicate problem ... though admittedly, without dualvar, ordinary Perl scalars can use undef to signal invalid input.

    I'm not an expert on any of this, just coincidentally happened to be looking at this topic while updating my notes on exception handling vs error returns in functions. :) Ideas/corrections and other cool references on this topic welcome!

    References Added Later

    • std::error_code (cppreference.com, since C++11) - a platform-dependent error code
    • std::expected (cppreference.com, since C++23) - an object of std::expected<T,E> holds an expected value of type T, or an unexpected value of type E; it's never valueless
    • std::unexpected (cppreference.com, since C++23)
    • Expect the expected (youtube) by Andrei Alexandrescu
    • std::variant (cppreference.com, since C++17) - a type-safe union; you can avoid exceptions by having a function return, for example, a std::variant<string, error_code> (similar in style to Haskell) - see Stroustrup, A Tour of C++, p 209 (superseded by std::expected?)
    • std::optional (cppreference.com, since C++17 - manages an optional contained value, i.e. a value that may or may not be present)

      > Save it for very special situations, like impressing people at conferences and cocktail parties.

      The use of "dualvar" in MCE::Shared::Cache is not due to schizophrenic behavior or trying to impress people. Nothing like that.

      There was a dated article on the web (I misplaced the link) where someone compared several pure-Perl caching modules. In addition to performance, the author of the article compared memory consumption. The thing that I remember is the extra memory consumption using a hash to store the expiration time.

      For me, "dualvar" solved a problem. I was able to add the "max_age" feature to MCE::Shared::Cache without greatly impacting performance or spike in memory consumption.

        The dated "perl-benchmark-cache-with-expires-and-max-size" article no longer exists on the web. Fortunately, Celogeek's GitHub repository exists, containing the article scripts. I made two test scripts locally, at the time, when developing MCE::Shared::Cache. What I recall from the article is the author captured memory consumption. Thus, the priority for me was ensuring low memory consumption first, then performance.

        test-cache-mce.pl

        $ diff test-cache-lru.pl test-cache-mce.pl 1a2,3 > # based on test-cache-lru.pl > # https://github.com/celogeek/perl-test-caching/tree/master 8c10 < use Cache::LRU; --- > use MCE::Shared::Cache; 17c19 < my $c = Cache::LRU->new(size => 500000); --- > my $c = MCE::Shared::Cache->new(max_keys => 500000);

        test-cache-mce-with-expires.pl

        $ diff test-cache-lru-with-expires.pl test-cache-mce-with-expires.pl 1a2,3 > # based on test-cache-lru-with-expires.pl > # https://github.com/celogeek/perl-test-caching/tree/master 8c10 < use Cache::LRU::WithExpires; --- > use MCE::Shared::Cache; 17c19 < my $c = Cache::LRU::WithExpires->new(size => 500000); --- > my $c = MCE::Shared::Cache->new(max_keys => 500000);

        Non-shared results:

        $ perl test-cache-lru.pl Mapping Starting Write: 629454.032475913 Read : 1121841.10542643 Found: 500000 Mem : 436666368 $ perl test-cache-mce.pl Mapping Starting Write: 662199.102659472 Read : 1054666.70019362 Found: 500000 Mem : 254976000

        Non-shared results with expires:

        Perl's dualvar capability is the reason why MCE::Shared::Cache is able to offer max_age capability without increasing memory consumption or greatly impacting performance.

        $ perl test-cache-lru-with-expires.pl Mapping Starting Write: 478838.905524077 Read : 726818.987997266 Found: 500000 Mem : 525479936 $ perl test-cache-mce-with-expires.pl Mapping Starting Write: 605342.765771411 Read : 994375.492182959 Found: 500000 Mem : 254980096

        Shared cache with expiration:

        The MCE::Shared::Cache documentation provides a parallel demonstration, using a shared cache. Here, I modified the code to match the Redis variant (i.e. set key-value pair with expiration).

        $ diff demo-from-doc.pl test-cache-parallel-mce.pl 17c17 < my $c = MCE::Shared->cache( max_keys => 500_000 ); --- > my $c = MCE::Shared->cache(); 29c29 < for ( @{ $chunk_ref } ) { $c->set($_, {md5 => $_}) } --- > for ( @{ $chunk_ref } ) { $c->set($_, {md5 => $_}, 600) +}

        Finally, the diff output for the Redis example.

        $ diff test-cache-parallel-mce.pl test-cache-parallel-redis.pl 10a11,12 > use Redis; > use Sereal qw/encode_sereal decode_sereal/; 17c19 < my $c = MCE::Shared->cache(); --- > my $c = Redis->new; 29c31 < for ( @{ $chunk_ref } ) { $c->set($_, {md5 => $_}, 600) +} --- > for ( @{ $chunk_ref } ) { $c->setex($_, 600, encode_sere +al({md5 => $_})) } 33c35,39 < for ( @{ $chunk_ref } ) { $f++ if ref $c->get($_) eq 'HA +SH' } --- > for ( @{ $chunk_ref } ) { > my $srl = $c->get($_); > $srl = decode_sereal($srl) if defined $srl; > $f++ if ref $srl eq 'HASH'; > }

        Results:

        $ perl test-cache-parallel-mce.pl Mapping Starting Write: 228731.311 Read : 143733.302 Found: 600000 $ perl test-cache-parallel-redis.pl Mapping Starting Write: 159626.727 Read : 164603.402 Found: 600000 $ perl test-redis-tcp.pl Mapping Starting Write: 57403.6057590594 Read : 60989.7109982115 Found: 600000 Mem : 270336