in reply to Re: checking a set of numbers for consistency mod p
in thread checking a set of numbers for consistency mod p

Thanks LanX, but it's not clear to me what algorithm you're proposing here. Taking my last example [1,0,0,1,0,0,1], 3, for example: you find that the biggest known entry is 1 (which presumably requires a pass through the values), now what?

Hey, number theory is off-topic!!!

Eh? It's just an algorithm question, right?

Replies are listed 'Best First'.
Re^3: checking a set of numbers for consistency mod p
by LanX (Saint) on Apr 09, 2022 at 20:46 UTC
    > Eh? It's just an algorithm question, right?

    I was joking.

    > Taking my last example [1,0,0,1,0,0,1], 3, for example: you find that the biggest known entry is 1

    take the pre-calculated sieve for 27 entries from my post and anchor the first 1 (marked ^) from both sequences for a one-on-one comparison

    [1,0,0,1,0,0,1] <0 0 1 0 0 1 0 0 2 0 0 1 0 0 1 0 0 2 0 0 1 0 0 1 0 0 3> ^ *

    and you'll see they diverge for the last entry (marked *) hence it's a fail.

    I hope it's clearer now. :)

    If you have billions of comparisons and max m=32 length sequences for a limited number of possible p, this should be very fast.

    (disclaimer: I'm not 100% sure I covered all edge cases caused by the holes)

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery

    update

    for the other examples given

    [0,1,0,2,0] <0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 4 0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 5> ^ -> PASS

    [1,0,5,0,1] <0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 4 0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 5 0 1 . +.. > ^ -> PASS

    [2,?,?,?,2] <0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 4 0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 5> ^ * -> FAIL

    update

    and if you have a sequence [2,[1,0,9,0,1]] with a max (here 9) which isn't in your pre-calculated sieve, identify it with the max from your sieve (here 5)

    [1,0,9,0,1] <0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 4 0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 5 0 1 . +.. > ^ -> PASS

    of course this would also work by identifying 9 with 4 or 3 or 2 because this sequence is so short. But with a possible m=32 you need that full length.

Re^3: checking a set of numbers for consistency mod p
by LanX (Saint) on Apr 10, 2022 at 11:24 UTC
    maybe easiest if you provided more test data, like a generator producing 1e5 true and 1e5 false examples.

    I'd implement it then.

    This could be used for benchmarking, too.

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery

      Generating sets that pass is straightforward: here's code that generates full sets for a given prime, and you can mask out a subset of the values with undef. All such sets should return true when queried with the same prime:

      #!/usr/bin/perl use strict; use warnings; my($prime, $range, $count) = @ARGV; for (1 .. $count) { # report the full range for a random start point my $start = int rand(2 ** 32 - $range); print join(' ', map val($start + $_), 0 .. $range - 1), "\n"; } exit 0; # return the $p-adic order of $n sub val { my($n) = @_; my $val = 0; ++$val, $n /= $prime while 0 == $n % $prime; return $val; }

      It's less obvious how to generate a useful collection of sets that should not pass. Best I can suggest is to take one of the passing sets, randomly increment or decrement an element (that has not been masked out), and see if the valid_set() function in my original post rejects it.

        > It's less obvious how to generate a useful collection of sets that should not pass.

        Actually I was aware it's not straightforward.

        I'd even say generating such a set is more complicated than solving the main task.

        Since you are the one with the requirement, you should know best what fits here. =)

        You should also know which primes to expect. Probably all less 32?

        Please be more explicit...

        > and see if the valid_set() function in my original post rejects it.

        Well provided valid_set() works flawlessly. ;-)

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery

        here my take on it:

        The basic idea is that the maximal known value in a sequence rules it's neighborhood.

        I probably over optimized it: I thought that comparing two arrays should be fastest and I'm caching those static sieves for comparison.

        Of course it's also possible to work without a sieve-table and calculate on the entries on the fly. (So maybe premature optimization on my part)

        This must be benchmarked and depends a lot on processor's caching of perl data structures.

        Please test and tell me if you find a bug.

        FWIW: I left a lot of debug code inside which is optimized away based on the constant DBG

        # https://perlmonks.org/?node_id=11142871 use v5.12; use warnings; use Data::Dump qw/pp dd/; my $range = 32; my %sieves; use constant DBG => 0; sub get_sieve { my ( $prime, $range ) = @_; my @sieve = 0; # p^0 = 1 my @pos = (0); while ( @sieve < 2*$range ) { @sieve = (@sieve) x $prime ; $sieve[-1]++; push @pos, @sieve-1; } pop @pos; # forget edge return { prime => $prime, sieve => [ @sieve[0 .. 2*$range-1] ], sieve_pos => \@pos, sieve_max => @pos-1, }; } sub valid_set_lanx { warn "call:",pp \@_ if DBG; my($p, $a_known) = @_; my $h_sieve_cache = $sieves{$p} //= get_sieve($p,$range); my ($a_sieve, $a_sieve_pos, $sieve_max) = @{$h_sieve_cache}{qw/sie +ve sieve_pos sieve_max/}; warn pp $h_sieve_cache if DBG; # --- find max_known my @known = @$a_known; my $max_known = 0; my $max_pos = 0; my $idx = 0; for my $e (@known) { next unless defined $e; if ($e > $max_known) { $max_known = $e; $max_pos = $idx; } } continue { $idx++; } # --- adjust into sieve range if ($max_known > $sieve_max) { $max_known = $sieve_max; $known[$max_pos] = $max_known; } my $idx_sieve = $a_sieve_pos->[$max_known]- $max_pos; if ( DBG ) { #warn pp [\@known,$a_sieve,$max_known,$a_sieve_pos->[$max_know +n],$idx_sieve,$max_pos]; warn (". "x $idx_sieve, join " ", map { $_ // "?" } @known); warn "@$a_sieve"; } my $ok = 1; for my $e (@known) { #warn (($a_sieve->[$idx_sieve] //"?"), " - ", ($e//"?")) if DB +G; next unless defined $e; $ok=0 if $e != $a_sieve->[$idx_sieve] } continue { $idx_sieve++ } return $ok; } # test the examples: expect [yes, yes, no, no] print valid_set_lanx(@$_) ? "yes\n" : "no\n" for ( [ 2, [0, 1, 0, 2, 0]], [ 2, [1, 0, 5, 0, 1]], [ 2, [2, undef, undef, undef, 2]], [ 3, [1, 0, 0, 1, 0, 0, 1]], [ 2, [1, 0, 8, 0, 1]], );
        OUTPUT:
        yes yes no no yes

        update

        holes are tricky, still need to cover this buggy edge case, which is legit but won't pass now.

        [ 3, [0, undef, 0, 0, 1]]

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery