in reply to Re^4: Using exists to check element in array
in thread Using exists to check element in array

Your table shows exactly a consistent behavior to exists when used with hashes.

No, it's not consistent with hashes. 

$ perl -e'
   my %h = ( a => 4, c => 6 );
   printf( "exists %s return false for elements found in a hash.\n",
      ( grep { !exists( $h{$_} ) } keys %h ) ? "can" : "didn\x27t" );

   my @a; $a[0] = 4; $a[2] = 6;
   printf( "exists %s return false for elements found in an array.\n",
      ( grep { !exists( $a[$_] ) } keys @a ) ? "can" : "didn\x27t" );
'
exists didn't return false for elements found in a hash.
exists can return false for elements found in an array.

[download]

The very existence of this inconsistency is reason enough for a warning.

$a[0] was never set, hence it doesn't exist.

No, that means it wasn't set. Setting or not setting has nothing to do whether an element is found in an array or not, and with what exists returns.

exists can obviously return false for elements that were never set, but it can also return true for such elements as the following demonstrates: 

$ perl -Mv5.14 -e'my @a; \$a[0]; say exists( $a[0] ) || 0;'
1

[download]

Your implication that exists checks whether an element was set or not is incorrect. As I already mentioned, exists checks if an element is within the array and if it isn't NULL.

I don't understand, probably above my expertise.

These are things that might unintentionally change replace the NULL with a (pointer to a) scalar. I didn't test.

Upd: I just did and they don't. But \$a[0] does, as shown above.

Replies are listed 'Best First'.
Re^6: Using exists to check element in array
by LanX (Saint) on Jan 31, 2024 at 11:51 UTC

    Time and again I see you changing your posts in retrospect without marking updates.

    For the sake of a decent discussion, please be consistent.

    Now, I'll split my reply into multiple posts, so lets target my first argument:

    Firstly:

    your original example was: 

    my @a;

$a[1] = undef;
$a[2] = 0;
$a[3] = 7;

    [download]

    and I replied

    > > Your table shows exactly a consistent behavior to exists when used with hashes. Unset values just don't exist.

    and here we go

    use v5.12;
use warnings;

my (@x,%x);

my $i=1;
for my $v (undef,0,7) {
  $x[$i]=$x{$i}=$v;
  $i++;
}

    [download]
    --- Index exists?
$i     @x     %x
0          
1     1     1
2     1     1
3     1     1
4

    [download]

    QED!

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    see Wikisyntax for the Monastery

[reply]
[d/l]
[select]
      Time and again I see you changing your posts in retrospect without marking updates

      Perhaps PM should enforce some text to explain node edits...

      Of course, it doesn't stop gibberish but the presence of gibberish will demonstrate that a change was made and imply that the poster didn't want people to know why they edited their post.

[reply]

      No idea what you're trying to say. You simply wrote a program that prints a subset of my table. Well, you showed the output of such a program, but didn't actually provide it. Nothing here contradicts what I said, but you claim it does?

[reply]
        I previously said it's consistent for hashes and arrays and proved it.

        Obviously, because I explicitly quoted the statement you denied.

        > Well, you showed the output of such a program, but didn't actually provide it

        I have no idea what that means, the code is provided.

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        see Wikisyntax for the Monastery

[reply]
Re^6: Using exists to check element in array
by LanX (Saint) on Jan 31, 2024 at 12:12 UTC
    Secondly

    > ( grep { !exists( $a[$_] ) } keys @a ) ? "can" : "didn\x27t" );

    Looks like keys @array was added to Perl in 5.12

    And it's obviously inconsistent to delete and exists .

    > The very existence of this inconsistency is reason enough for a warning.

    But it's even worse!!!

    keys is inconsistent to the behavior in hashes, even without using delete and exists and ever seeing any warnings !!!

    my (@y,%y);

$y[3]=$y{3}=1;

say "keys \%y: ", join ",",keys %y;
say "keys \@y: ", join ",",keys @y;

    [download]

    -->

    keys %y: 3
keys @y: 0,1,2,3

    [download]

    A consistent implementation of keys ARRAY should check with exists first before returning an index.

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    see Wikisyntax for the Monastery

[reply]
[d/l]
[select]

      And it's obviously inconsistent to delete and exists

      That's what I've been saying. exists isn't consistent. That's why there's a warning. And you're right, delete isn't either. We've proved this so many times already!

      keys is inconsistent to the behavior in hashes

      Next you'll say that scalar(@a) and scalar(%h) are inconsistent!

      Perl doesn't have sparse arrays. If $y[3] is in the array, so is $y[2].

      keys returns the elements found the structure for both arrays and hashes. keys is consistent for both arrays and hashes.

      Only exists and delete are inconsistent, as repeatedly demonstrated.

[reply]
[d/l]
[select]
        VAR / VARELEM@a / $a[$i]%h / $h{$k}
        VARELEM = 1May add multiple elements to the array.Adds at most one element to the hash.
        VAR in list contextThe value of each element.The key and value of each element.
        VAR in scalar contextThe number of elements.
        keys(VAR) in list contextThe key of each element.
        keys(VAR) in scalar contextThe number of elements.
        values(VAR)The value of each element.
        each(VAR)All elements.
        exists(VARELEM)False if the element isn't in array.
        False if it's in the array and the underlying pointer is NULL.
        True if it's in the array and the underlying pointer isn't NULL.        		False if the element isn't in hash.
        True if it is in the hash.
        delete(VARELEM)Sets the element to undef by setting the underlying pointer to NULL.
        Removes the element from the array if it's the last one.        		Removes the element from the hash.
[reply]
[d/l]
[select]
        > exists isn't consistent.

        No exists is/was consistent, keys ARRAY came later and is inconsistent.

        It doesn't make any sense that a $x[9]=1 on an empty array will make keys return 10 indices. And right after delete $x[9] keys will return none.

          DB<6> x @x
  empty array
  DB<7> $x[9]=1

  DB<8> p keys @x
0123456789
  DB<9> delete $x[9]

  DB<10> p keys @x

  DB<11>

        [download]

        > Next you'll say that scalar(@a) and scalar(%h) are inconsistent!

        No not next, I already said that scalar @array is inconsistent.

        I keep saying that all of this depends on the definition of exists and that orthogonality to hashes was respected.

        You keep insisting that your definition of exists is the only possible one.

        And you still haven't shown any code were delete and exists alone produce different results on arrays than on hashes.

        Even more your example with the reference is consistent for both.

        And keys ARRAY is obviously troubled.

        Actually what's happening here is that two inconsistent concepts were interwoven.

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        see Wikisyntax for the Monastery

[reply]
[d/l]
[select]
Re^6: Using exists to check element in array
by LanX (Saint) on Jan 31, 2024 at 12:35 UTC
    Thirdly:

    > exists can obviously return false for elements that were never set, but it can also return true for such elements as the following demonstrates:

    > 

    $ perl -Mv5.14 -e'my @a; \$a[0]; say exists( $a[0] ) || 0;'
1

    [download]

    now that's extra weird, because introspecting with Devel::Peek reveals that the implementation is (falsely) differentiating between different kinds of 0s and NULLs.

    I'm no C nor even XS hacker, but this looks like exists's understanding of NULL is just to narrow. ¹

    I can't see why this shouldn't be fixable.

    use Devel::Peek;

my @z;
$a = \$z[1];
say "\$z[$_] exists: ", exists( $z[$_] ) || 0 for 0..2;

say '---Dump @z';
Dump @z;

say '---Dump $z[0]';
Dump $z[0];

say '---Dump $z[1]';
Dump $z[1];

    [download]

    comments ### added 
    $z[0] exists: 0
$z[1] exists: 1
$z[2] exists: 0
---Dump @z
SV = PVAV(0x2770318) at 0x27a4930
  REFCNT = 1
  FLAGS = ()
  ARRAY = 0x2787080
  FILL = 1
  MAX = 3
  FLAGS = (REAL)
  Elt No. 0
  SV = 0                           ### 0 
  Elt No. 1
  SV = NULL(0x0) at 0x2783d70      
    REFCNT = 2
    FLAGS = ()
---Dump $z[0]
SV = NULL(0x0) at 0x7a91d0         ### NULL (WTF?)
  REFCNT = 2147483641
  FLAGS = (READONLY,PROTECT)
---Dump $z[1]
SV = NULL(0x0) at 0x2783d70
  REFCNT = 2
  FLAGS = ()

    [download]

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    see Wikisyntax for the Monastery

    ¹) a null understanding ;-)

[reply]
[d/l]
[select]

      now that's extra weird, because introspecting with Devel::Peek reveals that the implementation is (falsely) differentiating between different kinds of 0s and NULLs.

      The "other kind of 0 or NULL" is an undefined scalar.

      exists differentiates between what Dump shows as SV = 0 (underlying NULL in the array) and everything else (underlying non-NULL in the array).

      Say you have this variable: 

      my @a;
$a[1] = undef;
$a[2] = 123;

      [download]

      Devel::Peek shows 

      SV = PVAV(0x**6adf0) at 0x**99148
  REFCNT = 1
  FLAGS = ()
  ARRAY = 0x**8b9c0
  FILL = 2
  MAX = 3
  FLAGS = (REAL)
  Elt No. 0
  SV = 0
  Elt No. 1
  SV = NULL(0x0) at 0x**69538
    REFCNT = 1
    FLAGS = ()
  Elt No. 2
  SV = IV(0x**696f0) at 0x**69700
    REFCNT = 1
    FLAGS = (IOK,pIOK)
    IV = 123

      [download]

      (Replaced the common start of the addresses with ** to help with readability.)

      The underlying array buffer of @a looks like this: 

        ARRAY
  @ 0x**8b9c0
  +-------------+       Head
0 | NULL        |       @ 0x**69538
  +-------------+       +-------------+
1 | 0x**69538 --------->| NULL        | Body
  +-------------+       +-------------+
2 | 0x**69700 ------+   | 1           | REFCNT
  +-------------+   |   +-------------+
3 |             |   |   | ...         | FLAGS
  +-------------+   |   +-------------+
                    |   | SVt_NULL    | TYPE
                    |   +-------------+
                    |   |             | 
                    |   +-------------+
                    |
                    |   Head                  Body
                    |   @ 0x**69700           @ 0x**696f0
                    |   +-------------+       +-----------+
                    +-->| 0x**696f0 --------->| ...
                        +-------------+
                        | 1           | REFCNT
                        +-------------+
                        | SVf_IOK|... | FLAGS
                        +-------------+
                        | SVt_IV      | TYPE
                        +-------------+
                        | ...         | 
                        +-------------+

      [download]

      (Technically, the body of an SVt_IV scalar is part of the same memory block as its head, but illustrating that would just complicated things needlessly.)

      • SV = 0 indicates a NULL pointer in the array.
      • SV = followed by anything else indicates there's a non-NULL pointer in the array.

      What follows SV = is the type of scalar.

      • SV = NULL indicates a scalar of type NULL (SvTYPE(sv) == SVt_NULL). The only value a scalar of this type can hold is undef. my $x; creates such a scalar.
      • SV = IV indicates a scalar of type IV (SvTYPE(sv) == SVt_IV). It can hold one of the following: undef, a signed integer, an unsigned integer or a reference.

      The second number in SV = TYPE(0xBBB) at 0xHHH is the address of the scalar's head (i.e. of the scalar itself), and the first number is the address of the scalar's body. Scalars of type NULL have no body. As such, the pointer to the scalar's body in such scalars will be NULL (0x0).

[reply]
[d/l]
[select]

        Rewrote the parent since it was really badly written originally.

[reply]

      In hindsight, this existence-by-reference trick works for hashes too. Hence it's at least consistent.

      The big question is now, if and what rationale there this.

      Probably not really a problem at all.

        DB<1> %x=(a=>1)

  DB<2> $a= \$x{b}

  DB<3> p exists $x{b}
1
  DB<4> p exists $x{a}
1
  DB<5> p exists $x{c}

  DB<6>

      [download]

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      see Wikisyntax for the Monastery

[reply]
[d/l]
[select]

        As with all lvalue context operations, the rational is that it must exist so you can assign to it (e.g. $$a = 123;)

        Except not quite. Cause the element is evaluated in lvalue context in both for (@a) and f(@a) as well. So why don't they replace the NULL with a scalar?

        That's cause they create and return a proxy scalar containing a reference to the array and the index. Using magic, accesses to this scalar access the original array element. 

        $ perl -MDevel::Peek -e'sub { Dump($_[0]) }->( $a[0] )'
SV = PVLV(0x558b3a14b8a0) at 0x558b3a0f4538
  REFCNT = 1
  FLAGS = (GMG,SMG)                       <- Get and set magic
  IV = 0
  NV = 0
  PV = 0
  MAGIC = 0x558b3a119460
    MG_VIRTUAL = &PL_vtbl_defelem
    MG_TYPE = PERL_MAGIC_defelem(y)
  TYPE = y
  TARGOFF = 0                             <- index
  TARGLEN = 1
  TARG = 0x558b3a1241d8                   <- array
  FLAGS = 0
  SV = PVAV(0x558b3a0f6868) at 0x558b3a1241d8
    REFCNT = 2
    FLAGS = ()
    ARRAY = 0x0
    FILL = -1
    MAX = -1
    FLAGS = (REAL)

        [download]

        Creating this proxy for \ would cause issues.

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom