$#a returns a magical value in lvalue contexts. (It used to always returns a magical value, but this was optimized.)

$ perl -MDevel::Peek -e'@a=qw( a b c ); sub { Dump($_[0]) }->( $#a );'
SV = PVMG(0x1d36630) at 0x1cfcfd0
  REFCNT = 1
  FLAGS = (GMG,SMG)
  IV = 0
  NV = 0
  PV = 0
  MAGIC = 0x1cf6ab0
    MG_VIRTUAL = &PL_vtbl_arylen
    MG_TYPE = PERL_MAGIC_arylen(#)
    MG_OBJ = 0x1d07d10
[download]

The behaviour you observe is a missing SvGETMAGIC(sv) to handle such scalars.

use strict;
use warnings;

use List::Util      qw( max );
use Variable::Magic qw( cast wizard );

# Make $x a magical variable that always returns 5.
cast(my($x),
   wizard(
      get => sub { ${ $_[0] } = 5 },
   )
);

$x = 3;

if ($ARGV[0]) { no warnings 'void'; 0+$x }

print('max($x, 4)=', max($x, 4), "\n");
print('$x=', $x, "\n");
[download]

$ perl a.pl 0
max($x, 4)=4
$x=5

$ perl a.pl 1
max($x, 4)=5
$x=5
[download]