Can you please clarify? I have quickly concocted this:

use strict;
use warnings;
use feature 'say';

use Variable::Magic qw( wizard cast );
use B::Flags;

my $x = '1';
cast $x, wizard( 
#    get => sub { say 'getting...' },
);

say B::svref_2object( \$x )-> flagspv;
say 1 if $x > 2.2;
say B::svref_2object( \$x )-> flagspv;
say 1 if $x > 2;
say B::svref_2object( \$x )-> flagspv;
[download]

and output is as, I think, expected:

RMG,POK,OVERLOAD,pPOK,IsCOW
RMG,NOK,POK,OVERLOAD,pNOK,pPOK,IsCOW
RMG,IOK,NOK,POK,OVERLOAD,pIOK,pNOK,pPOK,IsCOW
[download]

but with getter definition un-commented:

GMG,POK,OVERLOAD,pPOK,IsCOW
getting...
GMG,POK,OVERLOAD,pPOK,IsCOW
getting...
GMG,IOK,POK,OVERLOAD,pIOK,pPOK,IsCOW
[download]

I'm not OP, just fooling around. My idea was:

use strict;
use warnings;
use feature 'say';

use Variable::Magic qw( wizard cast );
use B::Flags;

package no_int_please { 
    DESTROY { 
        say 'I\'ve been evaluated as integer, HOW DARE YOU?!'
            if B::svref_2object( @_ )-> flagspv =~ /IOK/
    }
}

my $x = '1';
cast $x, wizard( 
    get => sub { bless $_[0], 'no_int_please' },
);

say "value is $x, let's check it";
say B::svref_2object( \$x )-> flagspv =~ /IOK/ ? 'i' : 's';
say $x > 2 ? '>' : '<';
say B::svref_2object( \$x )-> flagspv =~ /IOK/ ? 'i' : 's';
[download]

which therefore won't work if comparison is to NV, but why?

If you want to prevent numification, you can use overload to provide a numifier that dies.