The following behaviour is observed (i.e. this isn't a guess):

• It compiles and runs. (I consider this a bug.) That's not the case for my argument list expressions that don't constant-fold into a list of simple variables.
• In the sub, $a and $b are declared as lexicals since the values of the package variables aren't used.

my has compile-time actions and run-time actions.

• The compile-time action declares the lexicals.
• The run-time action ensures the lexicals are cleared or replaced on scope exit.

From the observed behaviour, I deduce the following compile-time behaviour:

• When encountering a variable fetch, if parsing the argument list of my, treat it as a variable decleration instead.
• After parsing a my, check if its argument list is a plain variable list. If not, throw an error.
• Constant folding of my's argument list happens before the above check.

That makes

my (0?$a:$b) = 3;
[download]

equivalent to

(0 ? (my $a) : my $b ) = 3;
[download]

and

my $a if 0;
my ($b) = 3;
[download]

(minus the warning).