More is revealed if you try

  my $a='A';
  print "".$a, $a++;
  __END__
  AA
[download]

Indirectly, this goes toward explaining the behavior you're seeing.

What's happening under the covers in

  my $a='A';
  print $a, $a++;
[download]

is

• Push the variable $a by reference (not a \ reference, which would pass a reference to a reference).
• Push an object that represents the evaluation of the expression $a++, which has the side-effect of incrementing $a.
• Call print

print then gets a stack frame that consists of $a, which is now 'B' as a result of the postincrement, and 'A', which is the result of evaluating the expression $a++.

So how does print "".$a, $a++ help explain this? Why doesn't it also print "BA"? That's because what gets passed as a first argument is a reference to an object that represents the evaluation of "".$a. At the time the expression was evaluated, $a still held 'A'.

This behavior is hinted at in perlsub, which explains

The array @_ is a local array, but its elements are aliases for the actual scalar parameters. In particular, if an element $_[0] is updated, the corresponding argument is updated