Only auto-increment (++) is magical in that fashion and only when the variable is a "string". From perlop:

The auto-increment operator has a little extra builtin magic to it. If you increment a variable that is numeric, or that has ever been used in a numeric context, you get a normal increment. If, however, the variable has been used in only string contexts since it was set, and has a value that is not the empty string and matches the pattern /^a-zA-Z*0-9*\z/, the increment is done as a string, preserving each character within its range, with carry:

    print ++($foo = '99');      # prints '100'
    print ++($foo = 'a0');      # prints 'a1'
    print ++($foo = 'Az');      # prints 'Ba'
    print ++($foo = 'zz');      # prints 'aaa'
[download]

The auto-decrement operator is not magical.

|, &, ^ and ~ also work on strings.

>perl -e "print 'A'|'B'
C
>perl -e "print 'C'&'E'
A
>perl -e "print 'A'^' '
a
[download]

I wish << and >> did as well.

Documented in perlop

Yes, I actually noticed this node while searching for a solution to my problem. But isn't that more of an ASCII feature than a Perl one? I wonder why the deincrementing and bit shifting don't work.