& is a metacharacter on the substitution side of a s/// which must be escaped with a backslash, like this: \&

see perldoc perlre for more.

Edit:
Following my own advice, I tell you not to believe my first sentence. That would be true for the sed command, but not necessarily for Perl. In some circumstances however, when you use the /e modifier, it yields true again.

Cheers, Sören