Your problem is only partly that the string contains special Perl characters.

1. When you assign to $INVALID_STRING, make sure you protect yourself from interpolation. '"!"' etc. just don't end up in your variable. In this case, you can use the generic single quote op q:

$INVALID_STRING = q{!"$\%&'()*/@`~}; Here { ... } act as the delimiters, but many other options are available. See perlop for details.

2. Your main problem is that the *shell* is having a hard time dealing with the string, not perl. This is your main problem because it's harder to solve :-)

I'm afraid that this will depend a bit on which shell your system uses; experiment a bit with escaping ! and ' and putting everything in single quotes.