You could use \c

while (<>) {
   s/\cX//g; # removes ^X's
   s/\cM//g; # removes ^M's
   ...  
}
[download]