Performing substitutions on lines with special characters

wouldbewarrior has asked for the wisdom of the Perl Monks concerning the following question:

I have a question about running SED commands. I have a list of simple commands that I wand to run that look like this:

$trans =~ s/$item/$newitem/g;
[download]

and where the $item and $newitem elements are key-value pairs from a hash. What I have works fine for most characters, but I run into trouble with special characters like $ % &. I tried building an array of specials and including an 'if' statement to deal with the exceptions, such that the new code looks like this:

if( grep( $_ =~ $item, @special ) ) {
   $trans =~ s/\$item/$newitem/g;
} else {
   $trans =~ s/$item/$newitem/g;
}
[download]

But it still doesn't work, the escape character just seems to negate the '$' in $item. Any suggestions? Joe

20040407 Edit by BazB: Changed title from 'SED commands in Perl'

Comment on Performing substitutions on lines with special characters Select or Download Code

Replies are listed 'Best First'.
Re: Performing substitutions on lines with special characters by gjb (Vicar) on Apr 07, 2004 at 20:23 UTC
Have a look at the `quotemeta`Perl function. It will escape all characters that are "special" for regular expressions. Hope this helps, -gjb-	[reply] [d/l]
Re: Performing substitutions on lines with special characters by graff (Chancellor) on Apr 08, 2004 at 04:44 UTC
gjb is right -- if the problem is that the hash key contains special characters but you want these to be treated as literals (i.e "$" matches an actual dollar-sign character in the string), then just say `s/\Q$item\E/$newitem/` (you don't need to put "quotemeta" operators around the right-side of the expression). On the other hand, sometimes it's handy to make use of metacharacters, and always applying quotemeta operators would make this impossible. For example maybe the string is full-of-hyphenated-forms, and you just want to replace the hyphens with spaces - without removing "dashes" that operate as punctuation (having spaces around them instead of letters) - so a single regex with special characters is just what you want: `s/(?<=[a-z])-(?=[a-z])/ /g;` [download] In this case, the trick is to make sure that you get all the magical characters you want into the regex exactly as they should be, without them getting interpolated into something else -- and you have to make sure that things like dollar and percent signs that need to be treated literally are all provided with literal backslash escapes. For example: `my %replacement; $replacement{'\$34\.50'} = '$17.25'; # ... key is match, value is replacement for my $item ( keys %replacement ) { s/$item/$replacement{$item}/g; }` [download] Given a framework like that, do be careful about items that might interfere with one another (such that the final result might depend on the order in which edits are applied). This is the sort of thing where an AoA might be better than a hash: `my $replacements = [ ['S\&O 400', 'S&P 500'], ['\b400\b', '200'], # ... (or read these match/replace literals from a file) ] # this provides a stable, consistent order of application for my $edit ( @$replacements ) { s/$$edit[0]/$$edit[1]/; }` [download]	[reply] [d/l] [select]