akamboj84 has asked for the wisdom of the Perl Monks concerning the following question:

Hello Experts, I am kind of stuck in code. I am trying to do multi regex replacement in multiline file. However my scripts is unable to do replacement, I am not able to figure out why? Can someone help me please

->>Script Code

my $def="hashpatterns.txt"; my %dic=(); open(D, $def) || die "can't open definition file:$def\n"; while (<D>) { my ($oldp, $newp) = split /#/; $dic{$oldp}=$newp; } close(D); my $file="input.txt"; open(F, $file) || die "can't open definition file:$file\n"; open(W, '>out.txt') or die "can't write to file:$!\n"; my $line=join "", <F>; my $matchkey=join "|", keys %dic; $matchkey=qr /$matchkey/; $line =~ s%$matchkey%$dic{$matchkey}%g; print W $line; close(F);

--> hashpatterns.txt

\s+user\s"[^"]+"\s+password\s"[^"]+"\s+hash2\s+access(\s+console){2}(\ +s+new-password-at-login)?(\s+member\s"(default|engineer|networktest)" +){2}(\s+exit){0,2}#REPLACE1

\s+user\s[^"]+"\s+password\s[^"]+"\s+hash2\s+access(\s+console){2}(\s+ +new-password-at-login)?(\s+member\s"(default|READ-ONLY)"){2}(\s+exit) +{0,2}#REPLACE2

\s+user\s"[^"]+"\s+password\s"[^"]+"\s+hash2\s+access(\s+(console|snmp +|li)){3}\s+console(\s+new-password-at-login)?(\s+member\s"(default|LI +|li-prof1)"){2}(\s+exit){0,2}#REPLACE3

-->input.txt

user "testuser1" password "08Cl3V.leJKU/GskqArA0Yp4MFo" hash2 access console console new-password-at-login member "default" member "engineer"

user "v-test" password "VCp0GjSBK/KiWW.PgkQp7swXVMZ" hash2 access console console new-password-at-login member "default" member "READ-ONLY"

Replies are listed 'Best First'.
Re: Multiline Regex replacement in Multiline file
by Athanasius (Archbishop) on Sep 15, 2014 at 07:34 UTC

    Hello akamboj84, and welcome to the Monastery!

    You first create a dictionary mapping each search regex to its corresponding replacement text. Then, before searching, you change the search regex in two ways: (1) by concatenating all the search strings with |; (2) by applying the qr operator. So when you get to the substitution, if a match is found the dictionary lookup $dic{$matchkey} is guaranteed to fail, since the new value of $matchkey does not match any of the keys in the %dic hash!

    But even when this problem has been fixed,1 you will still be replacing the whole of the first user entry, including everything through to the final "engineer", with the single text REPLACE1. I doubt this is what you want. For the monks to help you further, you will need to specify the output you expect/desire.

    Update: 1For example:

    Hope that helps,

    Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

      Hi Athanasius, Thanks for that stuff. But I already have a working code like this. But I wanted to avoid the below "for" loop and instead do multi regex replacement.
      for my $matchkey (keys %dic) { $line =~ s%$matchkey%$dic{$matchkey}%g; }
      Also for each regex I have seperate set of pre defined replacement String not like Replace1 or Replace2( that was just used as an example) and I want replace the complete matched string with predefined Replacement String.

        Hello again, akamboj84,

        But I wanted to avoid the below "for" loop and instead do multi regex replacement.

        ...as my regex file is a long one and also i need to process a lot of files, so its kind of time/cpu consuming. I want to get rid of "for" loop...
        Re^3: Multiline Regex replacement in Multiline file

        There may be some clever way to do this using Perl’s extended regular expression patterns, but if so I haven’t found it. But deployment of the for loop can likely be made much more efficient. At present, the regexen are being applied to the whole input file; but if you can read in a paragraph at a time, this will save a lot of processing, as there will then be no need for the regex engine to re-search those parts of the input string that have already been matched and replaced:

        Hope that helps,

        Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

Re: Multiline Regex replacement in Multiline file
by Grimy (Pilgrim) on Sep 15, 2014 at 09:19 UTC

    Howdy akamboj84,

    In the replacement part of your regex, $dic{$matchkey} will always be undefined. $matchkey is the variable you defined above by joining all the keys in %dic, and is therefore not itself a key of %dic. What you actually meant was:
    $line =~ s%$matchkey%$dic{$&}%g; # Use the matched string as a key +in $dic $line =~ s%($matchkey)%$dic{$1}%g; # Same as above without the perfor +mance issues of $&
    See the relevant section of perlvar.

      Hello Grimy,

      This is a nice idea, but unfortunately, as long as the regex contains metacharacters, it can’t be made to work, because $1 will never contain those metacharacters:

      #! perl use strict; use warnings; use Data::Dump; my $re = 'aaa\s+bbb'; my %dic = ($re => 'new'); my $s = 'aaa bbb'; $s =~ s/($re)/$dic{$1}/; print "\$1 = $1\n"; print "\$s = $s\n"; print "\%dic = "; dd \%dic;

      Output:

      19:34 >perl 1011_SoPW.pl Use of uninitialized value within %dic in substitution iterator at 101 +1_SoPW.pl line 10. $1 = aaa bbb $s = %dic = { "aaa\\s+bbb" => "new" } 19:34 >

      Sorry, :-(

      Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

        Hello Athanasius, Is there any workaround for this ? as my regex file is a long one and also i need to process a lot of files, so its kind of time/cpu consuming. I want to get rid of "for" loop as mentioned in reply to your previous post