sdetweil has asked for the wisdom of the Perl Monks concerning the following question:

I have this function in one of my perl applications, what allows a user to use a regex to parse out some of the data, and it works fine
$data = "this is a test" $expression_text = "(.*)es" $data =~ $expression_text $result = $1 which is 'this is a t';
but if the expression is really a command, replace for example, then it fails..
$data = "this is a test" $expression_text = "s/es/dd/g" $data =~ $expression_text

a particular example is a file name string which was put into a file with single quotes, and on the commandline it needs to be double quotes..

'somefile.dat' = "somefile.dat", s// would do that fine..

but I can't seem to understand the nuance of getting perl to execute it properly.. using re debug,
it shows that the text in the right side variable is treated as the ENTIRE string to match against.
re eval doesn't help.
evl (var = var) doesn't do it.

oh sage ones, show me the light..

Replies are listed 'Best First'.
Re: using just variables in a regular exression
by davido (Cardinal) on Aug 30, 2012 at 23:24 UTC

    The former behavior is documented in perlop, under the "Binding Operators" section, where it states:

    If the right argument is an expression rather than a search pattern, substitution, or transliteration, it is interpreted as a search pattern at run time.

    This is a slightly dangerous behavior, though no more so than the act of passing a user-supplied pattern as $data =~ m/$user_pattern/. A user could supply a masterfully crafted (or simply grossly naive) pattern that results in massive amounts of memory consumption and near infinite looping inside the regex engine. And it's next to impossible to pre-process the user-supplied pattern in a way that will prevent this potential risk while at the same time retaining the legitimate functionality of the regex language.

    To get the ability for a user-supplied operator (such as s///) to be executed, you've got a couple of choices: One is to elevate your security risk a few levels higher by using string eval to evaluate the user-supplied code (this is usually a bad idea). Another is to create logic in your program that allows a user to specify whether he wants substitution or simple matching, and then supply the operators yourself. For example:

    use strict; use warnings; my $data = "Just another Perl hacker,"; my $action = 'subst'; my $pattern = 'hacker'; my $substitution = 'enthusiast'; my %actions = ( match => sub { my $rv = $_[0] = ~m/$_[1]/; return( $rv, [@-], [@+] ) +}, subst => sub { my $rv = $_[0] =~ s/$_[1]/$_[2]/; return( $rv, [@_], [@+] ) +}, ); exists $actions{$action} or die "Invalid action: $action"; my @results = $actions{$action}->($data, $pattern, $substitution); print $data, "\n";

    With a little thought, and a better understanding of the exact problem you're trying to solve, you could probably come up with even simpler dispatch table callbacks...or could add additional return values as needed. But the point is that here we're avoiding evaluating user-supplied code directly, although we still have the problem of the user possibly supplying a nasty regex.


    Dave

      Thanks, but
      Both of u coded the s// directly in perl source code. I only have two variables. One contains the data, the other contains the completely arbitrary expression. It could be a match m// expression, or a substitution s///, or some parsing expression. I have no idea.
      I do not want parse the expression contents in my own code to get parts to put back into some other instruction format in source code. That was the whole point of supporting regex. This s in a tool used by my product support teams to gather diagnostic data from a users system. Sometimes the product install design team didn't think how hard it might be to find/use the product install path in their saved config data.
      As an example, they recorded the direct path and name of the main install executable in single quotes, but we actually need the parent folder of the exe, with another subdirectory to get to the log files. 'C:/somedir/part2/part3/foobar.exe'
      My app lets the script writer parse that out.
      Path(str)='c:/somedir/part2/part3
      Path(str)='c:/somedir/part2
      Add /logs/*.log'
      But the ' kills it all.

      The s/'/\"/g would solve it. I don't have a built in replace function in my script handler.

        Then what you're after, as I mentioned, is the string version of eval:

        my $data = 'Just another Perl hacker,'; my $expression = 's/(another)/a/'; my $match_var; my $result = eval <<"END_EVAL"; my \$success = \$data =~ $expression; \$match_var = \$success ? \$1 : undef; \$success; END_EVAL if( $result ) { print "$match_var => $data\n"; }

        It would be better if instead of passing "$match_var out of the eval's scope you instead passed a reference to a copy of @- and @+; that way you could look at scalar @- to determine how many captures were produced, and then use substr on an original copy of the string to determine what those captures actually were. And while you're at it, you might also need to deal with named captures.

        But this opens your code to some serious problems. Consider the following:

        my $data = 'Just another Perl hacker,'; my $expression = 's/(another)/a/; print "Hello world!\n";'; my $match_var; my $result = eval <<"END_EVAL"; my \$success = \$data =~ $expression; \$match_var = \$success ? \$1 : undef; \$success; END_EVAL if( $result ) { print "$match_var => $data\n"; }

        Now by supplying a semicolon in the appropriate place within the code string, the user was able to move on, executing arbitrary additional code (in this case, print "Hello world!\n";). That's possibly innocuous, but how about 'while ( my( $key, $value ) = each %ENV ) { ... }', or how about backticks or a system call, or even unlink?

        Maybe you can trust your users. But I wouldn't even trust myself with code that exposes that sort of security flaw. No.....you're much better off staying away from the temptation to take the easy solution. It might save you an hour of coding now, but imagine how time consuming it will be when a user does something malicious because he felt snubbed by how you failed to notice him in the break room.

        The safest approach is to find a way to allow users to choose between different operations that you explicitly expose. Much less safe but still better than simple eval would be to use the Safe module's reval method. That will protect you against many malicious attacks, but won't protect from denial of service attacks. A user could still tie up a process forever and consume as much memory as you have available.


        Dave

Re: using just variables in a regular exression
by philiprbrenan (Monk) on Aug 30, 2012 at 22:55 UTC

    Please check: http://perldoc.perl.org/perlop.html#Regexp-Quote-Like-Operators

    if (1) {my $data = "this is a test"; my $expression_text = "(.*)es"; $data =~ $expression_text; say "1: $1"; $data =~ /$expression_text/; say "2: $1"; } if (1) {my $data = "this is a test"; my $expression_text = qr(es); $data =~ $expression_text; say "3: $data"; $data =~ s/$expression_text/dd/; say "4: $data"; }

    Produces

    1: this is a t
    2: this is a t
    3: this is a test
    4: this is a tddt
    
Re: using just variables in a regular exression
by ww (Archbishop) on Aug 31, 2012 at 02:42 UTC

    You have good answers to your question above. Now, please consider the manner of your asking.

    IMO, you're apt to confuse readers -- especially the newcomers whose education is a major goal of the Monastery -- by posting pseudocode without clear indication that that's what it is. And for something this simple, the Monastery's standard is (slightly paraphrased) post a snippet of compilable, executable code (cf: How do I post a question effectively?).

    Omitting semi-colons and regex delimiters merely obfuscates; including them (and the other niceties of compilable code) would cost you very little but would provide the community much greater ease of reading.
      Ok, I'll play along.. I don't like obtuse code, so I use a lot of whitespace.
      and my last issue was with the modifiers, which ended up needing eval().
      if (startsWithIgnoreCase(trim($expstr),"s") || startsWithIgnoreCase(tr +im($expstr),"m")) { # split the string my @t = split( /\//, $expstr); if(startsWithIgnoreCase(trim($expstr),"s")) { if(@t>3) { $t[3]=~s/e//g; SyntaxDebug("new regex modifier parameter=".$t[3]."\n"); if($debugFlag==$true) # 1.52 { use re 'debug'; eval("$expdata=~ s/$t[1]/$t[2]/".$t[3] .";"); $k = $expdata; } else { eval("$expdata=~ s/$t[1]/$t[2]/".$t[3] .";"); $k = $expdata; } } else { if($debugFlag==$true) # 1.52 { use re 'debug'; $expdata=~ s/$t[1]/$t[2]/; $k = $expdata; } else { $expdata=~ s/$t[1]/$t[2]/; $k = $expdata; } } SyntaxDebug("substitution=" . @t . " result=". $k . "\n"); } else { if($debugFlag==$true) # 1.52 { use re 'debug'; $expdata=~ m/$t[1]/; $k = $$expResult; } else { $expdata=~ m/$t[1]/; $k = $$expResult; } SyntaxDebug("match=" . @t . " result=". $k . "\n"); } } else { if($debugFlag==$true) # 1.52 { use re 'debug'; $expdata=~$expstr; $k=$$expResult; } else { $expdata=~$expstr; $k=$$expResult; } if (!defined($k)) { $k=$emptyString; } SyntaxDebug("parse expr='".$1."' k='".$k ."'\n"); }
      thank you Sam