Allasso has asked for the wisdom of the Perl Monks concerning the following question:

Hello,

I am trying to pass hex encoded characters as args from the command line, one to use as a search pattern, and one to use as a replacement in s///. While the script expands the hex code as desired to match the associated characted, the hex code is interpreted literally for the replacement.

I tried different ways of quoting to no avail. Anyone know why it does this and how to get it to substitute the character? It is a mystery to me why it works for the match, but not the sub.

#!/usr/bin/perl use strict; my ($arg_1, $arg_2) = @ARGV; my $var; $var = 'a'; $var =~ s@\x61@\x41@; # works print "$var\n"; $var = 'a'; $var =~ s@$arg_1@$arg_2@; # no work print "$var\n"; exit; ./arg.pl '\x61' '\x41' A \x41
  • Comment on hex code passed from command line is interpreted literally in substitution
  • Download Code

Replies are listed 'Best First'.
Re: hex code passed from command line is interpreted literally in substitution
by Corion (Patriarch) on Mar 05, 2011 at 20:44 UTC

    You will have to tell Perl that you want the number(s) passed on the command line interpreted as "hex code", whatever that means. Most likely, pack and unpack will do what you want.

      thanks, this works:
      $var = 'a'; $arg_2 =~ s@\\x([0-9]+)@pack("H*", $1)@ge; $var =~ s@$arg_1@$arg_2@; # works now print "$var\n";
      Two things I am wondering though,

      1) why is this not needed on the search pattern?

      2) what other kinds of surprises might I expect in passing regexes on the command line?

        why is this not needed on the search pattern?

        The regex is compiled after interpolation.

        what other kinds of surprises might I expect in passing regexes on the command line?

        You just asked why there was no surprise with the regex.

Re: hex code passed from command line is interpreted literally in substitution
by toolic (Bishop) on Mar 05, 2011 at 22:26 UTC
    The LHS of s/// is a regular expression, but the RHS is not. So, you can not expect them to behave the same way.

    Don't pass the '\x' on the command line for arg_2, then convert it to the character you want using hex and chr:

    $arg_2 = chr hex $arg_2; ./arg.pl '\x61' 41
      how about:
      arg.pl '\x61' '\x41' $arg_2 =~ s@\\x([0-9a-fA-F]+)@chr hex $1@ge;
      thanks, I was trying to use chr after Corion's suggestion to use pack/unpack, but I couldn't figure out how to make it work with hex (easily).

      EDIT: added a-fA-F classes.
      "The LHS of s/// is a regular expression, but the RHS is not. So, you can not expect them to behave the same way."

      Nevertheless, RHS accepts the \xNN format if it is hardcoded in the script. It is only when it is passed from the terminal that it doesn't work. Why not? What makes it different when it is passed from the terminal?
        Why not? What makes it different when it is passed from the terminal?

        Why does interpolation happen only once? Because that is how it works

        my $stuff = "\n newline gets interpolated at compile time"; my $other = "$stuff \n gets interpolated at run time"; @ARGV = qw( $stuff $other NOINTERPOLATION ); print "$stuff\n$other\n@ARGV\n" __END__ newline gets interpolated at compile time newline gets interpolated at compile time gets interpolated at run time $stuff $other NOINTERPOLATION
        See, $stuff and $other and @ARGV all get interpolated in that print call, but the values in @ARGV won't get magically interpolated once again

        What makes it different when it is passed from the terminal?

        It's not different.

        $ perl -E'$_="a"; $r=$ARGV[0]; s/a/$r/; say;' '\x41' \x41 $ perl -E'$_="a"; $r="\\x41"; s/a/$r/; say;' \x41
Re: hex code passed from command line is interpreted literally in substitution
by BrowserUk (Patriarch) on Mar 05, 2011 at 20:46 UTC

    Add /e to the substitution:

    $var =~ s@$arg_1@$arg_2@e;

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      "Add /e to the substitution:"

      that didn't work for me.

        Then I guess I misunderstand what you are trying to do:

        $find = "\x20"; $rep = "\x30";; $s = "The quick brown fox jumps over the lazy dog";; $s =~ s[$find][$rep]eg;; print $s;; The0quick0brown0fox0jumps0over0the0lazy0dog

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
Re: hex code passed from command line is interpreted literally in substitution
by ikegami (Patriarch) on Mar 06, 2011 at 00:50 UTC

    The core of the problem is that you don't realise that interpolation inserts the string in literally.

    $ perl -E' say "\x61"; $x="\\x61"; say "$x"; $x="\$x"; say "$x"; ' a \x61 $x

    If you want to treat the content of a Perl variable as Perl code, you need to use eval EXPR.

Re: hex code passed from command line is interpreted literally in substitution
by bart (Canon) on Mar 05, 2011 at 23:07 UTC
    Perhaps you're taking the wrong approach. If you want to take command line arguments and use them as parameters in an s/// statement, why not simply let the user enter the s/// statement as a whole? All you have to do is eval it.

    If you have to call it often (thousands of times), you can simply wrap it in a sub to compile it. $_ can simply represent the sub's parameter.

    my $perl = shift; $code = eval "sub { local \$_ = shift; $perl; return $_ }" or die "Syntax error: $@"; foreach my $var (@lots_of_data) { $var = $code->($var) }

    And in the end, you give the user the option to pass in other code than just a simple s///. This is something that the module File::Rename does, for example, a command line utility to rename files where you can define how the file name should be named in a Perl snippet that changes the value of $_ (the file name).

      Actually, it is only the RHS that the user is passing. I mislead you because of my example. I only used that because of my curiosity as to why it worked on one side but not the other.

      The pattern to be replaced is provided by the script, ie, the script searches for illegal character sequences and interactively replaces them with the user's input.

      What I came up with by the direction of others here seems to work fine. I'm happy so far.
      well the method I mentioned fell apart when trying to pass $1, $2 etc to be interpolated as captures, so you and ikegami are right, it really needs an eval. I did want to stick with the script 'pattern' 'replacement' format however (rather than script 's/pattern/replacement/') so I used this instead:
      #!/usr/bin/perl -wl use strict; my ($arg_1, $arg_2) = @ARGV; $_ = "apples"; my $expr = "s/".$arg_1."/".$arg_2."/"; eval $expr; print; ------- ./test.pl '\x61(.*)\x73' '\x41$1\x53' AppleS
        Now give your program these magic arguments "destroyer" "`rm -rf \x2f `"
Re: hex code passed from command line is interpreted literally in substitution
by AnomalousMonk (Archbishop) on Mar 06, 2011 at 06:52 UTC