GrandFather has asked for the wisdom of the Perl Monks concerning the following question:

hpunixguy2 has a list of files with shell meta characters in them. He would like to process the list so the meta characters are quoted so that the list can be passed to a shell command. How does he do that?

use strict; use warnings; while (<DATA>) { #fix the file name here print; } __DATA__ File1^test File2!test File3-test File2\test

He'd like it to print

File1\^test File2\!test File3\-test File2\\test

DWIM is Perl's answer to Gödel
  • Comment on The question hpunixguy2 was too scared to ask - How does he quote shell meta characters
  • Select or Download Code

Replies are listed 'Best First'.
Re: The question hpunixguy2 was too scared to ask - How does he quote shell meta characters
by ikegami (Patriarch) on Mar 07, 2006 at 21:44 UTC
Re: The question hpunixguy2 was too scared to ask - How does he quote shell meta characters
by graff (Chancellor) on Mar 08, 2006 at 02:05 UTC
    In order to pass a list of foobar file names to a shell command, I think the best approach does not involve perl at all (at least, not for unix/linux users, or people with proper unix-tool environments on their windows pcs):
    find path -type f -print0 | xargs -0 some_command -opts
    For example, I just tried this, and it worked fine:
    $ cd /tmp $ mkdir test $ cd test $ echo '12$34 12|34 12!34 12&34' | xargs touch # check that this really worked: $ find . -type f ./12!34 ./12$34 ./12&34 ./12|34 # now clean up the mess: $ find . -type f -print0 | xargs -0 rm # check that this really worked: $ find . -type f
    Apparently, using xargs on a shell command has the same effect as using the system() or exec() call in perl with arrays of args.
Re: The question hpunixguy2 was too scared to ask - How does he quote shell meta characters
by moklevat (Priest) on Mar 07, 2006 at 21:32 UTC
    This works for the limited test case, but as many in the CB noted, might not work for all possible cases that the shell would care about.

    #!/usr/bin/perl -w use strict; while (<DATA>) { chomp; print quotemeta($_)."\n"; } __DATA__ File1^test File2!test File3-test File2\test

    prints:

    File1\^test File2\!test File3\-test File2\\test
Re: The question hpunixguy2 was too scared to ask - How does he quote shell meta characters
by davis (Vicar) on Mar 07, 2006 at 21:32 UTC
    Hey there, GrandFather! Welcome to the Monastery. Here's one way:
    while (<DATA>) { #fix the file name here chomp; $_ = quotemeta; print $_, "\n"; }

    davis
    Kids, you tried your hardest, and you failed miserably. The lesson is: Never try.
Re: The question hpunixguy2 was too scared to ask - How does he quote shell meta characters
by Aristotle (Chancellor) on Mar 11, 2006 at 07:57 UTC

    Don’t escape particular characters. Protect any single quotes found in the data and then surround the whole string with single quotes, instead.

    while ( <DATA> ) { chomp; s/'/'\\''/g; print "'$_'\n"; }

    That is so simple you cannot get it wrong.

    Makeshifts last the longest.

      I find qq("\Q$x\E") much simpler. And even just "\Q$x\E" handles everything but newline and nul.

      - tye        

        Unfortunately, they’re both wrong. quotemeta is not a shell-quoting function, so the escaping agrees largely, but not entirely, which means the code is still unsafe. The second example, as you noted, will get newlines wrong. The first one gets lots of things wrong:

        $ echo '{}' {} $ echo "\{\}" \{\} $ echo "\\\\{\\\\}" \{\} $ echo "\\\\\{\\\\\}" \\{\\}

        With a bit of effort, you can use the discrepance in the last two demonstrations to cancel backslashes added by quotemeta. It’s tricky to exploit such an over-escaping vulnerability, but it’s entirely doable. In contradistinction, quoting shell strings by protecting single quotes and then single-quoting the entire string is absolutely airtight.

        The PHP folks had to learn the same lesson when it came to using addslashes to quote strings interpolated into SQL. Since none of the databases use the exact same quoting rules as PHP, SQL injection is still possible when quoting is done that way. It takes effort to exploit the weakness, but attackers will go to that length; so now PHP has a bunch of database-specific quoting functions.

        Makeshifts last the longest.