mirod has asked for the wisdom of the Perl Monks concerning the following question:

I want to split strings into arrays with the same rules as the shell.

I have a configuration file in which I want to allow users to enter options, possibly several options at once, that will be used in a system call. The think is, I want to use the list form of system (so I can accomodate filenames with fancy characters in them for example). So I think I need to parse the strings in the config file, and split them up as lists of arguments. In fact the exact same process that generates @ARGV from the command line.

I understand that this is shell dependant, so I really just want to allow the usual bareword and quoted arguments.

I have found 2 ways to do this: fork out a perl process and get the @ARGV back, which strikes me a extremely ugly, or use a regexp (see below).

My question is: is there a simpler (and already debugged!) way? I could not find anything in Regexp::Common, nor in any of the Getopt::* modules.

Here is a piece of code that seems to work, comments are also welcome:

#!/usr/bin/perl -w use strict; use Test::More qw(no_plan); while(my $args= <DATA>) { chomp $args; my @expected= map { chomp; $_ } `perl -e'print join "\n", \@ARGV' +-- $args`; my @parsed= parse_args( $args); ok( eq_array( \@parsed, \@expected), $args) or diag( " expected: ", display_array( @expected), "\n", " got : ", display_array( @parsed), "\n" ); } sub parse_args { my $args= shift; my @args; while( $args=~ s{^\s* (?: '((?:\\.|[^'])+)' # ' quoted args |"((?:\\.|[^"])+)" # " quoted args |((?:\\.|\S)+) # bareword args )} {}x ) { # only one of $1, $2, $3 is defined, get it my $arg=defined $1 ? $1 : defined $2 ? $2 : defined $3 ? $3 : +undef; $arg=~ s{\\}{}g; # unescape \ push @args, $arg; } return @args; } sub display_array { return '[empty]' unless @_; return "/" . join( '/ - /', @_), "/"; } __DATA__ -f --flag -f -l -fl -f val --file val -- file\ with\ spaces and more\ file\ with\ spaces ' +an other "file name" here ' "and here " foo\ foo\ bar 'foo bar' 'foo bar' foo\ bar "foo bar" "foo \" bar" foo\ bar 'foo bar' 'foo bar' foo\ bar "foo bar" "foo \" bar" foobar - f val

Replies are listed 'Best First'.
Re: Parsing strings into @ARGV
by eserte (Deacon) on Apr 23, 2004 at 15:03 UTC
Re: Parsing strings into @ARGV
by halley (Prior) on Apr 23, 2004 at 14:56 UTC
    You don't say WHICH shell you're talking about. Different shells will interpret command-line contents differently, and produce different underlying (char*)argv[] results (which Perl imports into @ARGV).

    On Windows, different C compilers (used to build the Perl.EXE interpreter) may even introduce variations, since the runtime library can re-interpret the (char*)argv[] to support filename globbing or other extended features.

    --
    [ e d @ h a l l e y . c c ]

      I know, I mentionned that. Basically I just want to be able to use "regular" arguments and quoted, or double-quoted ones. I don't think that's subject to too much variation, at least in the *nix world. I use bash usually BTW.

Re: Parsing strings into @ARGV
by Corion (Patriarch) on Apr 23, 2004 at 15:05 UTC

    CPAN suggests Text::ParseWords (found via search.cpan.org, looking for shell parse words, which pointed me to Shell::Words) - it seems to do what you want, more or less.

      Thanks. Indeed it should work, and it nearly does... but it fails on one of the tests (foo\ , with an extra space at the end of the name, which is mildly annoying as I do have files that end with spaces (before anybody mentions it: no I can't change that ;--). If I quote them it works OK though (.

      It might be a bug in the module and worth fixing though.

        Yes, it's a bug. This is the offending part:
        sub shellwords { local(@lines) = @_; $lines[$#lines] =~ s/\s+$//; return(quotewords('\s+', 0, @lines)); }
        The bug is the unrestricted deletion of trailing whitespace - it shouldn't delete whitespace that's preceeded by the right number of backslashes. I'll see if I can come up with a patch later tonight.

        Note that that's not the only case where Text::ParseWords and my shell (bash) disagrees. Given the string "foo\'bar", my shell parses that as foo\'bar, while Text::ParseWords turns it into foo'bar.

        Abigail

Re: Parsing strings into @ARGV
by meonkeys (Chaplain) on Apr 24, 2004 at 01:33 UTC
    The text you're splitting looks suspiciously like something that could easily be handled with Getopt::Declare.

    ---
    "A Jedi uses the Force for knowledge and defense, never for attack."