bliako has asked for the wisdom of the Perl Monks concerning the following question:

I am seeking Enlightnment with my favourite Monks on this bleak day

The short version is: I need to parse a command-line style string into an array. Each flag, option and argument must be an item in the array. Eventually I will use that array in a system(@cmd).

The long version: I have a perl wrapper to command (say ffmpeg). The wrapper processes its cmdline with Getopt::Long. I have the option of --extra-ffmpeg-params STRING-OF-ARGS. The wrapper does some preprocessing and finally calls ffmpeg with IPC::Run::run(@cmd). And @cmd should also contain the extra ffmpeg params passed at the cmdline. But they were passed as a string but I need to turn them into an array. Hmmm.

I have failed to find something like my @args = Getopt::Long::parse($string).

Right now a solution could be to specify the extra params not as a string which is convenient to the user but as a JSON array which is easy to convert to perl array. yikes!

bw, bliako

Replies are listed 'Best First'.
Re: Parsing a command-line string into an array
by Haarg (Priest) on Dec 22, 2023 at 14:03 UTC

      Hey! that works quite good despite its humble name - I never knew it existed.

      use Text::ParseWords; my @x = shellwords("-x '1 2 3' -y 4 5"); print join("\n", @x); -x 1 2 3 -y 4 5
        The down side is that
        -x '1 2 3' -y 4 5

        would have to be passed as

        --extra-ffmpeg-param='-x '\''1 2 3'\'' -y 4 5'

        You'd need nested quoting, which is very tricky.

        This can probably be mitigated by mixing quotes.

        --extra-ffmpeg-param='-x "1 2 3" -y 4 5'

        And this isn't just tricky to build manually; this is also tricky to build programmatically from the shell. Like, what if you wanted to wanted to do the equivalent of ffmpeg -x "$x"? It would look something like

        # to_shell_lit() - Creates a shell literal # Usage: printf '%s\n' "$( to_shell_lit "..." "..." "..." )" to_shell_lit() { local prefix='' local p for p in "$@" ; do printf "$prefix"\' printf %s "$p" | sed "s/'/'\\\\''/g" printf \' prefix=' ' done } --extra-ffmpeg-param="$( to_shell_lit -x "$x" )"

        (Source)

        This is part of the reason I suggested using multiple args.

        -X '-x 1 2 3' -X '-y 4'
        -X "-x $x" -X '-y 4'
        or
        -X 'x=1 2 3' -X 'y=4'
        -X "x=$x" -X 'y=4'
Re: Parsing a command-line string into an array
by jo37 (Curate) on Dec 22, 2023 at 16:52 UTC
    I have the option of --extra-ffmpeg-params STRING-OF-ARGS

    Is it really necessary to pass extra-options as string? You might use Getopt::Long's pass_through option. This would leave all unknown options in @ARGV. No double quoting required and options may be passed directly to ffmpeg.

    Greetings,
    -jo

    $gryYup$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p$
Re: Parsing a command-line string into an array
by hippo (Archbishop) on Dec 22, 2023 at 13:51 UTC
    I have failed to find something like my @args = Getopt::Long::parse($string).

    Perhaps I'm misunderstanding but don't you just need GetOptionsFromString?


    🦛

      Nope, it needs a spec which I don't have as these are ffmpeg's params I need to pass on. And it does not return an array but processes the args. I guess internally it does a good job in splitting the input string but I don't know how to access that.

Re: Parsing a command-line string into an array
by ikegami (Patriarch) on Dec 22, 2023 at 12:02 UTC

    You could split on whitespace if none the values are expected to contain whitespace.

    But let's say you want to pass -x 123 -y 'some val'.

    One approach you could take is

    prog [options] -X '-x 123' -X '-y some val' [options] [args]

    Split on the first instance of whitespace only.

    # my @extra_args = ( '-x', '123', '-y', 'some val' ); my @extra_args = map { split( /\s+/, $_, 2 ) } @opt_X;

      hmmm, thanks but that opens a different can of worms. I would prefer to rely on a module than string-splitting.

      Perhaps I can do something like this way-round-about way:

      use Data::Roundtrip qw/json2perl/; use File::Temp; use strict; use warnings; my $cmdlinestr = <<EOC; -x 123 -y 'some val' EOC my ($fh, $fn) = File::Temp::tempfile(); print $fh 'use Data::Roundtrip qw/perl2json/; print perl2json(\@ARGV)' +; close $fh; my $ret = `$^X ${fn} ${cmdlinestr}`; my $args = json2perl($ret); print join("\n", @$args)."\n";

      Basically, rely on shell/perl to split the string into @ARGV, by spawning a basic perl script with the cmdline, get a json back and have that back to perl array. A lot can go wrong in this as well.

        I would prefer to rely on a module than string-splitting.

        We don't need a module to parse the grammar I suggested. The grammar is just that simple. That's the point.

        All we're doing is parsing the provided value to separate it into a name and a value. And we know the names can't contains spaces or punctuation (except perhaps _ or -).

        This is the same approach curl uses.

        curl ... -H 'Header1: ...' -H 'Header2: ...' ...

        You could leave out the initial dash if you want (-X 'y'/-X 'y ...').

        You could use equal signs instead of spaces if you want (-X 'y'/-X 'y=...').

        Whatever. The specifics aren't important and are quite flexible since all you have to do is separate a name (with a very limited set of characters) from the rest.

        The solution I presented does not "open a different can of worms". It in fact avoids doing so.

Re: Parsing a command-line string into an array
by ikegami (Patriarch) on Dec 23, 2023 at 06:50 UTC

    Another option is to use the same approach as find -exec.

    prog [options] --ffmpeg [ffmpeg args] [sentinel] [options] [args]

    I'm not a fan of this approach because of its down sides.

    • Confusing. Options to your program and options for ffmpeg aren't distinguishable at a glance.
    • Requires coming up with a sentinel value.[1]
    • No support for this in Getopt::Long (as far as I know).


    Footnotes:

    1. ";" and "+" serve as sentinels for POSIX find -exec. ";" must be escaped since it already has meaning to the shell, though.