zeltus has asked for the wisdom of the Perl Monks concerning the following question:

I have a problem whereby my perl program might need to accept a lot of arguments. More than I am really comfortable with anyway - less than a hundred but more than say, four or five seems a bit on the outrageous side to me.

Anyway, there is nothing I can do about that, if this is the mechanism I am told to use, then so be it.

But is there a more efficient way of slurping the arguments into my perl program? I have considered pack/unpack, using mebbe uuencode/decode.

Any suggestions of comments, please?

Replies are listed 'Best First'.
Re: Lots and lots of arguments! (inefficient?!)
by tye (Sage) on Dec 11, 2014 at 17:38 UTC
    I have a problem whereby my perl program might need to accept a lot of arguments [...] less than a hundred [...] But is there a more efficient way of slurping the arguments into my perl program?

    The efficiency of passing 100 command-line arguments to a program is not something to be concerned about. Heck, command-line arguments are probably one of the most efficient ways to get data into a program (not that most of the other ways of getting 100 arguments to a program are inefficient enough to worry about either).

    Perhaps you could better explain what problem you fear will result and how uuencode or pack might help?

    BTW, if you use list-mode system (on a Unix-like operating system), then the only thing you have to worry about with command-line arguments are '\0' characters or perhaps the total length of the arguments adding up to more than some kernel constant (though I was not able to reproduce that old problem on my rather old but not ancient Unix system -- though 100 arguments isn't even close to that limit even on ancient kernels unless each argument is ridiculously long).

    - tye        

      "...Perhaps you could better explain..."

      :-)

      «The Crux of the Biscuit is the Apostrophe»

Re: Lots and lots of arguments!
by toolic (Bishop) on Dec 11, 2014 at 17:21 UTC
    An alternate to using a standard options module like Getopt::Long is to read in your arguments from a configuration file using something like Config::General.

      Chiming in because this is pet peeve. I dislike Config::General and still resent that it became the default for Catalyst. I would much rather see config with YAML or JSON or a plain .ini this = that style file or even a require "some-config.pl";. Config::General is the XML::Simple of configuration modules but moreso.

Re: Lots and lots of arguments!
by Corion (Patriarch) on Dec 11, 2014 at 17:21 UTC

    Have you considered just reading the arguments from a file?

      If you go this route (I consider it a good one, with the caveat that I don't understand what you're trying to accomplish clearly), I can heartily recommend YAML::AppConfig.

        Where I am, we usually use either YAML::Tiny or JSON::PP.

        YAML::Tiny is a very readable, light weight subset of YAML.

        JSON::PP partly because it is a core module. It is less readable, but handles complex structures better than YAML::Tiny.

Re: Lots and lots of arguments!
by graff (Chancellor) on Dec 12, 2014 at 02:49 UTC
    As indicated above by roboticus If you happen to store your list of arguments in a file, or if you happen to have a process that spews the list to stdout, and if you have a *n*x-like shell and associated utilities), then you can use xargs.
    cat file_with_many_args.txt | xargs your_perl_script # or process_that_spews_args | xargs your_perl_script
    xargs will normalize the whitespace that separates strings in its input, so whether its 50 strings on one line or 50 lines with one string per line, or anything in between with 50 strings, your script's ARGV will have 50 elements.

    Also, if you have the liberty of enhancing the usage syntax for your script, you might consider adding single option that allows for a large set of args to be read from a given file (e.g. your_perl_script -a arg_list.file

Re: Lots and lots of arguments!
by karlgoethebier (Abbot) on Dec 11, 2014 at 17:35 UTC
    "...accept a lot of arguments"

    How many? If it's < 100 Getopt::Long might be a solution;-)

    But please be aware that nobody likes such a command line interface. Perhaps except the users or of tar.

    Please provide some more details for more help.

    Update: Fixed typo.

    Best regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

      You've never used grep on even 100 files? I often pass many hundreds of arguments to command-line programs.

      But then, with a phrase like "four or five seems a bit on the outrageous side", I suspect that I have little idea what is really being talked about in the root node. :)

      - tye        

        Sometimes i wonder if a reply by you is an honor or a penalty.

        I think you mean something like: ./foo.pl a b c d e f g h (and many more).

        And me: ./foo.pl -a nose -b cuke -c foo -d bar | ./bar -t -A -e (and many more).

        "...a phrase like "four or five seems a bit on the outrageous side", I suspect that I have little idea what is really being talked about.."

        Aha. I see it clearly now. That means i don't have a little idea?

        Anyway, my best regards, Karl

        «The Crux of the Biscuit is the Apostrophe»

Re: Lots and lots of arguments!
by zeltus (Beadle) on Dec 12, 2014 at 11:07 UTC

    If it is reasonably efficient to pass lots of values to perl as a list of parameters, then that's fine by me

    I think I might have been affected by programming using old shells where a maximum of 9 parameters could be passed

    I can't use a file to hold anything... I just have to hookup to a web process that will trigger my perl proggie and send it lots and lots (well, a few dozen mebbe... but that is a lot, to me) or values.

    My question was, I guess, is there is a way to pack these into just a few parameters, and unpack them in my perl proggie to get the full list back.

    But if perl doesn't care how many it receives, I guess I shouldn't either... so I can ignore this idea and the overhead it would induce.

    Thanks for the replies, they included some interesting discussion points etc.

      My question was ... is there is a way to pack these into just a few parameters, and unpack them ... to get the full list back.

      Sounds like you have some input as to how the parameters are passed to your program.

      If the combined length is (or becomes) an issue (the number doesn't matter, it's the total of bytes that is limited), you might consider having the "caller" of your program pipe the parameters to your program's standard input (STDIN), one parameter per line.

      If a lot of the parameters are numbers, having the caller pack, then uuencode the parameters might compact the total number of bytes being passed, but this will add processing overhead.

      If that isn't enough, the caller could compress the parameters using bzip, gzip or similar, then uuencode that. But that will add even more processing overhead.

      The simplest, least overhead, way to get around command line limits would be to pipe the parameters, as I described.

Re: Lots and lots of arguments!
by crusty_collins (Friar) on Dec 12, 2014 at 18:14 UTC
    I find that this code is really all i need to read a file in. cat env.ini
    [DB] # PRISM PROD PRISMSID = XX PRISMSVER = X.XX.com PRISMUSER = X PRISMPASS = X PRISMPRT = 1522 [EMAIL] # EMAIL for the program emailSENDER = xxx@xxx.com emailRECIPIENTS = xxx@xxx.com,xxx@xxx.com ADDITIONAL_EMAIL = xxx@xxx.com # EMAIL for DIGITAL PRINT ACTIVATION_FILE_SENDER=xxx@xxxx.com ACTIVATION_FILE_MAIL_LIST= ACTIVATION_MAIL_SUBJ=(PROD) | DAILY DIGITAL PRINT ORDERS [PATHS] RETURNFILE=/mnt/pcard/Return RETURNFILEWIN=\\xxxx\pcard\Return
    Snippet of code
    sub getConfig { my $file = shift(@_) ; my $function = ( caller(0) )[3] ; print "$function \n" ; # get the script .ini file my $script = basename($0); $script =~ s/\.\///; $script =~ s/pl$/ini/; my @files = ("env.ini", "config/$script") ; foreach my $file (@files) { unless ( -f $file ) { print "Could not find $file.\n" ; next ; } print "Config File : $file\n"; open(FH , "< $file"); my @array = <FH>; close FH; my $group = 'default'; foreach my $line (@array) { chomp($line); next if ( $line =~ /^\s*\#/); if ($line =~ /^\[\w+\]$/){ $line =~ s/\[//g; $line =~ s/\]//g; $line =~ s/\s*//g; $group = $line; print "Found group : $group \n"; }else{ next unless ($line =~ /=/) ; my ( $key, $variable ) = split( /=/, $line, 2 ) ; $variable =~ s/\s+//g ; $key =~ s/\s+//g ; $ref->{$group}->{$key} = $variable; } } } return; }
    Outputs
    $VAR1 = { 'CLIENT' => 'X', 'DB' => { 'PRISMPASS' => 'X', 'PRISMPRT' => 'X', 'PRISMSID' => 'X', 'PRISMSVER' => 'X', 'PRISMUSER' => 'XX' }, 'EMAIL' => { 'emailRECIPIENTS' => 'xxx@xxx.com,xxx@xxx.com' +, 'emailSENDER' => 'xxx@xxx.com' }, 'PATHS' => { 'RETURNFILE' => '/mnt/pcard/Return', 'RETURNFILEWIN' => '\\\\xxx\\pcard\\Return' }, 'config' => { 'Date' => '20141212', 'ENV' => 'PROD' } };