lilgreen has asked for the wisdom of the Perl Monks concerning the following question:

I have a hash (%check) with the following content:

'DESCRIPTION' => 'Checks that various system filesystems are at the mi +nimum size' 'NAME' => 'filesys_sizes' 'TIP' => '(/<filesystem> is less than XGB, increase FS size to minimum +) The /tmp filesystems has a minimum filesystem size of 1GB Increase the filesystem to a minimum of 1GB chfs -a size=2097152 /tmp' 'TIP_NUMBER' => 37 'CHECK_CODE' => ‘#!/usr/bin/ksh93 typeset -A SIZE_MIN SIZE_MIN=([/var]=2097152 [/tmp]=1048576 [/home]=1048576) ISHACMP=$(lslpp -Lqc cluster.*.server.rte 2>/dev/null | grep -c cluste +r) for fs in ${!SIZE_MIN[*]}; do fssize=$(df -k $fs | grep -v Filesystem | awk \'{print $2}\') if (( $fssize < ${SIZE_MIN[$fs]} )); then print "BAD;$fs is less than minimum size ($SIZE_MIN[$fs]}), in +crease to minimum size" elif (( $fssize == ${SIZE_MIN[$fs]} )); then print "GOOD;$fs is at minimum size (${SIZE_MIN[$fs]} kb)" else print "GOOD;$fs is greater ($fssize) than minimum (${SIZE_MIN[ +$fs]})" fi done'

Here's the subroutine that's actually running the CHECK_CODE element:

sub exec_check { my (%check) = @_; my @return = `$check{'CHECK_CODE'}`; foreach (@return) { my ($status, $string) = split(";", $_); print "$status $string"; } }

I want to execute $data{'CHECK_CODE'} as a "chunk" - whatever is in it is run as a whole, and the perl script just handles the data returned from the chunk. I've tried and backticks and qx// and IPC::Run, and even open() but they all seem to have the same limitation - when I pass the hash element as the command to be executed it gets interpreted one line at a time. I always get the same error:

sh[3]: typeset: 0403-010 A specified flag is not valid for this comman +d.

On the system this being run on (AIX) this is a valid error for ksh because it doesn't have a -A option for typeset, but ksh93 does support -A so it's clear that each line of the code is being interpreted as a new child process. I've been fiddling with it for a while and am currently at the banging-my-head-against-the-wall phase, because I'm sure there's something completely obvious that I'm missing. It seems like it should be straight forward.

Replies are listed 'Best First'.
Re: Running blocks of shell script from within perl script
by Corion (Patriarch) on Apr 22, 2014 at 18:49 UTC

    If you want the hashbang to be respected, you need to write your code to a file, make the file executable and then run it.

    Most likely, it will be far easier to just launch the file specified in the hashbang and pass it the command(s) on the command line, most likely with the -c switch:

    my $shell= '/usr/bin/ksh93'; my $command= <<SHELL; typeset -A SIZE_MIN SIZE_MIN=([/var]=2097152 [/tmp]=1048576 [/home]=1048576) ISHACMP=$(lslpp -Lqc cluster.*.server.rte 2>/dev/null | grep -c cluste +r) for fs in ${!SIZE_MIN[*]}; do fssize=$(df -k $fs | grep -v Filesystem | awk \'{print $2}\') if (( $fssize < ${SIZE_MIN[$fs]} )); then print "BAD;$fs is less than minimum size ($SIZE_MIN[$fs]}), in +crease to minimum size" elif (( $fssize == ${SIZE_MIN[$fs]} )); then print "GOOD;$fs is at minimum size (${SIZE_MIN[$fs]} kb)" else print "GOOD;$fs is greater ($fssize) than minimum (${SIZE_MIN[ +$fs]})" fi done SHELL my @results= qx($shell -c '$command');
Re: Running blocks of shell script from within perl script
by Tanktalus (Canon) on Apr 22, 2014 at 22:20 UTC

    You have a few choices, as always.

    The simplest solution, the one more or less given by others, and the one I've been using, is to simply put the script in an external file, and run it. In my case, I just create a scripts subdirectory and run from there. The script is not embedded in my perl code. (I do have a lot of code to genericise this, including ensuring permissions, copying to remote filesystems, etc.)

    However, it's not the only solution. If it's important to keep the code and its metadata together, which is understandable, there are a couple more options I can think of.

    The first is to try to parse the first line of the CHECK_CODE directly, and if it looks like a shell, to use it with a "-c" flag to run the rest. This is not only non-trivial, but error prone. However, if you can pull it off, it requires no change to your existing data. You'd eventually be left with something like:

    my ($shell, $code) = mystical_extraction_goes_here($check{CHECK_CODE}) +; open my $fh, '-|', $shell, -c => $code or die "Can't run shell $shell: + $!"; my @return = <$fh>; close $fh;

    The second is to have the data do this for you. Have the shell, its options (including the -c), and the script, in your data. I don't know the original format of the data, but I'd suggest you'd end up with keys such as SHELL, SH_OPTS, and CHECK_CODE. And maybe CODE_OPTS. The _OPTS would be arrays. And then you'd get code like this:

    my @cmd = $check{SHELL}; if (exists $check{SH_OPTS}) { push @cmd, ref $check{SH_OPTS} eq 'ARRAY' ? @{$check{SH_OPTS}} : $ch +eck{SH_OPTS}; } push @cmd, $check{CHECK_CODE}; if (exists $check{CODE_OPTS}) { push @cmd, ref $check{CODE_OPTS} eq 'ARRAY' ? @{$check{CODE_OPTS}} : + $check{CODE_OPTS}; } open my $fh, '-|', @cmd or die "Can't run check code: $!"; #...
    What makes this solution cool is that a) you don't have to write a bunch of ugly guesses as to potential shells the next guy might want to use, and b) you allow that person to use other "shells". Like awk. Or sed. Or even perl. No limitation to just sh-compatible shells with the -c flag.

Re: Running blocks of shell script from within perl script
by RonW (Parson) on Apr 22, 2014 at 23:01 UTC

    If you want/need to avoid temporary files, you can use open2 to run the shell (or other program), pipe the script to it and pipe the output back:

    use FileHandle; use IPC::Open2; my ($shell, script) = split("\n", $data{'CHECK_CODE'}, 2); $shell =~ s/^#!\s*//; open2(RH, SH, $shell); print SH $script . "\n"; close SH; # is this the right place for this? my @out = <RH>; close RH;
Re: Running blocks of shell script from within perl script
by Laurent_R (Canon) on Apr 22, 2014 at 21:29 UTC
    Not sure this is what you are looking for, but you could use shell constructs such as parentheses to group your commands. For example:
    $ perl -e ' > my $sh_command = "( > echo foo > echo bar > echo baz > )"; > exec $sh_command; > ' foo bar baz
    Of course, the example above is a bit ridiculous, I would not call the shell from Perl to do this, but I was just trying to give an example of some possibility of multi-line shell command run from Perl. You can even run the command directly from Perl without having to use a variable:
    $ perl -e ' > exec "( > echo foo > echo bar > echo baz > )"; > ' foo bar baz
    Or even:
    $ perl -e ' > exec "( > echo bar > echo bar > echo baz > ) | wc > "; > ' 3 3 12
Re: Running blocks of shell script from within perl script
by pvaldes (Chaplain) on Apr 23, 2014 at 00:20 UTC
    fssize=$(df -k $fs | grep -v Filesystem | awk \'{print $2}\');

    Same as:

    fssize=$(df -k --output=size $fs);

    (but probably a little slower that this last line...)

    've tried and backticks and qx// and IPC::Run, and even open() but they all seem to have the same limitation - when I pass the hash element as the command to be executed it gets interpreted one line at a time

    thus forget about the for loop, forget about the bash array and do all in one line

    perl -e 'my $foo=`df -k --output=size /tmp /var /home`; $foo=~s/\n//g +; print "$foo\n";' returns --> 1K-blocks 12345 67890 1234567890

    You can now manipulate the string $foo directly in perl

Re: Running blocks of shell script from within perl script
by locked_user sundialsvc4 (Abbot) on Apr 22, 2014 at 20:52 UTC

    Yes, indeed.   exec, by any other name, can only “execute” one ... “something.”   Therefore, if you want that “one something” to consist of “executing many commands,” you need to persuade “the host, whoever it is ...” to execute “many commands” as “one something.”   :-D   In other words, you need to build some kind of a shell-script.   Construct such a script (if necessary).   Then, make it “executable.”   Then, execute “it,” and gather-up its [combined ...] output.   Q.E.D.

    “Perl or not ...” and, pretty much, regardless of operating system, this will be the way that you can get this kind of requirement done.

      Using this mentality I've got this to work:

      sub exec_check { my (%check) = @_; my $script = './fh.tmp'; open( my $out, '>', $script ); print $out $check{'CHECK_CODE'}; close($out); system("chmod 755 $script"); my @return = qx/$script/; foreach (@return) { my ($status, $string) = split(";", $_); print_output($status, $string); } system("rm $script"); }

      It still seems a bit inelegant, but maybe with as with so many perl things it's just as elegant as it's going to get :\

      “Perl or not ...”

      There you go again, quoting something that nobody said.

        Please ... just skip it.   Skip the personal jabs and stick to the subject of the thread.   My comment is very clearly meant to say that “in this case it does not matter whether you are using the Perl programming language ... or any programming language ... or not.”   I cannot help but conclude that my intentions were quite clear.

        Hmm.   Site logged me out.   Nearby comment obviously was mine.