cadphile has asked for the wisdom of the Perl Monks concerning the following question:

Fellow Monks,

We have a huge number of installed scripts, that all are written with #!/<perlpath>/bin/perl in line 1 of the scripts (we're in a Solaris environment, and have had an exclusive Sparc compute environment). We've recently added gobs of i386/Solaris servers in our server room, and I want to be able to have users run all these hundreds of scripts on the i386, or sparc platforms, without having to change all the scripts, and/or make separate i386 versions that point to a separate i386 perl build.

So, I've built the sparc, and the i386 versions *together*, and installed the binaries separately using --installbin=<perlpath>/i386/bin (for i386) and --installbin=<perlpath>/sparc/bin for sparc. Then in the directory "<perlpath>/bin", I made the file "perl" into a shell script, to figure out what machinetype the user is on, then exec to the ../i386/bin/perl, or ../sparc/bin/perl, respectively. So my single installation, has a separate architecture area for sun4-solaris-64int, and another for i86pc-solaris-64int. The non-architecture-dependent modules are all shared, and there are separate bin dirs for the perl binaries. The "perl-5.8.8/bin" directory then is just filled with wrappers that resolve the correct build bin...

So, this works fine, if the tool/script is written as follows:

#!/bin/sh -- # -*- perl -*- eval 'exec <perlpath>/bin/perl -S $0 ${1+"$@"}' if 0;
because the unix kernel is just doing a sh exec to the <perlpath>/bin/perl file, which is my bourne shell wrapper. No problem. The problem is that all the tools/scripts use the form:
#!<perlpath>/bin/perl
and in this case, the unix kernel is looking at <perlpath>/bin/perl, and noting that this is not a binary file, and that it doesn't return the appropriate "magic number", and so decides, that this file is not execl()-able. Therefore, it wrongly decides that it's a csh script (because it's char#1,col#1 character is "#"), and therefore begins parsing the script as a csh script, and quite quickly royally croaks....

OK, so (finally my question!) how can I trick the kernel into executing the target interpreter path of a perl script that begins with #!<perlpath>/bin/perl, when <perlpath>/bin/perl is a bourne shell wrapper? Here's the wrapper script, if you're interested:

#!/bin/sh ## this is the wrapper file: <toolroot>/perl-5.8.8/bin/perl die () { echo "ERROR: $1"; exit 1; } tool=`/usr/bin/basename $0` perlroot=<mytoolroot>/tools/perl-5.8.8 arch=`/usr/bin/mach`; ## try to exec the perl binary exe=$perlroot/$arch/bin/$tool if [ -x $exe ]; then eval 'exec $exe ${1+"$@"}' else die "Couldn't find $tool executable: \"$exe\"."; fi

Replies are listed 'Best First'.
Re: using "#!/<perlpath>/bin/perl" to exec a shell script?
by graff (Chancellor) on Oct 04, 2007 at 02:48 UTC
    We faced a similar issue in our local network: we have "project related" binaries (stuff that just doesn't go into /usr/local/bin on all the various machines) and these need to run from a "common path" on different OS's (solaris/sparc vs. freebsd/i386 vs. freebsd/amd64 etc) -- people using a given tool want to use it the same way, no matter which machine they happen to be sitting in front of.

    I don't know if this will help for your case, but the method we came up with was something like this:

    • There's a disk volume on a central file server that is mounted on all machines; it contains a path I'll call "/common" (we actually call it something else, but the point is, it's different from /usr).

    • There's a default, shared .bashrc that just about everyone includes in their shell login/startup process, and it includes, among other things, "/common/bin" being part of PATH.

    • In that central disk volume (mounted everywhere), /common/bin is actually a symbolic link to /etc/common/bin (given that /etc is always "local" to every machine, or to the "netboot" path for each diskless workstation).

    • In that same central /common path, besides the symlink pointing into /etc, we also have actual subdirectory paths like 'arch/sol_s64/bin", "arch/bsd_i386/bin", "arch/bsd_amd64/bin"; this is where we install binaries that don't belong in /usr/bin or /usr/local/bin, and we build for each distinct platform in the appropriate subpath.

    • Now, each machine is built with its own /etc/common directory, and inside that, we have a symlink called "bin", which points back to the particular "/common/arch/xxx_xxx/bin" appropriate to the given machine.

    Setting it up and keeping things straight as new machines are brought online is pretty simple once you get used to it, and maintaining the different binary installations is something we'd have to do in any case -- this arrangement actually makes that part a little easier to manage, I think.

    The same basic approach also covers "/common/lib" and a few other things where different versions of files need to be sorted out according to what kind of machine is using them.

      I like this idea. Basically you have the common path, linked through the local machine (and hence machinetype) to another link to the actual perl binary:
      ## common shared automount:
         /<perlpath</bin/perl -> /usr/local/perl588
      ## i386:
         /usr/local/perl588 -> /<perlpath>/i386/bin/perl
      ## sparc:
         /usr/local/perl588 -> /<perlpath>/sparc/bin/perl
      
      This would work if I could get our admin team to add these links by default in the standard setup of our servers. We have *thousands* of sparc servers however, running both Solaris9, and Solaris10, and about a hundred so far of the i386 servers. Unfortunately I can't modify the standard unix file structures.

      The basic mystery here is how to trick the unix kernel into thinking that a shell script is a legitimate binary file that returns the appropriate "magic number". The "magic number" apparently is an evaluation of the first two bytes in the file. Hence if the first two bytes of a file are #!, the file is a shell script; if they're %!, it's a postscript file, etc. Files that are "exec"-able from the kernel have "0x75" "E" "L" "F" as the first four characters.

      The more I discuss this, the more I'm convinced it's not possible to do what I'm trying to do. The real solution appears to be to just change all our scripts once, to invoke perl using

         #!/bin/sh -- # -*- perl -*-
         eval 'exec <perlpath>/bin/perl -S $0 ${1+"$@"}'
           if 0;
      
      This solution is actually relevant now that we really can't rely on the #! direct reference to the interpreter.

      But if there's some really really deep dark ancient magic that can work here, I am your humble adept...

        There may be another way:
        #!/usr/bin/env perl
        That is not ideal. It still involves updating all those scripts (but at least the update is a simple and constant string replacment). It puts the load on managing how the PATH, PERLLIB, PERL5LIB (and other?) shell variables are set for every user at every machine (but you might need at most half a dozen distinct settings, depending on how many distinct perl builds are needed).

        And of course, whenever you bring in a script that some else has written (e.g., as posted at PerlMonks), you'll probably need to edit the shebang line -- but I gather you might be doing this already.

Re: using "#!/<perlpath>/bin/perl" to exec a shell script?
by roboticus (Chancellor) on Oct 04, 2007 at 02:15 UTC
    cadphile:

    How about using symlinks? You might symlink /usr/arch/bin to /usr/i386/bin for your i386 boxes, and to /usr/sparc/bin on your solaris boxes. Then your scripts could all just use /usr/arch/bin....

    Untested, so it may not work all that well, but I think that's what I'd try first. If it works, you don't need to figure out the wrapper.

    ...roboticus

      Hi Roboticus, unfortunately, the perl installation is on a common automounted partition, that is shared across all platforms. We run almost entirely on servers, and don't have individual machines. Also, if I have a script that has
         #!/common/bin/perl
      
      and this resolves to:
        /common/bin/perl -> <perlpath>/bin/perl
      
      so that the common executable path is a link to the wrapper, you still have the same problem: i.e. the unix kernel can't exec the file <perlpath>/bin/perl, because it's not binary, and doesn't return the "magic number". So, the perl script just gets parsed by /bin/csh, which croaks...

      I'm really thinking there is no good solution to getting <perlpath>/bin/perl to trick the kernel into thinking that it's a real binary shell interpreter...

Re: using "#!/<perlpath>/bin/perl" to exec a shell script?
by perlfan (Parson) on Oct 04, 2007 at 03:59 UTC
    instead of relying on the she-bang, why not just call it with:
    % /the/one/you/want/perl yourscript.pl
    ?
      It actually *does* work beautifully if you call perl from the commandline. In that case, you're just calling the shell script, which exec's to the appropriate perl binary.

      It's the case of the hundred of perl scripts/tools which invoke the perl interpreter in the top line. The unix kernel performs magic to figure out how to handle the file you name there, or whether that file is a shell script, and then which shell interpreter to call.

      And it does seem to work if I call the perl script/tool as you say:

         <perlpath>/bin/perl myperlscript.pl
      
      This seems to work. Actually, if the tool is called through a wrapper, and that tool wrapper calls the tool as above, that will work! I think this is the solution!!
      ## my tool:
         perl-script-to-do-some-work.pl -> .multi-perl588-wrapper
      ## the wrapper finds the latest released version of the
      ## tool, and does an exec:
         PERL=<perlpath>/bin/perl
         EXE=<tool-root>/<tool-version>/bin/<tool>
         exec $PERL $EXE "$@"
      ## that $PERL does exec, if on i386:
         exec <perlpath>/i386/bin/perl
      ## if on sparc
         exec <perlpath>/sparc/bin/perl
      
      The key to this is to make sure the tool is wrapped, so that the wrapper can get both the tool release version, and also call it using an explicit perl override.

      Hey, that was the ticket!! All our tools are wrapped anyway, so this will work perfectly!!!

      Monks rock!

        As a matter of fact, I have the same shell problem - not related to perl - and this thread was basically the only place where I could find some useful info. --- cut here --- I solved by modifying my shebang line to: #!/usr/bin/env /path/to/my/wrapper # # and the script follows... --- cut here --- Thank you all