The "Perl script on command line works but as cronjob it fails" story revisited

karlgoethebier has asked for the wisdom of the Perl Monks concerning the following question:

Hi all, this is an old story, but it's still sometimes surprising:

The task

On a machine some files are uploaded via SFTP for further processing
Search for some files with a given name pattern and suffix (xml)
If the last file uploaded with name/suffix is older than n hours then send an email to customer and write the disaster to syslog
Write a status file (e.g. containing 0/1) to /tmp for each disaster found for Nagios monitoring via SSH

What i did

I wrote some perl....

disaster.pl --foo bar

...and a shell wrapper for cron:

#!/bin/bash
/usr/local/bin/disaster.pl --foo bar > /dev/null 2>&1
[download]

The scenario

I've got a test machine and a production machine which are totally identical.

It is impossible to test on prod.

OS is SLES11.

The observation

Running the perl script as well as the shell wrapper via commandline or cron on test leads to the expected results.

The UID on test is root.

On prod the UID is another, let's say karl - for some reason.

On prod the cron job (shell wrapper) gets executed and cron writes this to syslog but the perl script doesn't write to syslog and doesn't mail.

But when i change the shell wrapper on prod to...

/usr/bin/perl /usr/local/bin/disaster.pl --foo bar > /dev/null 2>&1

...it works.

The question

But why - what do i miss?

N.B.: Perl code, shell wrapper and crontab are OK, that's why i didn't post more details.

Update:

I tried all the given hints (most of them i did before posting) but i couldn't figure out the reason for this issue.

But fortunately i had solution: adding /usr/bin/perl in the shell wrapper did the job, although /usr/bin is already in the path when cron runs the script under my uid.

Thanks to all that helped.

~~Thanks for any hint and~~ Best regards, Karl

Comment on The "Perl script on command line works but as cronjob it fails" story revisited Select or Download Code

Replies are listed 'Best First'.
Re: The "Perl script on command line works but as cronjob it fails" story revisited by Corion (Patriarch) on Mar 05, 2013 at 11:08 UTC
There are a lot of things that you could still test in production to narrow down the error possibilities: Is `/usr/local/bin/distaster.pl` readable by `karl`? Is the `/usr/local/bin/distaster.pl` executable by `karl`? Can `karl` outside of `cron` run the script? Is the hashbang line (=first line) of the script `#!/usr/bin/perl`? Is SELinux interfering with command execution from `cron`?	[reply] [d/l] [select]
Re^2: The "Perl script on command line works but as cronjob it fails" story revisited by karlgoethebier (Abbot) on Mar 05, 2013 at 11:12 UTC
Hi Corion, i did this all...Regards, Karl «The Crux of the Biscuit is the Apostrophe»	[reply]
Re: The "Perl script on command line works but as cronjob it fails" story revisited by tobyink (Canon) on Mar 05, 2013 at 11:37 UTC
These things are normally caused by one of four things... Running under a different user, therefore different permissions. Running under a different user, therefore different "dotfiles". Running through cron, therefore different environment variables. Running through cron, therefore different working directory (avoid relative file system paths). `package Cow { use Moo; has name => (is => 'lazy', default => sub { 'Mooington' }) } say Cow->new->name`	[reply]
Re^2: The "Perl script on command line works but as cronjob it fails" story revisited by karlgoethebier (Abbot) on Mar 05, 2013 at 12:16 UTC
Hi tobyink, shure. But what really confuses me is: No mail, no syslog but my tmp files get written. Unfortunatly i can't remote this boxes cause i'm @home today... But i'll take a look in depth at it (env) when i'm back in office. I checked the other issues yesterday - OK And honestly, i don't have any idea why environment matters in this case. Perhaps i made some false assumptions: e.g. that this boxes are clones...(as been told by the admin who did the setup). Thanks and regards, Karl...and i'll give some feedback «The Crux of the Biscuit is the Apostrophe»	[reply]
Re^3: The "Perl script on command line works but as cronjob it fails" story revisited by roboticus (Chancellor) on Mar 05, 2013 at 14:33 UTC
karlgoethebier: In my (unfortunate) experience, clones are only identical the moment they're cloned. Normally there's an immediate tweak so they don't collide on the network. But soon after a package gets installed on one but not the other, or someone tweaks the "production" server and forgets to tweak the "backup" server. Or they make a test on the "backup" without cleaning up. The long and short of it is that I no longer trust anyone's statements when they tell me the system is a "clone". That's the reason you need to check the environment, permissions, account setups, etc. for everything that matters to your code, or the code that your code relies on. ...roboticus When your only tool is a hammer, all problems look like your thumb.	[reply]
Re: The "Perl script on command line works but as cronjob it fails" story revisited by choroba (Cardinal) on Mar 05, 2013 at 11:17 UTC
`say while $_ = join "=", each %ENV` on both machines and compare the (sorted) output. لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ	[reply] [d/l]
Re^2: The "Perl script on command line works but as cronjob it fails" story revisited by karlgoethebier (Abbot) on Mar 05, 2013 at 12:21 UTC
Thanks very much for help Choroba. Before i post something redundant (considered bad practice?), please see below: Re^2: The "Perl script on command line works but as cronjob it fails" story revisited. Regards, Karl «The Crux of the Biscuit is the Apostrophe»	[reply]
Re: The "Perl script on command line works but as cronjob it fails" story revisited by pvaldes (Chaplain) on Mar 06, 2013 at 01:46 UTC
check /etc/cron.allow and /etc/cron.deny check that the mail is not going to another mailbox (i.e the root of the prod machine), or that the mailbox of karl is not redirecting to another place, maybe out of the box. Try to put a CC order in the mail and see what happens and take a look at crontab -u root -l and crontab -u karl -l in "prod", the line of cron can be in karl or in root and both can act differently. The crontabs for root and karl are in different archives also.	[reply]
Re^2: The "Perl script on command line works but as cronjob it fails" story revisited by karlgoethebier (Abbot) on Mar 06, 2013 at 12:56 UTC
Hi pvaldes! Did it. Every thing is OK. Regards, Karl «The Crux of the Biscuit is the Apostrophe»	[reply]
Re: The "Perl script on command line works but as cronjob it fails" story revisited by graff (Chancellor) on Mar 06, 2013 at 04:22 UTC
When you say "It is impossible to test on prod," do you mean that you aren't able to use an interactive shell on prod to run the perl script on a command line (i.e. prod can only run your perl script via cron)? You didn't mention trying to run your script at the command line on the production machine. Given the evidence provided, I would expect that there might be something like a carriage-return character at the end of the shebang line in the copy of the script that sits on prod. (Have you checked byte-counts and/or md5 checksums for the two copies of the script? Does the test copy happen to have CRLF line terminations?) If there was that sort of difference between the test and prod copies of the script, it would explain why using /usr/bin/perl as the command executed in the shell script would get it to work via cron.	[reply]
Re^2: The "Perl script on command line works but as cronjob it fails" story revisited by karlgoethebier (Abbot) on Mar 06, 2013 at 12:51 UTC
Hi graff! "Does the test copy happen to have CRLF line terminations?" No it doesn't. If so, it won't run on Linux. Regards, Karl «The Crux of the Biscuit is the Apostrophe»	[reply]