Perl Module Not Working In Crontab

dailyoliver has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks - a little help here would be highly valued..

OS NAME: Red Hat Enterprise Linux 5 RHEL5

Perl version: v5.8.8 built for x86_64-linux-thread-multi

Module Version: Archive::Extract v?

Situation: A bash script 'my_bash.txt' invokes my_perl.pl - both scripts enclosed.

TWO Problems:

1. If I don't manually invoke 'my_bash.txt' from the same dir that it resides in - for example /usr/my_dir/my_bash.txt -

the Archive::Extract module in /usr/my_dir/my_perl.pl extracts the output to the root folder '/' and not the correct working directory /usr/my_dir/.

2. If I invoke /usr/my_dir/my_bash.txt via root crontab - the Archive::Extract module will not work.

Please could someone help as I have been at this maddening process for days..

BASH SCRIPT BELOW (this invokes my_perl.pl)


#!/bin/sh

#call perl script to download MODIS greening each decad 
/usr/bin/perl /work/locust1/swarmsdata/RasterData/greening/modis_green
+ing.pl
[download]

PERL SCRIPT BELOW (is called by bash script above)


#!/usr/bin/perl

use Archive::Extract;
$Archive::Extract::PREFER_BIN=1;

#if $inputZip exists unzip, store file correctly and remove date from 
+dekadalGet.txt    
if (-e $inputZip)
{ 

#unzip file
print $timeStamp.' -New zip file successfully downloaded, unzipping fi
+le: '."\n"."\n"; 

# create an archive object
my $archive = Archive::Extract->new(archive => $inputZip);

#extract the contents to the current working directory
$archive->extract() or die "Can't extract: " . $archive->error;
print $timeStamp.' -New zip file = '.$inputZip.' for '.$j.' successful
+ly unzipped -> deleting file.'."\n"."\n";

#delete the zip file
unlink($inputZip) or die "Zip file cannot be deleted.";  
unless (-e $inputZip)
{
 print $timeStamp.' -New zip file successfully deleted: '."\n"."\n";
}
[download]

Comment on Perl Module Not Working In Crontab Select or Download Code

Replies are listed 'Best First'.
Re: Perl Module Not Working In Crontab by graff (Chancellor) on Jan 06, 2012 at 07:46 UTC
The two key suggestions above that are most important: (1) create a minimal script that focuses on the main problem (what differs between interactive use and cron-job use), and (2) use strict. Apart from that, your script suffers from too much repetition of path information, ugly use of "." for string concatenation where a single double-quoted string will do, too many lines of code that could be removed by using a couple more core modules (e.g. POSIX, which provides the very handy "strftime()" function), and probably some errors that you aren't aware of. For example, you have some lines like this: `$geoTiff_delete_all = </work/locust1/swarmsdata/RasterData/greening/. +tif>; $zip_delete_all = </work/locust1/swarmsdata/RasterData/greening/.zip> +;` [download] Are you aware that if there are many .tif and .zip files, only one file name will be assigned to each of those variables? Is that your intention? Try this set of shell commands to see what I'm talking about: `cd /tmp touch foo1.foo foo2.foo foo3.foo perl -le '$f=<.foo>; print "scalar: $f"; @f=<.foo>; print "array: @f +"'` [download] UPDATE: Just to give you an idea of how coding style relates to ease of coding and maintenance, the following snippet of 48 lines should be functionally the same as (or better than) the first 186 lines of the OP perl script; I'm using strictures, consistent indentation, simplest possible quoting, no repetition of literal strings, fewer variables, no redundant comments, and a "for" loop that eliminates several lines of repeated code: Read more... (2 kB) I stopped at line 186 in your script ("Starting ... collection procedure") because after that point it was less clear to me what you were really trying to do (i.e. I'm not sure the OP code was actually doing what you intended). But I would expect you could get a similar (4-to-1) reduction for the rest of the script.	[reply] [d/l] [select]
Re: Perl Module Not Working In Crontab by ww (Archbishop) on Jan 05, 2012 at 23:24 UTC
Don't know, but line 20 of your shell script looks fishy to me. It doesn't invoke the Perl executable; merely references its location, so that part is either redundant or wrong. And despite your assertion that the modis_greening.pl runs but fails to use the module, I have to wonder if you have CHMODed your Perl script appropriately. Crontab won't be running under your id and you haven't cited any evidence that "the script works," whatever that means when you're telling us it doesn't do what you intended. Note that I haven't even looked closely at most of modis_greening.pl. As the advice on posting questions notes, post a minimal sample that reproduces the problem. 400 lines, even with generous whitespace is more than I'll bother to download, absent some far more intriguing problem description. The prior respondent may be onto sometime, but that suggests s/he did at least run your script against `perl -c ....`	[reply] [d/l]
Re: Perl Module Not Working In Crontab by Anonymous Monk on Jan 05, 2012 at 22:23 UTC
Read this if you want to cut your development time in half!	[reply]
Re: Perl Module Not Working In Crontab by i5513 (Pilgrim) on Jan 06, 2012 at 00:49 UTC
Check perl module not working in crontab, and let say us if it help. Regards,	[reply]
Re: Perl Module Not Working In Crontab by dailyoliver (Initiate) on Jan 09, 2012 at 18:34 UTC
Thank you all for your kind generosity and help - I have made the code a bit clearer and ONLY add the bits that are not working - basically the Perl module Archive::Extract works manually but will not work via crontab - there has been mention of environmental variables but I am not sure what I need to do here - any further suggestions with the simplified code would be great! thanks to you all-	[reply]
Re^2: Perl Module Not Working In Crontab by Anonymous Monk on Jan 10, 2012 at 09:06 UTC
basically the Perl module Archive::Extract works manually but will not work via crontab - :D "does not work" is not diagnostic :) Read more... attempts at humor (129 Bytes) turn on debugging, all things have debugging, even crontab $Archive::Extract::DEBUG = 1; Start reading these to see a typical crontab story :) The problem is usually basic path managment. A relative path to the program, wrong path to perl on shebang, an assumed Cwd, wrong module path ( %INC ) or other %ENV , wrong permissions, not checking for errors, not reading error messages, wrong path again :) system command is not working I have some problem with Crontab perl file doesn't execute script fails loading module version, but > version is installed CGI Help Guide , Troubleshooting Perl CGI scripts	[reply]
Re: Perl Module Not Working In Crontab by dailyoliver (Initiate) on Jan 11, 2012 at 22:26 UTC
Thank you all for your comments - The Archive::Extract module works fine if invoked manually. However it fails to extract using crontab. As suggested I have turned debugging on $Archive::Extract::DEBUG=1; <BELOW> are the results - would anybody have an idea what this means? I see the term 'write error (disk full?)' comes up several times??? Any help would be appreciated.. thanks again.. # Extracting with ->_unzip_bin Running [/usr/bin/unzip -Z -1 /work/locust1/swarmsdata/RasterData/gree +ning/MCD_GreenArea_20111211_LocustArea.zip]... MCD_GreenArea_20111211_LocustArea.tif Running [/usr/bin/unzip -qq -o /work/locust1/swarmsdata/RasterData/gre +ening/MCD_GreenArea_20111211_LocustArea.zip]... MCD_GreenArea_20111211_LocustArea.tif: write error (disk full?). Con +tinue? (y/n/^C) warning: MCD_GreenArea_20111211_LocustArea.tif is probably truncated Unable to unzip '/work/locust1/swarmsdata/RasterData/greening/MCD_Gree +nArea_20111211_LocustArea.zip': MCD_GreenArea_20111211_LocustArea.tif +: write error (disk full?). Continue? (y/n/^C) warning: MCD_GreenArea_20111211_LocustArea.tif is probably truncated at /usr/lib/perl5/site_perl/5.8.8/Archive/Extract.pm line 1106 Archive::Extract::_unzip_bin('Archive::Extract=HASH(0x1f96b990)') +called at /usr/lib/perl5/site_perl/5.8.8/Archive/Extract.pm line 415 Archive::Extract::extract('Archive::Extract=HASH(0x1f96b990)') cal +led at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl + line 340 at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl lin +e 340 # Extraction method failed # Extracting with ->_unzip_az IO error: write error during copy : No space left on device at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl lin +e 340 Extraction of 'MCD_GreenArea_20111211_LocustArea.tif' from '/work/locu +st1/swarmsdata/RasterData/greening/MCD_GreenArea_20111211_LocustArea. +zip' failed at /usr/lib/perl5/site_perl/5.8.8/Archive/Extract.pm line + 1166 Archive::Extract::_unzip_az('Archive::Extract=HASH(0x1f96b990)') c +alled at /usr/lib/perl5/site_perl/5.8.8/Archive/Extract.pm line 415 Archive::Extract::extract('Archive::Extract=HASH(0x1f96b990)') cal +led at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl + line 340 at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl lin +e 340 # Extraction method failed Extract failed due to errors at /usr/lib/perl5/site_perl/5.8.8/Archive +/Extract.pm line 439 Archive::Extract::extract('Archive::Extract=HASH(0x1f96b990)') cal +led at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl + line 340 at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl lin +e 340 Can't extract: Unable to unzip '/work/locust1/swarmsdata/RasterData/gr +eening/MCD_GreenArea_20111211_LocustArea.zip': MCD_GreenArea_20111211 +_LocustArea.tif: write error (disk full?). Continue? (y/n/^C) warning: MCD_GreenArea_20111211_LocustArea.tif is probably truncated Extraction of 'MCD_GreenArea_20111211_LocustArea.tif' from '/work/locu +st1/swarmsdata/RasterData/greening/MCD_GreenArea_20111211_LocustArea. +zip' failed Extract failed due to errors at /work/locust1/swarmsdata/RasterData/gr +eening/modis_greening.pl line 340. [download]	[reply] [d/l]
Re^2: Perl Module Not Working In Crontab by Anonymous Monk on Jan 12, 2012 at 08:12 UTC
What do you think this message is trying to tell you MCD_GreenArea_20111211_LocustArea.tif: write error (disk full?). Continue? (y/n/^C) ... IO error: write error during copy : No space left on device Are you ready to stop blaming Perl Module Archive::Extract?	[reply]
Re^3: Perl Module Not Working In Crontab by dailyoliver (Initiate) on Feb 07, 2012 at 11:53 UTC
Thanks you for your response - I eventually found out that I needed to add 'cd /my/working/directory/' into the bash script. This was a privilege issue - Archive::Extract module could not extract the file to the required directory and adding 'cd ...' to the calling script managed to solve it! Thank you to the guy that mentioned TURN ON DEBUG	[reply]