dtbach has asked for the wisdom of the Perl Monks concerning the following question:

Hello wise and knowledgeable Monks. I am VERY new to Perl and am seeking some wisdom. I have created a script to do what I want manually but I'd like to use a cronjob and automate the process. I want to extract all information from a file based off of the previous days date, (a cron job will run the script every Wednesday). So for instance cron job runs Wednesday August 17 and will include all entries from start of day August 9 to end of day August 16 then send that collected information to new file. The format of the data file being parsed is as follows: 20110802-23:42:01,4 20110802-23:45:01,3 20110802-23:48:01,1 20110802-23:51:01,1 20110802-23:54:01,2 20110802-23:57:01,3 20110803-00:00:01,3 When I tried to automate the process my OUTFILE had no information in it. Thank you for any assistance or suggestions that can be provided. I'm slowly working my way through the 5th edition of the Lama book and Perl by Example and the Perl Cookbook. Darryl
  • Comment on Extract data based on dates and place in new file

Replies are listed 'Best First'.
Re: Extract data based on dates and place in new file
by jethro (Monsignor) on Aug 10, 2011 at 18:15 UTC

    If I understand you correctly, your script works if you call it on the command line, but not as a cron job. That is usually because cron has a different environment, i.e. a different current working directory, a different PATH ...

    For example it might have problems finding executables the script is depending on

      The way the script works now it is using STDIN for starting and ending dates, so I input 20110809 and 20110816 (when I run it on 20110817) and it will extract the information between those dates and output to a file I call dfile. I want the cron job to call the script and automatically capture the required data and send it to the dfile without having to input the dates.

        Ok, so you need to change the script so that it calculates the dates of today and a week before today, instead of getting it from stdin, right? localtime(time()) will give you an array where you can extract year, month and day of today, subtract 7*24*60*60 from time() and you will get the date of one week before.

        If that isn't what you are looking for, be more specific what you want or what isn't working. Post the relevant code. Use the right tags so we can read it, How (Not) To Ask A Question

        "...send it to the dfile without having to input the dates. "

        Looks like this is a different question than originally posted?

        Seems that way to me.

        So, check CPAN for modules with "Date" and/or "Time" -- after all, something needs to tell the script that you want Cron to run which dates to check...

        Or, re-write your script with the ASSUMPTION that Crom will run it shortly after midnite; calculate the desired dates from gmtime or localtime, as appropriate.

Re: Extract data based on dates and place in new file
by Anonymous Monk on Aug 11, 2011 at 01:05 UTC
    An approach using Date::Simple
    #!/usr/bin/perl use strict; use warnings; use Date::Simple qw/ today /; my $yesterday = today() - 1; my $wk_before = $yesterday - 7; for ($yesterday, $wk_before) { $_ = $_->strftime('%Y%m%d'); } while (<DATA>) { print if $_ ge $wk_before && $_ le $yesterday; } __DATA__ 20110801-23:42:01,4 20110802-23:42:01,4 20110802-23:45:01,3 20110802-23:48:01,1 20110802-23:51:01,1 20110802-23:54:01,2 20110802-23:57:01,3 20110803-00:00:01,3 20110810-23:42:01,4
    This takes advantage of the fact that dates in this format, YYYYMMDD, are naturally sortable and can be string compared for greater than, less than or equality.
      With a necessary correction
      #!/usr/bin/perl use strict; use warnings; use Date::Simple qw/ today /; my $today = today; my $wk_before = $today - 8; for ($today, $wk_before) { $_ = $_->strftime('%Y%m%d'); } while (<DATA>) { print if $_ ge $wk_before && $_ lt $today; } __DATA__ 20110801-23:42:01,4 20110802-23:42:01,4 20110802-23:45:01,3 20110802-23:48:01,1 20110802-23:51:01,1 20110802-23:54:01,2 20110802-23:57:01,3 20110803-00:00:01,3 20110810-23:42:01,4
        This code worked EXCELLENT, thanks so much. I just added my in and out file handles to extract from the file then place the extracted data in another file and it worked fine. Now have to figure out using PAR or pp so I can include the Date::Simple module in the script, they don't want us installing additional CPAN modules on the machines we are using as storage real estate is at a premium running off of compact flash. Darryl