younggrasshopper13 has asked for the wisdom of the Perl Monks concerning the following question:

Hello, We have a perl script that ingests files from a remote server into a database based on their mtime and puts them into their respective location. This is due to not having standard naming conventions across log files. Problem is, that isn't always effective as mtimes can change. Is there a way in perl to head the file and tail the file to check the date in the first field and ingest it if it matches accordingly? The first column date format looks like 04/Jun/2017:11:04:55 say I need the perl script to match files for the last 6 days, it looks through the logs and checks the date, then ingests them?

Replies are listed 'Best First'.
Re: head/tail a log file for ingesting
by kcott (Archbishop) on Aug 08, 2017 at 05:07 UTC

    G'day younggrasshopper13,

    For the head part, follow the link already provided (perlintro). This will show you how to open a file; read as many head lines as you need.

    For the tail part, this will depend on the size and structure of the file. Here's some options:

    • For a small file, (<$filehandle>)[-1] will give you just the last line. If you want more than one tail line, reverse might come in handy, depending on how you write your code.
    • For fixed-width records, using seek might be best.
    • For large files, the builtin module, Tie::File, could be a good choice.
    • There's a variety of CPAN modules that might suit you better; e.g. File::Tail.

    You mentioned "first field" and "first column". If that's referring to CSV, see your earlier post, "parsing CSV"; if it's something else, you'll need to be more specific.

    — Ken

Re: head/tail a log file for ingesting
by Anonymous Monk on Aug 08, 2017 at 00:33 UTC

    Sure, see perlintro, and try writing something

      Thanks. You guys can just delete this thread. Don't mean to bother.