Re^3: Analyzing Files Within a Directory

"last read and/or written." that sounds like you need atime, access time. You could have say a file that has not been changed for a year, but was accessed just a second ago for reading. That might be true of a "work horse" program that is often used, but seldom modified.

For the second part, "parse out the results by field, e.g., the batch ID/job ID.". That sounds like you need some sort of regex to do file name matching? You might want to consider File::Find::Rule. In some more complicated scenarios, this can make the program logic easier to understand and implement. My requirements are usually straight-forward enough that I don't need it, but you should at least be aware of this option.

Update: A few more comments:

Will File::Find make it easier to find each with also specifying each individual file's batch ID/job ID numbers? File::Find solves the problem of writing the code of recursively descending through the the directory structure. This is well tested code that works. You can then focus on the job of deciding what to do with each file. In O/S file system lingo, a directory is actually just another type of a "file". The -f test will tell you whether a name is a simple plain file or not as opposed to a directory, or some kind of link. Not that the "directories of '.' and '..'" will occur, but are normally skipped.

Also note that huck made some good suggestions, although I am not sure if your level of experience allows you to completely understand his code. There are some "above beginner" aspects to it. Nothing derogatory is intended.

I suggest you start with my code as a prototype and see how you get on with that. By all means ask if you have questions.

Comment on Re^3: Analyzing Files Within a Directory

Replies are listed 'Best First'.
Re^4: Analyzing Files Within a Directory by afoken (Chancellor) on Mar 21, 2017 at 05:08 UTC
that sounds like you need atime, access time. Just a note: Updating atime may be "expensive" in some ways (e.g. the computer may need to spin up a laptop's harddisk just to update the atime of a file that is already in the buffer cache), so there are mount options to delay or completely prevent updating the atime. For linux, search for "noatime", "strictatime", "relatime" in the mount manpage for details. Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply]
Re^5: Analyzing Files Within a Directory by afoken (Chancellor) on Mar 24, 2017 at 07:28 UTC
A few more notes, after a msg by Marshall. `noatime` completely disables atime updates. This can be useful for files where atime is irrelevant, like everything in /bin, /sbin, /lib, /usr/bin, /usr/sbin, /usr/lib. No one cares when /bin/bash was read last. On a desktop / server, where disks rotate continuously, the atime update does not matter, but on a laptop or on flash media, like CF- (DIY firewall) or SD-cards (Raspi), each atime update means a disk update / a write cycle. So mounting those directories, or perhaps the entire root fs with `noatime` makes sense there. /home, /tmp and /var (or at least parts of /var) would live on a different filesystem with different mount options. If set up cleverly, the root fs could also be mounted read-only, preventing all updates including atime. `strictatime` is the classic unix behaviour. Update atime everytime a file is read. `relatime` is a clever hack. The atime is updated, but only if the previous atime was earlier than the current mtime or ctime. This helps programs like mutt to detect that a file has been read since the last modification. Newer Linux versions (>= 2.6.30) also update atime if the difference is larger than one day. Default since 2.6.30. Then, there is also `lazytime`. This updates atime, ctime, and mtime only in memory. For userspace, this should look like `strictatime`, but time changes require no disk access, so disks can be stopped, flash write cycles are prevented. Times are written to disk / flash only if something else changes in the inode, on fsync/syncfs/sync (e.g. from umount), when the (undeleted) inode is removed from memory, or more than 24 hours have been passed since the last write for the inode. Of course, if you need the atime/ctime/mtime in software, you don't want an unexpected power cycle or a crash, because all unwritten updates would be lost. Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply] [d/l] [select]