Re: using grep on a directory to list files for a single date

The key to a solution is reducing disk seeks. I don't know about NTFS, but in Unix there is the 'directory file' (name and inode number->inode (with most of stat() info)->file sectors hierarchy. NTFS should have something similar.

If you manage to get all of your information from the 'directory file' only, this can be done in one big read. E. g. you can use an existing naming scheme of the files.

The other thing is to play OS. If you need to read the (still some 1000?) filtered files, I'd stat them first. This requires an inode read. But before processing the files one by one, sort them by inode number. You help the OS to reduce disk seeks and to utilize read ahead caches. In one of my toy programms sorting dramatically changes disk sound from noisy screetching to a quiet toktoktok. And it was a lot faster.

Comment on Re: using grep on a directory to list files for a single date

Replies are listed 'Best First'.
Re^2: using grep on a directory to list files for a single date by markkneen (Acolyte) on Dec 02, 2004 at 10:15 UTC
Thanks - you have all been fantastic. Really greatfull for everybodies assistance I had a bit of a search around for info about NTFS file table (Master File Table) but not found much in the way of accessing through perl. There is a win32::adminMisc which sounds like it might be up to the job but still reading up on it :( Had to make a few changes to the script to get it to run `while (my $file = readdir(DIR)) {` [download] `while(<DIR>){` <- didn't work. (well not for me). Once again - Thanks... Mark K	[reply] [d/l] [select]
Re^3: using grep on a directory to list files for a single date by zejames (Hermit) on Dec 02, 2004 at 10:22 UTC
angle bracks `<...>` can only be used with file handles, not directory handles. -- zejames	[reply]