Re: Quickly reading the first line of a large number of files...

The list of files has already been striped on those ending in recognizable extensions.

I think it going to be out with benchmark and test perl open vs unix 'head -1'..

Thanks

--
The Snowman
snowman@notreally.co.uk

Comment on Re: Quickly reading the first line of a large number of files...

Replies are listed 'Best First'.
Re (tilly) 2: Quickly reading the first line of a large number of files... by tilly (Archbishop) on Aug 09, 2001 at 09:31 UTC
If all of the files are in the same directory with many others, then that may be one of your performance problems. Many file operations have to scan the directory listing, and as the listing gets long, this gets inefficient. If you can scatter your files across many different directories in some way, you will probably get a performance increase. I believe that the Reiser filesystem is also specifically designed to be fast in this case, and others probably are as well, so you may be able to improve performance at the OS level. (Then again, you may not, and that kind of performance tweak is likely to be forgotten when the next admin looks at the system.)	[reply]

Replies are listed 'Best First'.

Re (tilly) 2: Quickly reading the first line of a large number of files...
by tilly (Archbishop) on Aug 09, 2001 at 09:31 UTC

If you can scatter your files across many different directories in some way, you will probably get a performance increase. I believe that the Reiser filesystem is also specifically designed to be fast in this case, and others probably are as well, so you may be able to improve performance at the OS level. (Then again, you may not, and that kind of performance tweak is likely to be forgotten when the next admin looks at the system.)

[reply]