in reply to I need speed

To answer some questions: this is being done on a *cough* Win2k platform (no locate). I am a lowly knuckle-dragging grunt and access to a database server is not an option. This organization regulates that sort of thing tightly.

The bottleneck here seems to be processing speed, as opposed to memory or disk space. I suppose that could change if the index file grows to over 100M. It might increase efficiency somewhat to search a file containing only filenames, and then to retrieve path into from a DBM hash using the filename as a key, (with a little workaround for duplicate filenames in different directories). However, I think my initial idea of breaking the index file into separate, smaller indexes and choosing which to look through based on the search string may produce a greater increase in speed. If I didn't want to search for substrings in the filename this would be easy. The problem is that people don't always know the exact filename they are looking for, and it is useful to be able to display multiple matches (all files of a given extension, for example).

Replies are listed 'Best First'.
Re: Re: I need speed
by mirod (Canon) on Jun 21, 2002 at 19:28 UTC
    I am a lowly knuckle-dragging grunt and access to a database server is not an option. This organization regulates that sort of thing tightly.

    Would anyone object to you installing DBI with DBD::SQLite? It installs a data base server, but it's a really tiny one, and a DB is conveniently stored as a single file.In short it means no administrative hassle and probably no need to ask anyone about it. SQLite implements the LIKE operator and wildcards in queries, so it should help you with your problem.