You are doing a lot of opening and reading and closing and so on. Modern OSes have lots of diskcache, so it's not a problem, but it probably doesn't scale very well. Also, I personally find it hard to work with data that way.
I've always considered hashtables as the best thing since sliced bread. Hashtables were always hard to store on disk, but with the excellent DB_File module you can tie a hash to a diskstored db-database, and use it as if it were an in-memory hash table. No need to worry about the file format or loading-and-storeing the database. Just remember to untie it before you terminate the script.
And since it is a hash you can perform all the usual hashtabel operations, assigning, retrieving and deleteing (which solves your original problem). You can do foreach $key {sort keys %hash} and do a linear search if you want.

And then you can do hashes of different tied-in hashes, stored in different files. I find that a combination of perverse and convenient.