If you are not the one to decide switching business to a database, then you'd probably better use database anyway, but only for yourself. Import the data into a database from those files before even thinking about efficient algorithms. You'll save yourself a good amount of hair and blood.

You'll need some sort of timely update procedure, e.g. a croned incremental import script. That's the way.

chasing your update: Whoa, you have a whole hour when your data is stable! Import it into a database right after update and happily serve user search and browse requests from the database fully powered by sql or whatever.