Do you have to read all the text files? Is this all read-only in terms of the text files (i'm think you could dump it all into sqlite or mysql db)? I don't know how performance will differ, but maybe try DBD::CSV or DBD::AnyData to use DBI to access your data.. Might at least help in transitioning to a better solution.. and also if you can use Class::DBI there's Class::DBI::Cacheable so that you control cache'ing of retrieve calls..