Have you thought about doing this with a database approach using DBD::CSV yet? I have posted a working example of using the DBD::CSV driver here. Just thought you might be interested to have a look at this technique.