If it's supposed to output two files, what are the other two for? Also, you could make the code significantly simpler by loading both files at the start and removing the bad lines - do you expect to be handling huge numbers of records, or is this a possibility? I can't really get more specific without writing my own version of this script, and that's going to take a bit of time. Maybe after I get some sleep.