The usual answer for a task such as this would be to use a database. To get a flavor of database use you might like my tutorial Databases made easy. Note that DBD::CSV lets you treat a CVS file as a database!

Something to think about is how much information about the structure of your data is reflected by the code. Neither arrays or hashes are a great fit for storing your data and manipulating it which may be why you are having trouble thinking about and coding stuff up!