You better start by timing where the bottlenecks are, we can't tell from a distance how your system performs.

some ideas:

• try reading repeated entries (like the IDs) just once from the hash into a $scalar and reuse them
• measure if reading the whole table into a AoH at once instead of fetching every single row makes a difference
• consider fetching arrays instead of hashes
• consider SQL-queries/stored procedures which return the data directly in desired format

> to do what the following code does.

documenting / explaining what it "does" might facilitate getting help ...