Using arrays instead of hashes will save you maybe an order of magnitude of memory, if the indexes are really numbers in the ranges you show in your example. I don't know whether that would be enough.
How are you really using the data structure?
Caution: Contents may have been coded under pressure.