You are comparing apples and oranges. You are poking at the reasons for the disjunction between the two techniques when you look at what you need to do to handle an array reference in a hash - you special case the split/join code, but Storable just goes and does it. The cost of "just goes and does it" is a little execution time.

The surprise it not that Storable is slower, but how little slower it is compared to hand crafted code with up-front knowledge of the data structure to be serialized.

If you want to spend the time to hand craft code every time the data structure changes you can squeeze a little more speed out of the result for sure, but that becomes a maintenance nightmare and a really good way to lose data if you manage to get the in and out code out of sync with each other. Unless execution time is critical I'd strongly advise you go the Storable route.