Why doesn't perl optimize the former code to only produce one huge array?

Because no one has programmed it to do so. And that in major part is because there is no call for it.

The number of occasions when such an optimisation would be triggered is so few. Very few people would construct an anonymous array, from a list, from an existing array, in order to return a reference.

Conversely, the cost of checking for those extremely rare opportunities would have to be paid for every subroutine that returned an array reference.

Its a high-cost, low-benefit complication to the core that no one has seen fit to write.