When you use RETVAL, XS builds it in such a way that it's functionally the same as the first item on the return stack. Thus, you've effectively stuck an AV into $ents[0], which I'm guessing Perl DWIM's here and turns it into a reference to an array. Hence the additional level of references.

The correct way (I believe) to return a list in this situation is to declare your function with a return type of void and simply push each element onto the return stack instead of trying to return an array, using PUSHs I think.