They're about the same; almost irrelevant in any real program.

I propose that they're exactly the same.

$ perl -MO=Concise,-exec -e'$$ref'
1  <0> enter 
2  <;> nextstate(main 1 -e:1) v:{
3  <$> gvsv(*ref) s
4  <1> rv2sv vK/1
5  <@> leave[1 ref] vKP/REFC
-e syntax OK

$ perl -MO=Concise,-exec -e'@$ref'
1  <0> enter 
2  <;> nextstate(main 1 -e:1) v:{
3  <$> gvsv(*ref) s
4  <1> rv2av[t1] vK/1
5  <@> leave[1 ref] vKP/REFC
-e syntax OK
[download]

The normal case of rv2sv boils down to a couple of checks plus

sv = SvRV(sv);
[download]

The normal case of rv2av boils down to a couple of checks plus

sv = SvRV(sv);
[download]

So any difference in performance is not intrinsic to the operation.