Ok, your example made me realize that the best approach for now would be generating a unique reference using... eval:

my @arr;
push @arr, { xx=>eval '\99' } for 1..2;
say YAML::XS::Dump(\@arr);
[download]

Which prints:

---
- xx: !!perl/ref
    =: 99
- xx: !!perl/ref
    =: 99
[download]

which in place Loads fine, working around the problem.