A consequence of Perl's weak typing is that TIMTOWTDI applies equally well to negation of numbers. Since the XOR approach relies on the numerical model and current state of your variable, it is likely to be less portable and so is less desirable. See
perldata for some info on what constitutes a "number" in Perl.
And if your numerical operations need to squeeze out every bit of computational efficiency, you probably shouldn't be using Perl.