Just because 'xor' is low-precedence doesn't mean you can't use it. Just wrap expressions in parenthesis so that Perl knows what it needs to evaluate first.
if ($a xor $b) { ... }
if ($a || $b xor $c || $d) { ... }
if (($a xor $b) || ($c xor $d)) { ... }