I think in a lot of cases it doesn't matter (especially if die'ing), but there is a difference in precedence. perldoc perlop shows that || is higher than = and some other operators, and or has the lowest pecedence.
I've had cases where || return needed to be changed to or return so that Devel::Cover wouldn't complain about never getting a true value for the other half of the construct.
If you use 'or' here, you'll die upon failure of somesub (assuming it returns false to indicate failure). If you use ||, you'll die if $that is false. Big difference.
Just because you're invoking die after an || or a or doesn't mean you don't have to think about precedence. Consider the following:
somesub $this, $that || die;
If you use 'or' here, you'll die upon failure of somesub (assuming it returns false to indicate failure). If you use ||, you'll die if $that is false. Big difference.
If you call somesub with parens, the argument list is clearer to novice coders, the notation matches what they learned in grade 6 math class, and in this case, the precedence issue disappears.
In my experience, thinking too hard about precedence is usually a clue that you're being too tricky with your code.