I think the reason is in your own words :-)

doing math on numbers

The Perl compiler probably knows that $number % 2 is just "testing the last bit", which probably is more efficient than even the most optimized regex for testing "even"-ness. Plus you don't need to add 2 back in.

I assume, because $number is populated by counting, Perl doesn't even need to convert from string to number, so the other special cases are possibly also not an optimization.

For a human Perl Interpreter, of course, the situation looks different :-)