Install Math::BigInt::GMP and use Math::BigInt lib => 'GMP'; and things get much faster. You can distribute code in with that use, without the module locally installed everything still works, it just falls back on a slow Perl version. (Some may prefer Math::BigInt::Pari.)

Math::Big is meant to be a convenient wrapper around other modules. Use a faster one and it gets faster. Use a slower one, and it isn't.