I even edited the DH.pm file, so it only looks for Pari, still not luck

I think you'll also need to install Math-BigInt-Pari for that to work as intended. (It should be easy enough to build and install if you already have Math::Pari.)

Note that the message you referred to in your post is just an "fyi", not an error. However, without the Pari or GMP support, Net::SSH::Perl will be very slow.
Note also the advice already given by salva in this thread - imo, it's sound advice.

Cheers,
Rob