I probably don't know what I'm talking about, but I'm wondering if there is an issue with Math::Pari (and/or required libraries/modules/etc.) and having them in a 64-bit version.

For a comparison, I took a quick look at what Strawberry Perl includes in their distributions. For 32-bit 5.16.3.1 (release notes), they do include the Math::Pari module. However, for 64-bit 5.16.3.1 (release notes), they do not include that module. In the "Known Issues" section of the release notes, they have a link to a bug report (https://rt.cpan.org/Public/Bug/Display.html?id=52146). That bug report might give you some clues to help you with your situation.