Should we look to move to 64 bit Perl ( Perl 5.10) too?

It's hard to give a general recommendation without knowing the details.  The two main advantages of 64-bit perls are more addressable memory, and larger native integer numbers. Whether this is useful for you depends on the applications.  On the other hand, there are also some potential incompatibilities due to this — for example, see the discussion emulate 32-bit on perl 64.

But before migrating critical 24x7 applications, you would set up a test environment anyway, and thoroughly check for any incompatibilites... wouldn't you?

The most important question is: does your application support both 32bit and 64bit? e.g. Modern oracle only supports 64bit, which *requires* 64bit perl. If however you are depending on old9er) shared libs that are only available in 32bit, you'd have to stick to 32bit perl.

And TEST before you go live.

Side note, when moving, why not move to a more modern perl too? You'd have to test anyway. The most recent stable release is 5.14.2 (but I do not know what your distribution can offer).