You can't do that!

Either use a 32bit Perl or look for an Oracle client library compiled for 64bits