What version of Perl is this? Did you compile it with 64bit integers? Try building 5.6.1 with 64bit ints and see if that helps. I think it will but your system's underlying stat() might get in the way.
On my system (FreeBSD 4.5, x86) st_size is off_t which is a long long unsigned (64 bit). The perl version is 5.005_03, which is shipped as part of the os.
I will try some workarounds and if I don't succeed I will give 5.6.1 a try.