Perl has limited support for cross compiling, searching p5p for perl5-porters cross compile yields some threads where vkon participates (for WinCE) and this thread on cross-compiling on glibc for uClibc. But what I read there is not conclusive.

In theory (I've never done this myself), you should be able to follow the outlines of what vkon describes in this post.

My approach would be to try a "native" compile by either getting gcc to compile on the target machine or by using a VM to emulate a target machine with more RAM and CPU for compilation.

Maybe you can show us how far you get and where the problems are?