I'm not personally familiar with Citrus Perl that was suggested by someone else. For Windows, Strawberry Perl does offer a portable version that can be used on a flash drive.

For some info on how to create a portable Perl for Linux, someone shared some links on how to do this at Re^4: Advantages of Activeperl vs Strawberry Perl.