Slightly off-topic, but why wouldn't you use some very small linux dist like Damn Small Linux or Puppy Linux? They need only 50MB, that lets plenty of room to install Perl, your modules and your files on a 512MB key. BTW a stripped-down version of Perl can be automatically installed to Puppy Linux with the PupGet utility.
All that it does is to show the user a console with some text to which a special key is to be entered. After that everything else is automatic and the user must only acknowledge the end of the process before rebooting the machine. This is a dedicated application, not a general Linux. It could turn out that it takes equal time to strip down a DSL or Puppy (or Slax...) to make it as bare as possible. Also, that USB key is already in service and works well, albeit without Perl.
While adding Perl, I though of doing a bit of generalization of the key and derive it from LFS instead. I already build LFS systems, of course. If not, then _that_ would take a lot of time ! ;-)
I understand, I thought you wanted some sort of "pocket development environment", in which case something like a preconfigured PuppyLinux would rock! ;)