You shouldn't modify modules from CPAN. Otherwise, upgrading removes your changes. If you need to make changes, you should write a wrapper around it. If you can't, you should be working with the author to incorporate your changes back into the mainline.
If something needs compiling on a machine that doesn't have the tools, then you have a problem no matter what you do. Something on one machine will (almost) never work on another machine, particularly if it's a different OS (like compiling on Linux to work on Windows).
Subversion is the SCM (source control manager) that I use and that much of the Perl community is moving to. You might want to check out SVK - it's a wrapper around SVN that adds a lot of useful functionality.
Deployment is always a hard topic. The biggest key is being able to deploy with one command. At $work, we tend to deploy using SVN. But, that's not necessarily the best solution in all (or even some) cases.
A good testing infrastructure isn't something you can just throw together. Catalyst has an excellent testing infrastructure you might want to look at. It allows you to simulate various requests and see how your application code responds. This is a "Good Thing"™.
My criteria for good software:
Does it work?
Can someone else come in, make a change, and be reasonably certain no bugs were introduced?