Upgrades from external sources is a risk-laden activity.
Continuous Integration is a form of automation, and automation is dependent on a risk-averse environment.

I would not trust my product development process to be dependent on an automated CPAN upgrade/install/deploy process until my testing and monitoring infrastructure had proved themselves over time. Update: I also suffer no delusions that it would ever truly reach this state.

I think your local PPM cache is a brilliant work-around, and permits you to continue to develop and refine the automation of your CPAN upgrade and testing process as you see fit. If you ever get to the point where you trust that back end automation, you can choose when to integrate that into your PPM cache.

Until then, you're stuck with a mostly-manual process, substituting the time-honored "sweat and blood" for automation, all in the name of reliability. Theoretically, this should keep you motivated to work on those back end upgrade, testing, and monitoring tools. :-)

Or you could settle for releasing crap.

I do hope you saw that last line as sarcasm. :-)

Personally, I would look at CPAN::Mini or Pinto for maintaining an "approved" CPAN locally. The VM should only install from that approved CPAN.

You can take a look at the CPAN distroprefs for setting up your local compilation chain for problematic modules with non-CPAN extensions, but for those, I would consider building PPMs and install these. I don't know how Pinto handles non-CPAN binary blobs like PPM or other files.

My first reaction was: Don't do it!

Install and test your set-up very thoroughly and then; stick with it until you have a need to change it. Then install and test the set-up with the changes, very thoroughly and then test again. And only then move your workload(s) over to it. But keep the old one running in parallel, ready for a quick back out if the results from the two do not compare. Only when you've run that way for a month or so, do you take the old set-up down.

Continuous integration is change for the sake of change; and (IMO) bloody madness.

But, assuming that isn't an acceptable answer to your question; then I'd suggest a two stage solution: one is your production system; the other your integration&test system. You run both with live data in parallel. Make your changes to the I&T system on Friday's whilst hording the input data; leave it to catchup with the production system over the weekend.

If on Monday everything checks out; it becomes your production system and the production system becomes your I&T system. Make the same changes to the new I&T system, run in parallel for a few days, then add the latest set of changes on Friday; catchup over the weekend.

Rinse & repeat.

In this case, this is more about pre-release testing than about continuous deployment (ie we build and test every commit on develop, then every so often push a new version to prod, rather than pushing every commit straight to prod)

Still sounds like your allowing your process to be driven by every random change in every one of your dependencies; rather than by the need of your own code.

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority". I'm with torvalds on this

In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

I start with Module::Starter and then i immediately cd into the distro and initialize a local git repository and check in the code. Then i make my initial customization configuration changes and check those in. Then i hope on over to GitHub and set up a project and then i push my code to that project. Now i can work on any computer that i have registered my ssh keys with Github and push and pull accordingly. Most places of work have internal implementations of Github these days (like Stash or Gitlab) so security/IP issues should not be a blocker.

My development process is very much test driven. I do not add features without testing them. I also make sure my tests run in a very very short amount of time. Less than 10 seconds is my usual goal. I keep my tests simple and explicit, rarely using Object Oriented frameworks unless i really do have a lot of start up and tear down concerns. I never check in code unless all tests pass. If a feature changes then i make sure the tests are changed to reflect the new requirement(s).

Within Github i set up a travis-ci config file and i use travis to smoke test my CPAN module with 6 versions of Perl. Every push to the Github master triggers a new build at travis-ci. I also have my own Jenkins server running and i use that to build, test and create the tardist candidate to be uploaded to PAUSE. From here, i use perlbrew and cpanm to install and uninstall that candidate tardist for a final verification that it is ready to be promoted to "production." Perlbrew allows me to manage multiple versions of Perl locally, so i can easily smoke test by hand if CPANTS hands me back an unexpected failure report (usually days after the upload). Finally, i upload the built asset from my Jenkins server to PAUSE and the bump the version.

Jenkins saves all my successful builds and shows off the tests results thanks to help from TAP::Formatter::JUnit. I do not use a VM for this process because there is no need to. I would use a VM if i needed to build a server to provide a service, not to develop my code. If your current CPAN dependency chain takes 2 hours just to build, doesn't that mean that a client wanting to use your CPAN module will have to wait the same amount of time? You could always build an image of the "base" with something like Packer and use that to develop/test your code with as well. Many businesses are doing that currently because their build times are extremely lengthy.

L-LL-L--L-LL-L--L-LL-L--
-R--R-RR-R--R-RR-R--R-RR
B--B--B--B--B--B--B--B--
H---H---H---H---H---H---
(the triplet paradiddle with high-hat)

Nicely done.

I am curious how long it took you to get from first steps of implementation to the system you have today, and how much it has needed to evolve over time.

Care to share?

I think jeffa has your posts on ignore. Can you blame him with all the slandering you've done?

Ideas like using packer sound like what I'm after - if you're building a packed prerequisites bundle, would you rebuild it every time you modified the prereqs and store it centrally?

I’m surprised no one has mentioned Carton. You can control the versions of your dependencies. That or Corion’s suggestions which allow you even more stability. jeffa’s https://travis-ci.org/ related suggestion is also terrific but doesn’t in and of itself control dependency issues. You can mix the approaches to do so. But the continuous integration tests will almost certainly catch dependency problems if your test suite is half-decent. This will make you a better Perl citizen because you can feed back the breakages to the module owners in question and improve the quality and robustness of the whole chain you’re using.