I don't know if this'll help at all with your wanting to do incremental testing, but I started using Travis CI for build tests prior to uploading to CPAN to let a wider range of test platforms have at it. In doing this, I minimize to the best of my ability not uploading something to the CPAN that will fail because of something stupid, making publicly available a broken module, and taxing the CPAN Testers for nothing.

Just put your code on Github, sign up for Travis-CI, create a .travis.yml file in your distribution's root directory (example contents below), and then with every push, you get testing on a wide range of perl versions, across Mac and Linux (no Windows unfortunately... I have two Strawberry Perl VMs I test my modules on manually prior to upload to CPAN).

This is an exact working copy/paste of a .travis.yml file from one of my modules.

language: perl

perl:
    - "blead"
    - "5.22"
    - "5.20"
    - "5.18"
    - "5.16"
    - "5.14"
    - "5.12"
    - "5.10"

os:
    - linux

before_install:
  - git clone git://github.com/travis-perl/helpers ~/travis-perl-helpe
+rs
  - source ~/travis-perl-helpers/init
  - build-perl
  - perl -V
  - build-dist
  - cd $BUILD_DIR             # $BUILD_DIR is set by the build-dist co
+mmand

install:
  - cpan-install --deps       # installs prereqs, including recommends
  - cpan-install --coverage   # installs coverage prereqs, if enabled

before_script:
  - coverage-setup

script:
  - perl Makefile.PL          # or Build.PL if it exists
  - make                      # or ./Build
  - make test

after_success:
  - coverage-report
[download]

Here's an example Travis report on one of my modules. You can then drill down to each version of perl to see the test results themselves.