Got a git repo that we can point to for our own testing? I wrote Test::BrewBuild for testing these kinds of things (note that in win2k12+, the bbtester program can NOT be run in the background, one must use --fg mode. Background mode works fine on win2k8), and I'll gladly throw the tests against as many servers and perl configurations I can :)

You can test against all Perl/Berrybrew instances installed locally like this (inside of your distribution's root directory):

# test against all installed perls, without doing anything else

brewbuild

# -r removes all installed Perls, less the one you're 'switch'ed to
# -n installs N number of perls, randomly, and tests against them all

brewuild -r -n 4

# same thing, but after removing all Perls, install 
# specific versions to test against

brewbuild -r -i 5.22.1 -i 5.24.0 -i 5.10.1
[download]

Essentially, the ideal test is to wipe all of your perl/berrybrew instances away on each test run, so you are literally starting them from scratch.

When all else fails, email/contact the owner of the tester. This is listed in the Tester column under each OS/Perl ver test. For example, see this aggregate test report. Sometimes their email is listed in the end test results, other times, you can just drop their name into an Author search on MetaCPAN. If you can't find contact info on their Meta page, open up one of their distributions, and an email address can likely be found down around the Copyright section of the documentation.

People who operate testers for us are very friendly, and are willing to help diagnose problems you can't... such as running custom builds on specific configurations outside of normal CPAN Testers operations, providing feedback on their environment, or even looking at the code to see if they can spot issues you may have missed. (In fact, some of them are proactive; they'll email you about a test problem if they indeed know what the problem is before you've read your failure email the next morning).