The task of testing CPAN distributions is largely automated. What the Perl community is looking for from you is to provide hardware, the more unusual the better. However,there are certain platforms that CPANPLUS has issues on.

What tasks are you expected to perform?

Once you have CPANPLUS and associated modules installed, it's a matter of detecting new distributions that have been uploaded, and doing a CPANPLUS test install, mailing back the test results. All of this is automated.

You are expected to join and take part in the cpan-testers mailing list

What level of experience do you need to have?

In many ways, the naive user's approach with no technical input may be useful - finding issues that "seemed obvious", and detecting documentation errors. However, the ability to diagnose scripting and install problems is useful - differentiating problems on the test bed from problems with the module being tested.

what is the expected commitment in terms of time?

As the scripts are automated, you would be expected to support your test bed and deal with any idiosyncracies. Some of the more dedicated testers do analysis of failing tests, and give extra clues to the module author. This is something I regard as beyond the call of duty, but occasionally most welcome.