but this approach wouldn't work for the actual release

Sure it would, as long as its conditional :)

Make sure you run your author tests before make dist and uploading

Always test things that don't require credentials

If user is brave and provides credentials, run the tests that need credentials, otherwise skip these particular tests

Or you can mock (fake) these credentials-needing tests instead of skipping outright or running them live

See http://search.cpan.org/dist/Net-OAuth/MANIFEST

See Test::Kwalitee :)

Thank you for the various tips for the CPAN first-timer. Seems like making the credential-tests conditional is the way to go here. I'll also check out Test::Kwalitee and Test::XT as well.

I have some tests in modules that require user input. I put them in a test.pl file so that tests won't fail with auto build processes (like that of Activestate's PPM) but also allow the user to supply information if they want to run the full test suite.

Have a look at the test.pl script in the Cisco::Management module for example. The test.pl script asks for a router IP address and SNMP community strings to run the full set of tests. Pressing just enter skips the suite entirely. That may be a way to go - put your regular tests in a file and the ones that require authentication - put them in a test file that prompts the user for input (their own credentials) or allows them to skip the tests.

Thanks for your reply. I'll take a look at test.pl in the module and learn the tricks from there.

Don't.

Doing your own prompting and reading from STDIN is bad practice. Instead use ExtUtils::MakeMaker's prompt() subroutine. It will cope properly with things like buffering when CPAN.pm's output is being piped to tee(1) or if it is being driven by Expect, and with the user setting PERL_MM_USE_DEFAULT.

Say your module normally gets its credentials this way:

  my $obj = Your::Module->new($username, $password);
[download]

Rewrite Your::Module so that, when passed undef for the username and password, it reads a default username and password from ~/.your_module.json.

Then write your test cases to use undef usernames and passwords.

Lastly, at the top of each test file that needs credentials:

use File::HomeDir;
use File::Spec;
use Test::More;

my $creds = File::Spec->catfile(
  File::HomeDir->my_home,
  '.your_module.json',
  );
plan skip_all => 'Cannot test without ~/.your_module.json'
  unless -f $creds;
[download]

Be careful to ensure that none of your tests cases can result in dumping the credentials to STDOUT - people might not like their Yahoo account details being posted to CPAN Testers.