Say your module normally gets its credentials this way:

  my $obj = Your::Module->new($username, $password);
[download]

Rewrite Your::Module so that, when passed undef for the username and password, it reads a default username and password from ~/.your_module.json.

Then write your test cases to use undef usernames and passwords.

Lastly, at the top of each test file that needs credentials:

use File::HomeDir;
use File::Spec;
use Test::More;

my $creds = File::Spec->catfile(
  File::HomeDir->my_home,
  '.your_module.json',
  );
plan skip_all => 'Cannot test without ~/.your_module.json'
  unless -f $creds;
[download]

Be careful to ensure that none of your tests cases can result in dumping the credentials to STDOUT - people might not like their Yahoo account details being posted to CPAN Testers.