How can I stop this without editing the cpan package?

turn off debugging/logging option which sends messages to stderr by manipulating the appropriate variable or calling the appropriate method

See also STDERR going to string