This works great:

function perldoc() { command perldoc -n less "$@" |man -l -; }
[download]

Except it's not portable because the version of man included with the BSDs (including macOS) doesn't support reading manpages from stdin. Switching from a pipeline to temp files should be portable.

With a recent groff version (>=1.23), this also works:

GROFF_NO_SGR=1 perldoc -oman ...
[download]

Older versions of groff support the -c flag, but Pod::Perldoc::ToMan would need to be updated to allow the user to specify it.