LWP::Protocol::https is actually specified in the "suggests" part of the LWP Metadata. Perhaps you could trace through those areas as well when you are building the dependency tree?

Unfortunately, there is an implicit (but common) dependency, where LWP often depends upon LWP::Protocol::https ... implicit ...

There is nothing implicit about it, its simply optional https://metacpan.org/source/ETHER/libwww-perl-6.13/Makefile.PL

       prereqs => {
            runtime => {
                suggests => {
                    'LWP::Protocol::https' => '6.02',
                    'Authen::NTLM' => "1.02",
                    'HTTP::GHTTP' => '0',
                },
            },
        },
[download]

Thanks to you all for educating me about "suggests". Now I can go back and figure out how this analysis was run, and figure out how much additional baggage would be required if I included all of the "suggested" modules.

You shouldn't have to include all the suggested modules. As you said, this just happened to one of your customers in the field. They probably gave a configuration that triggered it such as providing a HTTPS URL to connect to something. Your test plan may have overlooked that so there is a hole in the test plan. I'm going to assume that during your build routine, there is a scan of the code or read prepared file listing the dependencies. Sounds like you need to modify that to add the suggested ones to the top level dependencies list.

Hi:) Some more specifics CPAN::Meta::Spec#dynamic_config, https://metacpan.org/pod/CPAN::Meta::Spec#Prerequisites-for-dynamically-configured-distributions ... cpanfile/cpanm/carton

Hmmm... if the OP says that the absence of this module caused “one of our products to break in the field,” maybe that dependency isn’t so “optional,” after all.

I didn’t even know that dependencies could be optional.   There certainly isn’t anything intuitive about that idea, at least to me.   Out of curiosity, how do CPAN-authors determine that such a status should be assigned?   Can it be overridden?   Can dependency-checking tools be instructed to include these “optionals” in their scans?

It sure does sound like the mechanism failed the OP and at least one of his customers.   So, what needs to be done about this?

I didn’t even know that dependencies could be optional. There certainly isn’t anything intuitive about that idea, at least to me.

Arguably, supporting HTTPS can, as of recently, be considered required. So, in the specific case of LWP::Protocol::https, perhaps that should now be required.

As for "optional features" in general, perhaps a better way of handling those? I suppose one could argue that optional feature "bar" of module "Foo" should be used via module "Foo::Bar". but, going back to the OP's use of LWP, would you really want to have to call different "get" methods for different URL schemas? Or would you rather one "get" that looks at the "https:" or "ftp:" at the front of the URL and figure out which protocol to use?

I suggest that, as an option, modules devs be able to specify "additional dependencies" as a hash of lists mapping "optional features" to the additional modules required to support said optional features. I'm not sure how best to to handle such options during module install, but having a feature/requirement map available will help with managing installed modules.

You're asking for a list of third-party plugins and other modules dynamically or conditionally used by commonly-used modules. A lot of modules have these. In fact, most large distributions probably do it.

Here's a few I've had to deal with:

• DBI uses database drivers (DBD::*).
• JSON uses JSON::PP or JSON::XS.
• LWP uses protocols (usually LWP::Protocol::*).
• syntax uses features (Syntax::Feature::*).
• XML::SAX uses of many SAX-based XML parsers.
• XML::Simple uses XML::Parser or XML::SAX.

And thats just on the perl side of things, Wx for example has different widgets available based on which version of wxWidgets you install