In my personal opinion, doing a dev release to CPAN without an initial release without having at least done your own suite of multi-environment testing borders on abusing the cpantesters network

My understanding was that we should always upload a dev release first!

I thought I got that from Re: What do I use to release a module to CPAN for the first time? but rereading it, that's not where it came from...
The point of this thread was to try to lessen the load on cpan testers, certainly not to abuse them.

Surely, if we upload a production release before a dev release, people can start using it before it is properly tested - or am I missing something here?

there's no reason for not doing a suite of tests long before any release or dev-release to CPAN, on at least Ubuntu and Windows

The tests were created ahead of writing the code. However, one test I couldn't write without the help from Testing image output. I had planned what I wanted to test but couldn't convert that visual test into a programmatic test. v0.01_1 omitted that test, subsequent versions included it and tried to get it to work in different environments. The tests were all run locally using prove and passed...I wouldn't have uploaded a module that failed local tests.

As a very recent user of GitHub, I had no idea I could test modules with it!

And as a final small piece of advice: always include the META_MERGE in your Makefile.PL

It's on my list of things to do...to work out what all these META files are in a distribution...

It strikes me as very odd that we have to run gmake disttest, then go and copy files from one place to another. Surely, the toolchain should do that for us if it always needs doing, shouldn't it? I always forget if I'm supposed to be copying META.yml or MYMETA.yml. I guess that if I understand the significance of these files, it will all be clearer.

Question...

Given where I am with 4 dev releases, what would you advise I do next?

Is it sensible to remove the tests that fail and upload a production release next and then develop it further from there?

My understanding was that we should always upload a dev release first!

The meaning of "first" is my quibble. If by "first", you mean "before I've done a initial public release", I have no problem with that (though I've never done that on my handful of CPAN modules, and don't remember seeing that advice). If by "first" you mean "before I've thoroughly tested my distribution", I would vehemently disagree with that understanding. :-)

if we upload a production release before a dev release, people can start using it before it is properly tested

Those who want a pre-release can install a pre-production module from GitHub, it doesn't have to be on CPAN for that.

However, one test I couldn't write without the help from Testing image output. I had planned what I wanted to test but couldn't convert that visual test into a programmatic test.

And CI would have helped you figure that out, even if your own machine isn't showing the error. Another vote in favor.

As a very recent user of GitHub, I had no idea I could test modules with it

Hence my post. :-) And hence my issue and PR, to help you get started with that. I got my start with stevieb's excellent Travis-CI test structure and extended that to AppVeyor (though I don't remember where I cribbed my first .appveyor.yml, sorry), then much later used his GitHub Actions test structure to start converting over to GitHub Actions instead. One usually needs a starting point for such things, unless one is very good at starting from raw documentation.

It strikes me as very odd that we have to run gmake disttest, then go and copy files from one place to another.

I've never had to do that with my gmake dist / gmake disttest . I vaguely remember being highly curious why you needed to do that, the last time you mentioned it, but didn't have time at that point to look into it, and I had probably hoped that someone else would disabuse you of that need. It might not be today, but I'll try to put that on my mental back-burner again. If no one else pipes up in the next couple hours, maybe you should create a new issue on your repo and @mention me over there (maybe even assign me, if it lets you), and that might serve to poke me over time.

Given where I am with 4 dev releases, what would you advise I do next?

Merge the CI environment I gave in the PR, and watch it fail. Then start debugging your test and/or module code until those CI tests are passing. You might want to create an issue in GitHub to track it, and if you're having problems getting it to pass, you could even show your exact steps that you have tests passing on your personal machine in a comment on that issue, so they can be compared to what the CI does.

I would suggest not doing another dev release to CPAN until you have a suite of tests that are working both for you and in the CI. Only after getting those tests passing would I say you should do another CPAN dev release. And given the number of problems you've had, dev release is still probably the best bet, until you've got a wider array of passing results. (I'm normally willing to do an initial full release after my personal tests are all passing, and my remote CI tests are passing, and I've done enough to convince myself that my distribution works well enough that someone in the wild would be able to get it to install and pass and do what my docs say it does -- having 100% test coverage helps give me those warm fuzzies, though it's not required in my personal workflow.)

Is it sensible to remove the tests that fail and upload a production release next and then develop it further from there?

IMO: Only if you are confident that the core features in your distribution actually work right, and that your documentation explains which features are not fully tested or might not work for others, even though they work on your machine.

I feel really weird giving advice like this. I'm just an amateur with a few CPAN modules that, as far as I know, no one but me uses (okay, I know of one other user of one of my modules, but one real user is statistically zero compared to CPAN norms). I'm not sure why anyone would trust any of my advice. And I definitely make no claim that it's "best practices".

Bod,

I looked into the META files issue a bit more.

I couldn't find the old post that your "META" comment reminded me of, exactly. (It may have been Re^3: What to test in a new module, but that was you calling it "META files" when it was actually talking about the META_MERGE parameter to MakeMaker, which defines some of the info that goes into the META.json/yml files. I thought maybe you had mentioned that copy back-and-forth before, but I cannot find it when I search your recent posts with "META". or it could just be that when you made the comment in this discussion, my mind invented a vague memory of some previous comment by you about having to do weird copy of META files. Sorry if my last post had a reply to a figment of my imagination.)

So I actually downloaded your repo, and went through the standard incantation:

perl Makefile.PL
gmake disttest
gmake dist
[download]

... I ran that with RELEASE_TESTING not set and with set RELEASE_TESTING=1 -- both passed° the disttest, and the distro .tar.gz that was created by gmake dist correctly included the META.json/yml files.

I then looked at the Re: What do I use to release a module to CPAN for the first time? post you linked to, and saw that step13 claimed you needed to copy the META.json/yml files to your main directory and manually adding those two files to the MANIFEST before doing the gmake dist : all of my experience and my experiments today say that is unnecessary. If I just follow the incantation above, then look in the .tar.gz, the META.json/yml are properly there -- I did't have to copy those files. Follow the standard incantation yourself, and see if you can see those files were properly created: if they do get created, then you can skip step 13 in your future releases, saving you time and effort. If not, please show the output from your sequence that you follow, and show the contents of the .tar.gz that is created.

---

°: where it "passed" all the tests except for the md5 hash tests (which I disabled, because they fail on my local PC as well) -- but the release-only tests were properly skipped when RELEASE_TESTING was not set, and properly passed with RELEASE_TESTING=1 ...

The meaning of "first" is my quibble. If by "first", you mean "before I've done a initial public release", I have no problem with that (though I've never done that on my handful of CPAN modules, and don't remember seeing that advice). If by "first" you mean "before I've thoroughly tested my distribution", I would vehemently disagree with that understanding. :-)

By "first" I mean after testing but before the public release...
However, until today, I wasn't aware that I could test a module in multiple environments other than spinning up a virtual environment. Given that this module is both passing and failing on Linux under the same version of Perl according to the test results, I figured I would have to spin up so many variations that it would be impossible, or at least impractical. Compounded by my servere lack of in-depth Linux knowledge.

Those who want a pre-release can install a pre-production module from GitHub, it doesn't have to be on CPAN for that.

The module was released to CPAN before it was uploaded to GitHub. Other than for Pull Requests, I've yet to see how GitHub is useful for Perl as people install them from CPAN. Given the number of Monks who use it and have advised me to do the same, I total accept it's the right way to go but I don't totally understand why yet.

I tend to do the development locally, test through example scripts, test through test files and prove them. Package and upload to CPAN then think about GitHub a few days later!

I vaguely remember being highly curious why you needed to do that, the last time you mentioned it, but didn't have time at that point to look into it, and I had probably hoped that someone else would disabuse you of that need.

I copy the META files because that's what's it says to do in Re: What do I use to release a module to CPAN for the first time?

Like I said earlier, I don't understand why I copy them or what they do. I really should but there is always something more pressing and, as it works, it is low priority to find out why it works...

Merge the CI environment I gave in the PR, and watch it fail

I've merged the META_MERGE Pull Request. My first time dealing with a PR from that end...I did one on Module::Starter under the excellent guidance of hippo but then heard nothing more. I'm looking at the CI PR now and will try to understand what it does and what I do with it!

I feel really weird giving advice like this. I'm just an amateur with a few CPAN modules that, as far as I know, no one but me uses (okay, I know of one other user of one of my modules, but one real user is statistically zero compared to CPAN norms). I'm not sure why anyone would trust any of my advice. And I definitely make no claim that it's "best practices".

I learnt a long time ago that you don't need to be very much further ahead to give valuable advice. Many times it's better if you are not too far further down the learning road as you haven't forgotten the challenges everyone goes through. Regardless of how weird you feel, your help is very much appreciated 👍

Merge the CI environment I gave in the PR, and watch it fail

I did watch - and observed something I had not seen before!

It fails on the horizontal image and passes on the vertical image...both on Ubuntu and Windows

My understanding was that we should always upload a dev release first! I thought I got that from Re: What do I use to release a module to CPAN for the first time? but rereading it, that's not where it came from...

Bod you're right, you didn't get it from the most excellent Re: What do I use to release a module to CPAN for the first time? by davido.

And you certainly didn't get it from my Writing Solid CPAN Modules ... because I'm a Dev release ignoramus :)

To remedy that I've just added some extra references to Writing Solid CPAN Modules:

• perlmodstyle (perldoc) - Perl module style guide (re making a dev release: if you want to release a 'beta' or 'alpha' version of a module but don't want CPAN.pm to list it as most recent use an '_' after the regular version number followed by at least 2 digits, e.g. 1.20_01)
• CPAN (perldoc) (FAQ 12) - How do I install a "DEVELOPER RELEASE" of a module? By default, CPAN will install the latest non-developer release of a module. If you want to install a dev release, you have to specify the partial path starting with the author id to the tarball you wish to install, like so: cpan> install KWILLIAMS/Module-Build-0.27_07.tar.gz
• Update (thanks hippo): CPAN::Meta::Spec (perldoc) - specification for CPAN distribution metadata (for release_status, if the version field contains an underscore character, then release_status must not be "stable")

Bod: note the leading zero in Ken's Module-Build-0.27_07.tar.gz ... I think you meant to use _01, _02, ... instead of _1, _2 in your Dev releases :)

More definitive references on CPAN Dev releases welcome.

More definitive references on CPAN Dev releases welcome.

Probably worth mentioning that CPAN::Meta::Spec now includes the release_status required field. The linked doc includes the proscription:

If the version field contains an underscore character, then release_status must not be "stable."

and finishes the section with:

Consumers may use this field to determine how to index the distribution for CPAN or other repositories in addition to or in replacement of heuristics based on version number or file name.

I see this as a belt-and-braces facility with the added bonus of being able to specify "unstable" as a value to indicate really alpha releases.

---

🦛