in reply to Re: testing/release advice [was: Stopping tests]
in thread Stopping tests
My understanding was that we should always upload a dev release first!
The meaning of "first" is my quibble. If by "first", you mean "before I've done a initial public release", I have no problem with that (though I've never done that on my handful of CPAN modules, and don't remember seeing that advice). If by "first" you mean "before I've thoroughly tested my distribution", I would vehemently disagree with that understanding. :-)
if we upload a production release before a dev release, people can start using it before it is properly tested
Those who want a pre-release can install a pre-production module from GitHub, it doesn't have to be on CPAN for that.
However, one test I couldn't write without the help from Testing image output. I had planned what I wanted to test but couldn't convert that visual test into a programmatic test.
And CI would have helped you figure that out, even if your own machine isn't showing the error. Another vote in favor.
As a very recent user of GitHub, I had no idea I could test modules with it
Hence my post. :-) And hence my issue and PR, to help you get started with that. I got my start with stevieb's excellent Travis-CI test structure and extended that to AppVeyor (though I don't remember where I cribbed my first .appveyor.yml, sorry), then much later used his GitHub Actions test structure to start converting over to GitHub Actions instead. One usually needs a starting point for such things, unless one is very good at starting from raw documentation.
It strikes me as very odd that we have to run gmake disttest, then go and copy files from one place to another.
I've never had to do that with my gmake dist / gmake disttest . I vaguely remember being highly curious why you needed to do that, the last time you mentioned it, but didn't have time at that point to look into it, and I had probably hoped that someone else would disabuse you of that need. It might not be today, but I'll try to put that on my mental back-burner again. If no one else pipes up in the next couple hours, maybe you should create a new issue on your repo and @mention me over there (maybe even assign me, if it lets you), and that might serve to poke me over time.
Given where I am with 4 dev releases, what would you advise I do next?
Merge the CI environment I gave in the PR, and watch it fail. Then start debugging your test and/or module code until those CI tests are passing. You might want to create an issue in GitHub to track it, and if you're having problems getting it to pass, you could even show your exact steps that you have tests passing on your personal machine in a comment on that issue, so they can be compared to what the CI does.
I would suggest not doing another dev release to CPAN until you have a suite of tests that are working both for you and in the CI. Only after getting those tests passing would I say you should do another CPAN dev release. And given the number of problems you've had, dev release is still probably the best bet, until you've got a wider array of passing results. (I'm normally willing to do an initial full release after my personal tests are all passing, and my remote CI tests are passing, and I've done enough to convince myself that my distribution works well enough that someone in the wild would be able to get it to install and pass and do what my docs say it does -- having 100% test coverage helps give me those warm fuzzies, though it's not required in my personal workflow.)
Is it sensible to remove the tests that fail and upload a production release next and then develop it further from there?
IMO: Only if you are confident that the core features in your distribution actually work right, and that your documentation explains which features are not fully tested or might not work for others, even though they work on your machine.
I feel really weird giving advice like this. I'm just an amateur with a few CPAN modules that, as far as I know, no one but me uses (okay, I know of one other user of one of my modules, but one real user is statistically zero compared to CPAN norms). I'm not sure why anyone would trust any of my advice. And I definitely make no claim that it's "best practices".
|
---|
Replies are listed 'Best First'. | |
---|---|
Re^3: testing/release advice [was: Stopping tests]
by pryrt (Abbot) on Sep 13, 2023 at 21:19 UTC | |
by Bod (Parson) on Sep 13, 2023 at 21:46 UTC | |
by pryrt (Abbot) on Sep 13, 2023 at 22:01 UTC | |
by etj (Priest) on Feb 22, 2025 at 13:15 UTC | |
Re^3: testing/release advice [was: Stopping tests]
by Bod (Parson) on Sep 13, 2023 at 21:34 UTC | |
by stevieb (Canon) on Sep 14, 2023 at 12:43 UTC | |
Re^3: testing/release advice [was: Stopping tests]
by Bod (Parson) on Sep 13, 2023 at 22:50 UTC |