Some of these kinds of problems, it never feels like there is a good solution. What I would probably try if it were my row to hoe is to just start writing unit tests, definitely not system tests, with the intent of just sketching, not trying to do finished work or even tests you'll necessarily keep. That forces the thing down to atoms and “I wish I could just do *this* instead!” and that might give you insight as to what belongs where and why; in practical instead of theoretical/philosophical terms.

I don't think there is a best practice here. I think a lot of large projects full of concerns that should have been separated get so balled up that they can never get split up properly. Every step into that world requires two steps to back out. The code base at work is a testament to this. I honestly think a total rewrite would be easier than realigning 20 years of disparate but deeply interdependent code.

FWIW, I'm playing around with my own micro-web-framework right now and it's my… probably fifth run at it. What is helping me decide how far to go or not go is trying to write consuming code and make it testable. Might just be another learning exercise in the end though. :P

The reason you are finding what you are trying to do, hard, is because it is a purely intellectual exercise, not driven by either need or urgency.

You'll probably take that as a criticism, but it isn't intended that way, we all have our back-burner projects. I've dozens.

IMO there are exactly two reasons to 'create a module':

1. It forms a 'natural' subdivision of the code.

   These 'fall out' of the designing, prototyping, refining or testing the project.

   You don't need to ask yourself those questions, because the answers are obvious.

2. You find yourself doing something in your latest project that is sufficiently similar to something you've done before at least once, and preferably twice, that a copy&paste would create technical debt that you will find yourself repaying.

   Ie. You are quite certain that the cost of extracting the code from the original project(s); (re)naming it; defining a common interface for it; and re-working those original projects will save you and/or your clients time/money in the future.

Any and all other reasons are 'costs'. To you and your life, and your company or institution. Investment, maybe; but only iff ....

And whether the ...s ever happen, whatever they may be, is unknown and unknowable. Like an old man putting money under the mattress for a rainy day that may never come.

That doesn't mean you don't do them -- if (for example) OSS is a hobby of yours, then 'somebody might find it useful' can be enough of a reason.

If you are young enough, and work (or aspire to) in a field where OSS contributions can be considered a part of your CV, better still.

But structuring and restructuring code to satisfy the intellectual need (your own, or that of others) to 'get it right' has been the ruination of vast numbers of commercial projects.

And if this is a purely personal project, the knowledge that you 'did it right' will be of scant comfort when you look back at all the things you didn't do to achieve it.

I mostly agree, but my question is less
• if I should refactor the code
• but rather how to refactor with a smart strategy.
When talking about CPAN we tend to mix the concepts of distribution and module, depending on context.

I'm intending to publish this distribution on cpan and am reluctant of handling dependencies between spin off distributions, while I'm still in the design phase.

Dependencies between modules are much easier to handle, because they can all be part of the same distribution.

And that's also showing the current result of my meditation, I'll gradually split into several modules inside the same distribution.

As soon as I feel the urge to publish one sub-module independently I'd check if it's mature enough (documentation, tests, naming) and do so.

Like this I keep full flexibility to continue developing the main project without being hampered by premature publications.

I hope it's clearer now.

For those wondering about my terminology:°

• a module is a .pm file which is found in @INC and use 'd
• a distribution is a complex directory structure with Perl files and meta info
Let's take Module::Starter for demo.

It's a module to create distribuitions, but is also available as cpan distribution.

The structure of the Distribution is seen in https://metacpan.org/source/DBOOK/Module-Starter-1.76.

The actual main Module is in https://metacpan.org/source/DBOOK/Module-Starter-1.76/lib/Module/Starter.pm

My main question now is how to best extend that directory structure to host sub-modules,

• An own top-directory /extra which holds stubs of these new distributions?
• Or just descending into my /lib/Module/Starter/ and adding new modules there?

( which is actually a strategy applied there, see https://metacpan.org/source/DBOOK/Module-Starter-1.76/lib/Module/Starter )

Most probably again a mix of both, I'll refactor into the same name space first, and will promote sub modules to own dists later on demand.

I hope my mediations are clearer now.

Cheers Rolf
_{(addicted to the Perl Programming Language :)
Wikisyntax for the Monastery FootballPerl is like chess, only without the dice}

updates

°) https://www.perl.com/article/96/2014/6/13/Perl-distributions--modules--packages-explained/