Code reuse and clean packages

FoxtrotUniform has asked for the wisdom of the Perl Monks concerning the following question:

This is more of a software-engineering question than anything else, but PM has some pretty good SE people. Thus:

I've written a lot of utility code for my own use. Some of it is probably genuinely useful to the rest of the world; some of it is a collection of reinvented wheels (although I like to think that my wheels are better suited to my style than what's already out there); and some of it is rather specialized to my toolkit and of limited interest to the rest of the world. A fair bit of this code is getting to the point where it should be shared between many of my projects, but I'm having trouble deciding how to divide it.

Right now, I have two major projects in usable states: a rather specific utility library (real-time 3d graphics; it's basically a wrapper around SDL and OpenGL with some extra features thrown in) and an application that uses said library. I have a few other projects that're just starting up, though, and some of the generic code in the library (things like logging routines, memory management, and so on) would be useful in those projects, although they aren't RT3D-related.

So what I have right now is One Big Library. It's useful for me, because it's easy to link to and stuff, but if I want to move my code to a different environment it all has to come along, and if I want to share my code with other people they may have to wade through a fair bit of unnecessary RT3D cruft to get the malloc() wrapper I used.

The obvious alternative is to break the One Big Library up into several smaller ones, each doing one thing (and, at least in theory, doing it well). That makes more sense from a distribution perspective, but it means that my build process gets a bit more complex, and that if someone needs all of my code they have more libraries to worry about. Breaking the code up into orthogonal, single-purpose chunks sort-of feels like the right thing to do, but it's going to add a lot of complexity.

Should I just bite the bullet and accept that I have a few weeks of refactoring ahead of me, or am I missing a third way? (And while we're at it, any pointers to papers, essays, or other literature on the subject would be greatly appreciated.)

-- F o x t r o t U n i f o r m Found a typo in this node? /msg me % man 3 strfry

Comment on Code reuse and clean packages Download Code

Replies are listed 'Best First'.
Re: Code reuse and clean packages by Zaxo (Archbishop) on Jun 09, 2004 at 22:42 UTC
I think you certainly should do the refactoring. Whether the factored modules should be packaged separately is a matter of what namespace they belong in. For example, if your logging module is distinct and useful enough that it could take a place in the Log:: namespace, it probably should. If not, perhaps your private logging should be rewritten to use something already in Log::*. That would add simplicity. I'd say the whole object of refactoring is to add simplicity. Doesn't mean it's not a complex task, of course. After Compline, Zaxo	[reply]
Re: Code reuse and clean packages by kvale (Monsignor) on Jun 10, 2004 at 00:03 UTC
If your library is seeming big and gnarly to you, it will probably be really hard to digest for someone who is seeing it for the first time. Creating a refactored hierarchy can go a long way toward helping someone make sense of your code and gives them a nice warm feeling that each component is a encapsulated piece, free of action at a distance in all the other code. It is true that that the build process becomes a little more complex, but once the makefile/autoconf scripts are written, it becomes simple for the user. Personally, if I see a C lib with an understandable layout and good docs, I'll live with whatever build process that is around. But the simplest build process imaginable won't compensate for a hard to understand library structure. I think an excellent exampe of this is the Qt library. It has a funky build system, complete with its own moc preprocessor, but the routines are arranged and described so nicely, I can deal with signals, slots, and mocs. -Mark	[reply]
Re: Code reuse and clean packages by toma (Vicar) on Jun 10, 2004 at 05:22 UTC
On-topic essays are plentiful at WikiWikiWeb. For example see BigBallOfMud and RavioliCode. Maybe YouArentGonnaNeedIt. It should work perfectly the first time! - toma	[reply]
Re: Code reuse and clean packages by McMahon (Chaplain) on Jun 10, 2004 at 14:58 UTC
One way to approach the problem is to refactor according to some QA-style questions: How hard would it be to debug this chunk'o'code? How hard would it be to explain this code to a newbie? How hard would it be to reproduce an error condition in this code? How hard would it be to test this chunk'o'code? Whenever the answer is "very hard"-- refactor; extract; simplify; document. From your description (useful code!) this might not be as hard as it sounds, once you dig in with a particular set of criteria in mind.	[reply]
Re: Code reuse and clean packages by paulbort (Hermit) on Jun 10, 2004 at 17:33 UTC
In addition to the very wise suggestions already made, I'd like to add an intermediate method, between the two extremes of 'leave it alone' and 'lots of little packages in proper namespaces'. First divide the functions into logical groups. Graphics in one pile, IPC in another, DB in a third, and don't be afraid to have one pile called 'general' or even 'misc'. If there is some code that might go to CPAN, and some that is proprietary, keep that in mind. Those piles are reasonable candidates for becoming libraries as they stand, with one extra consideration: You may want to re-arrange them to reduce dependencies. For example, if you put the function that filters CGI input in the HTML generation module, but an importer module needs to use just that function, and doesn't generate HTML, then that function should probably live in the 'general' library. Depending on how you build the libraries, you might not have to do weeks of work to make this happen. My first cut at this for my current project involved making modules that were just a big pile of `sub foo {...}`, and a simple `use MyModule` gets all of those functions into your namespace. So you can automate some of the conversion process by removing the `use` you already have, doing a syntax check, and then adding the new `use` for the individual modules needed. Unless you have thousands of scripts, this shouldn't take weeks. (N.B.: A powerful editor will help. Notepad is not a powerful editor.) -- Spring: Forces, Coiled Again!	[reply] [d/l] [select]