(OT) Open Source Needs Open Discussion

Months ago I researched an open source laboratory information management systems (LIMS) project hosted on Sourceforge (Gnosis LIMS). I work customizing a commercial LIMS so I was excited to see a fully open sourced system. I did not ask to participate in the project mainly because it was Java based and already had lots of developers and two admins. I left this issue alone for six months and came back to see how the project was doing. It went nowhere. Even with half a dozen developers and a project plan no work whatsoever was actually done code-wise. The principal of the project either could not or would not surrender the project to someone able to bring it to completion. My e-mail discussion with him revealed that he was considering giving the project to someone else but had not come to any decision. That was a year ago.

In another instance I found an open source shopping cart system (AgoraCart) that needed what I perceived to be a change to its core code. The script was using $ENV{'SERVER_ADMIN'} for error messages when failing to load libraries rather than using the variable for the admin e-mail in the setup. (On a shared server you have to call the system administrator to change the environment variables passed to your script versus you changing it yourself in the script's configuration file.) When I asked the maintainer to make the change he balked that he would not. I explained that the configuration item in the script was there for this reason. He did not seem to care.

You may be thinking, "If the project is open source, why not just make your own project?" I can answer that with an example: Imagine that there were one hundred Apache projects all using slightly different versions of the same source. Chaos would ensue. Users would become angry. The whole project would dissapate. Open source projects should be like rivers. As they grow they should encompass the knowledge of many participants. A change in the name of a project for a mid-level enhancement divides mindshare. This is because people know something by its name -- it is only human. If the project is known as Quickie-Foobie and you branch off too many times you lose the force of acceptance the product has already gained. You are effectively starting over. So it is best to change those things about the core source of a product that others have found can be improved as time and skill allow.

I have learned the following from my experiences:

A. Popular open source projects are victims of their development strategies.

Gnosis seemed to suffer from "analysis paralysis." There was documentation early on and no actual code or prototype. I think a project that starts purely with documentation in the open source community is doomed to failure. There needs to be a combination of planning and reality. Had they utilized some open type of discussion methodology right from the start along with periodic prototypes they would probably have a working product today.

B. The best strategies utilize structured come-one-come-all approach to enhancements.

Agora is a good example of a script that has captivated many different prosumer type users. One of the most often heard statements from their Yahoo group list participants is that they are not developers. The actual code could use some rewriting but it seems to work. There are quite a few modular enhancements written by the user community, particularly for gateways to credit card processing centers and for shipping, but attempts to actually rewrite the core are largely ignored.

C. There needs to be a framework (or a tool) for open source projects to use that allows for high level discussion of the application

Internet Request for Comments (RFCs) are a good example of this process. Everything is open to discussion and debate. The best technology meets the (hopefully) best minds and is reshaped, not ignored. Many open source projects are ignorant of this long standing example.

We need to have a solid plan for reviewing and accepting issues/problems with core product code. This goes beyond bug tracking. Problems with architecture do not directly map to bugs, but rather are the root cause of many performance and enhancement issues. Open source projects should judge change requests purely on technical merit. The open source community needs to focus more on RFC-like methodologies for source code improvements as well as for actual project planning. This practice needs to be taught alongside the seemingly endless discussions of the benefits of open source development methodology. In short, open source is of limited value unless you have an open type of development coupled with some structured manner of review.

Celebrate Intellectual Diversity

Comment on (OT) Open Source Needs Open Discussion

Replies are listed 'Best First'.
Re: Open Source Needs Open Discussion by chromatic (Archbishop) on Sep 11, 2003 at 20:23 UTC
If you're saying "Having open source code doesn't alleviate the need for good development practices," I agree. I said something similar in Five Lessons Open Source Developers Should Learn from Extreme Programming. I hesitate to go any further, though, because several projects have survived despite doing things that seem really really wrong. Don't discount volunteerism and stubbornness — they are terrible trumps for planning commercial development, but they work, occasionally, in open source projects.	[reply]
Re: Re: Open Source Needs Open Discussion by Anonymous Monk on Sep 11, 2003 at 23:12 UTC
I hesitate to go any further, though, because several projects have survived despite doing things that seem really really wrong. Is there actually a single, well-known example of an open source project that did things right? I can't think of one. Everything I've seen indicates a moderately distributed brute force method. Not that there's anything wrong with that :).	[reply]
Re:(3) Open Source Needs Open Discussion (participants over planners) by ybiC (Prior) on Sep 12, 2003 at 01:14 UTC
I recall Clay Shirky as having said "Quality participants are more important than quality planners", which seems relevant to this thread. cheers, ybiC striving toward Perl Adept (it's pronounced "why-bick")	[reply]
Re: Open Source Needs Open Discussion by perrin (Chancellor) on Sep 11, 2003 at 21:24 UTC
I'm not sure I follow your conclusion. It sounds like the problem with LIMS is too much talk and not enough action, while the problem with Agora is lousy developers. Neither of these things will be improved by more discussion. Discussion is always the first thing to try, but if that fails and you end up in a deadlock, the next step is to put your code where your mouth is, and write it yourself. If you offer a patch and the maintainer won't accept it, you can always fork the project or offer your patch separately. Eventually, either social pressure will cause the forks to merge or one of them will just die off. I do have a lot of sympathy for project maintainers who come across as difficult because they take a "patches welcome" attitude towards feature requests. There are always a lot more people asking for things and using a project than there are contributing. That's why I try not to whine about anything too much unless I'm willing to write code for it. After all, as a user I am getting something for nothing until I actually write code. You also have to consider that open source projects often succeed due to the strength of one or two core contriutors with a vision. It's precisely this lack of formal RFC methodologies that allows tremendous progress to be made on a project. You have an idea, you code it. It sounds to me like LIMS is all RFC and no project, and that's the root of the problem. Incidentally, I just glanced at the AgoraCart code, and I think that if I ever did something based on this I would be eager to distance myself from the name of this thing. This is horrible code: Perl 4 style, cgi-lib.pl, hardly any CPAN modules, no package declarations at all in the Agora code, and NO STRICT!	[reply]
Re: Open Source Needs Open Discussion by adrianh (Chancellor) on Sep 11, 2003 at 21:25 UTC
In short, open source is of limited value unless you have an open type of development coupled with some structured manner of review. What - like CPAN you mean :-) Yes a lot of open source projects fail for various reasons. So do a lot of non-open projects (often in a less public manner). As chromatic says, being open-source doesn't free you from having a sensible development process. I really don't think most open source projects fail because of a lack of an open development process. They fail because of lack of time and will, rather than due to any lack of input from the world in general. People can suggest options and ideas all they like. In the end somebody has to do the work and implement the feature. If they don't want to do it - if it does supply that vital itch that they need to scratch - then it won't get done. No matter what the theoretical technical merit may be.	[reply]
Re: Open Source Needs Open Discussion by hardburn (Abbot) on Sep 11, 2003 at 21:08 UTC
Imagine that there were one hundred Apache projects all using slightly different versions of the same source. Chaos would ensue. Poor example. Apache is a healthy, growing project. The two other projects you mention sound like they are stale with developers who are unwilling to make changes and equally unwilling to allow someone else to step in. There's no reason to fork a healthy project, but stale projects are why Free Software license allow forking in the first place. I do find the example of Apache kinda ironic--its orginal goal was to bring together the hundreds of patches made for the old NCSA server into a single project. So its really the reverse of your example. ---- I wanted to explore how Perl's closures can be manipulated, and ended up creating an object system by accident. -- Schemer Note: All code is untested, unless otherwise stated	[reply]


Come for the quick hacks, stay for the epiphanies.
	PerlMonks