As the link above states, the OpenBSD team continuously systematically audits their software. However, as the BSDs have so much cross-pollination in their codebases, it ends up being an interesting hybrid approach. I can recall a few instances where bugs that slipped by OpenBSD's security reviewers were caught by folks on the NetBSD team doing non-systematic general reviews.

If nothing else, it proves that more eyes on a piece of code leads to more secure code, systematic reviews or not, which is one of the real strengths of open source.

One would expect that the code in Open Source projects is constantly inspected by the users of it

In a recent effort at code review on an Open Source C++ project I had some difficulties that eventually drove me away from the project. I would find something that could be easily re factored to fix a bug (one documented in the code itself) but was not allowed to fix it because it had originally been the way I proposed and then at some point had been changed. The svn entry for the change didn't mention why (but did mention the bug it introduced), but since no one could remember why the code was the way it was, they weren't willing to change it. I know that came out kind of garbled but the crux is that no one knew the code well enough to explain why it was written with the bug in it in the first place, and they where afraid fixing that bug would introduce a worse bug. So in this case I'm not sure what to think, just figured I'd share my miserable experience and ask if anyone else had similar cases and how they combat that? I'm sure its related to the pride (i wrote it that way so it should stay that way even if it added a bug) but I'm really kinda clueless. Eventually (very very soon after starting with the project) it drove me away from the project entirely.

That experience highlights the importance of regression tests for fixed bugs.
--
Online Fortune Cookie Search

If the ego have a strong play in OS project, and I believe that you hare right, then I don't see that the review many success.

Code review it's, clearly, less motivating than write a new one, so ..

I think it's pretty much empty sophistry to suggest that open-source projects (those of real importance) get to where they are today “by luck and by chance.” The important stuff, that we actually seriously rely upon, gets subjected to intense peer-review as it moves its way along the source-code control systems on the way to a “blessed” release.

Even for CPAN there is “CPANTS,” the amusingly named CPAN Testing Service which systematically self-tests everything on everything. If you don't submit a module with a good set of automated test-suites, it's not going to attract much attention, if it even makes it into the library at all.

Serious open-source-ware is a professional product, which is going to be employed in professional work and which therefore must be up to that standard of excellence.

Seasoned developers quickly learn that skimping on testing is like skimping on automotive maintenance:   “you can pay me now, or you can pay me later.” Testing has to be something that you do while you are developing the project. Planning the whole thing has to be something that you do before(!) you start to build anything at all. Good documentation has to be part of it. Go the whole distance or don't go at all. Nobody wants – nobody can afford – “half-ass-ware.”

We don't teach that sort of thing in school. We should, but for the most part we don't. There is actually not too much instructional-time available for the degree-track curriculum within a standard two- or four-year degree program, so most of the time gets spent just pointing out the features of the machinery and teaching the student just enough that he can use the machine without slicing his fingers off. Finishing a homework assignment is an accomplishment. But when you move into the profession itself, everything changes.

Well - I don't want to discuss if there are OS project using code inspections. I am asking a practical question - show us examples, tell us what works for a typical OS project, what are the political and psychological tricks to use.

The procedure described in Code Complete requires physical presence in one place - this is not feasible for most of the OS projects that I know. So some changes are inevitable. Maybe it is not a big philosophy - maybe it is just about starting doing it by yourself and sending the defects you found together with tests proving your points.

Did I miss where the OP claims otherwise? I wouldn't say that automated testing is the same as code reviews, both are useful and serve different purposes.


___________
Eric Hodges

By the way, Code Complete is one of several books that was at one time bundled together by Microsoft Press as Software Engineering Classics. And... they are. Read 'em all.

For all the warts we all may have with Microsoft's marketing department, their publications house is very good.

There is a perl-based alternative to reviewboard, CodeStriker. I don't know if it's any better but, hey, that's made of good Perl with no stinkin' Python inside :)

I think that the most effective form of code-review in any project, OSS or otherwise, derives from the notion of a distinct “development vs. production” barrier that developers cannot cross by themselves.

Development code, and the development team itself(!), does not have read/write access to the development code-libraries, nor any access at all to the production databases and files. This is enforced by management decree.

As an example of how this sort of thing can be done, look at Linux itself. “If you want to get something into ‘the” Linux distribution, you've gotta get it past Linus Torvalds.” Or, you gotta get it past somebody (else).

There is also very-often a rule that “any source-code change that is built ‘to address an RT ticket’ must strictly address the defect that is described in the (approved...) ticket.” {RT is a commonly used problem-reporting system...} The change will then be associated with that ticket, and managed with it.