Wanna learn software management? Go to Joel on Software. Now. He has tremendous insight on software development and management and I have a lot of respect for what he writes. That being said, I had an issue with his article about never rewriting a large software application. Rather than sum up his excellent arguments, I suggest you read his article before replying to this node (and you should read it even if you don't want to reply).

I should also mention that this is a follow-up to Rewriting a large code base. Thanks to the information garnered in that node, we have a much better plan of attack for the rewrite, but I'm hoping to abuse Monks for even more perspective :) As a result, feel free to not vote for this node.

Despite having read his article, I still called for (and got approval on) a rewrite for the code base. What follows is a brief description of the code and my reasoning for a rewrite. The code is a Web-based product that allows for complete manufacture to retailer to consumer product ordering and distribution with integrated inventory management and drop-shipping programs. Without going into detail, it's huge, complex, and the specialized industries that we cater to are very pleased with its potential. I say "potential" because it's not living up to it. The software works well so long as no one does anything unexpected (and it's a Web-based application that relies on cookies and Javascript for much of its functionality!).

Here's why I reluctantly have asked for a rewrite:

  1. Massive system with massive security holes. I've plugged as many as I can, but I know there are probably more out there.
  2. The database was poorly designed and does not properly support our business rules. Further, the current schema cannot support changes that our customers have insisted upon.
  3. There is extensive use of global variables which has made the system inherently difficult to maintain and has driven up our maintenance costs considerably.
  4. This system was never tested. You will not find a single line of test code and because the code is not orthogonal, fixing one bug often causes all sorts of other bugs to crop up. Since there is no test code, finding other bugs is trial and error.
  5. This system is poorly documented. Often, what little API documentation there is does not accurately describe the API in question. Further, we have many, many similar objects but with radically different APIs.
  6. Extensive creation and use of global temp tables (which are rarely deleted) in our database has caused the database to crash frequently. Because these global temp tables store information for so many of the global variables, trying to pull them out has proved extremely difficult.
  7. Many tricky programming features were added because the IT director (who is no longer with us) thought they were "neat".
  8. We need to port the code to mod_perl to increase performance, but this is literally impossible due to the code design.

I did not design the code base and was brought in to work on it near the end of the project. I have built up knowledge of its functionality only through painful trial and error. We do not have a single person in-house who completely understands the system and after reviewing our options, we felt it would be more cost-effective, in the long run, to rewrite the code base from scratch.

All code would have unit tests written *prior* to coding. Automated test suites would run against all builds to validate functionality. All undocumented code is automatically rejected. User and developer manuals would be written concurrent with code generation. Tests would be written for every bug to ensure that we never manually catch another bug. Further, no bug can be marked as "Fixed" in the bug database (we use Bugzilla, darn it) unless a test has been written for it.

We currently have at least one hundred clients who have asked how soon they can use our system (and another 700 who have expressed high interest). Unfortunately, we can't support that many clients. There is an upper limit of only four or five clients before the system becomes unuseable! Rewriting the code base is extreme and we might lose some clients, but those are clients we can't even accept given our current limitations.

In short, I do think there are times that a large, working application needs to be rewritten from scratch (we can't even reuse any old code due to extensive side effects).

I'd be interested in hearing discussion about this. Have any monks had similar experiences (either with or without the rewrite)? How did you deal with them and what sort of issues did you face? If you didn't rewrite such a large system, how would you go about fixing a problem this huge?

Cheers,
Ovid

Vote for paco!

Join the Perlmonks Setiathome Group or just click on the the link and check out our stats.

In reply to (OT) Rewriting, from scratch, a huge code base by Ovid

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.