little_mistress has asked for the wisdom of the Perl Monks concerning the following question:

This really isnt a question, nor is it a meditation, nor does it really belong in the Perl Monks Discussion. So, I will put it out here.

I ran across an interesting bug at work today in someone elses code. The bug has been there for two years unnoticed untill this week.

Durring my experiments to debug these three regular expressions I simply commented them out to see what effect it would have if they were not doing their job at all. To my surprise, it fixed the bug. In other words they were totally unnessisary.

This brings up the question of "Over Programming". When is enough enough and when is it too much? Obviously I experienced an example of that today, but, and i grant that this is not a direct perl question, however the nature of Perl makes over programming easy to do, what sorts of design methods does one need to use to curb over programming?

Replies are listed 'Best First'.
Re: Over Programming
by chromatic (Archbishop) on Apr 26, 2000 at 00:48 UTC
    In my experience, Over Programming is caused by two often-conflicting things in the mind of the programmer:
    • An incomplete understanding of the problem or the dataset
    • The desire to write a quick bugfix, rather than overhauling the whole function
    It's my opinion that sitting down with a few blank pieces of paper away from your computer for an hour to hash through a couple of designs and to draw some diagrams is the best programming discipline I have. If I can't list the basic steps I need to take, I don't understand what I'm trying to do.

    Only after I can fit the whole conceptual model in my head am I able to deal with the exceptions that make code so crufty and weird most of the time -- they let me refine the model to where it works best, most of the time.

    That doesn't mean there are still tweaks and rewrites ahead, just that I know what's important and I can code for that, instead of throwing things here and there, hoping they'll work. (Okay, I'm bitter at some existing code some days that isn't so much designed as it is blenderized.)

Re: Over Programming
by comatose (Monk) on Apr 25, 2000 at 03:47 UTC

    Two words: peer review

    Of course, coming back to your own code a few weeks or months later is good, too. Personally, my Perl coding ability is always improving. I might look at the code I wrote six months ago and say, "Jeez, this really sucks," and make it better.

    Otherwise, I'm not quite sure if there's a lot you can do in the process of coding. One possibility might be benchmarking but that won't always fix "over coding" as you describe it.

Re: Over Programming
by perlmonkey (Hermit) on Apr 25, 2000 at 03:47 UTC
    Enforcing 'over commenting' at my work seems to work. Let the coders know that they must comment the purpose of every block of code. That way they think about every line again when they go into a documentation phase of the project.

    Of course I assume there is a documentation phase? If not, that might be your problem.

    Peer review is also a great weapon against bad code.

    Thats my two bits anyway.
      I agree with this completely. Since I starting forcing myself to over comment my code, my bebugging cycles have gotten much shorter. Not only does it mean less bugs get out the door, but it means if something is too ugly to admit I'm doing it that way, I can't just pretend it's not there; I have to admit it sucks, or do it right.
      Also, it helps to _find_ bugs. Something may be obvious, but that doesn't mean a person should have to parse it all to find the stop they're looking for.
      And since we all improve over time, what was obvious when I wrote the function might not be obvious at all when I'm debugging it. More like, "did I really do that?"
RE: Over Programming
by Anonymous Monk on Apr 25, 2000 at 17:17 UTC
    Another way that has always worked for me in the last *gulp* 25 years of my programming carreer is to code by the twin principles of economy and elegance. There was a push several years ago toward the concept of "self commenting code", ie. code that would not have to be burdened by comments because the function would be evident by the way in which the code was written. This gave rise to OOP, among other practices.
    The problem is that a language powerful enough to be self-documenting (like Perl) is also prone to over-economizing and obfuscation. When one gets so "elegant, cute, and economical" that you no longer understand what it was that you (or someone else) was supposed to be doing, one tends to patch, rather than fix.
    It makes me wonder how many thousand lines of my code havee been commented out by those who came after? I'm sure I wouldn't understand what I was trying to say those twenty odd years ago. But you see, that's why we have powerful language constructs now, there's more than one way to do it. I agree that over commenting is a good way to make code more understandable and supportable, especially when it is necessary to streamline the code due to memory or clock tick concerns. There is a fine line between efficiency and obfuscation, and sometimes the comment is the only thing that saves you. But at the same time, I believe that the code should comment on itself, by being well written and understandable.
    Of course, I have yet to see a self - documenting regexp. Commenting seems to be the only thing that works for me there.
    Well this old man has rattled on long enough . . .

    Simplicus
      If you want to vote on the above, do so here. I didn't notice I was anonymous. Oh well:

      "To lose one's mind is a terrible thing,
      or not to have a mind at all"

      - Dan Quayle.
Re: Over Programming
by jbert (Priest) on Apr 25, 2000 at 19:12 UTC
    I don't like the 'it seems to work' school of program analysis, unless it is backed up by a (by definition) complete set of regression tests.

    There is such a thing as defensive coding.

    i.e. Parsing a config file.

    while( <FILE> ) { s/#.*$//; # Lose comments s/^\s*//; # Lose leading w/s s/\s*$//; # Lose trailing w/s next unless $_; ... do stuff }
    This will function fine (modulo bugs...the above is just typed). It will continue to function fine if you remove the w/s stripping lines and the set of test data doesn't contain any lines consisting of only w/s.

    However, if you do that and then someone adds a line containing w/s to the config file then it will barf. Of course, if your environment defines that failure in that case is acceptable since that is not a legal line in the config file then you have done nothing wrong.

    IMHO, a (small, controllable) bit of over-engineering is OK, if done sensibly.

    Then again, there are always cases where this would be the wrong thing to do. So shucks, its not a universal rule.

Re: Over Programming
by little_mistress (Monk) on Apr 26, 2000 at 00:15 UTC
    OH yes the regular expressions i commented out did do something if the found a match.

    Let me give an example:

    When the user entered text into one of the text areas that was parsed by these regular expressions it would find any metacharactors and insert a perl escape charactor in frount of them. This is to alow a user to use metacharactors when naming something, if they so choose. Lets imagine for the sake of argument that a user wanted to use an equals sign "=" in the id of an object that the web interface allows them to configure. In the config file you would end up with something like this:

    filterid = id\=5

    All well and good however when it was read back from the configuration file and displayed in the UI ... it continued to show up as "id\=5" rather than "id=5" due to perl's wacky interpolation rules. Which, in this case, the programmer was working against not with.

    Now, when it came to debugging the problem with what perl was giving to the UI I first had to determine if we really neaded the metacharactors to be writen to the configuration file with a perl escape charactor. From all my experience you don't, however in this case perl was putting together html pages full of javascript that were being spit out to a browser. There was the possiblity that the escape charactor was ment to be aliteral slash in perl and an escape charactor in the javascript it was writing. Not the case, as I found out when I totally commented the regular expressions out and not only did the bug dissapear, but no new bug was added. By removing totally the three lines of code and not replacing them at all with anything solved the problem.

    When I finally sat down and talked the the original developer about this a 20 minute discussion of how making sure that you handle metacharactors is importaint, silly girl go back to your cube. (thats alittle unfair of me to say, I'm well respected here.) But no lie it took 20 minutes to convince this fellow that overcoding was causing the problem. I showed him on his copy of the software on his workstation we came back to my cube and I showed him on my installation, we went to the lab and installed a build from one month ago and I showed him that the solution worked there. I grant that erroring on the side of caution is the way to go, however, that doesn't always make for sane code.

    Perl, due to its forgiving nature, I think is more prone to over coding than C, Java, Corba, Python ... the list goes on. It seems to be the case that over coding is a bug in the progam, caused by a bug in the programmers. ;-)

    little_mistress@mainhall.com

Re: Over Programming
by mt2k (Hermit) on Apr 26, 2000 at 04:58 UTC
    For those who think their code is bug-proof, I suggest stepping through your code and taking a second glance.

    Almost ALWAYS, when you think your code is efficient and bug-free, don't be surprised to find a few inefficiencies, bugs, and a lot of uneeded code snippets.

    In the past, I have seen people rewrite their code, deleteing tons of lines, replacing 10 lines with 1 or 2 and a lot more.
    This is usually when someone tries to fix a script they wrote when they began perl (Same thing with any other language).

    Here's what I say: Beginners it's okay to overdue it; you don't know better!
    Advanced: look over your code and make it better!

Re: Over Programming
by Specimen (Acolyte) on Apr 25, 2000 at 17:19 UTC
    I would be very worried if I commented out something and it didn't change the way the program worked. Are you sure you tested it thoroughly and that it really doesn't do anything ? Sounds a bit like you didn't give it data for which the code did something deliberate ? You could always post the relevent code here