Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

My number 1 tip for developers.

by BrowserUk (Patriarch)
on Sep 12, 2003 at 01:06 UTC ( [id://290885]=perlmeditation: print w/replies, xml ) Need Help??

Always start with working code!

Sounds dumb right? How can you always start with working code? If you are just starting a new project, you have no code, so you can't start with working code. Wrong. It's easy. Start1 by typing as much as code as you know will work. Eg. Many file processing and filter type scripts I will start with:

#! perl -slw use strict; while( <> ) { print; }

1 You have to start somewhere. If you type just while( <> ) { and tried to compile it, it would fail. So don't do that:)

I haven't tried to compile or run that, but I am fairly confident, that it will run without errors on my machine! Of course, the shebang line means that it will likely fail on other setups, but that is true for many variations of shebang line, even between different installations of the same OS.

Start small

The point is to start small and then add stuff in small steps. Don't move on until you have made the current step work. As you gain experience, you will be able to type more each step with a reasonable degree of confidence that it will work, but don't get over confident. Any time you type a line of code, and you are not sure if it is correct, stop. Try it.

Trying it can mean doing several things. Often a good first pass is to do perl -c yourscript and see whether that produces any errors. If you are unsure of the syntax of something, typing perl -de1 will give you a debugger session in which you can type simple one liners and have the syntax checked immediately. I've got a really simple script which reads from the keyboard and evals what I type which I use all the time. It has a few extras, like it imports several utility modules like Data::Dumper, List::Utils and my own personal utils module. It also is set up so that I can end a line with \ and it accumulates the lines until it gets one without a \ before evaling them which is useful. I often thought of trying to clean it up a bit to make it more useful, but I run it many times every day, and only get around to adding to it when I need to. It serves my purpose very well. It's only about 10 lines long.

If your lucky enough to be working on a fairly simple piece of code like above, then you can try it on your test data. You do have a file of test data don't you?

Test data

If you don't STOP. Start another editor session and create one. If your editor has macro ability, this maybe all you need to produce a reasonable facsimile of your data. If not, you may need to write another short script to generate it. rand is great for this. It doesn't need to be too exact, but just close enough to exercise your program. And it needn't be too big, just big enough to contain one or two examples of data that will exercise each path of your code. Even this isn't necessary for your first pass, you can always modify and improve your test data as you go, making it more sophisticated as you code each new path of your program.

Save often

Each time you test your program and it works. Save it. That is, if your editor doesn't handle it for you, then make a backup. I tend to keep at least 3 ordered copies of each file of any serious project. If your editor doesn't do this for you, make yourself a small batch, shell or perl script to do it for you. If the way you work is that you never leave your editor, or close your file when developing, then create a macro or shortcut or whatever that allows to backup your file with the latest set of changes when you choose to, Ie., not every time you save the file before testing it. Yes, in most decent code editors you can always use undo to move backwards, but

  1. You may not remember the right place to go back to.
  2. There is always the temptation to think that you knew what change you made that broke the code and only back-step part way. Then you think you know what you did wrong last time, and so you make knew changes. They don't work.

    So now you have a hybrid situation. Maybe some of your new stuff does work, but it comes after some other stuff that doesn't. Now your in the situation that you have stuff in your undo buffer that you want to keep, but it comes after stuff that you want to undo. Plus you can't remember how far back you need to go to get to the point where it was working.

Have a way of committing a backup that is independent of the normal save function/ key/ command.... And use it! But only when you have working code.

Throw it away!

When you add a section of code and it just point blank refuses to work. Bin it! Retrieve the last working backup and start again. Do not be afraid to throw away a piece of code you just spent the last 4 hours working on. Don't think it was a waste, it wasn't. Often, your second pass comes out much better than your first. You now have the experience of doing it the first time and when your writing the second, this will be fresh in your mind. The good bits will come back naturally. The bad bits will tend to stand out as such. Don't try keeping the non-working code and modifying it. This invariably leads you into just repeating the same mistakes. If you have more than 4 hours worth of code since you made your last backup of a working version, you are waiting too long between testing! If you have the discipline to throw away and start again, you'll rapidly cure yourself of going too far before testing:) Personally, I rarely spend more than 20 minutes or half an hour coding without performing a test, but different folks, different strokes.

Try it! It works.

It's not only for small projects

That's all well and good I hear (somebody) saying, if your program is a silly little one file script with a single source of data, but it wouldn't work for my project because it consists of many modules (classes etc.), and it is impossible to run one module in isolation of the whole suite. And anyway, the module I am working on has dependencies upon other modules that I haven't coded yet, so I cannot test until I have coded them also. Wrong!

Design first

When coding a module, you should have a fairly clear idea of the interface to that module. That means you should know, before you start coding pretty clearly what functions, procedures or methods will be available for use by callers of the module. You should also have a fairly clear idea of the parameters they take and the sort of results you should expect. So code a "caller" that uses that interface. In perl modules I tend to code this as a "main' program within the same file.

package doit; sub new{ return bless {}, $_[0]; } sub enumerate{ my( $aref ) = @_; return @$_; } .... return 1 if caller(); package main; use strict; my $doit = doit->new(); my @data = 1..10; print $doit->enumerate( \@data );

Dummy up

In this way, I can just run the module to test the code. It makes writing a "working" module easy. Just dummy up all of the interface calls and have them return something vaguely sensible, and write code in the main package to call those dummy functions. As the code in the module gets more sophisticated, so the test code can be made so. Or maybe do it the other way around. As you increase the reality and complexity of the test code, so you update the module code to suit. I prefer the former, but it's not a hard and fast thing, just a preference.

Not only does this answer the "can't be run in isolation" problem, it also answers the dependencies problem. If your module has dependencies, it takes very little time to code that module using placeholders for the real code that don't even inspect their inputs, but simple return a hard coded (or even randomly chosen) "correct" answer when called. This allows you to get on with coding and testing your current module. It also acts a "working starting place" when you get around to coding the dependant module for real. Once you have completed (to some definition) the functionality of your current module, you have a ready made test script for the dependency module.

Finish well or throw it away

Finally. I always try to finish, for the day, night, week etc, with a working program. If I haven't, I don't throw what I have away there and then, but I do seriously consider doing so when I start again the next session. If I cannot see the problem within about 20 minutes (maybe a little longer if I've just woken and am caffeine deficient:), then I will save the current stuff under a different name and revert to the previous working version. Almost always, I will never refer back to the renamed, non-working version and it ends up in the bit bucket, but just occasionally, I get to a point when re-coding where I remember having found a neat solution to some bit, but can't remember the details. Then I refer back to that bit of the code. I NEVER revert to the non-working version in it's entirety!

This was prompted by various things, mostly my having spent too long and become really frustrated trying to get something to work before realising I should have followed my own advice above and binned the non-working code and started again. When I did, I had the new version coded in about a quarter of the time I had already spent trying to get the previous version to work.

Well, there it is FWIW. Rather longer than I intended, but maybe it will help someone.

Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
If I understand your problem, I can solve it! Of course, the same can be said for you.

Replies are listed 'Best First'.
Re: My number 1 tip for developers.
by adrianh (Chancellor) on Sep 12, 2003 at 07:21 UTC
    Start1 by typing as much as code as you know will work

    In my opinion an even better option is to start by writing a test for what you want to happen, then write the code to make that test pass. Repeat until code done.

    Test Driven Development takes a little getting used to, but works very well in my experience. Advantages include:

    1. You get pretty much 100% test coverage for free.
    2. You always know when you have got a piece of code working.
    3. You always know if a new piece of code breaks something.
    So code a "caller" that uses that interface. In perl modules I tend to code this as a "main' program within the same file.

    You can also do this sort of thing with Test::More and friends - and have the advantage of having something you can move to a test script later on. So instead of:

    package main; use strict; my $doit = doit->new(); my @data = 1..10; print $doit->enumerate( \@data );

    do something like:

    package main; use strict; use Test::More 'no_plan'; isa_ok my $doit = doit->new(), 'doit'; my @data = 1..10; is_deeply [$doit->enumerate( \@data )], [1..10], 'enumerate works';

    Then you don't have to think about whether what's being printed out is correct.

      I have yet to be convinced about the use of Test::More and it's brethren.

      I've seen signs of people using these modules within the code being tested. That means that it either has to be removed for "production purposes" or disabled. If it is removed, then you risk its removal changing the nature of the code. If it is disabled, then it means that the production version carries the weight of the testcode. Neither is satisfactory IMO.

      Using it the way you showed above, seperate from the real code, just used to simplify the validation of results makes some sense, but I'm still not sure.

      My other basic qualm is that it tends to render the testcases to a series of individually descrete steps. Essentially unit tests of a very fine grain. I prefer my testcases to be more realistic, by which I mean, I like them to be as close as is practical to a real world usage of the code under test. This tends to do somewhat more than just exercising the API under test, in that it also tends to highlight design issues. If the API has been design such that it is awkward to use, this tends to become obvious. We've all encountered APIs that may look great on paper, from the designers perspective, but that force the coder to jump through hoops in order to use them in the real world. (Anyone used Java?:).

      Coding the testcase as a realistic 'caller application' has the benefits of:

      • Highlighting design flaws or impracticalities.
      • Serves as a working--and maintained--programmers documentation example. So much better than the glorified 'prototype' examples that usually exist and often fall out of maintainance as soon as the documentation goes to press.

      I can see the benefit of using the Test::* group to provide easy notification of where & why the overall testcase fails, but I still have qualms about their more abstract effects upon the testcases.

      • How do they effect timing issues?
      • How do they intereact with threading?
      • Can I easily disable the test verification code whilst leaving the testcase code they exercise in place?
      • Or do I have to code the tested code twice so that the exercised code will still be exercised (and its results and sideeffects still be available to the next part of the overal testcase)?
      • Can I easily reuse the testcase (with its embedded but disabled Test::* code) for profiling and benchmarking purposes?

      From my breif exploratory visits to the Test::* group of modules, none that I have seen, yet satisfy these criteria.

      I still don't have a satisfactory way of ensuring that my module-embedded testcases are not loaded into memory when the module is used. I have a very hooky method, but it has many flaws. If anyone has any suggestions on how to do this I'd love to hear them.

      Examine what is said, not who speaks.
      "Efficiency is intelligent laziness." -David Dunham
      "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
      If I understand your problem, I can solve it! Of course, the same can be said for you.

Re: My number 1 tip for developers.
by simon.proctor (Vicar) on Sep 12, 2003 at 08:27 UTC
    My tip: Learn to solve your own problems

    Before I consider posting to Perlmonks (or bothering co-workers ;) ) I always try to write a smaller program that illustrates my problem. I give it the data (hardcoded where possible) that I expect to get (or to compare with the data I expect from my code) and work from there.

    This allows me:
    • Controlled space where I can experiment
    • Clean brain space to concentrate on the problem
    • Knowledge that I can't break anything
    • I can have finer control over running the test area
    I say try it (if you don't already).

    Just my 2p
Re: My number 1 tip for developers.
by tantarbobus (Hermit) on Sep 12, 2003 at 02:04 UTC

    I would have to say that it is better to find out how you work best as a programer, and tailor habits so complement the way you program. If you work best by starting with a full application in non-working psudo-code and refactoring it into a working application, then do that. If you programming best by writing the application on a white-board and then typing it in, do that. Programming is an art, and you have to learn what makes the muse sing for you, and continue to do that.

    One other thing: I, for one, could not throw away a non-working bit of code without first finding out why it did not work because, if the code is not working I made a mistake. I screwed up. And if I thow that piece of code away before I learn what I did wrong, how am I to make myself a better programmer? How am I to learn from my mistakes if I do not even know what they are?

Re: My number 1 tip for developers.
by mildside (Friar) on Sep 12, 2003 at 06:07 UTC
    Regarding Test Data:
    And it needn't be too big, just big enough to contain one or two examples of data that will exercise each path of your code.
    This may be obvious, but I'd like to add that it is important that the test data includes data that is out of spec. I learnt this the hard way, on a project where the data we get from the client is always going wrong in some way or another. You must always assume that you will get badly behaved data as well as well behaved data. Bad things can happen if you don't. I now check all data fields with a specific regular expression for that field for sanity checking and dump any data that doesn't conform to a rejects file to send back to the client.


Re: My number 1 tip for developers.
by hding (Chaplain) on Sep 12, 2003 at 03:43 UTC

    'Tis one of the reasons I like Lisp and Smalltalk so. You get to start with an image with a boatload of working code already in it and from there on it's simple gradually to add the code you need to make a finished application.

      When I first started using SmallTalk, I really liked that too. However, as I started working with it for real and trying to develop projects requiring half a dozen developers, it became a pain.

      The first problem was that if you had to sit down at another developers machine and try to work, it was awkward because they had invariably customised the heck out of their "image". EditPanes, keyboard definitions, the works. The nearest analogy I can think of is sitting down and trying to work with an AZERTY keyboard, or on a machine where the OS is configured for a different language.

      You can have the same problem with highly configurable editor (like emacs). You sit down to review someones code or see if you can spot their bug, hit Escape-Meta-Alt-Control-Shift-D to scroll the code down, collapsing the previous indented block you just read and expanding the next--just like god designed that key sequence to do:)--only to find that the guy you were trying to help has that key sequence defined to remove all line breaks, comments and extraneous white space, and reduce all variable names to single or double characters.

      It's his fault that he didn't code his macro to a) save the code first; b) Checkpoint the changes into the undo stack. :)

      The second problem is trying to share code between multiple developers. That the code becomes a part of the environment rather than living in external files makes it quite difficult to synchronise classes between developers. I do remember my client spending a quite large sum of money to purchase a class to synchronise changes from individual developer images to a central "code warehouse"--essentially a CVS type of thing. The only problem was that it took so long for it to search the whole damn class hierarchy to locate and save the changes, this was back when 20MHz 386's where state of the art, that it became a pain to do it regularly, consequently, we spent crap loads of time trying to merge changes that had been effected by two (and more) developers. Painful.

      I've never used Lisp, but I heard that you can get a similar situation. A bit like I now have with my site/lib/* tree. lumps of "standard" code that I've modified for one reason or another to suit my purposes, but which make it scary for me to upgrade because of all the little tweaks I've come to rely upon.

      I really like the idea of a intereactive development environment that allows for deep introspection and modification of the "system" and its classes/functions, but I would like one where the system itself performs diff-ing and versioning so that a) it becomes easy to back out changes and/ or isolate user modifications from the original system. b) allows me to upgrade the system and have any changes I made to the original system be merged (assuming that the new base system is compatible) automatically and preferably, prompts me when it encounters conflicting changes.

      With modern cpu speeds and memory sizes it ought to be possible for this to happen pretty much transparently to the developer.

      One final thing that I felt was missing from the versions of SmallTalk I used was some way of automatically removing unused classes from a final application image prior to distribution. Memory is less critical these days, but back then, system managers freaked when you handed them a stack of five 1.44mb floppies for even the simplist of applications. Five MB apps were unusual then. Even Word was only a coupe of hundred Kb at that time.

      Who knows. Maybe such an environment will develop for Perl 6--but I won't hold my breath waiting for it:)

      Examine what is said, not who speaks.
      "Efficiency is intelligent laziness." -David Dunham
      "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
      If I understand your problem, I can solve it! Of course, the same can be said for you.

        I don't use Smalltalk in a professional setting, but I do believe that the larger commercial Smalltalks (e.g. VisualWorks) have developed systems for allowing multiple developers that seem to please Smalltalk developers. Also Smalltalks (and Lisps) these days typically have image strippers that can be used to remove anything unnecessary from the image before deployment.

Re: My number 1 tip for developers.
by zby (Vicar) on Sep 12, 2003 at 07:26 UTC
    Start1 by typing as much as code as you know will work.
    Usually there is one most important/most complicated/new for me CPAN module that serves as a basis for my code and for the start I just copy some example from the documentation.
Re: My number 1 tip for developers.
by Jenda (Abbot) on Sep 14, 2003 at 15:49 UTC

    I guess the methodologies and all this stuff was discussed sufficiently by others so I'll add just a few shameless plugs :-)

    1) I agree it's very nice to have some kind of interactive interface to Perl for testing tiny snipets of code. I don't really like the perl -de1 hack os I also wrote my own interactive interface allowing me to enter more lines at once and so for. You can find it on my page. It evaluates the code only after you enter a line that ends with a semicolon, allows you to execute external commands or print the value of something, allows you to paste whole subroutines without trying to eval them in pieces (in here-doc like syntax) and so forth. I find it most helpfull. (Yes I know there is some completely unrelated PSH on CPAN. My PSH was released first, but I did not get the namespace registered with and didn't upload without registration.)

    2) The backups. Of course you should not write anything without a version control system, but of course you can't check in every single change. Having to remember to make a backup every time you make some change is tedious and not all editors will do that for you. In case you are using Windows you may find my helpfull. The script will backup everything you change in the directory every time you save it. Of course you'd better make sure you do not forget to turn this off ;-)

    Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
       -- Rick Osborne

    Edit by castaway: Closed small tag in signature

Re: My number 1 tip for developers.
by Anonymous Monk on Sep 12, 2003 at 06:43 UTC
    Start1 by typing as much as code as you know will work.
    Bad idea. Follow the basic software engineering steps
    • spec - obtain a spec
    • analysis - understand the problem/spec ... maybe prototype (this is where you throw stuff away)
    • design - pseudocode
    • implement - write it
    • test and verify - you can do this while implementing
    • maintain and update
    Your way sounds like it would take twice as long as anyting else.
      That was the preferred way of doing things when I started - way back in the early '70ies.

      Things have progressed since then!

      The recommended way is now a variant of Test early, test often mixed with Iterate - either XP, Agile or perhaps some form of RUP (for the BSDM fans) or whatever the current name is.

      Nowadays you prototype, testing as you go along, releasing incrementally and refactoring when needed, all in an iterative way. All this requires a strong test-suite, which is buld preferebly before development. Testing is a lifelong process, and the testsuite keeps growing - often it is larger than the application - but it keeps everyone happy, as we all know that the system is working as expected.

        That's why I say test and verify - you can do this while implementing .

        Basically I agree with this other AM at Re: Re: Re: My number 1 tip for developers..

        You and BrowserUk both seem to have a different understanding of the steps. Perhaps I should have emphasized basic in the basic software engineering steps. There are no subcategories. There are no metrics.

        BrowserUk says The whole point of the tip is the idea that you learn to recognise (as early as possible) when you are wasting time trying to 'get something to go', and that's a good tip if you get writers coders block. I suppose my only uneasiness with BrowserUks original node is that he first tips on the mechanics of banging a keyboard, when spec/analysis and then design is what you always need to start with.

      You've just describe (in breif) the waterfall method. As beloved of CompSci faculties and IT departments from the late 60's on. It Was then, Is now, and ever shall be the worst way to develop software ever devised (IM(NS)HO:).

      The biggest problem is that the main criteria that it satisfies is based upon flawed metrics--kLOCs, function points etc. Project progress is measured in terms of how much code has been written. That's equivalent to measuring one's progress on a journey in terms of how far one has traveled, instead of how much closer you are to you destination.

      You can travel a very long way, and do so very fast, but unless you have regular and mandatory checks to ensure that you are travelling in the right direction, measurements of how far you have travelled are entirely bogus. But even simple checks of progress are not enough. It is entirely possible to complete a journey from A to B by spiralling around A in an ever increasing arc (Archemedian Arc?) until you arrive at B. If you take your progress measurements at inopportune points, it may seem like your making steady progress, and unless you have an overview, you won't see the flaw.

      I was required to use this methodology,and still have a plastic laminated reference card outlining the "10 stages of The Project Development LifeCycle" which breaks these 10 stages into 102 substages. I keep it on my wall as a reminder of the "Bad Ol'Days", the mid to late '80's in my case.

      I worked on a project that had 1000 developers working at 8 sites across 2 conglomorates in 6 countries. Millions of lines of code developed "To a spec", and then saw the huge mess when they tried to integration test the results.

      No thanks:)

      I'll stick to those bits of the half-a-dozen-or-so methodologies I've seen become the latest fad, and then fade, that made the transition from a rule written on paper, to something I personally found useful enough that I continued the practice even after I no longer had to do it.

      As for the term "software engineering". In my opinion, that is an oxymoron. See Re: How do you view programming for an explanation of why.

      Your way sounds like it would take twice as long as anyting else.

      The whole point of the tip is the idea that you learn to recognise (as early as possible) when you are wasting time trying to 'get something to go', and start again. It's my personal experience that doing anything the second time is easier than the first. I've also found that trying to fix-up a first attempt usually takes longer than starting from scratch. So the purpose of only typing as much in each go as you know will work, and then testing that it does, is designed to minimise the amount of code that you need to revisit at any stage.

      Writing code is easy, it's finding the bugs that is hard! So don't try and find them. Throw the bugs away and write again. IME, this gets you to the end of the journey much faster, because you spend much less time retracing your steps.

      In the end, this was only my tip, offered on the basis that some might grasp the logic and find it helped them, but like all tips and advice, it's entirely ignorable:)

      Examine what is said, not who speaks.
      "Efficiency is intelligent laziness." -David Dunham
      "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
      If I understand your problem, I can solve it! Of course, the same can be said for you.

        I learned the Classic Development Cycle quite differently from most people (it seems). My high school Comp Sci instructor (1979) presented this basic software development flow and then immediately used it to introduce concepts of iteration, recursion, and feedback (on the model itself). I never felt that the Classic Development Cycle approach was flawed, just most people's interpretation and implementation of it (as a one-way monolithic process). To my mind, modern methodologies like XP seem like natural extensions or revisions of the classic model (as I learned it), not radical departures.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://290885]
Approved by Enlil
Front-paged by Courage
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others about the Monastery: (2)
As of 2024-04-21 17:18 GMT
Find Nodes?
    Voting Booth?

    No recent polls found