Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical

Self-improvement and TMTOWTDI

by Tanalis (Curate)
on Jan 24, 2003 at 14:00 UTC ( #229598=perlmeditation: print w/replies, xml ) Need Help??

A couple of things this morning set me thinking about Perl and its nature, and the methods it provides to complete tasks in more than one way.

I'm sure I'm not alone here in that I code Perl for a living, 8 hours a day (theoretically *g*), 5 days a week.

This morning, though, I was looking through a module I wrote about 4 or 5 months ago, that works perfectly well doing what it's doing (interfacing with one of the corporate databases and processing information that comes from it). Completely on the spur of the moment, I decided that I could improve one of the methods in the module, and started to rewrite it (I had time on my hands). About half-way through, I realised I wasn't improving anything much at all: I was largely just changing code so that it looked different - I wasn't adding anything or taking any functionality away.

Now, I started thinking about this, and I finally came to the conclusion that because I can do something differently, as my tastes for code and style change, I logically do do things differently. The method that caused offence this morning was quite nicely written, given my tastes that month, but it just happened to be incompatible with what I'd decided I liked this month. Sounds quite shallow, and I guess it is, but there's no other explaination (at least, that I could think of) for me picking up on a couple of things and subconciously thinking they're "bad".

I'm slowly coming to the conclusion now that what happened this morning is quite natural, and not as shallow as I originally thought. 4 months ago I was relatively new to Perl, relatively new to the concepts I was trying to implement. I guess it's only natural that I'd have gotten something working (regardless of how) and left it at that.

Coming back to the code now, with a much better knowledge of both Perl and the DB that I was slowly learning then, a couple of ways of writing code that were perfectly acceptable to me then just aren't now, because of what I've learnt and the methods I now know how to use.

The whole concept of there being MTOWTDI has a flaw: if you're out to seek personal improvement, as either a byproduct of learning Perl, or as part of it, it's inevitable that you (and your perspectives) change, in some ways. I think me looking back at old code critically, simply because I wouldn't do it that way now, is a part of this, at least for me.

Don't get me wrong here, I think that the whole concept of having more than one way to do stuff is great. My point, I think, is that the knowledge you need to confidently be able to do things differently, and to evaluate the different ways of doing things for clarity, efficiency and readability, takes a very long time to gain. This, though, is the same for anything - until you have the necessary basic knowledge to be able to evaluate your options effectively, decisions you take are by definition flawed, simply because you can't fully explore all of the possibilities.

This, in my mind, is one of Perl's best points: you never stop learning, and coding with the language can always be challenging and interesting, but also one of it's worst points. Where do you draw the line? With (most) languages, it's possible to step back at some point and say that you know it inside out. I'm not sure that's possible with Perl.

I'm interested to know what other people think about this - is there an attainable point that would cause you to think that you know "all there is to know" about Perl? (Is this even possible?) What's your take on this?

A few random thoughts (that became a crazily long post) .. excuse the rambling :)
-- Foxcub

Replies are listed 'Best First'.
Re: Self-improvement and TMTOWTDI
by Heidegger (Hermit) on Jan 24, 2003 at 14:19 UTC

    Today I had a talk with a friend of mine, who's a Java programmer. We kind of agreed with him that by looking at the Java code you can moderately quickly realise if the person who wrote it is a decent programmer.

    Java doesn't let the programmer so much of TMTOWTDI as the Perl does. When I'm programming Java I usually end up by looking up a method in the Java docs.

    Also, as experience of a Java programmer grows, he makes more and more reusable object-oriented code.

    Why I'm comparing these two languages? Again, I want to stress my statement, that it's easy to tell if a Java programmer is good or not. To do the same with a Perl programmer is more difficult.

    As Foxub wrote in his meditation, the style means more to the Perl programmer. And the programming style can change from month to month. The Java programmer's path is more straight, with no curves.

    So, just after lurking into a Perl programmer's code, would you take him/her into your project? Or dismiss him at once? Do you need a Perl programmer or a hacker?

      Why I'm comparing these two languages? Again, I want to stress my statement, that it's easy to tell if a Java programmer is good or not. To do the same with a Perl programmer is more difficult.

      Got to admit I disagree. Having seen perl code produced by job seekers (and having to deal with other peoples code in general) I find that you can very easily see whether a developer is a reasonable programmer - just as you can with any other language.

      There are more differences in minor areas of style, but as long as you know perl you can understand when something is sensible.

      Possibly we're talking about two different things. There is being a good programmer (being able to produce sane, comprehensible code). There is also having a knowledge of all the perl idioms and the various odd language corners.

      The latter does not imply the former, and the former does not imply the latter.

      I've seen code that obviously shows a deep knowledge of the various obscure corners of perl, but written in a completely non-maintainable "selfish" style. Sign of a bad programmer. Usually impossible to fix. CV goes directly to the bin.

      I've also seen nice code, but that is obviously written by somebody who has come from another language. Lack of knowledge can be fixed by training. CV stays on my desk.

      Obviously, the ideal is to have both :-)

      Also, hopefully obviously, you never hire on code alone. Somebody can write great code and be lousy in a team.

      I would not take a programmer just based on looking at their code. I might reject someone after seeing (bad) code, but a code sample does not answer the crucial question of whether, among the many ways to do it, that programmer can live with settling on the way(s) shared by the team who has to live with his or her code.

      This is not to say that a good programmer would not have a significant role in deciding what the shared ways to do it are - but teamwork needs willingness to live with other people's limitations.

•Re: Self-improvement and TMTOWTDI
by merlyn (Sage) on Jan 24, 2003 at 16:03 UTC
    I've been programming for 32 years, and for over a decade in Perl.

    When I write code the first time, I'm usually drawing on known good patterns that I'm able to recall in the past. Perl's flexibility gives me the options to give the right balance between named and unnamed things, processes versus data, and organization and optimization for various goals, such as efficiency or maintenance.

    But after the program is working, I usually think of three other things I also want it to do. At that point, a rewrite has a different goal, and might even result in "oh yeah, I also want to do this and that, so that would better be handled by a different abstraction".

    This is a natural part of the learning process, stretching both me as a programmer, and in particular my skills in programming in Perl. It's also partially an acknowledgement that I slowly learn more about the particular problem domain as well, by solving one piece of it, and then realizing what the next problem in the problem domain might be.

    Most of the patterns I carry in my head are now "native Perl", but there are still things I recall from Icon and Smalltalk that have no good Perl representation yet. Maybe someday soon. {grin}

    -- Randal L. Schwartz, Perl hacker
    Be sure to read my standard disclaimer if this is a reply.

      there are still things I recall from Icon and Smalltalk that have no good Perl representation yet. Maybe someday soon. {grin}
      What do you miss?
        Nothing that isn't already in Perl6. {grin}

        From Icon: Generators. Co-routines. Structures-as-hash-keys.

        From Smalltalk: a true integrated development environment, like we had in 1980. {grin}

        -- Randal L. Schwartz, Perl hacker
        Be sure to read my standard disclaimer if this is a reply.

Re: Self-improvement and TMTOWTDI
by adrianh (Chancellor) on Jan 24, 2003 at 14:43 UTC
    This, in my mind, is one of Perl's best points: you never stop learning, and coding with the language can always be challenging and interesting, but also one of it's worst points. Where do you draw the line? With (most) languages, it's possible to step back at some point and say that you know it inside out. I'm not sure that's possible with Perl.

    Personally, I find this is true of all languages - not just perl. It's been my experience that you cover the same ground learning any language - from complete novice to somebody who spends their time poking and refining their knowledge of the corners of the language (including its idioms and libraries)

    I've never reached the stage of "I know all I need to know" in any language. Mostly because the language is not really the most important thing. Programming is. You learn new progamming techniques and then have to figure out the best way to apply them in a particular language.

    Actually, to be mildly heretical, I don't think perl offers more "extra" ways of doing things than many other languages. The difference is that the perl culture treats this as a good thing ;-)

      Actually, to be mildly heretical, I don't think perl offers more "extra" ways of doing things than many other languages.

      Somebody _had_ to call you on this. :-)

      I can think of a disturbing number of ways to iterate over the elements in an array in perl. In Pascal I can only think of a few, likewise for VB, and similarly for C/C++ (I mention this one last because I know it the least well and so might be suprised.)

      Without using procedure calls I can think of the following notationally distinct means (and thats just at the moment, no doubt there are more, and i'm even ignoring the 'for' eq 'foreach' aspect that adds even more)

      # 1. ... map { ... } @array; # 2. ... grep { ... } @array; # 3. for my $item (@array) { ... } # 4. ... for @array; # 5. for (my $i=0; $i<@array; $i++) { ... } # 6. my $i=0; while ($i<@array) { ... $i++ } # 7. my $i=0; while ($i<@array) { ... } continue { $i++ } # 9. my $i=0; ... , $i++ while $i<@array; # 10. my $i=0; MARK: ... $i++<@array and goto MARK; # 11. my $i=0; { ... $i++<@array and redo; } # 12. ... foreach 0..$#array; # 13. foreach (0..$#array) { ... } # 14. Blame [tye] for( pipe(IN,OUT) && do { local $,=$/; print IN @array,'' } && <OUT> ) + { ... }
      That makes thirteen so far.(!) And I bet as I ride the train home a few more will come to me too. (I will update this node with any contributions that are made so please send em in. Id like to see if we can hit 20. :-)

      So at least in this regard I think I would have to disagree with your claim.

      Incidentally, I think that this is an interesting juxtapositioning (maybe the wrong word ill grant) of two key perl concepts. The first is "everything that is different should probably look different if at all possible", and the second is that "for everything that you can do there should be a few different looking variants available (TMTOWTDI)", which combine together nicely to match another generally held but certainly perlish imperitive which is that "code should look like it does what it does". So if we need to transform every element of a list to create a new one we use a different construct than we do if we want to apply a simple function to every element of the original, which is different from when we want to do something complicated with every element, and is also different from when we want to use the index, yada yada yada...

      All in all i think this is great once you become used to it. When I see

      my @array; $_++ foreach @count;
      I instictively understand that the intention of the other programmer was very different to
      my @array=map $_+1,@count;
      which I find very useful in dealing with old code.


      --- demerphq
      my friends call me, usually because I'm late....

        Just a couple of ideas ..
        my $i = 0; do { $array[$i++] .. } until $i > $#array; do { $array[$i++] .. } while $i <= $#array; JUMP: do { $array[$i++] .. } goto JUMP if $i <= $#array;
        I reckon that's 17 .. 3 to go :)

        -- Foxcub

        Update: Couple of corrections.

        Some more...
        #1 while (@array) { my $elem = shift @array; ... } #2 foo(@array); ... sub foo { my $elem = shift; ... foo(@_) if @_; ... } #3 foo(@array); ... sub foo { my $elem = shift; ... &foo if @_; ... } #4 { my $elem = shift @array; ... redo if @array; }
        Duplicate these for using pop or splice rather than shift if you want. Also with your existing ones do you count for as different than foreach? Beginners do.

        I think we passed 20. :-)

        There's probably a few variations on this theme?

        @a = 1 .. 10; $i = $ENV{myI} || 10; print $a[$ENV{myI} = --$i]; do $0 if $i>0;

        Examine what is said, not who speaks.

        The 7th Rule of perl club is -- pearl clubs are easily damaged. Use a diamond club instead.

        Somebody _had_ to call you on this. :-)

        I knew somebody was going to slap me!

        How about Common LISP (gotta (love (those (brackets))).

        At the very least we have: count, count-if, count-if-not, every, fill, find-if, find-if-not, map, map-into, merge, mismatch, notany, notevery, nsubstitute, nsubstitute-if, nsubstitute-if-not, position, position-if, position-if-not, reduce, remove, remove-duplicates, remove-if, remove-if-not, replace, search, some, substitute, substitute-if and substitute-if-not.

        That's 30 - and I've probably missed some - different way of iterating over a sequence of elements.

        (in case anybody hasn't realised - I am not being entirely serious here :-)

        It would be interesting to see whether all these different ways to iterate over the elements of an array compile more or less to the same underlying code.

        Now that would be some optimising compiler!


        "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

        > Without using procedure calls I can think of
        > the following notationally distinct

        Notationally distinct is not terribly important. A number of your "different" ways are extremely minor variations. For example, 3 and 4 differ only in terms of the variable used and whether the condition is placed before or after the block; they not only do exactly the same thing, they do it in exactly the same way. Counting them as different is pedantic in the extreme; if you really think that's clever, I can easily think of a couple hundred ways to do it in line number BASIC.

        More, some of your ways to do it are simply taking more general mechanisms and applying them to a problem normally solved with a more specific type of construct. For example, using a while loop with a counter at the end of it that is not used for anything else other than to count out the iterations of the loop is something you can do in any language that has while loops, because the for loop is conceptually a special case of the while loop. Congratulations, you discovered a general principle that applies to all languages.

        Perl does have a few more than other languages, but it's not quite like you make out. Further, I'd say that these extras (like map) are not "extra" per se but are part of what makes a rich language such as Perl rich. I use Emacs lisp, and it too has some of these higher-level constructs that are essentially a specialised case of something that with a bit more work on the programmer's part could be done with standard flow control mechanisms, but the higher-level features, where appropriate, are less work to use. (And speaking of lisp, does anyone else wish Perl had an equivalent for rassq? Or does it, and I'm missing it for lack of knowing where to look?)

         -- jonadab
Re: Self-improvement and TMTOWTDI
by dragonchild (Archbishop) on Jan 24, 2003 at 14:27 UTC
    Heh. When you get to a certain point, then you'll constantly be writing the same bit of code over and over cause you haven't decided which of the 4 ways that work is the one you like this minute. Welcome aboard! :-)

    Update: I was reminded of a node I once wrote called Iterative learning... that expressed this sentiment to a 't'.

    We are the carpenters and bricklayers of the Information Age.

    Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

Re: Self-improvement and TMTOWTDI
by gjb (Vicar) on Jan 24, 2003 at 14:44 UTC

    In my opinion this is true for most learning processes of "interesting" things. You can always learn something new.

    It is important to realize that the amount of new things one learns about a topic is not constant in time. Typically a lot of new things are learned initially, while over time less and less new insight is acquired. To put it in mathematical terms: initially the learning curve is pretty steep, but after a while it gets more and more flat. Or to rephrase it in terms of Pareto's law: the first 80% are learned in 20% of the time, the remaining 20% take 80% of the time (except that it shouldn't add to 100% in this case ;-)

    For me this implies that after a while I've to switch to something new to learn in order to keep learning sufficient amounts. Is it worth to invest a lot of effort for the remaining 10% or 5% or 1% one doesn't know yet? This depends on how proficient one needs to needs to be at that particular topic, but in general I find it's more worthwhile to know, say 75% of a lot of things rather than 95% of just a few. The good news is that the former can be achieve with about the same effort as the latter.

    Just my 2 cents, -gjb-

    PS: any numbers cited are purely for illustration, obviously ;-)

Re: Self-improvement and TMTOWTDI
by l2kashe (Deacon) on Jan 24, 2003 at 14:56 UTC
    I do it constantly. I have been banging out code in perl ( as opposed to perlish code), for around 3 years now. I look at some of my earlier works and shudder. But at that point my goal was A) learn how to code, and B) code everything to be as clear to the next maintainer as possible. Verbose comments, always creating bracing groups for conditionals, avoiding tie and map and grep, cause they were perlish, and not very clear on what they do in given contexts compared to other looping structures. I always used foreach $something (@blah), as opposed to any other loop because it was very clear what was being looped through, and what contained the element, etc.. you get the drift.

    Anywho, I have been really hunkering down with perl over the last few months, and I've noticed that it takes me longer to make the code overly verbose and clear (as to function of the code) for the next admin, than to just bang out my code and be done with it. I have started moving to more useage of the perl idioms cause I can, and its how I now "think" as I'm coding.

    I think the point you are raising, everyone acknowledges along the way. On the whole, how many ways would you like to skin you cat *today*. I like the variety of perl. It allows me to make simple (and even moderately complex) loops simple, and not take up huge amounts of space. If give me multiple ways of expressing my thoughts in regards to the data set being processed.

    In my mind, it probably is easier to 'master' another language, you can maybe even master perl I mean look at tye, sauoq, and merlin, to name just a few, and their depth of perl knowledge. But I've found a funny thing happening in my career as I gained knowledge. There is ALWAYS something else to learn within any given aspect of this thing called computing. A few people I know simply state, "You need something done, no matter what, he can get it done" and my jaw kinda drops and I babble "but I dont know half of this or even a quarter of what I should of that, and nothing about this" and they laugh a say "what ever your the man, you know it all"...

    So I guess what I'm trying to say is, hopefully you will never become complacent in regards to the quality of your code, and that you will more than likely continue to realize that there are whole new problems that you have never thought of solving, or even knew existed. Along the way, sometimes you will feel giddy with the knowledge you have accrued, other times you'll gnash your teeth at the lack of progress you seem to be making (or is it not making). But in the end, looking back at whatever point, you should be able to review and notice what you have learned since then, and how you applied it from that point forward. And you may end up rewriting something a dozen, or even 2 dozen times, and other times you'll sit down, bang it out, reveiw in 3 or 6 months, nod and say "yup still looks good"... For me thats just programming

    /* And the Creator, against his better judgement, wrote man.c */
Re: Self-improvement and TMTOWTDI
by hardburn (Abbot) on Jan 24, 2003 at 15:24 UTC

    Like you, I will sometimes find code I wrote a few months ago and wonder what I was thinking. How could I have ever come up with something so awful?

    I think this proves that we learned something during the time we spent not looking at that code. If we didn't do that, we should wonder what we are doing wrong right now.

    In a Slashdot interview a while back, David Korn, inventor of the Korn shell, said that he often looks through the code he wrote for the Korn shell about two years ago, wonders what an idiot he was, rips it out, and starts it over again. He's been doing this every few years since he started the project (15-20 years or so).

    It at least sets my mind at peace to know that master programmers do this, too :)

Re: Self-improvement and TMTOWTDI
by artist (Parson) on Jan 24, 2003 at 16:37 UTC
    I'm interested to know what other people think about this - is there an attainable point that would cause you to think that you know "all there is to know" about Perl? (Is this even possible?) What's your take on this?

    Well, just like life and other organic systems, we find that our perl knowledge is constantly evolving. Our better way of perl coding comes out of necessity. Necessity is the mother of invention. In our case, invention means enhanced perl knowledge. Now if you increase your necessity you are bound to find new things in perl. If you think that core structure cannot accomplish that, you will even build a new version of perl.

    Good thing about perl is we have great CPAN repository and sites like PerlMonks. It also represents the efforts of thousands of people ( Perlmonks alone has 6200 writers in total). Users around the world constantly find or write shortcuts (Modules and other titbits) to fulfill their necessity and is readily usable by others when it becomes their necessity. Thus for your own code you may find better ways to accomplish that because your thoughts are joined with thoughts of thousands of others over time, which already has enhanced your thinking and ways of doing the things. There could be minute things for which you think 'traditionally' or accept subconsciously, others would put a systematic efforts and make it better. When your eyes see that, you adopt that easily and thus improve your code. I think it is synonymous which the fact that we in our daily life use hundreds of things invented by others to improve our own life continuously.

    We have adopted TMTOWTDI for perl, and thus a constant background thinking that 'there is another way to do it and which might be better' is always there. People are motivated by this vary formula to search for the new things. (ex.. Columbus) In addition to that, Perl has various means of symbols and thus we can see the same thing is used in different context and light. (Obfus are good examples of this.) And with thousands of simultaneous efforts we see more and more of that. This fact also increase our understanding about that same thing and subconsciously aids our efforts in learning perl emphasizing the fact that there is much more to learn.


Re: Self-improvement and TMTOWTDI
by Limbic~Region (Chancellor) on Jan 25, 2003 at 03:44 UTC
    Perl is constantly humbling me!

    I considered myself a decent programmer before I started working with Perl about 8 months ago. I almost never read a book on a new language I was learning other than to get syntax. I mean - there are standard control structures almost every languages has right? You just learn how this language does it and you continue programming the way you always have right? - WRONG!

    In this thread it is said that in Java, it is easy to tell if someone is a decent programmer, but with Perl's Tim Toady approach, the waters are murky. I would agree that is is easy to tell the green newbies as their code looks much like a first year student learning a foreign language. The problem is - telling the difference between a mediocre Perl programmer and an expert.

    There is a scene in Good Will Hunting where the professor is telling Will that there are only a handful of people in the world that can tell that Will is a thousand times the mathematician than he is. Unless you are a professional programmer that understands how the interpreter, optimizer, compiler, etc works - it is very hard to see why "Will's" 2 line code difference from the proffesor's makes it 10 times faster and/or more efficient.

    A while back, I posted this node about the value of "RE"-coding for a newbie. I guess I should have also added how important it is to RTFM from cover to cover. Sure you can learn how to use proper syntax and get the job done, but without getting into the guts of how Perl works under the hood - a newbie will never be Mr. Will Hunting.

    Cheers! L~R

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://229598]
Approved by hding
Front-paged by djantzen
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others musing on the Monastery: (8)
As of 2023-12-05 22:29 GMT
Find Nodes?
    Voting Booth?
    What's your preferred 'use VERSION' for new CPAN modules in 2023?

    Results (29 votes). Check out past polls.