But as far as development process, for many of my own 'personal' progs, I go back and revise them over time depending on my changing 'whims' the evolving language and my understanding thereof -- but only insomuch as I want to upgrade/update a program.
This prog started out as a 1 liner shell script /w perl filter several years ago:
Then within the past 3 years successive versions added support for:iostat 2 -dkx |perl -e '$|=1;while (<>) { /(?:Dev|[a-s]{3}\s+)(.*)$/ a +nd $1 =~ /[1-9s]/ and print ;};'
Then was unhappy w/with it not showing any of my dynamic disks (they were all grouped under 'sdX'), so added that over the weekend and due to the increased display length, decided to go to a replot-display vs. scrolling.
A few days ago I "folded" shell portions into the perl and made it a native perl script.#!/bin/bash -e Time=2 IostatArgs="-dk" NewIostatArgs="-m" while [[ "$#" > 0 ]]; do if [[ ${1%[0-9]} != $1 ]] ; then Time="$1"; else if [[ -n $NewIostatArgs ]]; then NewIostatArgs="$NewIostatArgs $1" else NewIostatArgs="$1" fi fi shift done if [[ -n "$NewIostatArgs" ]] ;then IostatArgs="-d $NewIostatArgs" fi let Time=$Time+0 #comment out sd[cdef] -- part of md0 (2010-03-03) iostat $Time $IostatArgs | \ perl -e ' $|=1; $clear=`tput clear`; $at00=`tput cup 0 0`; while (<>) { if (/(^.*(?:Dev|[hs]d[abcdefg-z]\s|md[0-9]\s)\S*\s+)(.*)$/) { my ($dev, $rest) = ($1, $2); if ($dev =~ /Dev/) { print $clear.$at00.$_; } else { print unless $2 =~ /^[0\s\.]+$/; } } } '#!/bin/bash -e Time=2 IostatArgs="-dk" NewIostatArgs="-m" while [[ "$#" > 0 ]]; do if [[ ${1%[0-9]} != $1 ]] ; then Time="$1"; else if [[ -n $NewIostatArgs ]]; then NewIostatArgs="$NewIostatArgs $1" else NewIostatArgs="$1" fi fi shift done if [[ -n "$NewIostatArgs" ]] ;then IostatArgs="-d $NewIostatArgs" fi let Time=$Time+0 #comment out sd[cdef] -- part of md0 (2010-03-03) iostat $Time $IostatArgs | \ perl -e ' $|=1; $clear=`tput clear`; $at00=`tput cup 0 0`; while (<>) { if (/(^.*(?:Dev|[hs]d[abcdefg-z]\s|md[0-9]\s|dm-\d+)\S*\s+)(.* +)$/) { my ($dev, $rest) = ($1, $2); if ($dev =~ /Dev/) { print $clear.$at00.$_; } else { print unless $2 =~ /^[0\s\.]+$/; } } } ' # vim: ts=2 sw=2
The same day I got unsatisfied with the all numeric display, -- when it's changing every 2 seconds, it's really hard to take in the data and see what's going on. So I decided to move to an ascii graphical display to better give me an idea of what was going on. and begin an extemporaneous rewrite toward that goal. It was in the middle of doing this that I also realized that 'dm-x' was too cryptic for me, so I'd need to add the dm->mount-name mapping.
the extemporaneous rewrite was fatiguing to look at -- and I needed to think about refactoring which caused my earlier booboo.
I also decided that if I was going to do a graphical display, I wanted it to appear to update with minimal blank time, OR w/no blinking at all. This meant I needed to pre-buffer the output until I had a new page ready, then issue the clear screen and dump out the new data, but that got too complicated to complete in my head w/no output, so I decided to get what I had 'working' so I'd at least have something tangible working -- a 'prototype' of sorts. That's what I submitted as my problem code.
Also, last night figured out if I was going to map the 'dm' names, I probably should try to get partition i/o and their names instead of just looking at the devices -- haven't even started w/that.
I also came to realization that all of the stats I needed could be gotten out of /proc/diskstats, rather than using 'iomon'... Another "rewrite" pending... though I am debating about whether or not to continue to develop and get working the 'iostat' usage case as a 'fallback position' should /proc/diskstats not be available (i.e. if I wanted to make this script more generally useful than just for my systems).
I'm leaning toward making it work as well, but may shift that to a back burner and move getting stats directly out of /proc/diskstats a 1st priority -- along with I/O breakdown by partition rather than just device.
So you see, with some of my progs (I go through MANY iterative rewrites as I develop the code, since as I begin to prototype, I start realizing things that don't work and things that do, or even hit on other ways to do things that might be considerably more efficient.
A simple filter on 'iostat' made sense as a 1-liner, but as complexity grows, its good to review such decisions in light of evolving needs and see whether or not the original paradigm (for getting io stats) is still the best one for the job at hand.
OTOH, it would have been VERY VERY difficult for me to sit down and plan all this out in advance -- since much of it I didn't know I wanted until I actually started to see output and decided I wanted something different...
Unfortunately, that's **often** been my design process -- I can write down in very high level language what I want "do see where I/O is going on, on my system", but exactly what's the best way to see that is not something I can know in advance of actually doing it -- unless someone else has already done it -- in which case, I could just copy their code! ;-)
That's why managers don't get that programming isn't an engineering discipline but an artistic/creative one. In engineering, you build tools out of existing widgets and it's known about how long it takes to put those widgets together, but if you are creating something that hasn't been created before, then you can't know how long it will take.
That's one of the key underlying points, I believe behind MIT research that being able to schedule computer code is a 'myth'. And that people who claim to be able to do so, are being disingenuous. The only way to possibly be good at scheduling is to put in huge multipliers about what you think it will take, then adapt your time schedule to fit the predicted schedule. This has the problem of sometimes allowing too much slack time (though, arguably, **some** slack time is needed to prevent burnout).
It's when managers 'micro-manage' that such an approach becomes problematic -- with the result, In my experience, being that a micro-managed project will come out with fewer features, less quality in the same time as one that isn't (presuming the engineer isn't a newbie and is capable of managing their time and getting work done). When micromanagement is in play, then all the things that you would likely have done to make the project better, DON'T get done because they are not explicitly listed in the micro-manager's schedule.
Then, of course, they 'blame you' for those things not getting done -- when you tried to explain to them that if they wanted to to micromanage, then you can't expect the person to think for themselves lest they be considered 'off target' or doing superfluous work (which really happened to me more than once when I tried to go beyond what was called for in the 'plan', but was still within schedule).
Even when such extra work was done in *extra hours* (staying late @ work, working on weekends), the pinhead literally told me that if I had the extra energy to be working on such tasks, I should tell him so he can allocate the time where he sees fit. Unfortunately, if it isn't something _I_ want to work on, I'm not going to have the 'extra energy' to want to do such work.
I took such an attitude as being defiant and sabotaging -- like I was somehow wrong to choose how I wanted to spend my extra time, just because I happened to be at work (this *wasn't* about me doing personal projects @ work, but was about going beyond his 'plan' on work projects -- like 'stress testing' a parallel, multi-threaded project to attempt to break it via obscure timing problems, etc...
I think I got a bit off of my original topic...but wanted to respond to the idea of not 'second guessing' or that second guessing is always a bad thing, when 'refactoring' and redesigning 'on the fly' is more often my native mode of development. I think a close cousin, if not the same thing, got a buzzword applied to it a decade or so back and became 'trendy' -- 'eXtreme programming'. It seemed to be creating a movement that was trying to formalize and stratify what was, for me, a dynamic process, like a manager trying to get such a development practice "under control" -- but somehow whenever I read the ideas of that, they seemed so 'rigid' -- and time consuming, whereas I guess I prefer the freedom to use that mode when it's appropriate but not have the mode for a project dictated by someone who isn't doing the development. But I never worked w/a manager who actually used that style, so I can't really say....
Anyway, back to more diddling...either that or a nap...hmmm....(naps are an important part of the creative process!) :-)
In reply to Re^4: foreach(@input_lines) giving array ref not scalar? Confusion
by perl-diddler
in thread foreach(@input_lines) giving array ref not scalar? Confusion
by perl-diddler
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |