software development is not the firm's core business :D :D, but it's becoming important, so they're starting to realize the importance of a solid design.

…maintain real code in real world, according the engineering principles it's very very hard. Not impossible, but difficult.

You are completely right but not because code or engineering is hard. You are right because doing a good job at anything is hard. Being a good teacher? Good God, it’s Sisyphean. Being a good musician? 6+ hours a day for years. The code world is special perhaps in that it’s still so wide open. It’s been talked and written about at length; managing code processes can make things worse, some hackers are geometrically more productive than others and it’s hard to see why, adding manpower to a code project can dramatically slow it down.

Anyway, you said nothing controversial. Being good at anything is difficult. It’s why so few things in life are nearly as good as they could be. And why, back to musical examples, some makers can churn out musical instruments that won’t last a year in real use while others makers can and have created instruments that have already lasted hundreds.

I commend this to you as an exploration of some of the governing issues: Thinking about Thinking.

Homo homini deus est, si suum officium sciat.
[download]

Cecilio Stazio
Ie: Man is god for other men, if he knows well his job.

L*

There are no rules, there are no thumbs..
Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

Homo homini deus est, si suum officium sciat

Hm. Not nearly as powerful as the original:

Homo homini lupus

Nor doesn't it ring true as strongly. It's not so easy to improve on the classics :)

I commend this to you as an exploration of some of the governing issues: Thinking about Thinking.

Thanks, I will read the article as soon as possible!

There used to be an idea to design, create pseudo-code, and real code would be a small percentage of the development process. While the idea is great for designers, it is terrible for programmers. It removes most of the excitement of coding and the magic of making things work. Design is a great idea that doesn't work in the real world for most programmers.

Instead, the idea of frameworks and standards took hold. Though frameworks and standards are misapplied and often just plain wrong, it does give some form to the code and is at least sometimes better than complete freedom. A compromise of sorts.

For those inclined toward personality typing, on the MBTI, in my experience, programmers are Ps, designers, Js. Ps are better programmers and making things work. Js can do that, but not as well. Ps also have an easy time making disparate things work together, often leaving Js in the dust. Js are better designers and will prepare fallback plans well before Ps even recognize there may be an issue. Recognizing others strengths and compromising is the best way to complete our goals.

Today, rapid development has taken on new meaning. It means, as stated, the customers don't even know what they want, and yet the code must get started right away. One solution has been new approaches to writing fast code. But, while this works great for the UI, it wreaks havoc on good code. To make matters worse, managers often do not understand the difference between good and bad code. This leaves the programmer with no recourse, and management with "unforeseen" problems.

If the programmers are trusted by their managers, such as can happen over time, there's good chance the programmer's opinion will be trusted and more time can be asked for and granted. Perhaps not always, but often enough to make it workable.

If the programmers are trusted by their managers, such as can happen over time, there's good chance the programmer's opinion will be trusted and more time can be asked for and granted. Perhaps not always, but often enough to make it workable.

How true!

You require the customer to sign off on what they want before you begin, and to sign off on the completed project if it does that, even if it doesn't do what they want! Once your customers see you are serious about this, you will have fewer cases of it. They will be more careful about what they ask for. You need commitment to follow this policy supported at the highest level, or else people will complain high enough to get around it. Yes, there will on occasion need to be emergency patches, but even then (especially then!) you can still maintain programming standards.

Once your customers see you are serious about this, you will have fewer cases of it.

Fewer cases because they might not be continuing your employment. It probably depends on the relationship.

You require the customer to sign off on what they want before you begin

Even when there is sign-off, what it means is entirely a new thing. I'm on a project right now where there is sign off on multiple levels, yet almost noone knows what is happening. The customer thinks one thing, the BAs something else, the programmers have to go through multiple revisions before they get it right (according to the team lead), and the team lead overrides the customer's stated wishes by convincing them that what they want is impractical.

What you want is right, but are you sure its practical?

It is completely practical - it's how it was done at Master Lock. Developers could update the version control system but could not promote code to production. Only the gatekeeper could do that (in our case, the DBA since it was mostly Oracle PL/SQL), and he required the developer to show a signed acceptance from the customer before he would do so. The customers expected the developer to demonstrate the code in front of them, and could access the development system to test for themselves, in the case of forms or reports.

Dum Spiro Spero

Sign offs are good in theory. In my experience, when purchasing software/systems from external contractors, extras really cost extra and really extend schedules, which tends to control scope changes. When doing internal development, the business unit management usually has more clout than the IT management, so their lack of due dilegence in reviewing and understanding the spec/scope/schedule, just turns into your 16 hour days or hunt for the next job.

James

How do you manage this situations, what are your experiences about?

The Boy Scouts have a rule: "Always leave the campground cleaner than you found it"

What if we followed a similar rule in our code: "Always check a module in cleaner than when you checked it out"

-- The Boy Scout Rule (O'Reilly)

At work, we try to follow The Boy Scout Rule. Unfortunately, due to time pressure and deadlines, the code is often only a teensy bit better. Still, if you're swimming day after day in a foul-smelling swamp of legacy code, you need something to keep you going, to keep your head above water. Having the feeling every day that you've made the code better, rather than worse, does make it easier to cope.

This is a really hard problem and there are no silver bullets. And it is not just commercial time pressure and deadlines that leads to horrendous legacy code. After all, even Perl itself and the blessed PerlMonks code base have not been spared, as described at Nobody Expects the Agile Imposition (Part VI): Architecture.

(Cautiously sticking my head out a little bit [farther ...?] here), I actually don’t agree with that “rule.”   No, not at all.

I want to look at the revision logs in the version control system and see that every commit is clearly tagged to an open-and- approved-for-action ticket in the issue tracking system.   Then, I want to see that every revision corresponds only, and very exactly, with the specific issue that was described authorized in the approved ticket.   I never want to see a change being made that is not directly related to the ticket, no matter how badly someone felt that the “legacy” code smelled.

If you want to “clean up your campsite,” then ... open a ticket which describes exactly how you intend to do that and exactly what the business impact of the change will be.   If that change is approved, then you may proceed, as you would with any other software change and with an equal amount of probity.   Otherwise, and until then, the answer is “no.”   Even though you might be supremely confident that your “new and improved” code will be exactly equivalent to the code that it replaces, not to mention “–er,” you’re still making changes to source-code that is working now.   Even though you were not asked to do it, nor authorized to do it.   You’ve subjected the enterprise to the unplanned-for and unrecognized risk that you might screw-up, and you did it unannounced and on your own (non-)authority.   So what if you’re good; so what if the change is small.   That’s not the point.

“Change,” to software source-code, “is risky.”   There must be no exceptions to that rule.   Even if a carpenter thinks that a wall sux where it is, and would be so much –er if it had a nice window in it ... that does not authorize the carpenter to take it upon himself to install one.

Yeah, I know that this notion or this position won’t “sit well” with a lot of folks ... but I’ll stick to it, nonetheless.   This sort of discipline would be de rigueur in any other form of engineering undertaking, and I submit that Software Engineering should be no exception.   I didn’t start out with that point-of-view.   I came to it by decades of cleaning-up after the consequences of it not being done.

Many businesses follow the process model.

Established businesses see the benefit regularly, but their margins are slimmer.

Few start-ups succeed with it.

I suspect the reason is because the start-up often is doing something new and exciting, attracts quality people who make a sufficient number of good decisions that the end result is passable, was built at low cost, and can reap the kinds of margin needed by investors.

I have now seen three companies attempt to migrate from the start-up mentality into the more mature process model; two are still flailing with it after years and years, and one is actually grinding through it with a fair bit of success.

I used to favor the open and artsy approach, but that was effective because most software developers were engineers, and evaluated the whole problem as best they could. With the passing of time, I see the software development environment has become less about engineering and more about having a job. The need for process has increased to compensate for the absence of engineering discipline in the marketplace.

I begrudgingly admit that for most businesses, you have it right. The fixed process, cumbersome though it be, allows risk-averse delivery. Software is so commonplace now, this is needed far more than I could have ever wanted to admit.

I believe there are places where trusting your developers to make the right choices, and artsy/freelance approaches like the Boy Scout Rule, have a benefit. But the number of places where such things work well for a business are shrinking rapidly. And it would not suprise me that you don't see it. On that, we are likely to simply disagree, and I'm okay with that.

Process is great until it gets in the way. Someone once shared their managerial philosophy: If you can't measure it, you can't manage it. Most process-oriented people see this as an absolute, but the reason is simple: A single failure is unacceptable in their world.

Without risk, profit margins are reduced. Do you go with safety, or margin?

I have come to the conclusion that the answer to this is not as cut and dried as most people with an opinion on the matter would have us believe. There is a time and a place for each. But where software development tends to take place in business these days, the vast majority, unfortunately, must favor safety.

And the sun still comes up tomorrow.

In my experience, it is a business-process issue, and this is the biggest point that I stress in consulting/turnaround engagements.   The project faltered, not because the programmers were incompetent nor because the requirement was impossible, but because of unstructured and undisciplined business process.   Many people presume that software, because it “merely” consists of source-code files, is easy to change.   Not so.   Computer software is actually a frighteningly-complex machine that operates autonomously, and whose thousands of parts are all tightly coupled.   Any change could break the whole ... and you are extremely fortunate if you realize that this has occurred.

This means that the business process of managing the software ... of any changes to it or to its requirements ... must be formalized, and that formality must be adhered-to.   You can’t simply get a cool idea in your head, walk into the “dungeon,” and order your programmers to stop whatever they’re doing and do it.   The programmers, in their turn, can’t simply “make it all up as they go.”   Change, itself, is seen to be business-risky no matter how large or how small it appears to be.

Many companies now have a “change approval board,” or CAB, such that nothing may be done to any system (whether in production or in development) without the express approval of the Board, which reports to the CIO/CTO if s/he does not directly sit on it.   Completed changes are reviewed to determine that they correspond to an approved change-order and that they change “nothing more and nothing less” than what is called-for.   Programmers lose the autonomy, the power, and the discretion that they might be accustomed to.   But, the programming environment is vastly improved.

Good software process management also relies upon source-code control, a ticketing system, and the discipline to effectively use both.   It relies paramount on clarity and effectiveness of human communication.

Many people presume that software, because it “merely” consists of source-code files, is easy to change

You're absolutely right.
Unfortunately, writing code it's considered just writing. When discussing, I often use the metaphor of building an house: If you want a kitchen on the second floor, and after the work ends you realize that is better to have a kitchen on the first floor, if you design a familiar house but after you want an open space, it's easy to realize that throw down everything and redo it's a cost of time, money, and resource.

I'm completely agree with your analysis.

The analogy I like to use is one that’s lifted from a little Kindle-only book, Managing the Mechanism, available (only) on Amazon.   The analogy is that software is an autonomous, self-directing machine.   Acting only on the yes/no instructions of which it consists, software must direct the digital computer (which knows nothing at all) to do exactly the right thing in all of the circumstances it might encounter.   All without any direct influence by the programming team that built it.   The software must not only play football, but win the game, and all this with no humans on the field.

And the complexity of that software contrivance is ... basically, infinite.   Almost any part can interact with almost any other, and interactions occur both across “logic” and over time.

You would never build a physical device, nor any sort of building, with such complete lack of process discipline.   (Building and engineering codes, laws, etc. would not allow it ... and of course it would be “obviously folly.”   Well, the “folly” in software is many times greater ... but, somehow, not “obvious.”   Project-management of a software project is like no other form of project management, because the nature of the project being managed is like nothing else in the world.

I think it’s coming out on the Apple iBook-store soon.   I see no reference to a printed edition.