I even know of organizations where design and development of the testing environmment begins at the same time as — or even well before — application development

Well, to be quite frank, I know of these organizations much more by rumor than by personal experience. But I have faith that they exist... somewhere...

---

Give a man a fish:  <%-{-{-{-<

Usually only in an authors head when trying to flog you their latest "Better software quality with {hype-technology-of-the-week}" book.

As soon as you get into interactive web pages with lots of daily changing dynamic data in a dynamic environment, the best test is a second pair of eyes during development/code changes and than re-test the changed part on a dedicated test server manually after importing the latest database backup.

perl -e 'use MIME::Base64; print decode_base64("4pmsIE5ldmVyIGdvbm5hIGdpdmUgeW91IHVwCiAgTmV2ZXIgZ29ubmEgbGV0IHlvdSBkb3duLi4uIOKZqwo=");'

"I even know of organizations where design and development of the testing environment begins at the same time as — or even well before — application development."

This is personally how I approach nearly each piece of software I write (whether it be Perl, Python, C, C# etc), and most of my code has 98%+ coverage (with coverage being covered in dozens if not hundreds of different angles).

That said, the vast majority of companies I've worked for or contracted for don't take this approach as it's too costly. Businesses need to get product out the door, so proper Test Driven Development (TDD) adds a massive delay to realizing Return on Investment so it's mostly eschewed for develop it now, hope it works, but if we have to fix a bug later, THEN add tests for that one piece.

It's a sad reality that profits outweigh quality, but that is the world we live in, unfortunately.

Now, if you know of a specific shop that does use TDD by default, please let me know who :)

…it's too costly.

It’s a false economy though. Maybe not 100% but usually if there is some balance between total coverage TDD and shipping early/often. Like buying something on credit with a large downpayment, you halved, at least, the total cost when the interest is paid off. You, in my view, take the right approach and I expect the hackers who work with you or inherit your code appreciate it. That said, “profit” is not a dirty word. Without profit, there is nothing to bank. Without savings banked, we’re back to toiling in the fields or on the seas and hoping the larder doesn’t turn before the spring.

I even know of organizations where design and development of the testing environment begins at the same time as — or even well before — application development.

Those are the ones that never finished their original project, because they spent all the time debugging their tests that have been written by the same people as the actual project code.

That's not true, actually. At work, we write lots of tests: unit tests, integration tests, and API tests, we have manual testers and QA engineers. Maybe it wasn't the case at the very beginning, but nowadays, any new feature or change must be properly tested. And our original project is kind of finished, or at least our customers pay for it and use it (we've just had the best quarter results in the whole history).

($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
[download]

I also write a lot of tests. Sometimes before a new feature. It does take quite a bit of time; more time than the actual code in most cases. It also protects against shipping bugs, losing customers, late night emergency patches, the next dev’s insanity, regressions, bad upgrades, doc errors, impossible refactoring time…; and in my current $work industry in particular, it prevents lawsuits. After a year or so, it has amply repaid the initial time sink.