Re^3: Does anybody write tests first?

Me neither.

I'm utterly convinced of both the need for, and the benefits of testing during the development process. My argument is entirely aimed at the methodology and tools being advocated and used.

As for the slides, I got as far as page 12 and 13 and just stopped reading:

Redundancy Is Good.
No! Redundancy is redundant.
I'll put that another way. It costs time. And time is money. Redundant effort is therefore, wasted money.
There are no stupid tests.
Rubbish! Of course there are stupid tests.
Testing that perl can load a module and converting the result into a 0.05% change in a statistic is stupid. When left to it's own devices, perl will report a failure to load a module with a concise and accurately informative warning message...and stop. Preventing that stop is pointless. Every subsequent test of that module is, by definition, going to fail.
I did go back and scan on, because I felt I ought to. And I could pick out more, but I won't.

When testing tools or suites are more complex and laborious than the code under test; when they prevent or inhibit you from using simple tools (like print/warn/perl -de1), they become a burden rather than a help.

In the car industry (in which I grew up), there are many ways of testing (for example) the diameter of a hole in a piece of sheet metal. You might use a lasar micrometer. You might a set of inner calipers and a set of slip gauges.

The favoured test tool? A simple, tapered shaft with a handle and clearly marked, upper and lower bounds. You simply poke it in the hole and if it passes the lower bound and stops before the upper, the hole is within tolorance. It is simple to operate, Is extremely robust in use. And takes no time at all to use. It's easy to use, so it gets used.

The Occam's Razor of test tools. I seem to think that Andy has an appreciation for OZ. He should consider applying it in this case also.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Comment on Re^3: Does anybody write tests first? Download Code

Replies are listed 'Best First'.
Re^4: Does anybody write tests first? by kyle (Abbot) on Feb 22, 2008 at 14:08 UTC
Testing that perl can load a module and converting the result into a 0.05% change in a statistic is stupid. A simple sanity check before deployment might simply check that everything loads correctly. As you know, use does more than just load. It actually executes code, in some cases. If the environment has changed, the config has changed (as it often does when you go to production), just loading a module might break. One might have modules that are rarely used, or loaded dynamically via require. If your other testing isn't very good, at least you can check these rare cases with a quick `Test::More::use_ok`. If someone broke a module such that it doesn't load, I want to show that before I get ten minutes into another (manual) test that depends on it. Certainly there are ways to misuse a simple "does this load" test, but they are not without value.	[reply] [d/l] [select]
Re^5: Does anybody write tests first? by BrowserUk (Patriarch) on Feb 22, 2008 at 18:18 UTC
A simple sanity check before deployment might simply check that everything loads correctly. As you know, use does more than just load. It actually executes code, in some cases. If the environment has changed, the config has changed (as it often does when you go to production), just loading a module might break. But don't you see that all `use_ok` does, is allow the test script to continue when the use has failed. It doesn't test anything extra. It doesn't tell you any more. It doesn't verify exports, or configuration, or tell what piece of code failed. Indeed, if any warnings or errors are produced that might tell you what failed and why, it hides them from you. And, unless you are testing more than one module from that test script (which I assume no one does), there is nothing else useful to do, because if the use failed, none of the other tests are viable. Literally all you've done by using `use_ok` is allow the running of further tests that cannot possible succeed. Oh. And allowing the harness to count more imaginary numbers. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l] [select]
Re^6: Does anybody write tests first? by kyle (Abbot) on Feb 22, 2008 at 18:50 UTC
unless you are testing more than one module from that test script (which I assume no one does) Sorry this was unclear. When I wrote of a "simple sanity check before deployment", I actually did mean a test script that does nothing but `use_ok` over and over with no other tests. When I've had a project with numerous modules, I have found it useful to have `use_ok` tests for all of them in one place. `my @modules = get_module_list(); plan 'tests' => scalar @modules; foreach my $module ( @modules ) { use_ok( $module ); }` [download] If `get_module_list()` is bright enough to use File::Find, it will pick up modules without me having to explicitly list them. I get a sanity check on even those modules for which I haven't written more extensive tests. As I said, if your other testing isn't very good, checking that use works is a start.	[reply] [d/l] [select]
Re^7: Does anybody write tests first? by BrowserUk (Patriarch) on Feb 22, 2008 at 21:09 UTC
Re^4: Does anybody write tests first? by chromatic (Archbishop) on Feb 22, 2008 at 08:32 UTC
Testing that perl can load a module and converting the result into a 0.05% change in a statistic is stupid. And yet I've seen it catch real bugs in real code.	[reply]
Re^5: Does anybody write tests first? by BrowserUk (Patriarch) on Feb 22, 2008 at 11:31 UTC
And yet I've seen it catch real bugs in real code. Sorry, but you'll have to explain to me how testing that perl can load a module detected a bug, that you would have missed by allowing perl to report: `Can't locate object method "new" via package "Some::Module" (perhaps you forgot to load "Some::Module"?) at...` [download] ? Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l]
Re^6: Does anybody write tests first? by chromatic (Archbishop) on Feb 22, 2008 at 18:57 UTC
Sorry, but you'll have to explain to me how testing that perl can load a module detected a bug... Module loading in Perl 5 is not transactional. If there's a compilation error halfway through the code and the rest of the module fails to compile, all of the symbols defined in the first half and all of the side effects executed as a result of compiling the first half are still in effect. The best way I've found to detect that is to use the return value of `use_ok`: `use_ok( 'Some::Module' ) or exit;` If the module fails to load completely, the test file will stop execution right there.	[reply] [d/l] [select]
Re^7: Does anybody write tests first? by BrowserUk (Patriarch) on Feb 22, 2008 at 21:00 UTC
Re^8: Does anybody write tests first? by chromatic (Archbishop) on Feb 22, 2008 at 21:28 UTC


The stupid question is the question not asked
	PerlMonks