in reply to Philosophical question about testing

As said previously, you're a great person for wanting to write tests.

If you find you're rewriting your code to test your code, you haven't stepped far enough outside the mindset of your program yet. Your tests want to make sure all the parts work, so you need to figure out how to make each part work in isolation.

Let's say you have a main program that calls read_input(), calculate(), and write_output(). What you need to test is does each function do what it is supposed to? and if the functions all do what they are supposed to, does the main program work?

For instance, let's take write_output(). It should take some existing data and write that out to a file. To make this easy to test, you'll need two things: a way to provide data to the function, and a set of example output files to check the function's output against.

Let's assume that you pass a data structure and a file handle in to write_output() (this arrangement makes it particularly easy to test, since we have control over where the input comes from and where the output goes - writing code to be easy to test is never a bad idea!).

Now start enumerating the possibilities - what output should I get if I pass undef? a null string? an empty array/hash? a data structure containing one good entry? (use Data::Dumper to get the data structures to pass in, and use File::Temp to capture your output and Test::Differences to validate what you got vs. what you want).

You can test the input function and the calculate function in much the same way: given these inputs, what outputs do I get? Test::Deep and is_deeply from Test::More are both useful (Test::Deep is more detailed) to compare two data structures.

That question - "given this input, do I get the expected output?" is the key question to ask when writing a test. It sometimes takes a certain amount of thought to figure out how to set up the inputs and outputs - for example, when testing database functions, it's sometimes necessary to load dummy data into your database and test with that (say you want to check your error handling of bad or missing data - your code may not allow you to create the bad data, but it should be able to handle things if the data is bad for some reason.)

To test your main program, you need to eliminate complications from the functions and just see if the main program does what it should. If the program's been written as a standard script, (i.e., code flows from top to bottom, and all functions are defined in place), you'll only be able to feed it inputs and look at its output to see if it works - it's technically possible to "mock out" the functions (that is, replace them with dummy versions that do a straight mapping from "I got this" to "I return that", eliminating the possibility of logic problems in the functions as contributions to problems - and since you've got inputs and outputs already from the function tests, this should be lots easier to do!), but if you've isolated the subroutines in a separate package both testing and mocking them becomes way easier.

This is also one of the reasons it's simpler to write tests first, because you don't find yourself in the position that you have code that is hard to test, because you've designed in ease of testing from the start.

  • Comment on Re: Philosophical question about testing