in reply to Testing: To Mock or Not to Mock?

I come down on both sides of the question. I mock-out a lot of stuff when building my unit-tests. The purpose of my unit tests is to verify the *my* code is doing the right thing, right? I can just take the documented results of those External (to my code) Bits and replace them with stubs, right? Sort out my bugs and get on with it.

But then I go back and add an integration test(or tests) that set up the appropriate environment for the External Bits and validate that the combination of my code and the External dependencies all work as advertised. Both sets of tests have to pass before I have a 'Clean Run'(tm) of my test-harness.

In the process of setting up the integration test, I may find that External Bit phoo has a dependency on another External Bit baht, so I mock that dependency and roll forward writing my test.

Once I have a clean unit+integration test, my level 1 test harness; I build build a level 2 integration test, replacing the mocked baht dependency with baht itself and write the integration tests for that. Rinse and repeat, until I have worked myself up to the top of the stack.

Yes it is a LOT of work. But, the first time that your test-harness fails because someone made a 'minor change' change four levels up the stack -- it's worth every hard fought second. And that's the real utility of a Test-Harness. When Someone makes a change, I get early warning of any issues before the Code reaches the User. Testing in depth and nightly cycles are the way to go.

----
I Go Back to Sleep, Now.

OGB

Replies are listed 'Best First'.
Re^2: Testing: To Mock or Not to Mock?
by agianni (Hermit) on Nov 09, 2007 at 14:17 UTC

    I agree that exhaustive testing along those lines would be ideal, but it's not usually a reasonable option. My current project includes over 40k lines of Perl code and nearly that many lines of tests and I can't imagine how much more time it would take to implement full-stack integration testing.

    I'm curious, though, about the value of integration testing to full depth. For example, if I have some code where sub A calls sub B, which in turn calls sub C, why do I need to test the full integration of these? Rather, why can't I just test the integration between A and B and the integration between B and C and suggest that I have testing the integration between A and C transitively? Might there really be situations where that would not suffice?

    That said, backing up your mocking at least one level for the sake of integration does make a lot of sense, and I hadn't thought of it in such an specific manner before. Thanks for your input! 

    perl -e 'split//,q{john hurl, pest caretaker}and(map{print @_[$_]}(joi
+n(q{},map{sprintf(qq{%010u},$_)}(2**2*307*4993,5*101*641*5261,7*59*79
+*36997,13*17*71*45131,3**2*67*89*167*181))=~/\d{2}/g));'

    [download]
[reply]
[d/l]

In Section Seekers of Perl Wisdom