Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

Re: Test Technique: Self-removing test data

by perrin (Chancellor)
on Jul 18, 2007 at 03:51 UTC ( #627184=note: print w/replies, xml ) Need Help??

in reply to Test Technique: Self-removing test data

We did something like this on my last project, and got pretty far with it. Eventually though, I started to wonder why we were bothering. For one thing, it's still not really safe to run it on a production database. What if a test dies? Mess.

In the end, there were enough special exceptions and additional things to clear that it probably would have worked better to just wipe the whole database before each test script.

  • Comment on Re: Test Technique: Self-removing test data

Replies are listed 'Best First'.
re: test clean up vs. test database
by markjugg (Curate) on Jul 18, 2007 at 13:25 UTC
    That's the other major model to use: a dedicated test database. I recall that's what Ruby on Rails sets up default. I have to confess to not trying that approach in earnest because of a perception that it would be too difficult to get started with and maintain as the "fixture" data and data model change over time.

    I agree that it would certainly have the benefit of allowing you to (more) safely exercise the production code-line.

    Are they any specific tools you are using to make this approach easier?

      I haven't tried tearing down the database between scripts. I use the method you described. However, I think the only fixture data you would need is whatever you already have to create a new database (in my case, a script that drops and recreates the tables, and fills in lookup values). The test-specific data is added by the test in either approach.

      Even with an END block, it's possible for your test to die in a way where it won't be able to effectively remove the data you added. That's what makes this approach unsafe for use on a production database.

      It's still faster than dropping the database and recreating it, but as things got more complex, I spent a lot of time troubleshooting problems with deleting the test data, and I think it would have been wiser ultimately to trade a little test speed for the saved debugging time.

      Another wrinkle is web testing with Mechanize on code that creates data. If your web tests cause data to be added to the database, it won't be in your stack to delete, so you end up with manual deletes, and END blocks that complain and crash if the script dies before the data was added. It gets messy.

        We tear down and recreate the test database between different test programs. On the current test setup we're using, here's the timing information for our test suite:

            Files=38, Tests=1811, 106 wallclock secs (49.47 cusr +  8.39 csys = 57.86 CPU)

        While that's certainly not as fast as some folks would like it, we tend to run individual test programs and when we're done with a particular feature, bug fix, whatever, we run the entire test suite. Previously we used other methods (rollback, mock database handles, manually deleting information, etc.), between test programs, but we ran into so many weird edge cases that going ahead and dropping and recreating the test database was far and away a huge win for us.

        Heck, even if a test died catastrophically halfway through and the test database didn't get dropped, the next test program still issues a drop command before trying to create a database, thus ensuring that we never have to worry about data spilling over between tests. It's also trivial to temporarily disable the drop on destroy command just to see if any test scripts aren't cleaning up properly, as that can often hint at bugs in code.

        Update: I should also mention that Test::Class can also speed things up quite a bit in many area. With Test::Class, if you have 30 test classes, you're still only loading the Perl interpreter once, related modules are usually only loaded once, and many test suites can speed up quite a bit if you have slow loading code. Of course, the test organization and inherited tests also great. Test::Class still needs some work, but it's pretty damned good.


        New address of my CGI Course.

      We work on a test database, on which we apply the same upgrade script as the one being offerred to clients. So maintaining the database is part of the tests too. Apart from that, I think the two methods are more complementary than supplemantary in nature, i.e. even on a test database, it is a good idea to distinguish between data which is part of the environment and data introduced by the tests and to clean up the latter, for which the method under discussion is a good idea.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://627184]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (7)
As of 2023-05-29 15:05 GMT
Find Nodes?
    Voting Booth?

    No recent polls found