in reply to Test code that makes filesystem modifications?

Should I knowingly put code in a test suite that intentionally makes changes to the state of a system?
That depends what you mean by changing the state of a system. If it's just a matter of creating (and afterwards deleting) of files, and it's done in the tmp directory, go ahead, that's what the tmp directory is for. But if it's anything non-trivial, like opening ports, or doing something that would require root/admin priviledges, never, ever do something without 1) clearly stating to the user what you are about to do, and 2) explicitly asking the user for permission. And by all means, try to determine whether you are run from an interactive session or not, and if not, don't do the tests.

Abigail

  • Comment on Re: Test code that makes filesystem modifications?

Replies are listed 'Best First'.
Re: Re: Test code that makes filesystem modifications?
by tovod-everett (Sexton) on Mar 04, 2004 at 17:43 UTC
    It is just a matter of creating and deleting files in the tmp directory, although the following negative possibilities exist:

    • Some sort of Ctrl-C event prevents file deletion at the end.
    • Permissions get set to something really ugly on an object, preventing deletion until the permissions are reset to something cleaner. Since the person executing the code is the owner of all objects they create, they retain the right to change permissions on them, but my cleanup code doesn't attempt to reset permissions, especially since it'd sort of be relying on the code it's testing in the first place.
    • I do have to write the test suite to behave differently depending upon whether the user has admin or not (and actually, ideally, the test suite would be run under both situations, which is obviously _not_ something I can fake in the test suite:).
    In any event, I've decided to do this:
    • Release the .t file in some sort of disabled form.
    • Put a message up high in the README asking for people to run the disabled test suite and let me know how it goes
    First off, this side steps my nervousness. Second, the test suite will probably comprise somewhere around 50,000 to 200,000 tests, and may take on the order of an hour to run! I've decided that one of the purposes of an exhaustive test suite is to be very sure there are no latent bugs in my core logic. I've found so many bugs caused by my insufficiently thorough understanding of exactly how Microsoft implements permission inheritance that I've written fairly efficient code to test the biggest conceivable envelopes. As in, here's 40 permissions. Apply every possible combination of them to a directory tree that's three deep and check that my code _never_ reports a wrong or missing inherited permission. 40*40*40 = 64,000. I may scale this back a little:).

    --Toby Ovod-Everett

[reply]

In Section Meditations