Here's a basic list of the discrete tests:
- Basic functionality (Does a single level hash or array work?)
- Embedded functionality (Does a HoH, HoA, AoH, AoA, and deeper work?)
- What about wide hashes and arrays (4000+ keys)?
- What about deep hashes and arrays (4000+ levels)? Does every level behave appropriately?
I then want to cross-reference those tests against:
- What happens if we use filters on the keys? The values?
- What about changing some of the internal key values?
- Does changing the hashing function make a difference?
- Does cloning work?
- What about importing and exporting from standard Perl data structures? tied datastructures?
- How about if I turn locking on and off?
- What about creating the db using tie vs. DBM::Deep->new?
- How about concurrent access?
- What about turning autobless on and off? What about locking? autoflush?
It's almost a N-dimensional matrix of possibilities, and I want to be able to cover as many as possible.
Now, 99.9% of these tests will not be run when the user installs. I plan on picking some representative examples that provide 95%+ code coverage in under 30 seconds on a standard machine and using those for installation tests. That suite will also be the tests that I run on a standard basis before committing changes.
However, I need a suite of tests that I can run overnight (if necessary) that will completely and utterly crush this code and show me that every single edge case I can think of has been covered. Then, when a bug is found, I can demonstrate the the bug has been covered in every one of these scenarios. I need to do this so I can have the level of confidence in DBM::Deep that I have in MySQL or Oracle.
A perfect example is the autobless feature. I found a few bugs in autobless in a few situations, so I fixed them. Recently, however, I found a bug with autobless when it came to exporting. Had I been using this comprehensive test suite, I would have found that bug already (and others like it, I suspect).
My criteria for good software:
- Does it work?
- Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
Read Where should I post X? if you're not absolutely sure you're posting in the right place.
Please read these before you post! —
Posts may use any of the Perl Monks Approved HTML tags:
- a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
| |
For: |
|
Use: |
| & | | & |
| < | | < |
| > | | > |
| [ | | [ |
| ] | | ] |
Link using PerlMonks shortcuts! What shortcuts can I use for linking?
See Writeup Formatting Tips and other pages linked from there for more info.