As my interest and skills at automated testing in Perl have grown, I've started applying my test-driven development approach to how I diagnose and report bugs in other people's modules. I think a test-driven bug report has several benefits.
First, by phrasing my bug report in the form of a test, I give the original developer a head-start on fixing the bug and confirming the fix. Second, once I'm already working with someone's test files, I'm more confident in exploring possible fixes, as I can quickly try something out and see if it fixes the issue I found without breaking anything else. Whereas previously I might have just reported an issue and waited for a fix, I now find myself taking a quick peek to see if I can actually fix it now.
It's a wonderful feeling to patch a bug you found and watch the rest of the test suite just hum along smoothly. If you're desperate, you can even install that privately fixed version immediately, knowing that it's just as functional as an unpatched version.
Of course, this only works for modules with good coverage and clean style in testing, but the prevalence of good unit tests and tools like Devel::Cover, plus the strong testing culture in the Perl community make it likely that you'll find at least some tests for any major module you encounter. And for module developers, this is a reason to write your tests cleanly and refactor them periodically -- wouldn't you rather receive bug reports in the form of a test file?
Here's a quick checklist for getting started with test-driven bug reporting. It should be within the capabilities of most intermediate Perl programmers, presuming some familiarity with Test::More and other major test modules, as well as distribution build systems like ExtUtils::MakeMaker and Module::Build.
Download and unpack the distribution tarball. Hint: with CPANPLUS, use the cpanp command line tool to download it quickly:
$ cpanp -d --fetchdir=. MODULE
If you use a version control system (e.g. Subversion, SVK, etc.), import the downloaded distribution into a private working area so you can track your changes and easily compare them to the original to produce patch files.
Write a test file or patch a test file to demonstrate the bug -- include any additional details in comments in the test file.
If you feel confident, look through the code to see if you see a possible fix. If you don't know where to start, try stepping through the test file in the debugger and watch how your test executes.
If you find a fix, try it out against the full test suite and see if all the tests pass. If so, great! If not, make a couple more attempts if you can invest the time in it.
Post your tests (and patch, if you wrote one) to RT, to a mailing list for the module, or to the author's CPAN address (or to all three, plus any other support forum that may exist).
Of course, it's not always possible to figure out someone's testing approach or to replicate a complicated bug quickly in the time you're willing to allot to it. However, if you can, I think you'll find it's a much more satisfying contribution than posting a simple complaint. And if you can only send a test and not a patch, I still think you're more likely to see a quicker fix to your bug.
Happy testing,
-xdg
Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.
In reply to The value of test-driven bug reporting by xdg
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |