...A rhetorical question brought about by the situation on my current engagement whereby the mandated test 'tools' for perl scripts & modules include the aforementioned
Test::Unit.
I feel the need to post what follows - being a (nearly un)conciousness stream) to an, if not appreciative, then certainly knowledgeable audience - because any attempt to unburden myself elsewhere i.e. on the current engagement, inevitably results in almost suicidal frustration (due to the lack of knowledge & understanding)...
Within the context of a test team using perl to automate the real-time testing of hardware (& software) products, as I've said earlier, my current client engagement mandates the means of testing of modules (& to some degree, scripts) using the following...
- Test::Unit for white box unit testing
- Black box testing is accomplished by running a separate monolithic script, in which the black box tests for all scripts and modules are defined
Numbered amongst the reasons cited (for the above) are...
- it's always (well, for the past year or so) been done this way
- To introduce something new would be to confuse the script &/or module authors
- Both the production and the unit test code are contained in the same file - thus easing the jobs of both the maintainer & reviewer of a module
- The maintenance of 2, or more, scripts in parallel with the scrip(s)/module(s) under test (as in Test::Builder based testing) wouldn't be likely to happen
In retrospect, this is, AFAICT, the complete list of reasons
Whilst any testing has to be better than none at all, I would and indeed do, contend that this [the above] sucks (and then some!!!), for the following reasons...
- It [Test::Unit] isn't standard (nor for that matter, best) perl practice - Test::Class would appear to be the way to go for white box testing
- It [Test::Unit] doesn't (fully) support TAP
- It [Test::Unit] is legacy, AFAICT no longer maintained, code
- To work properly, it [Test::Unit] inspects the package @ISA to determine the classes from which the class under test inherits - hence it doesn't work, out of the box, with Object::InsideOut - another mandated CPAN library
- Moreover, Object::InsideOut doesn't properly support inheritance of the methods declared in non-OIO classes - specifically the test methods declared as the Test::Unit API
- Object::InsideOut problems aside, it [Test::Unit] doesn't play nicely with other of the mandated CPAN modules e.g. Error & Exception::Class
- Since both test & production code _are_ contained in the same file, the file rapidly becomes (to my mind) unmanageably large (some of the module files are already > 10k lines long before any unit test code is taken into consideration!)
- The error messages generated on unexpected exceptions e.g. compile errors, are frequently absorbed by Test::Unit such that the error is reported in/from/by the Error module c/w the compiler error and full stack trace
- ...confuse the script &/or module authors - given that very few of the scripts/modules actually have unit tests, there's no difference between learning how to use either Test::More or Test::unit
- As for the maintenance of 2, or more, separate files - there are quality standards that can and indeed should, be updated to reflect new & improved working practices. Moreover, is it easier to scroll (sometimes > 9k lines) between a change to a sub/method and the corresponding change to its test method (in one diff viewer), or to have the 2 changes side-by-side in 2 viewers ?? Hmmm, no-brainer !!
Well, thanx for listening/reading - I feel a whole load better now - suffice it to say that, at work, it feels like the quip on my favourite mug is so pertinent to be eerily redolent i.e. I refuse to have a battle of wits with an unarmed person.
A user level that continues to overstate my experience :-))