Different behavior running tests under prove vs. MB/EUMM

dragonchild has asked for the wisdom of the Perl Monks concerning the following question:

BrowserUk and xdg have been helping me debug a problem with the tests for DBM::Deep on Win32. In working on it, they both noticed that running tests under prove showed a different set of broken behavior than running the tests under MB. Specifically, the number of open filehandles at the end of the test was different.

The problem seems to be getting solved (opening an already open filehandle in Win32 is bad - who knew?!), but the different behavior bothers me. I use prove in my vim macros because it works independently of MB, EUMM, MI, or a lack of anyone of them. If it doesn't behave the same, is it testing the same thing?

My criteria for good software:

Does it work?
Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

Comment on Different behavior running tests under prove vs. MB/EUMM

Replies are listed 'Best First'.
Re: Different behavior running tests under prove vs. MB/EUMM by BrowserUk (Patriarch) on Nov 03, 2007 at 14:30 UTC
I made a discovery. Try adding `-b` to prove. It adds the blib directories to @INC. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l]
Re^2: Different behavior running tests under prove vs. MB/EUMM by dragonchild (Archbishop) on Nov 03, 2007 at 14:34 UTC
I prefer using -l instead. That adds the lib directories. My standard is "prove -lv t/(file).t". My criteria for good software: Does it work? Can someone else come in, make a change, and be reasonably certain no bugs were introduced?	[reply]
Re^3: Different behavior running tests under prove vs. MB/EUMM by grinder (Bishop) on Nov 03, 2007 at 18:28 UTC
I have no doubt that you understand why `-l` works for you, but just to provide a counterpoint, I don't use `-l` for my own distributions, as I like putting the main .pm file in the base directory of the unpacked distribution, for the simple matter that it makes it a tad easier to pull into an editor. There are also distributions that munge the .pm significantly on its flight from `./lib` (or `.`) to `./blib`, or even generate it ex nihilo. In these circumstances, `-l` doesn't produce the same results as `-b`. That said, I understand why you like `-l`. It means you can edit the source .pm file, and run the `prove` test immediately, without having to run `make` (or `./Build`) in between. There's nothing worse than making a change to the module, and then spending considerable time wondering why it didn't fix the test suite :) • another intruder with the mooring in the heart of the Perl	[reply]
Re^3: Different behavior running tests under prove vs. MB/EUMM by BrowserUk (Patriarch) on Nov 03, 2007 at 14:49 UTC
Doesn't that mean you have to do a make install before you can run the tests? Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^4: Different behavior running tests under prove vs. MB/EUMM by dragonchild (Archbishop) on Nov 03, 2007 at 14:50 UTC