Whe you say "I'm installing modules that I know on an OS that I have used them on before, so I don't consider it risky.", do you mean the SAME MACHINE that the tests have passed, or the fact that the tests passed on one machine, so install on the other with the same OS should be ok?

I understand the argument, and agree with it in situations like developer tests vs. install tests, or where tests don't ship with the modules. But surely this is the exception rather than the rule?

The majority of CPAN modules don't fall into this category and failing 'make test's almost always mean that the modules aren't going to work as expected and really should just as well be considered build failures.

For those who didn't want to always run the tests during 'make' or 'Build' there would always be the command line option to not run tests.

I'm sure I'm in the minority, but I always get mixed signals from the people more important than I in the community about it. Everyone likes to say that having makeing and using tests is of the utmost importance; and everyone like to jump on on the "did you actually run make test" bandwagon when someone reports a module malfunction, yet running the tests is still an optional proposition for the person installing the module. I've always thoug (in the case of most CPAN modules that ship the tests as part of the dist) that a test failure should constitute the same as a build/make failure; and as equals; tie them together.

As I said, I have tests for my own application that exercise the parts of the modules they use that I care about.

The majority of CPAN modules don't fall into this category and failing 'make test's almost always mean that the modules aren't going to work as expected and really should just as well be considered build failures.

My experience has been just the opposite -- failing tests on CPAN modules that I've installed and used successfully before hardly ever indicate anything useful. They usually indicate a non-portable assumption in the tests themselves. I run the tests when I'm trying new modules, or when something doesn't work for me, but I don't run them to verify that a configuration I've used before still works.

(You know you're talking to the guy who built EToys practically by himself, right?)

The point is that if you have a homogenous system (all from the same manufacturer, all the same model, all built off the same install disk), you don't have to run the tests. And, if you don't have a homogenous system, what on earth are you doing? You can't write good code for a system you aren't testing on.

failing 'make test's almost always mean that the modules aren't going to work as expected and really should just as well be considered build failures.

When I was running MP2, I had a test failure nearly every time I installed it. I never used those features. Does this mean that MP2 didn't work for me? The websites I ran on it would bely that assessment ...

---

• In general, if you think something isn't in Perl, try it out, because it usually is. :-)
• "What is the sound of Perl? Is it not the sound of a wall that people have stopped banging their heads against?"

I appreciate the vote of confidence, but I don't want to take credit for building eToys by myself. We had a pretty large (20 or so) team of coders working on the site, including some of the most talented people I've had the pleasure of working with.

The point is that if you have a homogenous system (all from the same manufacturer, all the same model, all built off the same install disk), you don't have to run the tests.

Does that same install disk include the EXACT copy of perl with all installed modules as production. or was perl and the other modules installed manually on each server?

I understand the theory that they "should" be the same, but that's usually not the real world. Shit happens. I'd much rather find out something is broken, missing, out of place, borked on an install while running tests than when it's in production and I didn't run the tests.

And like I said, I'm not a "always run make test" saint. But assuming things are kosher just because it works on the three other same-OS machines installed form the same disc seems like an accident waiting to happen. THe whole point of tests is to point out problems before they effect production. Sometimes those are tests in the dist. Sometimes they're not. But to each their own.

Many modules on CPAN have inappropriate tests for many enviornments such as asking questions or connecting to remote hosts. An automated enviornment would need additional knowledge for each module to handle the test questions, behind "make; make install".

Of course, that gets me going into the route of remote connection tests shouldn't be part of the default test suite, and that is a battle left for another day.