Re^2: Order of tests

I suppose the best approach is a blended one. Run the full suite at intervals but not after every tweak of the new code. The time saved by not running tests which more than likely are going to pass anyway is a real savings. You just need to balance that against how much coding and testing time (possibly the entire time between now and when you last did the full test!) you'll lose if you fail to detect a break in the older code.

But God demonstrates His own love toward us, in that while we were yet sinners, Christ died for us. Romans 5:8 (NASB)

Comment on Re^2: Order of tests

Replies are listed 'Best First'.
Re^3: Order of tests by stevieb (Canon) on Mar 11, 2016 at 19:46 UTC
This is why it's important to separate your tests into separate files for different areas of functionality. You can run a single test file to ensure you haven't broken the functionality immediately surrounding the code you're editing, but you don't need to run the full suite every time, and periodically run the whole test suite. That said, if you only run your entire suite periodically, the time spent figuring out which update broke a remote part of your project might be more costly than just running the whole suite on each change. Typically, I run my full suite on each change. On my bigger projects with dozens of test files with thousands of tests, I at minimum run the suite on each commit prior to pushing, then on each push, Travis-CI runs the gamut on 4-8 versions as well.	[reply]

Replies are listed 'Best First'.

Re^3: Order of tests
by stevieb (Canon) on Mar 11, 2016 at 19:46 UTC

This is why it's important to separate your tests into separate files for different areas of functionality. You can run a single test file to ensure you haven't broken the functionality immediately surrounding the code you're editing, but you don't need to run the full suite every time, and periodically run the whole test suite.

That said, if you only run your entire suite periodically, the time spent figuring out which update broke a remote part of your project might be more costly than just running the whole suite on each change.

Typically, I run my full suite on each change. On my bigger projects with dozens of test files with thousands of tests, I at minimum run the suite on each commit prior to pushing, then on each push, Travis-CI runs the gamut on 4-8 versions as well.

[reply]