Re^3: Inscrutable test failure

Static data only gives you reproducibility, not correctness. If you use a static timestamp for testing, and manage to get the DST changes out of the way (or in the past and thus at fixed dates), you will get consistent errors or successes in your test suite. This does not buy you correctness in the live program as the timestamps of the future and conditions on the production machines might well differ.

Obviously, your client doesn't have a firm grip on the identical setup of test and production machines either as the (supposed) differences in timezones between test and production environment show, but maybe you still can get a VMWare image (or Xen, or Parallels or whatever the VM container du jour is) to run your tests in an environment as close to production as possible before approaching your client for installation.

Comment on Re^3: Inscrutable test failure

Replies are listed 'Best First'.
Re^4: Inscrutable test failure by dsheroh (Monsignor) on Oct 26, 2007 at 21:59 UTC
I think I'm still missing some aspect of your suggestion to use a static timestamp in the tests. If I had it test `is_weekday(1193435566)` (that being the current time as I type this) and the test object was configured such that Friday is a weekday and Saturday is a weekend, then it would respond by saying that the static time is a weekday when run in the US and (correctly, but not consistently) that it is not a weekday when run in Australia.	[reply] [d/l]
Re^5: Inscrutable test failure by Corion (Patriarch) on Oct 26, 2007 at 22:06 UTC
My point was that your test results will always remain reproducible when you use fixed timestamps. But your explanation also showed me another aspect that I hadn't fully grasped yet - using timestamps in GMT will always need careful adaption to the local timezone because the (calendar) date will be different between the two timezones.	[reply]
Re^6: Inscrutable test failure by dsheroh (Monsignor) on Oct 26, 2007 at 22:23 UTC
Yeah... That (calendar) date issue is why I introduced Date::Manip and dynamically-determined dates to the tests in the first place. I'd really like to get rid of it without either introducing dependencies on which timezone the test is run in or reproducing the `is_weekday` logic in the test (which seems like it would defeat the test's purpose, since it would also reproduce any bugs in the original), but I have no idea how to do so.	[reply] [d/l]