in reply to End to End testing
in thread Trojan Perl Distributions

For a full test you really need some End to End testing. That is a general rule....
That is not really true. You should be testing interfaces. If each interface receives and transmits data as you expect, you need not do full End to End testing. It's the kind of thing I would only do in development. The DBI/DBD modules are tested very well, so all you need is to ensure the SQL query you send is as expected, and the test data returned is processed correctly. Both of these can be handled very well with mocking.

As for coding for different databases, then this should be part of your development testing, and not part of the install testing. If you are writing to enable the use of different databases, by all means use DBI to see if the appropriate drivers (and versions) are loaded. However, you shouldn't need to test whether the database handles a specific syntax at the install stage.

Regards transactions, this is fine if the database you are testing can handle transactions. Unfortunately there are several people who are running non-transaction enabled databases, such as MySQL (although I believe version 4 has can handle them), and authors who are coding their modules specifically for them.

--
Barbie | Birmingham Perl Mongers | http://birmingham.pm.org/

Replies are listed 'Best First'.
Re: Re: End to End testing
by zby (Vicar) on May 07, 2004 at 07:32 UTC
    I don't understand why I should test only the interfaces. Do you argument that it is only interfaces that can fail when installing software tested in one environment in another unknown environment?
[reply]
      There are lots of things that can fail. The network, your network card, the database daemon among many other things. You shouldn't be testing those so why test whether DBI can do them? If your interfaces are correct, then the fault will lie elsewhere and is down to the user to fix. If the database daemon isn't running for whatever reason, why should that mean you have a failed install? If you are correctly testing that the functionality of your modules is correct, then you shouldn't have to worry about whether DBI can talk to a live database.

      If you ever get involved with designing a large system, then testing the interfaces is all you can do. The GEC Telecommunications project for designing a System X telephone exchange, took 3 days to do an end to end run. If we had ever had to install it elsewhere, doing an install test like that would been laughed at. As such we tested what went in was processed correctly and what came out met the agreed expected output, both for garbage and accurate data. It meant we could run the regressions tests on small parts of the system that could be slotted in when bugs were fixed, and we knew it would work within the whole system, because the interfaces were correct.

      --
      Barbie | Birmingham Perl Mongers | http://birmingham.pm.org/

[reply]
        I agree that the installation testing should be more timely than a full development testing. I just want to suggest that End to End testing is the most effective way of testing - so I would do some End to End rather then interface testing during installation because this can catch errors faster than the other way.

        If I install some software and connect it to a database that that is not compatible with that software I'd like to be told about that during the installation proces. Otherwise I would use that software and possibly not even notice that it fails in some subtle way.

[reply]

In Section Meditations