in reply to Trojan Perl Distributions

I don't think a mock object will do. For a full test you really need some End to End testing. That is a general rule, going to the more specific case of databases, you can never garantee that the generated SQL will work with a particular database engine, there are many subtle differences in syntax and semantics of SQL implemented on different databases, there are bugs specific to each platform that you need to code around.

Said that I really believe we need standards as to how test scripts should interact with the environment. In my current module I create the tables and do the tests in one transaction and roll it back at the end of the script, for databases with transaction this seems to be a good solution.

Replies are listed 'Best First'.
Re: End to End testing
by barbie (Deacon) on May 06, 2004 at 11:40 UTC
    For a full test you really need some End to End testing. That is a general rule....
    That is not really true. You should be testing interfaces. If each interface receives and transmits data as you expect, you need not do full End to End testing. It's the kind of thing I would only do in development. The DBI/DBD modules are tested very well, so all you need is to ensure the SQL query you send is as expected, and the test data returned is processed correctly. Both of these can be handled very well with mocking.

    As for coding for different databases, then this should be part of your development testing, and not part of the install testing. If you are writing to enable the use of different databases, by all means use DBI to see if the appropriate drivers (and versions) are loaded. However, you shouldn't need to test whether the database handles a specific syntax at the install stage.

    Regards transactions, this is fine if the database you are testing can handle transactions. Unfortunately there are several people who are running non-transaction enabled databases, such as MySQL (although I believe version 4 has can handle them), and authors who are coding their modules specifically for them.

    --
    Barbie | Birmingham Perl Mongers | http://birmingham.pm.org/

[reply]
      I don't understand why I should test only the interfaces. Do you argument that it is only interfaces that can fail when installing software tested in one environment in another unknown environment?
[reply]
        There are lots of things that can fail. The network, your network card, the database daemon among many other things. You shouldn't be testing those so why test whether DBI can do them? If your interfaces are correct, then the fault will lie elsewhere and is down to the user to fix. If the database daemon isn't running for whatever reason, why should that mean you have a failed install? If you are correctly testing that the functionality of your modules is correct, then you shouldn't have to worry about whether DBI can talk to a live database.

        If you ever get involved with designing a large system, then testing the interfaces is all you can do. The GEC Telecommunications project for designing a System X telephone exchange, took 3 days to do an end to end run. If we had ever had to install it elsewhere, doing an install test like that would been laughed at. As such we tested what went in was processed correctly and what came out met the agreed expected output, both for garbage and accurate data. It meant we could run the regressions tests on small parts of the system that could be slotted in when bugs were fixed, and we knew it would work within the whole system, because the interfaces were correct.

        --
        Barbie | Birmingham Perl Mongers | http://birmingham.pm.org/

[reply]

In Section Meditations