in reply to (OT) Help with test design

One of the methods is an implementation of the minimax algorithm. How would I go about testing this?

I'm a bit surprised by this question. My first reaction would "test it like you always test", but apparently, that isn't working for you. Which makes me wonder, how *do* you test? I test by giving the functionality I want to test input of which I know the output should be, and whether what I get is what I expect. Why wouldn't that work for a minimax algorithm? That algorithm doesn't depend on the phases of the moon, does it?

It's the same for the XS code. Test it the same way as you test Perl code. For the user it doesn't matter whether the implementation is pure Perl or XS, so why should it matter for the tests?

Abigail

Replies are listed 'Best First'.
Re: Re: (OT) Help with test design
by dragonchild (Archbishop) on Sep 11, 2003 at 14:38 UTC
    The issue I'm grappling with is figuring out input and output for a function that depends on nearly every capability of every other class in the system. Now, that may be a design concern that's coming out in testing. If that's the case, then I need to redesign, and that's ok. But, I've never written tests for a function as ... far-reaching as my implementation of minimax seems to be. My largest concern is that I don't know if I'll have enough coverage in the tests. But, maybe I should start writing tests and see what happens. I just wanted to design at least one portion of the system before writing it. *sighs*

    The XS portion wasn't a matter of how to test it, but that I want to write in XS, so I need the tests. :-)

    ------
    We are the carpenters and bricklayers of the Information Age.

    The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6

    Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

[reply]

      The way to test an abstract class is to test one or more simple concrete examples of that class. For Game::Die you would test a few easy types of die: 1d6, 1d10, 2d20, etc. For Game::AI, you may have to make a tiny example game that uses all the features of an AI. It doesn't have to be a fun game, or a real game. Find a Tic Tac Toe game and implement that AI.

      I don't know what the allure for XS would be in a minimax framework. It's so thin, and won't be the real work of any minimaxable game. Build a tree of foreseen board positions, using one AI per player. One method on AI can be coded to simply return a list of all legal possible moves, instead of rushing to commit the actual board to a single move. Then for any given board position, you have to rank the "value" of that position or the move that achieved it. The only real "work" involved would be to weed the tree of disastrous courses of action before you spent a ton of time asking the AI to think farther along those lines.

      --
      [ e d @ h a l l e y . c c ]

[reply]
      i think the problem you describe is a significant one, encountered less by modules which are less ambitious, algorithmically speaking:

      it is really to do with the fact that your problem is dependent on a very large number of system states.

      yes, you can test a few of these states for validity, whether the game plays well, but the combinatoric explosion results in test coverage that is less than statistically adequate. this has important implications for unit testing, for this type of problem.

      see http://satisfice.com/testmethod.shtml for a perl implementation of one approach to solving the problem of testing for systems with large numbers of states - tho i believe your problem complexity bigger than those *it* tests.

      at the site above, two different types of testing, exploratory versus scripted, are distinguished. i believe you need exploratory, tho most of our harnesses are geared towards 'scripted'.

      perhaps halleys suggestion of forcing play for a small game works best, tho to do this with the range of games you have is a significant overhead.

      best of luck, anyway.

      ...wufnik

      -- in the world of the mules there are no rules --
[reply]

In Section Seekers of Perl Wisdom