but what would go in a test suite?

Anything that is important for your product. Here are a few points you may want to think about. Not all points are relevant in all cases though.

1. Correctness is usually important, and one of the first things that are tested. This could range for simple things like "if I plug X and Y in this function, the result should be Z" (typically the things that are tested when "make test" of a CPAN module is run) to "if I push the red buttons after I disabled the blue switch, while standing on my head humming the national anthem, does it make a decent decaf".
2. System resources. How CPU intensitive is it? How much memory does it use? If your product has always hummed along on a Sparc 5, but a new minor release suddenly requires a 6 processor 3800 than your customers won't be happy, even if the product passes all correctness tests. Besides CPU and memory usage, there's disk usage, I/O usage, shared memory usage, etc, etc to consider.
3. Robustness. What does your program do when something unexpected happens, like a file that suddenly disappears, a network that goes down, a remote server that no longer answers, a disk that's no longer repsonsive? How does the program behave when it receives signals? What does it do when I cut-and-paste a million characters in a small text field? How does it behave on garbage input data? Or on no input data? What happens if the program crashes?
4. User interface. If you have a GUI program with many screens, is the user interface consistent, or does each screen need its own manual? Is the user interface simple? Can I do things that need to be done often with one keystroke or mouse-click, or do I need control-alt-shift-meta-cokebottle all the time? Does your product confirm to the house style? Is each background the right shade of orange, or do you still have screens in green, the house style of 2001. Are all the logo's correct? [1].
5. Regression tests. Adding a feature in one part of the software shouldn't have an effect on another, unrelated, part of the software. You can lose customers that way, because people do get upset if the software suddenly behaves differently. I'd be mighty upset if the next release of vi suddenly gave a different meaning to 'a'.
6. Stress testing. Testing a single instance of a product is one thing, but what happens if two people use the product at the same time? Or 10? Or 1000? How does the product behave if run continuously for 24 hours? Or a week?
7. Security. Can the software be used to comprise security in one way or the other? This not only means gaining unlawful access to the box the machine runs on, but also access to data (for instance, in a database).

[1] Don't laugh. I've worked for a company that was once threatened with legal action because a button displayed the old name of the product - a name that our company no longer had the right to use.

Abigail

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.