I work for a catalog/web retailer. Each year we produce many catalog mailings. Each mailing consists of many different versions of a catalog (testing different offers, different covers, customer segments, etc.), possibly up to 100 different versions.

I developed a perl system to simplify and automate much of the design process. The user can specify an arbitrary number of required components of a catalog (eg each book must have a cover, an order form, signatures 1 thru 8, etc), the different versions of each component (eg the fall catalog cover component has three versions: red, blue, or new customer), the different catalog types (eg the new customer catalog version gets the new customer cover, the discount card, an order form, the sigs, etc), and tests we're planning on running (eg test sending the new customer catalog to a random 100K recent prospects vs. sending the standard catalog to a random 100K of recent prospects).

The system reports if the tests are valid (which elements differ across each test), the minimum detectable statistically signif response lift (based on the sample sizes), production quantities for all production elements (eg across all versions, how many red covers do we need), input parameters for our test tracking system, input parameters for our operational systems, production instructions that get fed into the printer's production systems to drive a demographic bindery line.

Took about a week to build in Perl. Documentation via POD. Heavy use of OO. Nothing pretty, no GUI or web frontend -- just a command line script with three input files that writes multiple output files and a log.

Removes horrendous drudgery, removes chances for costly errors, and lends itself to better marketing as designing tests is simplified.

The code was the easy part; understanding and abstracting and reengineering the process took the hard thought.