At CAST2009, I took James Bach’s “Teach Yourself Testing” course. Part of his course was to have the testers play a deduction game with a variety of dice. The testers present a set of dice to the instructor, who responds verbally. The object is to figure out what the instructor’s response is going to be, and write down the algorithm that you deduce is being used to generate the response. The rules are you have to present a set of 5 dice, which you have rolled so their values are random.

I was paired up with another tester. Paul Holland, acting as Jim’s assistant, gave us our pack of dice. There were regular old 6-sided dice with dots, and also 4-, 6-, 12-, and 20-sided dice with Arabic numerals, and 6-sided dice with symbols such as exclamation point, spiral, explosion, skull, flower. One likely glowed in the dark.

My partner and I grabbed a random handful of dice, and rolled them, and asked Paul for his response. “Red blinking zero,” he said. We rolled again. “Red blinking zero,” he said. We rolled another set of dice. “Red blinking zero,” he said. Slowly, I began to detect a pattern…

Jim Nilius and I will be taking our act on the road again, this time to CAST2009. Our discussion at WREST 2 in Indianapolis on why regulated testing does a poor job of testing software produced a lot of great ideas for how to clarify the problems with using 19th-Century test methods to evaluate computerized voting systems.

Jim and I are preparing a strong argument for the Election Assistance Commission to overhaul the NIST certification standards to help certification labs handle testing complex software systems much more congruently, and we are taking this argument for a test drive at CAST. So come on over and test it, test us, rip it to shreds. We will rebuild it. We have the technology, we have the capability to make it better than it was before. Better, stronger, faster…