Another of Becky Fiedler’s videos from CAST 2009 in CO Springs. When “Feature Creep” was over, I had enough time in my Lightning Talk slot for an encore. Jim Bach wanted to hear “Rapid Tester”, so I played it.

Becky Fiedler shot this performance of “Black Box” after dinner was all over and we were just hanging out in the hallway. It pretty quickly turned into a Jonathan Coulton love-fest after the testing songs were played…

I performed this a month or so ago at CAST2009 in Colorado Springs, where Becky Fiedler recorded it. The intro to this song went, “This song is written from the perspective of a piece of bloatware that used to be sleek and clean.” I did this during a Lightning Talks session, meaning I had to bring it in under 4 minutes. That’s why there’s little time for dramatic pauses…

Jim Nilius and I will be taking our act on the road again, this time to CAST2009. Our discussion at WREST 2 in Indianapolis on why regulated testing does a poor job of testing software produced a lot of great ideas for how to clarify the problems with using 19th-Century test methods to evaluate computerized voting systems.

Jim and I are preparing a strong argument for the Election Assistance Commission to overhaul the NIST certification standards to help certification labs handle testing complex software systems much more congruently, and we are taking this argument for a test drive at CAST. So come on over and test it, test us, rip it to shreds. We will rebuild it. We have the technology, we have the capability to make it better than it was before. Better, stronger, faster…

Displaying sound judgment in a complex, dynamic environment is a hallmark of wisdom.

-From the Wikipedia entry for “Sapience”

At WREST 2 in Indianapolis on May 14, I interviewed Jim Nilius, former Director of Testing at Systest Labs, a very tightly regulated shop in Denver CO that tests electronic voting systems. We discussed the applicability of the theme of the workshop, “Beyond Scripted Testing”, to the line of work his lab did. In his view, there is no space for unscripted testing in a regulated environment, simply because unless the tests are scripted they cannot be reviewed prior to being run, and if they cannot be reviewed then you cannot maintain your accreditation and your company will cease to exist.

I am not familiar with this line of thinking, but it got a lot of nods from around the table so it is probably a true dynamic. Which seems pretty scary and counterproductive, because it entirely stifles innovation and sapience in testing and drives it underground into what Jonathan Kohl refers to as a “shadow process“. I drew a system diagram that I believe represents the general state of a regulated testing shop.

(Here is the PDF. Contact me via gmail for the original ODP, editable in OpenOffice, if you wish to modify or upgrade this document under the publishing rules of WREST.)