Software Tasting

OK, so you have relaxed, you have found your feet, you stand straight up. How do you move? You can learn a lot about how you move by watching other people move and walk around. Notice where their movement originate in their bodies, and how the impulse flows through them. Some people lead with their head, like they are on stilts and constantly trying to catch up with their noggin. Some people lead with their shoulders, hunched up, or defensive. Some people lead with their knees and shins, banging into things. Some lead with their chest, puffed up and top heavy. T’ai chi practice is to initiate all movement in the center of gravity, called the t’an tien. The masters were not fooling around when they said “all movement” - all physical movement, breath, emotion, thought, everything was moderated by the t’an tien. In fact all sensory intake is channeled to the t’an tien, where it is processed into life energy and initiates our reactions.

The Rapid Tester’s t’an tien is her mission, the goals of the organization for which she is testing. The mission is at the center of the tester’s decision-making process, and everything that she learns is evaluated in light of the mission, and her responses are chosen to further the mission.

You have seen testers who are focussed on everything else. The toolchasers who automate everything regardless of utility towards the mission, are like the guys who walk around with their energy in their heads, lurching after everything that catches their eye. The ones who do just enough to get by and are afraid of anyone scrutinizing their work, keeping their shoulders hunched around their ears. The blunderer, who makes a lot of noise about testing but somehow always forgets to put the “where” clause in the “delete from” statement, and wipes out the test database again. Or the puffed-up process jockey who is quick to ask for the repeatable test script that you used to find the bug, and to knock you if you can’t comply.

I’ve been all of those testers, and many more. But I forgive myself, and remember my mission focus, and return to being an effective Rapid Tester. My first t’ai chi teacher, Dan Ogrydziak, constantly exhorted us, “Focus in the t’an tien at all times. There is nothing you can do that would be better if you were not focussed in the t’an tien. If you forget, that’s OK. When you remember, then relax and focus again in the t’an tien. The important thing is not to remember, but to remember to remember.”

Remember to remember the mission of testing (the t’an tien), let go of all preconceived notions of what a tester has to be doing (relax), and do whatever you think needs to be done right now.

Taoist philosophy links heaven above with earth below via the upright human. Since tai chi is the embodiment of Taoism (at least in part - Professor cheng Man-Ching said that his tai chi was 30% Lao Tzu (Taoist) and 70% Confucius (Confucianism)), the principle is interpreted very literally. The spine is said to link heaven and earth, so one carries oneself as though the top of the head were suspended from above on a string, and the sacrum was attached to a thousand-pound weight. The spine itself has separation between the joints, is not kinked or bent sideways (though the natural front to back curves remain), and offers the most space in the torso for the organs to function. The image is a string of pearls suspended from above. It maintains its shape as it moves through space, the pearls do not lean on one another, and it can move freely in any direction according to any force applied to it, and returns to the straight form after the force is gone.

When we are physically straight, our limbs are free to move in any direction and manner, so we can respond in any way necessary. Our lungs have maximum capacity for best functioning, and all our organs have space to move around and get a healthful little massage as you go through the day. When I lean over with hands in pockets, I may look cool (or not!), but I have a lot of work to do to get back into position to respond to my environment. And I have lost my connection to the earth - see the post on single-weightedness.

In testing, straightness comes into play when we enter a test open to any possibility of system behavior. If I have the feeling of straightness, I can deploy resources as soon as they are needed, whether it is tools or data or asking help from my professional contacts. If I test with purpose, committing the full weight of my intellect to the test, I am straight, not leaning over and nonchalalantly testing with my psyche half-engaged.

With straightness, I can link the micro activities of testing with the macro activities of model building and contextualization: by maintaining my connection to the ground/yin (my mission and models) and heaven/yang (the details of day to day testing). This is embodied physically by using the lower body to connect to the ground and control your own body’s position in space, while using the hands and arms to connect to your partner in practice, listening to their balance and transmitting the force of your movements. They can’t work together without straightness.

The notion of straightness has a lot of connotations in English, one of which is “integrity”. Great testing requires integrity of thought and action, and also integrity of reporting. Practicing physical straightness helps me get the feeling of mental and ethical straighness as well so I can tell when I’m not really flexible, really engaged, really forthright, and stop leaning over, take my hands out of my pockets, look the app in the eye, and engage.

In tai chi, as a martial art and as a healing art, the source of power is the ground. Our connection to the ground through our feet is the foundation of everything that we do, so severing that connection is (almost) always a bad thing. That is why there are so few jumps in the tai chi form, and why when we practice together the goal is to cause your partner’s feet to leave the ground. The principle of yin and yang says that one foot must be the dominant connection to the ground. There is no such thing as equal connection to the earth through both feet simultaneously. That’s as sensible as saying the moon has two dark sides or two lighted sides. Equal weight means no connection at all, and this is very dangerous, so we strive to maintain single-weightedness, and when shifting weight, to minimize the time we spend at or near 50 / 50 weight distribution.

So the foot that has 51% through 100% of the load of the body is the weighted foot, and it is the foot on which we focus our attention to connect with the power of the ground. The other foot is free to do other things - step, kick, sweep, whatever - but it is empty, and should not distract us from the central concern which is to connect to the earth through the weighted foot.

In testing we very often need to tap into the power of the ground, the power of thoroughly knowing the territory we are in, so our tests are grounded in valid, factually correct system models, exploring a particular coverage area, with oracles that we understand - or if we don’t understand them, we understand the schema of them and can identify the holes we are trying to fill. We need to work to solve one problem at a time, which leads us to control the variables in our tests and vary them systematically. We need to have a leg to stand on, which in testing terms means we need to have a reason for testing where we are, a risk model and theory of error that locates our testing in high-value space.

Most of all, though, what I bring to testing from training in single-weightedness is the feeling of committing to the problem. All of my attention and effort is going into understanding this exact spot in the app, and grounding myself in the problem space so that when I have to tell the quality story, it comes all the way up from the ground, not just from the top of my head.

In tai chi we are taught to “step empty”, to place the foot on the floor without committing any weight or energy to that foot. Only when the foot is flat do we shift our wieght. This way we remain single-weighted (another principle for another post) and balanced, and keep our options open about what to do with that empty foot, either shift into it, leave it empty, or move it without disrupting the rest of our body.

To illustrate: Which two animals habitually step empty? Some common guesses are cats and tigers, because they have soft footfalls. This is not correct. Empty is not the same as soft: cats tread softly, but they also jump a lot, which obviously means you will land with weight. Tigers have very heavy tread, though they appear lithe. Other guesses were birds (they have to jump up to fly, and they have to land, though perhaps softer than cats), and gazelles (the leaping again). Not dogs, hippos, wildebeest. Here’s the first animal: the full-grown elephant habitually steps empty. Why? With so much mass, its bones cannot withstand a hard fall, so it takes very few chances with its footing. Even though the likelihood is relatively small that it will step on unstable ground, the consequences could be dire, so it moves deliberately. By the way, this is why it’s no great trick for the circus trainer to put her head under an elephant’s foot. No elephant will step on such an unstable surface. (Notice I said “full-grown elephant” - when they are small elephants can run more because their mass is not so great. Even full-grown elephants run on occasion, but not habitually.)

The testing lesson here is that there are times when the project or the testing has the characteristics of an elephant, and the risk then is that any fall will really hurt. When failure is expensive, even when unlikely, step empty. Make sure you have a recovery plan in place in case the system crashes, so you can retain data, system state, and configuration. Make small controlled changes so that it is easy to trace the variation that caused unexpected behavior. Test a new area lightly at first, then commit more and more effort to it as it bears the weight of scrutiny. Then it will become a new trusted base state and you can step again.

So now we have seen one risk model: a failure, though unlikely, will really hurt, so step empty. The other animal has the variables swapped around. Every step may be onto unstable ground, so that even though it is light and durable, the animal has so many opportunities to fall that it confirms every step. Give up? It’s a pack mule. The reason that mules were used in the Grand Canyon, with its treacherous gravel switchbacks and a mile to fall if you put a foot wrong, was that horses weren’t careful enough and kept getting themselves killed. The peculiar gait of a mule is from lifting one foot at time and putting it down in its new place before shifting any weight, and shifting back and replacing the foot if there is insufficient stability. And the classic pose of the mule resisting moving forward another inch, despite all the tugging and kicking and cursing? It can’t find a trustworthy place to step, so it’s not going anywhere.

In terms of testing, we can be like mules by being prepared to find an alternate route if our first step is blocked, to look for traction in an app, and finally if in our professional judgement we can’t move further, we stop and say “no” to requests to continue down a dangerous or unstable path. When I am testing a flaky app that has lots of failure potential, and I want it NOT to crash, that is when I step empty and set all my variables to normal values, follow the safest path to the destination, and present as little load as possible.

At the time I first took a class with James Bach on Rapid Software Testing (RST), I had been studying t’ai chi chuan for about 7 years. Throughout the class I kept hearing James say the same things my instructors told me: relax, be open to the energy of what’s around, listen and adapt, and keep your balance. Of course, he didn’t use those exact words. What he said was, “To do Rapid Testing, you don’t need permission. It is completely within your own control.” Keep your balance. “Rapid Testing is like team sports: quickly establishing the proper relationships with your team, handling the ball properly when it comes to you, passing it off to the right person at the right time, staying aware of the time and the score, and furthering the goals of the team.” Adapt to your situation. “Give up responsibility for impossible testing tasks like ‘complete coverage’ and ‘zero defects’, and instead do your best with what you have and be prepared to explain the choices you make.” Relax. “Do everything you can to improve your skill as a testing problem-solver and teller of the quality story.” Practice, practice, practice. T’ai chi will aid everything else you do.

Key principles taught in t’ai chi class are Relaxation, Straightness, Separation of Yin and Yang, Empty Steps, and Focus in the Tan T’ien. I’d like to share how I apply these to Rapid Software Testing.

Imagining Strength, the Relaxed Way

My first t’ai chi teacher, Dan Ogrydziak, made a point about strength. He said that strength is not just in how much we can do, but in how easily we can do it. Imagine, he said, a cartoonish strongman and a pencil-necked geek each lifting a 10 lb barbell. They each could do so, probably, but the stronger one will be much more relaxed about it. Now imagine the weaker man practicing lifting that barbell with as much relaxation as possible, with the goal to be as relaxed as the other man - what would his body look like when he was done? What would lifting that barbell feel like? What else could he lift, or do, more easily? T’ai chi is a path to explore the benefits of this re-definition of strength, strength through softness rather than through the usual method of over-work and damage and rebuilding the muscle tissue, which leads to hard, stiff, bulging muscles.

I had to make a similar transition between the goals I wanted to achieve when I began practicing rapid testing. I’d been told that I was “in charge of quality” and that “we should have bug-free software”, so I was pretty stressed: bugs were my responsibility, since I was in charge of quality, but I didn’t have any power over the bug-insertion process. I looked at the software and decided the UI was too complicated, so I tried to get the design team to change how it did its business, and promptly created significant ill-will with that team. I did not understand the central engine of the software, and did not want to appear dumb by asking too many questions, so I remained ignorant and covered my tail by finding low-hanging bugs. At the same time the code was being developed and morphed at a dizzying rate by a young, talented design team. I was in over my head, and tried to control the chaos by doing what I had been told: Requirements - Test Matrices (“Where are the requirements docs? How can I work like this?”), being the Gatekeeper (“We don’t release it until I say it’s fully tested!”), trying to nail down a Dev Process (“Which bug was this fix for? No bug, no fix!”). This is like being the weak guy, and coveting the way the strong guy looks, with his big bulging, inefficient muscles and his injury-prone joints.

The RST class gave me a new frame of reference for my success: actual value to the team, instead of control of the product. I began by throwing away the process documents and matrices, and asking the people who were in charge what they wanted from me right now. I kept lists of concerns raised by people who mattered, and followed up on them as well as I could. I swallowed my pride and got training in the innermost workings of the system, and ultimately decided that it was not worth my time to test it explicitly, since the developers were using TDD and xUnit - but at least I had done so rationally and with a clear conscience. And I involved the whole team in testing their product with several after-hours bug hunts where the Design team and I assigned user personae to the team and we all acted out scenarios - this built bridges back to the Design team, and also gave the developers a taste of testing, immediately improving my credibility when I asked them for testability suggestions and improvements. In other words, I gave up trying to control everything, and redefined my job into something that I could do where I was with what I had.

I resumed sleeping at night, another fabulous side benefit of practicing relaxation.

Letting Go of Preconceptions

When we relax under our current burdens, we become usefully strong in all the ways that can help us do what we are doing right now. The most important thing that relaxation does for us is to open the pathways of chi energy in our bodies. The feeling is immediate and clear: better blood flow, more energy, better coordination, more sureness and precision in movement. The same physical letting go I do when I relax can have immediate mental and emotional effects too: a clearer sense of the din, the energy right around me, less anxiety, a more appraising eye, more self-confidence and humility. This in turn leads to considered action which is more likely correct or correctable.

In the RST course, James spoke about this by saying, “Testing is applied epistemology.” If we want to know how a thing works, we have to open our eyes and ears and gather evidence, and open our analogic mind to create testable hypotheses which interpret that evidence. Then to test the hypothesis we need to separate what we want to keep stable from what we want to vary, and introduce those variations in a controlled, observable way. The enemy of this process is preconception. Tension, whether mental, emotional, or physical, is the result of anticipating something happening to us like something that happened in the past, and being on hair trigger alert to respond in the pre-planned fashion that our tension facilitates. We become predisposed to interpreting any stimuli as being whatever we are anticipating, whether it really is or not, and once we do that, we trip the trigger of whatever response we have already decided on and have prepared to do. Obviously, this is terrible testing practice. How many entire problem spaces have I ignored because I was not open to seeing any problems there?

James also addressed the idea of using the benefits of relaxation when he spoke about testing in a “flow state”, usually during exploratory testing. When we have granted ourselves the freedom to act as we see fit and to follow up on any leads we see, even within the bounds of a charter, we have chosen to test with our minds open. We can use our enhanced awareness of the energy of the team, of the project, and of the software itself to guide our tests to focus on the greatest risks or the most pressing needs. This is important when we are going to report our testing status and need to be accountable for our decision-making. It was easy when I had my matrix to be able to say, “I had to cover squares B5-C12 today. It isn’t my fault no one cares about the contents.” Now I need to be able to tell my team that I spent the week trying to calm down this particular customer, even at the expense of following up on that bug fix that the developer wanted feedback on. I find that I am in much closer touch with my leaders when I use this approach, because I have to make sure that they buy into my read of the situation. I find I’m usually right, and when I’m not, I am able to change course and accept the new direction with equanimity because my criterion for success is value to the team, not control of the product.

Avoiding Injury

Another important benefit of relaxation is that you avoid injury from failing at something that is too much for you. Tendon and ligament damage only happen when you cannot lift something with some relaxation. By working to do everything with relaxation, we start to notice when we can’t be relaxed, and perhaps ask ourselves what the problem is: too heavy a burden? Too hard an assignment? Too contrary a lover? Too high an expectation? Then we can work to shed the excess and recover our marginal capacity, and our relaxation, and our productivity, and our health.

The analog in the RST course is the Status Report game. “On a scale of 1 to 100”, asks your boss’s boss, “how done is testing?” The sort of answer James was looking for is, “0 to 100 what? I have a better idea, let me tell you about our product and our testing.” The usual answer, of course, was something like “85 - We’ve done most of our testing, there’s still a bunch to go, and 80 sounded too small.” People fall into the trap of playing up to the expectations that they believe others put on them, and then they paint themselves into a corner. Much better is to tell the story of quality, plainly and openly.

In terms of project work, the analog is to say, “I don’t know how to do that (ensure zero defects / program in PHP / make sure we are CMM-Level-2-compliant)” or “You might want to find someone else who can help you with that (gin up numbers for a sales brochure / close out defects to hit a quota / give the thumbs-up for a release)”. Otherwise I just take on stressful jobs that I can’t do.

Better Listening and Responding

One interactive t’ai chi activity we practice is called push hands or listening hands. We stand a few inches apart and place our hands lightly on one another’s arms. Then we relax and listen to each other’s bodies as we move our weight back and forth, with the goal to unbalance your partner while remaining balanced, and to use as little force as possible. It is a lot like trying to hack a system and keep from being hacked, and relaxation is your firewall and also your penetration strategy. With more relaxation you can hear your partner’s body, its limitations and its intentions, sometimes even before it physically moves. At the same time your body loses its tendencies and predictability, gains flexibility in the joints, and is able to sink lower and lower to be more stable, so you are harder to unbalance.

James and Michael Bolton have come up with a vast array of lists of options that are available to a tester: test techniques, analytical methods, planning methods, logistical issues, reporting techniques, and on and on, coalescing the hard experience of decades of testing and thinking into a diamond-dense pith of instantly-accessible heuristics to apply. Much of the RST course is built on these lists, and the goal is to open the student’s eyes to the incredible array of possibilities at their disposal without making it seem like to landscape is so limitless that they freak out. For some reason, my mind does not do well trying to memorize these lists (even though I can remember thousands of lines of song lyrics - strange). But I’ve developed a working familiarity with them, so that I understand when I’m in a situation where one might come in handy.


The most satisfying testing, though, comes when my interaction with the software takes on the characteristics of push hands, and I am working on its balance with variations of data and timing and environment, and it is challenging me with nuances of responses that require deep analysis to be sure what they are saying. Was that “success” message really valid, or was it given to me erroneously? Did it round down properly, or just truncate the response? One false assumption, and I miss a lead, and down I go on my butt when a customer calls with a a bug report. Or, when I write the exact right oracle utility (using perl, of course) for a new feature, and find the hidden showstopper - boom, down the app goes on its butt.

RST = Relaxed Software Testing

A Rapid Tester relaxes and remains useful while situations change around her. She has good work capacity and precision, dampens stress and anxiety, and inspires confidence. She appraises the energy around her and is able to respond appropriately: this rumor is a new risk item, this nagging flicker might be a bug, this email is a charter, this design tidbit is a model update. The Rapid Tester sheds what she cannot lift: she knows when to ask for guidance or state that she does not know how to accomplish an objective. She does her best and reports plainly on her progress. She knows her tendencies, her strengths and weaknesses, and exploits them and those of others in service to her mission. She can pinpoint the source and nature of the issues she encounters, and can imagine options to choose from in how to solve them.

Does this sound like you?