Software Tasting --by Geordie Keitt » t’ai chi

Testing In Principle: Root

Geordie Keitt — Tue, 25 Nov 2008 03:58:23 +0000

T’ai chi hurts to do. The benefits are enormous, and the cost is pain. The nice part is, the same deal applies to most things in life. If you want to be a great guitar player, your fingers will hurt. If you want to be a great athlete, your muscles will hurt. My teacher Chris Casey says, “There is no way out. But there is a way through.”

What hurts is letting go of the tension in my body and feeling that tension come into consciousness. When I work this while standing and imagining that I am connected with the earth through the soles of my feet, the pain is incredible in my feet and legs, the burn of chi blowing through parts of my body that I’ve denied it access to for many years. The gains of course are commensurate with the effort I put in. The key win while rooting is that I am connected to something much larger than myself, a thing to which everything and everyone else is connected as well: the earth. When I am participating consciously in that connection I have insight into the connections others have with the earth as well, and can analyze and manipulate it to my benefit in tai chi, and to their benefit too as a friend. The pain comes from losing the shell of ego I have built that separates me from my surroundings.

The testing parallel is obvious from my other blog entries: the bare facts of your context are the ground you stand on when choosing what to test and why. A great Rapid Tester does not arrive in a context already thinking she knows all she needs to know, fully formed, individuated and full of opinions about what is best. She learns. She crawls, then walks, then runs. Asks questions, appears clueless, and opens herself to learning about everything from everything. She molds herself into a teammate in order to be useful, takes on all the different viewpoints of the stakeholders, and also learns to “think like the code” (model the system) as well.

This is a painful process for those of us with egos that tell us to look like we have the answers. It is hard work to catalog my biases and weaknesses, and to systematically counteract them. One of the best methods for analyzing my tendencies is to participate in friendly competitions and debates with fellow testers. I learn that all of my favorite tricks have pitfalls, and there are a lot of great ways to test and manage testing that I have not yet thought of. I have become more comfortable with that fact. And I have become comfortable in my community, a vibrant and living aspect of my context that I am very happy to be able to tap when necessary.

One more point about Root. The tai chi player with more root always wins. That is because they can extend their root beneath their opponent and cut them off from the earth. A good player can literally bounce me like a basketball when they sink below me, test my connection to the earth, and find the flaws in it. We have the same job as testers, to probe the structure of our systems and the strength and depth of their logic and implementation, and to work on the flaws we see or suspect, until we break the illusion that the system “just works” and can bounce it around like a basketball. When I do good Rapid Testing, I am sinking deeply into the problem space, asking the hardest questions, getting at the heart of the matter, and feeling the burn.

Testing In Principle: Work Directly Toward the Center

Geordie Keitt — Sat, 18 Oct 2008 01:19:36 +0000

When I worked with James Bach on a Satisfice project he would ask me uncomfortable questions about what I was doing and why I was doing it. He’d ask, “Tell me what this bug is at its core.” I’d say, “When I click this button, I get an error message instead of the dialog I’m expecting.”

“Why do you expect a dialog?”

“Because that’s how the app is supposed to work.”

“How do you know?”

“It worked that way another time.”

“Ahh, so there is an inconsistency between how it worked before and the way it works now. What is different? How do you know that they are not both right?”

What he needed me to get at was the clearest possible understanding of the issue, and the clearest and simplest expression of it. He imposed a 75-character limit on bug report summaries.

A typical progression of a bug report might go like this:

Error message appears after user selects calendar icon on application page

Calendar sometimes fails to load on application page

Calendar sometimes fails to load on any page

Calendar sometimes fails to load on low-RAM system (256MB)

All Java calendars fail to load when less than 50MB RAM available

I have found that I know when I have hit the core of a bug (or at least close enough). My mind comes to rest. Robert Pirsig describes the feeling very nicely in “Zen and the Art of Motorcycle Maintenance”: “the material and the craftsman’s thoughts change together in a progression of smooth, even changes until his mind is at rest at the exact instant the material is right.”

In t’ai chi we focus on the center of everything. The center of your weight, the tan tien, is the source of all your power. If you can control your opponent’s center then you can control his balance in push hands and you will win. When you shift weight you will shift it to the center of your foot and the center of your knee will travel straight over the center of your foot.

I read a book about Abraham Lincoln that described his mental processes as slowly but inexorably penetrating to the heart of every matter. He paid attention to all the sides of an issue, peeling back the layers of emotion and baggage that people allow to accumulate on an issue such as the continuance of slavery in America, while aiming over and over and over for the heart of the matter. He didn’t mind if he didn’t fathom it all at once, he was patient and confident that when he arrived at the core it would be evident. It is this grasp of the issues that led him to be such a fierce debater. His stance was always more deeply rooted than his opponent, who inevitably would grasp at an argument thinking it was the core that Lincoln had already peeled back for himself and exposed as more mere baggage.

Rapid Testers directly approach the areas of greatest risk, greatest uncertainty, least clarity, greatest fear and greatest neglect. These are the leverage points where we can be the most use in the least time. The proper attitude is humble and fearless. Like a great martial artist we understand that we must dispatch these enemies of quality as quickly as possible because although only one thing happens at a time, there are no guarantees about when the next thing will happen. There are no shortcuts to the center though. You aim and release your best energy at it – tests, weight shift, tai chi push – and then adjust as your feedback returns to you.

Testing In Principle: Focus In the T’an Tien

Geordie Keitt — Sat, 26 Jul 2008 17:20:48 +0000

OK, so you have relaxed, you have found your feet, you stand straight up. How do you move? You can learn a lot about how you move by watching other people move and walk around. Notice where their movement originate in their bodies, and how the impulse flows through them. Some people lead with their head, like they are on stilts and constantly trying to catch up with their noggin. Some people lead with their shoulders, hunched up, or defensive. Some people lead with their knees and shins, banging into things. Some lead with their chest, puffed up and top heavy. T’ai chi practice is to initiate all movement in the center of gravity, called the t’an tien. The masters were not fooling around when they said “all movement” – all physical movement, breath, emotion, thought, everything was moderated by the t’an tien. In fact all sensory intake is channeled to the t’an tien, where it is processed into life energy and initiates our reactions.

The Rapid Tester’s t’an tien is her mission, the goals of the organization for which she is testing. The mission is at the center of the tester’s decision-making process, and everything that she learns is evaluated in light of the mission, and her responses are chosen to further the mission.

You have seen testers who are focussed on everything else. The toolchasers who automate everything regardless of utility towards the mission, are like the guys who walk around with their energy in their heads, lurching after everything that catches their eye. The ones who do just enough to get by and are afraid of anyone scrutinizing their work, keeping their shoulders hunched around their ears. The blunderer, who makes a lot of noise about testing but somehow always forgets to put the “where” clause in the “delete from” statement, and wipes out the test database again. Or the puffed-up process jockey who is quick to ask for the repeatable test script that you used to find the bug, and to knock you if you can’t comply.

I’ve been all of those testers, and many more. But I forgive myself, and remember my mission focus, and return to being an effective Rapid Tester. My first t’ai chi teacher, Dan Ogrydziak, constantly exhorted us, “Focus in the t’an tien at all times. There is nothing you can do that would be better if you were not focussed in the t’an tien. If you forget, that’s OK. When you remember, then relax and focus again in the t’an tien. The important thing is not to remember, but to remember to remember.”

Remember to remember the mission of testing (the t’an tien), let go of all preconceived notions of what a tester has to be doing (relax), and do whatever you think needs to be done right now.

Testing in Principle: Straightness

Geordie Keitt — Wed, 16 Jul 2008 14:25:46 +0000

Taoist philosophy links heaven above with earth below via the upright human. Since tai chi is the embodiment of Taoism (at least in part – Professor cheng Man-Ching said that his tai chi was 30% Lao Tzu (Taoist) and 70% Confucius (Confucianism)), the principle is interpreted very literally. The spine is said to link heaven and earth, so one carries oneself as though the top of the head were suspended from above on a string, and the sacrum was attached to a thousand-pound weight. The spine itself has separation between the joints, is not kinked or bent sideways (though the natural front to back curves remain), and offers the most space in the torso for the organs to function. The image is a string of pearls suspended from above. It maintains its shape as it moves through space, the pearls do not lean on one another, and it can move freely in any direction according to any force applied to it, and returns to the straight form after the force is gone.

When we are physically straight, our limbs are free to move in any direction and manner, so we can respond in any way necessary. Our lungs have maximum capacity for best functioning, and all our organs have space to move around and get a healthful little massage as you go through the day. When I lean over with hands in pockets, I may look cool (or not!), but I have a lot of work to do to get back into position to respond to my environment. And I have lost my connection to the earth – see the post on single-weightedness.

In testing, straightness comes into play when we enter a test open to any possibility of system behavior. If I have the feeling of straightness, I can deploy resources as soon as they are needed, whether it is tools or data or asking help from my professional contacts. If I test with purpose, committing the full weight of my intellect to the test, I am straight, not leaning over and nonchalalantly testing with my psyche half-engaged.

With straightness, I can link the micro activities of testing with the macro activities of model building and contextualization: by maintaining my connection to the ground/yin (my mission and models) and heaven/yang (the details of day to day testing). This is embodied physically by using the lower body to connect to the ground and control your own body’s position in space, while using the hands and arms to connect to your partner in practice, listening to their balance and transmitting the force of your movements. They can’t work together without straightness.

The notion of straightness has a lot of connotations in English, one of which is “integrity”. Great testing requires integrity of thought and action, and also integrity of reporting. Practicing physical straightness helps me get the feeling of mental and ethical straighness as well so I can tell when I’m not really flexible, really engaged, really forthright, and stop leaning over, take my hands out of my pockets, look the app in the eye, and engage.

Testing In Principle: Single-Weightedness

Geordie Keitt — Wed, 16 Jul 2008 12:53:14 +0000

In tai chi, as a martial art and as a healing art, the source of power is the ground. Our connection to the ground through our feet is the foundation of everything that we do, so severing that connection is (almost) always a bad thing. That is why there are so few jumps in the tai chi form, and why when we practice together the goal is to cause your partner’s feet to leave the ground. The principle of yin and yang says that one foot must be the dominant connection to the ground. There is no such thing as equal connection to the earth through both feet simultaneously. That’s as sensible as saying the moon has two dark sides or two lighted sides. Equal weight means no connection at all, and this is very dangerous, so we strive to maintain single-weightedness, and when shifting weight, to minimize the time we spend at or near 50 / 50 weight distribution.

So the foot that has 51% through 100% of the load of the body is the weighted foot, and it is the foot on which we focus our attention to connect with the power of the ground. The other foot is free to do other things – step, kick, sweep, whatever – but it is empty, and should not distract us from the central concern which is to connect to the earth through the weighted foot.

In testing we very often need to tap into the power of the ground, the power of thoroughly knowing the territory we are in, so our tests are grounded in valid, factually correct system models, exploring a particular coverage area, with oracles that we understand – or if we don’t understand them, we understand the schema of them and can identify the holes we are trying to fill. We need to work to solve one problem at a time, which leads us to control the variables in our tests and vary them systematically. We need to have a leg to stand on, which in testing terms means we need to have a reason for testing where we are, a risk model and theory of error that locates our testing in high-value space.

Most of all, though, what I bring to testing from training in single-weightedness is the feeling of committing to the problem. All of my attention and effort is going into understanding this exact spot in the app, and grounding myself in the problem space so that when I have to tell the quality story, it comes all the way up from the ground, not just from the top of my head.

Testing In Principle: Stepping Empty

Geordie Keitt — Wed, 16 Jul 2008 12:41:10 +0000

In tai chi we are taught to “step empty”, to place the foot on the floor without committing any weight or energy to that foot. Only when the foot is flat do we shift our wieght. This way we remain single-weighted (another principle for another post) and balanced, and keep our options open about what to do with that empty foot, either shift into it, leave it empty, or move it without disrupting the rest of our body.

To illustrate: Which two animals habitually step empty? Some common guesses are cats and tigers, because they have soft footfalls. This is not correct. Empty is not the same as soft: cats tread softly, but they also jump a lot, which obviously means you will land with weight. Tigers have very heavy tread, though they appear lithe. Other guesses were birds (they have to jump up to fly, and they have to land, though perhaps softer than cats), and gazelles (the leaping again). Not dogs, hippos, wildebeest. Here’s the first animal: the full-grown elephant habitually steps empty. Why? With so much mass, its bones cannot withstand a hard fall, so it takes very few chances with its footing. Even though the likelihood is relatively small that it will step on unstable ground, the consequences could be dire, so it moves deliberately. By the way, this is why it’s no great trick for the circus trainer to put her head under an elephant’s foot. No elephant will step on such an unstable surface. (Notice I said “full-grown elephant” – when they are small elephants can run more because their mass is not so great. Even full-grown elephants run on occasion, but not habitually.)

The testing lesson here is that there are times when the project or the testing has the characteristics of an elephant, and the risk then is that any fall will really hurt. When failure is expensive, even when unlikely, step empty. Make sure you have a recovery plan in place in case the system crashes, so you can retain data, system state, and configuration. Make small controlled changes so that it is easy to trace the variation that caused unexpected behavior. Test a new area lightly at first, then commit more and more effort to it as it bears the weight of scrutiny. Then it will become a new trusted base state and you can step again.

So now we have seen one risk model: a failure, though unlikely, will really hurt, so step empty. The other animal has the variables swapped around. Every step may be onto unstable ground, so that even though it is light and durable, the animal has so many opportunities to fall that it confirms every step. Give up? It’s a pack mule. The reason that mules were used in the Grand Canyon, with its treacherous gravel switchbacks and a mile to fall if you put a foot wrong, was that horses weren’t careful enough and kept getting themselves killed. The peculiar gait of a mule is from lifting one foot at time and putting it down in its new place before shifting any weight, and shifting back and replacing the foot if there is insufficient stability. And the classic pose of the mule resisting moving forward another inch, despite all the tugging and kicking and cursing? It can’t find a trustworthy place to step, so it’s not going anywhere.

In terms of testing, we can be like mules by being prepared to find an alternate route if our first step is blocked, to look for traction in an app, and finally if in our professional judgement we can’t move further, we stop and say “no” to requests to continue down a dangerous or unstable path. When I am testing a flaky app that has lots of failure potential, and I want it NOT to crash, that is when I step empty and set all my variables to normal values, follow the safest path to the destination, and present as little load as possible.

Testing In Principle: Relaxation

Geordie Keitt — Thu, 15 May 2008 04:17:26 +0000

At the time I first took a class with James Bach on Rapid Software Testing (RST), I had been studying t’ai chi chuan for about 7 years. Throughout the class I kept hearing James say the same things my instructors told me: relax, be open to the energy of what’s around, listen and adapt, and keep your balance. Of course, he didn’t use those exact words. What he said was, “To do Rapid Testing, you don’t need permission. It is completely within your own control.” Keep your balance. “Rapid Testing is like team sports: quickly establishing the proper relationships with your team, handling the ball properly when it comes to you, passing it off to the right person at the right time, staying aware of the time and the score, and furthering the goals of the team.” Adapt to your situation. “Give up responsibility for impossible testing tasks like ‘complete coverage’ and ‘zero defects’, and instead do your best with what you have and be prepared to explain the choices you make.” Relax. “Do everything you can to improve your skill as a testing problem-solver and teller of the quality story.” Practice, practice, practice. T’ai chi will aid everything else you do.

Key principles taught in t’ai chi class are Relaxation, Straightness, Separation of Yin and Yang, Empty Steps, and Focus in the Tan T’ien. I’d like to share how I apply these to Rapid Software Testing.

Imagining Strength, the Relaxed Way

My first t’ai chi teacher, Dan Ogrydziak, made a point about strength. He said that strength is not just in how much we can do, but in how easily we can do it. Imagine, he said, a cartoonish strongman and a pencil-necked geek each lifting a 10 lb barbell. They each could do so, probably, but the stronger one will be much more relaxed about it. Now imagine the weaker man practicing lifting that barbell with as much relaxation as possible, with the goal to be as relaxed as the other man – what would his body look like when he was done? What would lifting that barbell feel like? What else could he lift, or do, more easily? T’ai chi is a path to explore the benefits of this re-definition of strength, strength through softness rather than through the usual method of over-work and damage and rebuilding the muscle tissue, which leads to hard, stiff, bulging muscles.

I had to make a similar transition between the goals I wanted to achieve when I began practicing rapid testing. I’d been told that I was “in charge of quality” and that “we should have bug-free software”, so I was pretty stressed: bugs were my responsibility, since I was in charge of quality, but I didn’t have any power over the bug-insertion process. I looked at the software and decided the UI was too complicated, so I tried to get the design team to change how it did its business, and promptly created significant ill-will with that team. I did not understand the central engine of the software, and did not want to appear dumb by asking too many questions, so I remained ignorant and covered my tail by finding low-hanging bugs. At the same time the code was being developed and morphed at a dizzying rate by a young, talented design team. I was in over my head, and tried to control the chaos by doing what I had been told: Requirements – Test Matrices (“Where are the requirements docs? How can I work like this?”), being the Gatekeeper (“We don’t release it until I say it’s fully tested!”), trying to nail down a Dev Process (“Which bug was this fix for? No bug, no fix!”). This is like being the weak guy, and coveting the way the strong guy looks, with his big bulging, inefficient muscles and his injury-prone joints.

The RST class gave me a new frame of reference for my success: actual value to the team, instead of control of the product. I began by throwing away the process documents and matrices, and asking the people who were in charge what they wanted from me right now. I kept lists of concerns raised by people who mattered, and followed up on them as well as I could. I swallowed my pride and got training in the innermost workings of the system, and ultimately decided that it was not worth my time to test it explicitly, since the developers were using TDD and xUnit – but at least I had done so rationally and with a clear conscience. And I involved the whole team in testing their product with several after-hours bug hunts where the Design team and I assigned user personae to the team and we all acted out scenarios – this built bridges back to the Design team, and also gave the developers a taste of testing, immediately improving my credibility when I asked them for testability suggestions and improvements. In other words, I gave up trying to control everything, and redefined my job into something that I could do where I was with what I had.

I resumed sleeping at night, another fabulous side benefit of practicing relaxation.

Letting Go of Preconceptions

When we relax under our current burdens, we become usefully strong in all the ways that can help us do what we are doing right now. The most important thing that relaxation does for us is to open the pathways of chi energy in our bodies. The feeling is immediate and clear: better blood flow, more energy, better coordination, more sureness and precision in movement. The same physical letting go I do when I relax can have immediate mental and emotional effects too: a clearer sense of the din, the energy right around me, less anxiety, a more appraising eye, more self-confidence and humility. This in turn leads to considered action which is more likely correct or correctable.

In the RST course, James spoke about this by saying, “Testing is applied epistemology.” If we want to know how a thing works, we have to open our eyes and ears and gather evidence, and open our analogic mind to create testable hypotheses which interpret that evidence. Then to test the hypothesis we need to separate what we want to keep stable from what we want to vary, and introduce those variations in a controlled, observable way. The enemy of this process is preconception. Tension, whether mental, emotional, or physical, is the result of anticipating something happening to us like something that happened in the past, and being on hair trigger alert to respond in the pre-planned fashion that our tension facilitates. We become predisposed to interpreting any stimuli as being whatever we are anticipating, whether it really is or not, and once we do that, we trip the trigger of whatever response we have already decided on and have prepared to do. Obviously, this is terrible testing practice. How many entire problem spaces have I ignored because I was not open to seeing any problems there?

James also addressed the idea of using the benefits of relaxation when he spoke about testing in a “flow state”, usually during exploratory testing. When we have granted ourselves the freedom to act as we see fit and to follow up on any leads we see, even within the bounds of a charter, we have chosen to test with our minds open. We can use our enhanced awareness of the energy of the team, of the project, and of the software itself to guide our tests to focus on the greatest risks or the most pressing needs. This is important when we are going to report our testing status and need to be accountable for our decision-making. It was easy when I had my matrix to be able to say, “I had to cover squares B5-C12 today. It isn’t my fault no one cares about the contents.” Now I need to be able to tell my team that I spent the week trying to calm down this particular customer, even at the expense of following up on that bug fix that the developer wanted feedback on. I find that I am in much closer touch with my leaders when I use this approach, because I have to make sure that they buy into my read of the situation. I find I’m usually right, and when I’m not, I am able to change course and accept the new direction with equanimity because my criterion for success is value to the team, not control of the product.

Avoiding Injury

Another important benefit of relaxation is that you avoid injury from failing at something that is too much for you. Tendon and ligament damage only happen when you cannot lift something with some relaxation. By working to do everything with relaxation, we start to notice when we can’t be relaxed, and perhaps ask ourselves what the problem is: too heavy a burden? Too hard an assignment? Too contrary a lover? Too high an expectation? Then we can work to shed the excess and recover our marginal capacity, and our relaxation, and our productivity, and our health.

The analog in the RST course is the Status Report game. “On a scale of 1 to 100”, asks your boss’s boss, “how done is testing?” The sort of answer James was looking for is, “0 to 100 what? I have a better idea, let me tell you about our product and our testing.” The usual answer, of course, was something like “85 – We’ve done most of our testing, there’s still a bunch to go, and 80 sounded too small.” People fall into the trap of playing up to the expectations that they believe others put on them, and then they paint themselves into a corner. Much better is to tell the story of quality, plainly and openly.

In terms of project work, the analog is to say, “I don’t know how to do that (ensure zero defects / program in PHP / make sure we are CMM-Level-2-compliant)” or “You might want to find someone else who can help you with that (gin up numbers for a sales brochure / close out defects to hit a quota / give the thumbs-up for a release)”. Otherwise I just take on stressful jobs that I can’t do.

Better Listening and Responding

One interactive t’ai chi activity we practice is called push hands or listening hands. We stand a few inches apart and place our hands lightly on one another’s arms. Then we relax and listen to each other’s bodies as we move our weight back and forth, with the goal to unbalance your partner while remaining balanced, and to use as little force as possible. It is a lot like trying to hack a system and keep from being hacked, and relaxation is your firewall and also your penetration strategy. With more relaxation you can hear your partner’s body, its limitations and its intentions, sometimes even before it physically moves. At the same time your body loses its tendencies and predictability, gains flexibility in the joints, and is able to sink lower and lower to be more stable, so you are harder to unbalance.

James and Michael Bolton have come up with a vast array of lists of options that are available to a tester: test techniques, analytical methods, planning methods, logistical issues, reporting techniques, and on and on, coalescing the hard experience of decades of testing and thinking into a diamond-dense pith of instantly-accessible heuristics to apply. Much of the RST course is built on these lists, and the goal is to open the student’s eyes to the incredible array of possibilities at their disposal without making it seem like to landscape is so limitless that they freak out. For some reason, my mind does not do well trying to memorize these lists (even though I can remember thousands of lines of song lyrics – strange). But I’ve developed a working familiarity with them, so that I understand when I’m in a situation where one might come in handy.

The most satisfying testing, though, comes when my interaction with the software takes on the characteristics of push hands, and I am working on its balance with variations of data and timing and environment, and it is challenging me with nuances of responses that require deep analysis to be sure what they are saying. Was that “success” message really valid, or was it given to me erroneously? Did it round down properly, or just truncate the response? One false assumption, and I miss a lead, and down I go on my butt when a customer calls with a a bug report. Or, when I write the exact right oracle utility (using perl, of course) for a new feature, and find the hidden showstopper – boom, down the app goes on its butt.

RST = Relaxed Software Testing

A Rapid Tester relaxes and remains useful while situations change around her. She has good work capacity and precision, dampens stress and anxiety, and inspires confidence. She appraises the energy around her and is able to respond appropriately: this rumor is a new risk item, this nagging flicker might be a bug, this email is a charter, this design tidbit is a model update. The Rapid Tester sheds what she cannot lift: she knows when to ask for guidance or state that she does not know how to accomplish an objective. She does her best and reports plainly on her progress. She knows her tendencies, her strengths and weaknesses, and exploits them and those of others in service to her mission. She can pinpoint the source and nature of the issues she encounters, and can imagine options to choose from in how to solve them.

Does this sound like you?