Testgasm

This is a blogpost to describe what was going on in the Rapid Software Testing class held by James Bach. This is not meant to be a comprehensive analysis on what happened or the lessons I learned but few highlights and hindsight analysis on what happened.

The dice game

...this is just a few of what we had...

Those that have been attending the dice game know that the initial exercise is hard. When one gets the idea on how the algorithm goes it’s easier to play variations on the game by group of friends or colleagues (not to say these are necessarily different). I (thanks to Michael Bolton’s course), Henri Hannuniemi and Sami Lehtonen (thanks to Antti Niittyviita) had played the game earlier and so we were grouped together and given a different algorithm than the others. I was so thrilled by the idea that I could use all the skills that I had learned to crack that nut open.

The game started and we were given a bunch of dice: a handful of regular, different colour/size dice; few D20 dice; few D10 hand drawn dice; a die that was in a transparent die; some “poker dice”, etc. We had so many variables that we became a bit confused. I started a list of different things that we could be analyzing. The size, the colour, dots or numbers, amount of dice, type of dice, arranging of dice, “zero is not zero” etc. We then arranged them so that there were different D6 coloured dice stacked together in different ways, five groups of five. James came to the table, we said “2” as a guess for every single dice pile and James replied “0”.

We had the miscellaneous dice lying in the stack nearby and I thought I ask a reply on that also. I said it “2” also and James replied “1”. A one? We had a huge number of different types of dice lying there. What could it be?

We proceeded to arrange the dice in a fashion that there were 2 or 3 special dice and regular dice bundled up so that all special dice were in use. All but one group ended up in a “0”. The one that was the “1” had the die where there was a dice within a dice. A frantic math calculation ensued. We tried to sum up the two dices together, but the result was always a “1”. So I took the die in hand and turned it so that James could see only “1” and a “2”, and we thought it was a “1”. The answer was “2”. Frustration!

Testgasm

We then analyzed the steps we took to get to that “2”. We had a couple of theories, but when the die was on the table, it was always a “1”. Heureka! We took five dice off the table and said “5”. And a five it was!

The thought process took 3 persons 20-25 minutes. The sparring between the team enabled us to try different even crazy-sounding ideas to good extent without exhausting our innovation. We simplified the data and made it more complex. We tried to look for changes in the output by varying the input. We used the different models of problem solving from our lives to figure out the pattern. We also solved a pattern composing of vocal input pattern (I save that for one of my exercise in the company) and one with a difficult mathematical pattern composing of differently grouped dice.

"Give me your hardest problem!"

Like James said, we were doing testing Kung-fu! Solving problems like matrial artists! Bonk! Blam! Thwaak! Zlott! Swoorsh! Phatam! Ka-pow! Like Batman fighting crime, we fought problems! (I’m getting a bit excited here, if you can’t read it from between the lines.) We all felt like we were invincible and we had the tools to crack every problem in the world! “Give me your hardest problem and I will solve it for you!”

The high after testgasm

After we got all the patterns solved we were all ecstatic about what we had done. We were in a problem-solving high! The word to describe the feeling would be “testgasm”. And truth be told, after a serious testing session where you find something awesome, something really important; you will get a testgasm. I’m not sure if other that testers understand the rush after a successful testing session; it is hard to explain. The joy of completing a hard task and feeling joyous about it in ways you never thought you could be. That feeling is something that testers all around are looking for when they test. Your face may light up and you yell “Yes! I got it!” and other people in the cubicle stare at you like you’re deranged.

Focus/Defocus

I had heard the concept of focusing/defocusing in Michael Bolton’s class before but I never truly understood the meaning behind it. ("If focusing is focusing, defocusing is the opposite of that") I tried to look for material online, but it was all vague and didn’t make an impact. The way that James described the method was simple: “When confused, focus; when frustrated, defocus.” Wow! When testing one sometimes loses momentum and struggles with a single piece of data for long time. Using input patterns that don’t find the problem may lead into frustration. I felt exactly that during an exercise about systematic testing.

James had us testing a piece of software that had a bug in it. We were asked to find the bug and then try to figure out why it fails. The input was a valid IP address. You know what I figured out? I have so strong built in mental models about meaningful number combinations that I was grinding my teeth and sweating to break that pattern. I had tried a pattern by testing the high and low numbers, duplicate numbers, you name it. Could I just be blind to the bug? Then James said “Look into your data. Can you see a pattern?” Yes I did. And lots of them! “Now try to find an input that is as different as possible from previous data.” And guess what IP address I used? “1.2.3.4” That’s right! I broke my pattern by trying “1.2.3.4”! What was I thinking?!?!

James was like “Dude! What the hell?” and I sat there frustrated and confused. Then I realized what he had meant. If I had drawn a line to represent my test data, it would have been like this:

I just needed to break the pattern and be more random and use data from between my clustered data pattern. And with first random IP address I put in, I found the bug. Later James explained why the program behaves like that, but I can’t remember the true root-cause of the bug, but I did learn a lot about focusing and defocusing.

Hung over (from learning)

It has now been two weeks since the course. After the class I was in a high that lasted through the weekend. I thought about giving a thorough analysis about the course but I see no need for that. I had the time to let the learning infuse to my spine and now I feel that the stuff I learned make even more difference. I did have an information hangover after the class and it was hard to get back to grips with non-testing work again. I was glad, however, that I could attend the Intensive course the next week so I wasn’t that bummed.

I recommend the class for everyone willing to improve their critical thinking, testing skills and/or argumentation skills. The exercises were good and to the point. The hot seat treatment James gave to some of us gave us experience to stand pressure and perform well when in a stressful situation. I also learned a lot about myself and the way I learn. It was also cool to hang out with great testers like Samuli Elomaa and Aleksis Tulonen.

I still have a long way to go and a lot of learning to do. (After James mentioned it) I began to think myself as a constant student, always learning. I promise James (and I will talk about this to Mr. Bolton) that I will try to compare the classes done by him and Michael. That way both of them could learn what could be done better or differently. That however will have to wait a few days (i.e. weeks) as I have a growing backlog of blog posts.