One down two to go!

And no, I will not link the King Diamond song here (even tho it’s cool). This blog post is about my experiences from the first (my second first day) at the Rapid Software Testing course by James Bach. I seem to be one of the lucky few who actually are able to have two goes at this amazing adventure to the deep dungeons of an exploratory testing mind.

Like I tweeted earlier, I fell into traps; the most obvious traps that can be set for a tester. I know I can do better so here’s a quick analysis on what happened and what could I have done better:

http://morningandotherstories.wordpress.com/2011/12/

Context: When I was confronted with a testing problem without a context, the first thing to do is to determine the context. Right? Here’s what I did: I thought I might sound too clever on asking and challenging, so I answered the question without determining the context. This lead into questioning my answer and making me realize that “we’re shooting live ammo here”. So I decided to sharpen up.

• I could have done what I (try to) do every time I’m facing a problem: Understand the context, the people involved, the background, the current status, the mission, etc; communicate with the client, stakeholders, testers, etc. That could have brought the whole problem into a new light. I fell into a clever-trap. 

Pattern and mental models: I had the advantage/disadvantage to see the formula for a pattern beforehand. The task was to determine what the pattern was. Those that have taken the RST course will remember this, right? I remember the point of this exercise being to break the assumption and to focus and defocus. Again, my previous knowledge bias struck in: I completely ignored testing the product and did some of the procedures I had seen done in previous classes. The pattern was the same, however, so basically I didn't lose anything…

• …EXCEPT FOR EXPERIENCE! I could have spent the time testing in ways I had not tested before. Now I had the time to do something different. Instead I tried to look good by using Excel (like James Whitaker, I think) and I made a simple script to produce test data. I didn't even bother to check the logs, as I had checked them numerous times before. Should they have changed the behavior even a bit, I wouldn't have been able to identify the pattern. I fell into a showman-trap.

http://members.multimania.co.uk/mrmal13/saints/amardas.htm

Documenting: The task was simple: Write tests and justify all of them. James specifically asked everybody whether they had made a justification for ALL the test they were going to run. An exploratory tester documenting test cases and justifying them all with specific, detailed texts? Can you picture that? I could hear myself saying within my head “Yes, James” without even questioning my so obvious bias. I tried to be a good boy and wrote them cases and then feel pride.

• The exercise was about justifying the testing but in a different way. My doing inexpensive testing and discovering something worth discovering, one can justify a more expensive testing. Like James said: “The cheap tests pave the way for the more expensive ones. They are the scouts that look for a good spot to place the artillery.” That comment alone is worth its own blog post. But instead of being inquisitive, exploratory tester, I became a test designer that is going to get fired. I fell to the guru-awe-trap.

So my first day was a success: more learning than I hoped for. I also thought that I had some form of advantage of knowing the stuff from previous, but it seems that I had misunderstood (or belittled) some basic principles of Rapid Testing. I felt some of the stuff explained more comprehensively, with sidetracks to bulk up the areas where I thought the content was thin (namely the modeling and project elements of Rapid Testing). I do wish James would have covered the modeling heuristics with more depth, but I will leave room for other areas for the next two days.

What I’m looking forward is the process of Rapid Testing itself. I somehow missed a whole lot on that subject when Michael Bolton was teaching the course. Also to hear more about the oracles would be nice. We scratched the surface today but we’re still to cover some of the mnemonics that I frequently use, like CIDTESTD, HICCUPPS, etc., but it was nice to see an update to SFDPOT with the Interface addition. I will be using that when I reinvigorate the workshop in near future.

The importance of the context

So as I’m heading home, I will share this one particularly funny event that happened in the beginning of the class. Here’s the story without context:

James Bach spit on Aleksis Tulonen.

If you add context, here’s what really happened: Some poor fellow was getting given the “Socrates treatment” by James and he had an act prepared for the response the guy was going to give. He asked a question and set a trap onto that question. He knew the answer would be something “horrifying”. So he took a sip of water after asking the question. When the poor fellow answered James theatrically spit the water from his mouth as if being so stupefied by the answer that he just couldn't bare it. Unfortunately the aim of the jet of water was a bit off. Aleksis, who happened to sit in the front row next to me, was a victim of the poor aim. Aleksis’ phone and computer got some droplets on them as collateral victim of this splendid act. Hopefully tomorrow’s newspaper doesn't have a title “A testing consultant got spit on by a guru”.

The Adventures of Exploratory Monkey

This is a summary of Sami Söderblom’s track about exploratory testing and finding the inner adventurer. He presented this at the Nordic Testing Days 2012 in Tallinn. I found this track one of the most interesting ones as this cuts to the core of testing by stripping the excess and unnecessary activities away and focuses on the essence of truly good software testing.

Before I start ripping the track apart (well actually quite the opposite), I want to mention that this is somewhat in line with James Whitaker’s views* about exploring the software. After watching a video about large scale exploratory testing** I saw great many similarities in Sami’s track. I wonder how much of an influence has James Whitaker been to this track…

Sami had some great stuff going on in the track about exploratory testing. Great to me, because he tackled some of the issues that I’m struggling with when I try to explain exploratory testing to people new to the concept. The loop of Design, Execution, Observe, Learn and Adapting was the kind of thing people should be familiar with. By removing some action from the testing, we will result in a cumbersome approach that hinders the effective testing that we are trying to achieve with exploring. Also it was great thing for others so that they can get a firm grasp of what exploratory testing is in its most fundamental level.  Sami had great examples using the Battleship metaphor, the Great explorers of different domains, and the way of surviving with as little documentation as possible.

We have all played battleship game, right? B4 and c8, hit, destroyed, and so on, you know the drill. Some people think testing like the Battleship game and rely that if we cover all the grids by test cases we will find bugs. But the bugs don’t play by the rules! There are ships outside the grid! There are ducks instead of ships! There are mines within the grid! Why should we play by the rules? Why not do everything it takes to find all the ships? Use hammers instead of “C4” (or even real C4). Use a vacuum cleaner to suck in all the ships. Use an umbrella over the mine so you won’t break the environment by accident (and thus hinder your testing effort). Use whatever tools and techniques you have to get the job done! Cheating is allowed in software testing!

Sami introduced a few cool people from real world and from fictional world as his favorite explorers. Dr. House is an explorer, there’s no doubt about that. He tries all things he knows and utilizes all possible resources, techniques, tools, medicine, etc. to get the problem solved. He might try things that don’t necessarily cure the illness but that uncover more information about the problem. Sami also told about Sherlock Holmes, Gordon Ramsay and Jeremy Clarkson, and made cool examples how they solve problems or challenging tasks. This made me want to read more about this Sherlock Holmes, and I already started recording the House tv-series from the 1st season.

He talked also about reporting bugs. This is quite contradicting to the track made by Ainars Galvans, as Sami sees that everything must be reported as they might be a manifestation of something more severe. The root-cause is what matters! So if we don't know the root-cause, we might miss a terrible bug by thinking it's trivial.

All in all Sami managed to bring new information about the world of exploratory testing to the audience and about the building blocks of quality. I’m happy that he showed the Venn-diagram that had "testing" and "verification" in separate circles AND emphasizing that both of them need the other to be at their most effective.

I recommend reading Sami’s blog about exploratory testing. He has great thoughts about testing and quality. He will eventually write report of the conference, so follow the blog and hear his thoughts about his own performance and of others.

"All testing is exploratory!" -Sami Söderblom

*) “Exploratory Software Testing: Tips, Tricks, Tours, and Techniques to Guide Test Design” by James A. Whitaker 
**) YouTube video about large scale exploratory testing by James Whitaker