Mushroom-picking heuristic

Imagine this: You are in a forest and you are trying to find juicy mushrooms. You'd like to find some porcini, black chanterelle, maybe some yellow bearded milk-cap. Edible mushrooms nonetheless. Before you go, do you need something? Perhaps the following:

• the calendar which shows different mushroom appearance into your local forest (not too close to habitat because mushrooms ingest heavy metals)
• maybe some research on what the mushrooms look like that you are trying to find (also pictures thereof so you don't pick something poisonous that looks vaguely like edible mushrooms)
• some knowledge on how to pick mushrooms (pick them up intact and one piece either by twisting or pulling)
• maybe choose the weather when you go mushroom-picking (the mushrooms should be rather dry when being picked up)
• prepare the mushrooms as soon possible (remove sand, pine needles, moss, etc)
• use proper tools (a knife, a brush, a basket instead of a plastic bag)

These are things that you might want to consider when going out for mushrooms. Now, these are all preparatory things, bits of knowledge you might want to have before actually going to the forest. Obviously you can go without knowing these things, but you may end up with no mushrooms or worst case poisonous bastards like amanita or cortinars. The trip to the woods might have been productive nonetheless - you got fresh air and some exercise, maybe you had a good chat with a fellow mushroom-picker. Not all unprepared trips are totally worthless, but you need some preparations to achieve good results.

How does this translate to testing? You prepare yourself to the testing task by doing almost same things you do when you go mushroom-picking. Like so:

• You check the schedule for the most convenient time to do specific kind of testing. (In January you might want to ice-skating instead of mushroom-picking, which might still be fun.) If you're doing usability testing, you might want to choose a time when there is something someone can actually use. When doing penetration testing you might want to pick a time when there is something to penetrate. Cuz if you choose the timing badly you might not achieve the best results. Also checking the schedule gives you clues on how to time-box the testing.
• You might want to research the subject of testing. What should you be looking for, what things you expect to discover, what are the risks that are already known. Perhaps you want to learn more by exploring the product, by trying things, playing and clicking around, banging the keyboard with a shoe. You might have pictures of the GUI or architecture schematics, maybe a person to help you go about the product. Like mushroom-picking, you can stumble on interesting things, but to recognize the important, the critical, the alarming stuff you might need to do some research.
• You might need knowledge on how to do software testing. By clicking around without a purpose might not be good testing. A good tester has a skillset that she utilizes to perform good testing. It is also important to know the domain, how to perform testing in that particular product/domain/service.
• Choose the weather when you go testing... Urhm... Testability and configuration, choose the best starting conditions, datasets, timeslots, loads, etc. to achieve the best testing performance and to find interesting things. You might want to do testing when the backend is performing poorly or the network connections are bad. Maybe choose a dataset that is production like or maybe it should be a fuzzing test to generate weird data to the APIs.
• Prepare your mushrooms. Deliverables! You should take notes on your testing performance. This is to help you steer your testing, generate ideas that couldn't yet be executed, dot down risks and bugs you find. You can then explain to other people what you did, why you chose to do specific tests and checks, what did you find. If needed, prepare a useful report on the testing you did.
• Use proper tools. Testing is about using tools, obviously. Most important tool is your brain! Use it. Also use tools to help do things that are difficult or time-consuming, keep your concentration while testing. Use scripts when they are useful, record your screen, have quick note taking equipment. Use other people! Two brains are sometimes better than just one... A person can be a tool!

(This is not comprehensive list in any case. You might want to take other context variables into account to achieve the best result when you actually go testing.)

I'm now in the woods with my wellies on, my trusty 'shroom-basket, a J. Marttiini mushroom knife, and a backpack willed with sandwiches, a “Book of Mushroom and Black Magick”, coffee and some chocolate. Maybe a map, a compass, some survival gear if I get lost in the woods. I want to be prepared and tooled up. How the hell do I find those mushrooms?

This is where a Lévy Flight heuristic kicks in (a heuristic mentioned by James Bach at CAST 2014). It is an algorithm by which animals (and humans) go foraging. "When defined as a walk in a space of dimension greater than one, the steps made are in isotropic random directions." says Wikipedia. Essentially doing something in one spot until you move to other area to do something there. So, I stand on the road and head to the woods. I try to look for sweet spots in the woods, like decaying fallen trees, dry mossy areas under pines or furs. If I have dome my preparations correctly I look for these areas and I might know where to find them. So I start roaming in the woods and stumble on an area that has something interesting. A fallen tree! Yay! Maybe I'll find some black chanterelle there. So I look closely at the area and spend time there, perhaps picking some mushrooms or just scouting the area for clues where I might find some. After a while I head to a new location keeping in mind where I am in the forest.

So I move around the wooded area in a pattern that tries to achieve the best coverage of the important areas. The pattern might seem random, but I have a mission which I am trying to fulfill with the choices of direction to wander. When I am halfway on my walk, I might want to go towards the road so I don't end up too far when the sun goes down. I spend time on areas that are either rich in mushrooms or interesting places in themselves. Maybe I learn something while scouting for porcini, something about birds or types of moss I tread on. Maybe I note down areas that have some other mushroom that I am not intending to pick (you don't want to mix mushrooms in the same basket - I don't know why...). I take mental notes, maybe write stuff down, mark places on my map. All the while trying to achieve a goal I set for myself before I went foraging. On my way back I stumble on an abandoned building. Cool! I might take a look inside and maybe I find something interesting. Maybe an old newspaper or a book? I might spend time there even though I was set out to pick mushrooms. This is interesting new place and I want to know more of it. So I deviate from my initial mission and investigate the building. It's apparently someone's old home and its walls have sunken in to the ground. Maybe the architecture of the pre-WW2 era interests me, maybe the newspaper has some information from the "ye olde times". Maybe the book has a letter tucked in between the pages. I allow myself to deviate because this might be more important than mushroom-picking.

Back to the testing world. The woods turn into APIs, GUIs and code. The moss becomes the date on which I tread on. The mushrooms... They're not bugs, if that's what you thought. Mushrooms are information, relevant information. It can be about a behavior that is annoying the user, an error message in the wrong place, a risk that needs to be communicated to the stakeholders. There might be bugs, but there is so much more. The you go foraging in the software you can apply the Lévy Flight  heuristic either by accident or purposefully. It is called focusing and de-focusing. You focus on some area to find interesting things for some time. Then you move to other area. If the area you first stumble upon is hugely rich in things waiting to be discovered, you might spend most of your time there. Or you might just stop there briefly and look for more important things to discover.

You start with a mission and you head out to accomplish that mission. You take notes and notice things. You investigate things that look and feel important. You forage information on the product under test.

Here's a scenario that might give clues to choosing mission for your foraging.  On Monday you wonder if you should go picking mushrooms. It's a fine day, but instead of going head first into woods you barely know, you go investigate. You take your dog with you and go scouting the forest. You find a batch of blueberries and eat a few. Oh, they're so nice! The dog, Rover, eats some also. On Tuesday it rains. Bummer! So you decide to go to a shop and buy some equipment for the trip as soon as the rain stops. You decide to get a basket that has compartments for different mushrooms. You didn't even think of it before talking to the shop keeper who's apparently an expert on the matter of picking mushrooms. Great find! On Wednesday you are called to the office for an important meeting. A nice, sunny day wasted in meeting.  No time for foraging today. On Thursday you get your gear, your dog and head out to woods. You check the sweet spots you discovered on Monday and pick delicious mushrooms and even some blueberries. On Friday you make delicious mushroom stew with some potatoes and carrots. Then, to top it off, you make a blueberry pie. All the recipes for these you checked online but used your own twist to make them your signature dishes. A perfect way to start a weekend.

To sum it up, Mushroom-picking heuristic is a two-fold heuristic:
- First it is a preparation heuristic. It helps you create a mind model that allows you to plan for the up-and-coming testing session, testing phases, etc. To an acceptance testing session one might prepare differently than to a security testing phase. Nonetheless testing needs preparation and a good way to tackle the preparation is the Mushroom-picking heuristic. You should make the preparing heuristics to match your own context. Think of tools, background knowledge (oracles), time-frames and constraints, people attending the sessions, bug reporting procedures, facilities, etc.
- Second it is a testing management - a steering heuristic. It helps you move from one focus area to another. Focusing and de-focusing is one aspect. With note-taking you can keep track on the areas you have covered and to remind if there's need to return that area. Keep in mind the Stopping-heuristics so you won't get stuck too much in one area.

The Mushroom-picking heuristic is not comprehensive nor should it be. It is a model that might work or you may find it useless. Perhaps you might try it and give me feedback on how it worked and how I should improve it. It is a work in progress.

Have a tasty spring!

- Pastori

Let’s Test 2015 – conference at a glance (from a distance)

Just to get up again and write something, I decided I’d do something I’ve done before. I really want to join the Let’s Test 2015, but it seems I cannot. This blog post is similar to those I’ve done before for it is a “Conference at a Glance” kinda thing. So this is

Let’s Test 2015 – conference at a glance (from a distance)

It seems the conference is a 3 day thing with 2 key notes, 33 sessions or various kind. Since I’m a lazy person, I shall do the following:

• Tackle both key notes
• Choose one session from each day based on my familiarity of the speaker
• Choose one session from each day based on my interest in the title
• Choose one session from each day that I pick randomly

That should bring me to a total of 11 sessions that I try and grade. If you (as a conference speaker or as anyone else) feel like I should do more, just ask me in the comments section.  I try and keep it simple.

I will use a heuristic grading system (introduced here) to determine what would be the best session for me. I will grade the stuff with Angry Birds ™ grade – 0-3 stars per area – on five areas:

• Person-to-person (How will the person and his/hers work affect/inspire me or the people I know?), 
• Session value – short time span (How much can I get out of the session tomorrow – next year?), 
• Session value – longs time span (How much can I implement o my work and teach to my colleagues, my community?), 
• Steal-ability (How much of it am I willing to borrow and further develop to make it better and, more importantly, mine?), and 
• Challenge-ability (My past knowledge on the topic and my willingness to challenge the session contents.)

Ben Simo’s “There was not a breach – There was a blog”

Based on the description on the webpage, I quickly came to think I should have done something like that. I should have started blogging about something that is affecting major public. Like the Finnish Railway renewal or something else of a similar matter. Having not thought of the idea, I shall keep my eyes open the next time something big comes up. So “Thanks Ben for giving me a great idea. Don’t mind me copying it in the future!”

As we’re talking about a keynote here, I don’t think during-session-challenging will occur. As this is kind of an experience report, I feel I can absorb huge amount of wisdom and ideas from it. If only reading through the description gave me so many ideas already, attending the keynote might blow my mind. Alas, it might not happen. So my head is safe for now. Challenge-ability might be a bit low, but if you think the challenging as in self-challenging, things I can challenge in my own thinking processes, ways of working, how I present myself to others. On those parameters I see a lot possibility to challenge.

On that note, the steal-ability just went through the roof. As for long term value, I believe that short term value comes from bringing the conversation to the coffee tables for those who are not American nor have the exposure to HealthCare.gov. The long term effects lie in the the steal-ability I mentioned before. If I can introduce some public service kind of attitude towards my own behavior, it’ll really make a difference.

P2P-level is a bit tricky. Ben has been on my radar for years. I’ve been following him and reading his blog sporadically, but I never really came to realize how much of an influence he has been. There hasn’t been too much communication between him and me save for a few tweets now and then, or some random facebook comments to each other’s posts. I must say I should have been more in his face about stuff. I shall change that and get to know him better!

As for score (should I be at the conference I would attend the keynote whatever it might be):

• Person-to-person: * (I wanted it to be ***)
• Short time value: *
• Long time value: ***
• Steal-ability: ***
• Challenge-ability: ** 
• Total: 10/15 stars

Antti Karjalainen’s “Detecting the Heartbleed Vulnerability”

The description might not provoke my vastest interest, but I was part of the Heartbleed scene in a sense. While working at F-secure I heard about it weekly during the “hip season”. I didn’t particularly have anything to do with fixing nor working with it, but I was there to spread knowledge about it.

The thing is, I am the kinda guy who shies away from über cool tools, fancy technology and protocols. I have some interest in security testing, fuzzing and analytics, but I’m more comfortable to leave those to the people who know them better. When it comes to knowing what fuzzing does, I’m comfortable in what the Wikipedia says. That is to say with no disrespect towards the fine men and women who dapple with such technologies. Hats off to all of you!

P2P is a challenge for me, since Antti is a Finn and I might have run into him at some other conference. I can’t put my finger on it, however. I have never talked with him about fuzzing nor about tools (“the shying” and other excuses). The thing is, I don’t think I could talk with him about the fancy stuff. I might have a few cool comments like “that looks cool” or “whaddaya know”. The thing is, however, that I cannot say for sure. I wish there would be a common ground on which we could build a conversation and then work from that.

The short term value might be in form of an interest into fuzzing tools. Or to some yet unknown aspect of fuzzing I could use in my testing. I feel that the topic is so tool/technology oriented, I might not get enough. It also affects the long term value for I don’t think I am capable of transferring that knowledge to my community or colleagues. I don’t say there won’t be any inspiration during the keynote – I might change my way to think testing and tools.

And since my knowledge on the technology and the tools are so diminutive, I feel the steal-ability and challenge-ability are scored low also. This isn’t to say that Antti doesn’t rock most of the listeners’ world! I believe that he is the kinda guy we need more of. Just like Ben, helping regular people with their daily lives is what counts!

• Person-to-person: *
• Short time value: *
• Long time value: *
• Steal-ability: **
• Challenge-ability: *
• Total: 6/15 stars

1st day sessions:

Ilari Henrik Aegerter’s ”A Tester's Walk in the Park” (based on familiarity to speaker)

Ok. Having read the description, I’m torn in half: A session without a clear outtake or a fountain of good ideas. I don’t know quite yet. I do appreciate that the problem solving is the key to it, but are we talking about artificial, abstract problems like “where is testing going”, or concise practical problems like “how can I convince my manager to pay my trip to Sweden”? The uncertainty intrigues me in a way that I wouldn’t dare miss this session. If we can coax people like Michael Bolton or Ben Simo to join in with their problems, it can be a hoot. It could be a hoot with just me, Ilari and three guys I don’t know. The thing is “I don’t know”.

I have spent some time chatting with Ilari. He has coached me on different things, latest today (the April 1st 2015) on how to approach a testing communication problem. I know this guy and I like him. I have to admit I haven’t paid too much attention on his whereabouts the last 18 months, but I reckon that all will be changed.

The value of the sessions is a tricky one. I cannot say what value I can get for I don’t know the contents that well. On short term it might arouse good conversations and comradery between the people attending. Experience reports in the form of problem solving might be a good take-away. In longer time span, the technique itself might be a good thing to learn. The introduction of philosophical thinking and approach to software testing is actually quite intriguing. Challenging the session might be a tough one, since it feels like an experience report of a sort.

• Person-to-person: ***
• Short time value: **
• Long time value: **
• Steal-ability: **
• Challenge-ability: *
• Total: 10/15 stars

John Stevenson’s ”A Journey Towards SelfLearning” (based on interest in the title)

WOW! This sure sounds cool! A sessions in a form of a game show! Count me in! I am interested in learning and how people learn. I am a continuous learner myself and I think it is time to up that interest in me once again. If this session could make my learning more structured, I would get more out of the time I have at hand.

I know next to nothing about John Stevenson, but lately I have spotted some of his tweets. He could be one to chat more deeply about learning. Him and James Bach might be a good dinner guests if I wanted to talk about learning and teaching. Since I hope to be a teacher of a sort someday, I think they could give me valuable ideas.

The value I see from this session is vast, both long and short term. While stirring me short term, it could make me think about my life on a longer run, my education and my striving towards being a teacher (though Finnish teachers are paid really poorly). Since I know something about learning, I think I might even be able to challenge John on his ideas. Though I don’t think I can introduce any particularly new and fancy to his curriculum, I might be able to increase the value of the session by making him express thing in different ways to be more easily approachable.

• Person-to-person: *
• Short time value: ***
• Long time value: ***
• Steal-ability: **
• Challenge-ability: **
• Total: 11/15 stars

Louise Perold ’s "Non-violent Communication" (random pick)

Well well well. NVC. I must say I had need for those skills today when I argued with a developer on why the test cases written for manual execution are a poor excuse of a test automation. I did seek some coaching from Ilari (like mentioned before) but I wasn’t able to convey my point to him in a way that would have left both him (the dev) and me in a mutually enjoyable place of mind. Having said that, Louise’s session might be the one for me.

I don’t know her at all, though I might have traded tweets in the past. I’m intrigued meeting her, though. Should I not be able to come to conference, I really want to talk to her about Mortimer J. Adler’s book “How to Speak How to Listen” which I’m reading sporadically now and then. Also what I found out from Daniel Pink’s book “To Sell is Human” might contribute to the Non-violent communication.

Values from this session might be really high in short term and long term. I might not be able to transfer them to my community in a way Louise could, but I bet the example and behavior might influence my co-workers and other people as well. My knowledge on listening and conversation skills might enhance the challenge and steal –abilities, but I would have to let the time indicate what’s beneficial and what’s not.

Non-violent starring would be as follows:

• Person-to-person: *
• Short time value: ***
• Long time value: **
• Steal-ability: **
• Challenge-ability: **
• Total: 10/15 stars

2nd day sessions:

Laurent Bossavit & Michael Bolton’s ” "Defense Against The Dark Arts” (based on familiarity to speakers)

Critical thinking, they say. Very well. I would expect something like this from Bolton and Bossavit. Given Laurent’s book “Leprechauns of Software Engineering” it is about time to teach us Earth dwellers them skills to tackle possibly harmful (and perhaps even rigged) information.

I know both of the speakers and I love spending time with them. Both incredibly intelligent fellas with excellent ideas and views of the world. It seems almost a loss that I might have to miss their workshop. But I shan’t weep! I find their session really intriguing so I might badger for a coaching sessions should I need to miss the event.

As for value of the session, I see high value altogether. Critical thinking skills and the practical application of it are worth gold in the testing industry. Whatever I can bring home from that session would be valuable material for my colleagues in ways to think critically.

Since I have some former knowledge on critical thinking, I feel the challenge-ability is high. Also I see that the whole session is about challenging, I feel compelled to challenge much of their material. Steal-ability is high in a sense that I want to educate my colleagues on that particular subject.

My god, I’m looking at quite a session:

• Person-to-person: ***
• Short time value: ***
• Long time value: ** (it’s really 3 stars, but I cannot give the highest score yet)
• Steal-ability: ***
• Challenge-ability: ***
• Total: 14/15 stars

Huib Schoots’ “How to be an Explorer of Software” (based on interest in the title)

Mr. Schoots talks about creativity? The guy who had a conference sessions in which he mostly played music talks about creativity? I think this is the stuff everyone needs to see. The title pushed me in a quite different assumption on the contents of the session. I thought that it would be about hands on exploring something, but since the description kinda gave the impression that it’s about how we document and observe, it actually fulfills the title quite well. Interested I surely am.

Huib is a great fella and he could be one of the top 3 reasons for me to attend. He is the bloke that makes people smile on their worst day. All that and a world-class tester! Say no more! I have seen Sami “the Monkey” Söderblom talking about exploring, read (the beginning) Elisabeth Hendrickson’s book on exploring, so it would be nice to see another take on the subject. The value is hard to define, since I feel the values are more personal than community wide spreadable thoughts. To be able to more concisely test and document a software is a great skill to have.

To challenge Huib is something I think he would enjoy more than anything, so I think I must get in touch with him and get my hands on the material if I can’t attend. I might also be able to steal something that I later present something as my own, but I ain’t gonna let him know. It’s hard to put a finger on the stuff that I’d like to steal, but I bet there are loads of stuff.

The exploration of stars…

• Person-to-person: ***
• Short time value: ***
• Long time value: **
• Steal-ability: **
• Challenge-ability: ***
• Total: 13/15 stars

Erik Davis’ "Effective Practice Manager" (random pick)

A practice manager. That’s a new term for me. Based on the description I didn’t get what is a practice manager. Is it like a skill coach or a competence manager? Either way I feel this is an experience report with some educational features. Oh it’s an experience report with discussion. Maybe we have some cool practice manager problems we can help solve for him. Challenging might come to play while there’s a discussion in the session, but I do know so very little about the subject beforehand.

I was intrigued by the “increase their own impact at work” bullet point. That is something I want. I want autonomy and to see my own passions and skills realize in my daily work. In that sense this session might be a good catch. It would definitely have long term effects but I couldn’t get the short term value out of the description. Maybe it lies in the “whatever else comes out of the discussion”.

I don’t know Erik at all. He might be one of those blokes who have avoided my radar so far. Based on his personal description I think I he would be the guy to get to know. Maybe I’ll ask him for a beer at the evening activities. His interest in educating testers and building their skill set is something I want to do also.

• Person-to-person: **
• Short time value: *
• Long time value: **
• Steal-ability: **
• Challenge-ability: *
• Total: 8/15 stars

3rd day sessions:

Jari Laakso’s ”Security Testing” (based on familiarity to speakers)

Before I have even checked the description on Jari’s session, I must say I am intrigued by this. He is my first touch on security testing on my Finnish blog years back. Unfortunately, I haven’t met the guy. I know he’s intelligent and capable tester, but I haven’t been in contact him for some time. Maybe I have to rekindle that connection to get the latest.

Ok. The description. Very well! Hands on security testing! Where do I sign? This is like one of the essential stuff that people are asked to when doing exploratory testing. SQL injections are in every text book on testing, but this guy actually shows how to do that. Nice! I can see the value skyrocket! The thing is, to be able to share this knowledge I should know even more about the subject. I might lack the interest in security testing in general, so I might not be able to teach these skills to others (I’d ask Jari to do it for me).

To challenge him might be difficult due to the fact that we are talking about technical stuff once again. Things like cross-site scripting are founded on protocols, REST-calls and whatnot, and I don’t feel I can challenge him on those. I might wanna try, tho. ;)

• Person-to-person: ***
• Short time value: ***
• Long time value: *
• Steal-ability: *
• Challenge-ability: *
• Total: 9/15 stars

Alexandra Casapu’s “Examine Your Testing Skills” (based on interest in the title)

Hands on testing and discovering our testing skills. I like it. In EuroSTAR test lab I had the chance to dapple with Mr. Lyndsay’s machines and I really want to have another go! Besides I’m interested in how my skills map out. I don’t know Alexandra, but I’ve heard of her. She is one of those people I want on my “meet these people in the Testing Scene” list.

The idea of having various non-technical skills to help you with testing is interesting. I would like to know other peoples’ skills and if I can borrow their passions and learn what they know. I’d be able to steal ideas from the participants and from Alexandra. Very nice! Since I have been doing some reading on skills, I might be able to challenge her and my previous views quite easily. And skills practicing in practice is always something to take home to.

• Person-to-person: **
• Short time value: **
• Long time value: **
• Steal-ability: **
• Challenge-ability: **
• Total: 10/15 stars

Scott Barber’s  ”Experiencing Product Owners”  (random pick)

Random in deed… Scott, if you’re reading this, you might want to check the description on the website. ;)

A big star on the effort, though.

Conclusion

Having surpassed the pain of starting to write again, I feel joyous on how I managed to rate 10 (Scott’s doesn’t count) sessions on five heuristic parameters. The toughest thing is ahead of me, though. I need to convince someone to pay my trip to Sweden.

Vi ses!
Peksi

Doodling exercise

I was at the EuroSTAR conference 4th November in James Bach’s tutorial. I had not attended a full day training, talk or tutorial for a long time and my mind was still a bit rigid. I had a blast listening to James, but I noticed that from time to time my mind began to wonder. I started doodling on my note book. I didn’t pay attention. I was sitting between Laurent Bossavit (https://twitter.com/Morendil) and Kristoffer Nordström (http://contextdriven.blogspot.fi/). I know those guys are brilliant minds and I was actually a bit ashamed for doodling and losing my focus. So I doodled “in secret”.

At the coffee break I talked to Kristoffer about doodling and he explained it to me something like this: When listening to the talk and engaging one side of your brain, the other side starts to get bored. It begins shouting “Booooooring! I wanna do something creative!” and it makes you lose focus. Kristoffer told me to keep on doodling as part of keeping your focus. I did after that and I was able to keep my focus and come up with some more or less important ideas about testability and rapid test management – which was James’ topic on that day.

Kristoffer challenged me to write a blog post about doodling within one week, but I wasn’t able to tackle it before. Now I am, so I conducted an exercise. I tried to take notes on something that I have little or no interest in and tried to keep my focus on the job by trying to take notes and occasionally doodling. I did this by watching the most tedious video I can find from Youtube and I tried to summarize it using my doodles and notes. I chose “Intelligent Design and Creationism in the Classroom” (http://www.youtube.com/watch?v=x2qIAjtrNdY). It’s a 40 minute talk about something I try to stay away – religion and the teaching thereof. Here’s what I managed to do:

I started watching the video and thought there would be some fundamentalist talking about religion with such a zeal I couldn’t finish watching the video. Actually I couldn’t, because the video broke for some reason at 10:36. I did however rethought my previous attitude towards the video.

The beginning of the video was actually quite concise and gave things out in a quite unpassionate, to-the-point kind of analysis. I got a better understanding on why legislations on religion and marriage are the way they are. The context, I believe, requires some understanding of the government process and immigration policies. Also as a Finn, some of the things felt odd, like narrowing the gap between church and government.

More on the exercise. I tried to use doodling to keep me focused, but actually I had no trouble focusing. Taking notes was time consuming so I veered towards pictures to remind the topic that was talked about. It gave me something of an emotional attachment thus making the notes more understandable and clearer. The problem with this exercise was that it wasn’t authentic. I tried to make notes for the sake of note taking. I could formulate questions about the topics I doodled about, and in that sense I might have already felt that I must doodle about topics that I have some interest in already. I am keen to find out more about the “10 Commandments monument vs. non-Christian monument”.

As a conclusion, doodling is a good way to focus AND de-focus. I was able to shift my focus from the talk to the topics I got interested in. Also the doodles highlight the important stuff. Underscoring, circling, etc. make the text more dynamic and one can more easily find the areas of interest from the notes.

Thanks for challenging me, Kristoffer. It was a good thought exercise in addition to making me want to learn more about doodling. I still have a long way to go in note taking, but at least I know where to turn to get better exercises on note taking.

- Peksi

Note to self: read the following before going to the next meeting, session, conference or tutorial:
• http://sunnibrown.com/doodlerevolution/bootcamps/visual-notetaking-101/
• http://www.eurostarconferences.com/media/149386/alan-richardson_virtual-conference.pdf
• http://erik.brickarp.se/2012/10/practice-1-note-taking.html

Conference at a Glance, part II – My glance on the Tuesday AM tutorials

This is the second part of my series of posts about EuroSTAR 2013 conference. I apply the same method of evaluating as I did previously, so read the “My glance on the Monday tutorials” before this post, if you haven’t done that already.

Ian Rowland’s “Thinking Outside The Locks”

I have not heard of Ian Rowland, but I must say I’m intrigued. A magician? The biography in EuroSTAR page makes me want to know more about this fellow. In fact, I googled his name and got to his website. I’m really looking forward to see Ian do his stuff. I would guess that humor is involved in addition to mind blowing approach to thinking outside the box (or locks as he says).

To be honest, I think critical thinking, unconventional approaches, non-rational thinking are the tools of my trade. I would be able to use those both daily and they would make me become a better tester both short and long term.

I would recommend this to all my colleagues. In fact I’ve been thinking about a 30 minutes workshop on out-of-the-box thinking and using unconventional methods to solve a problem. If I get enough ideas for my workshop I might just do a workshop, and I hope Ian can help me generate ideas for it.

To be honest, I can’t think of anything to disagree from the summary of the tutorial, so I might not be able to challenge him. I’m curious to see how the methods he uses actually project to software testing. I might steal some of the details and trick he uses to my own work, like I mentioned before.

The tutorial seems very interesting as a light weight beginner for the conference (even better after a full day of Monday tutorial), but is this the best option out of the cast of many great speakers? I would be the one doing a half day tutorial on the subject, considering I don’t have that much experience in coaching out-of-the-box thinking.

After the smoke clears and the magician bows, I would like to see/hear/learn about implementing the skills and theories Ian shows to us. Theory is good and all but I would like to see results in testing craft to be happy with the tutorial.

On my Birdy scale, Ian Rowland’s “Thinking Outside The Locks” would scale as follows:

• Person-to-person: **
• Short time value: ***
• Long time value: **
• Steal-ability: *
• Challenge-ability: 
• Total: 8/15 stars

Prof Harry Collins and James Bach’s “Using Sociology To Examine Testing Expertise”

I think I said enough of James Bach on my previous blog post, so I will concentrate on Prof Collins. I find his resume quite impressive. He has (co-)written books that we testers should (I haven’t) be reading including the “Tacit and Explicit Knowledge” and “Rethinking Expertise”. I am eager to hear what he and James Bach have come up with. The duo of unschooled (but not untaught) and a University professor could spell doom to us mere mortals. I’m truly eager to hear their tutorial.

I am really eager to listen to stuff about meta-knowledge (or knowledge about knowledge). I do not, however, see a short term benefit from it. It will eventually develop my sense of self analysis. I’m very interested in any studies about testing and testing methodologies and this tutorial taps to that – using the tacit knowledge in addition to radiant expertise.

I don’t know straightaway how I could harness the tutorial for the benefit of my fellow testers. If the tutorial addresses social tacit knowledge, I could be able to make the company benefit from acknowledging that knowledge.

I am keen challenge the fact that there are skills that no person possesses but a group of people. Let’s say I invite a group of people to my house and I want to learn Chinese. None of these people know Chinese, but as a group we might be able to possess the skills to communicate in Chinese? Am I on the right path here? Or are we talking about more-than-sum-of-its-parts mentality, where we all would know just a little Chinese or a language close to Chinese?

I would refrain myself from teaching this. Maybe I could mention and guide people to seek for material appropriate to this experiment. I have no previous knowledge in this kind of study as I lack the university background.

Just like in Ian’s tutorial I would like to see/hear something that I could implement to my own work. What do I do with the information about what skillset does the teams have?

On my Birdy scale, Harry Collins and James Bach’s “Using Sociology To Examine Testing Expertise” would scale as follows:

• Person-to-person: ***
• Short time value: 
• Long time value: **
• Steal-ability: 
• Challenge-ability: **
• Total: 7/15 stars

Peter Zimmerer’s “Questioning Testability”

I begin to wonder what people think about me as a community member when I don’t know most of the people making presentations and more importantly tutorial speakers. This pre-analyzing the tutorials also helps me to familiarize myself with the people so I could recognize them at the event location in Gothenburg. I believe I have a lot to talk about with Zimmerer from all kind of things, but I believe we can make a conversation out of his topic also.

Testability is a freaky subject for me. I might be living in a bubble where we almost automatically plan our products with testability in mind. We aim to make the testing as fast and efficient as possible, so this tutorial might not give me too much in short term. I do believe that testability is one of the key things to enable efficient testing, so I more than recommend this tutorial to everyone!

I do believe that I could benefit from revolutionary points of view, which I hope Peter will provide. At some point when testability becomes more a worry for me, I might need the skills. Also the ability to promote testability could be important for me in this company. At some point the leading testability evangelists might leave the company, so we need as much tacit knowledge on testability as possible.

I’m expecting a lot of practical examples to be able to share myself (possibly after altering them to my own flavor). In that sense, the stealability is quite high in tutorial. I would focus mostly on practical appliances of testability, because testability as theory is quite trivial. People seem to have trouble in understanding how they can make stuff happen in practice.

When it comes to questioning, the words “step-by-step” rose a red flag. Is this method an omniscient, all-encompassing process? I hope this tutorial doesn’t turn into “do this and everything will be fine” but into “apply these skills where it’s reasonable”. If I am to join the tutorial, I will definitely challenge

On my Birdy scale, Peter Zimmerer’s “Questioning Testability” would scale as follows:

• Person-to-person: *
• Short time value: *
• Long time value: **
• Steal-ability: **
• Challenge-ability: ** 
• Total: 8/15 stars

Anne-Marie Charrett’s “Coaching Software Testers”

James has mentioned Anne-Marie a couple of time in our conversation and praised her coaching skills. Then again, I must admit that I have not made myself too familiar with her work. I’m looking forward to seeing her and possibly having a chat at some point. Hopefully she’ll be able to donate some of her time to me.

Actually the coaching method described here is something I have already done a few times. First James Bach coached me using the Socratic Questioning and then I used it to coach Erik Brickarp, Jari Laakso and Aleksis Tulonen among others. The amount learning on BOTH parties was phenomenal. I would love to gain more skills in this area to be able to continue my journey as a coach. This is something that both my colleagues and my fellow crafts(-wo-)men will benefit. I have some skills to begin with so I will employ them in future to the benefit of all, including me.

Having said that, I will try to steal as much as possible from this session and to mold it to my own. I recommend this session – yes, without having yet attended it, but having faith in it like in no other! Coaching skills are paramount on testers skill set if they ever want to become true professional.

I find it hard to challenge this on two reasons: I would consider myself as a member of coaching congregation and I would find it hard to challenge something I have unquestionable faith in. I am willing to try challenging for the sake of argument, but facing Socratic Questioning arguing for argument’s sake might be my downfall.

On my Birdy scale, Anne-Marie Charrett’s “Coaching Software Testers” would scale as follows:

• Person-to-person: *
• Short time value: **
• Long time value: ***
• Steal-ability: ***
• Challenge-ability:  *
• Total: 10/15 stars

James Christie’s “Questioning Auditors Questioning Testing”

James Christie (How many people called James are presenting in this conference?) is one of those that fall on the same category as Anne-Marie – I would love to talk to them for what I have heard from my fellow community members – but I have never delved into James’ work in-depth. Maybe I can sneak into his lunch table and steal a minute to talk about testing. ;)

In the past, I was working in a company where an audit was held. I was part of a group that coached the people getting audited to answer the questions correctly to appease the auditor. Wrong approach to audit – we do not always act according to the documented process but according to our best knowledge on the situation. The audit was about the documentation. The auditors were held in so high authoritarian position that they were not challenged – I was not allowed to talk to the auditors. ;)

I don’t see a short term benefit on this tutorial, however. I’m not currently in a position to be part of audits currently at F-secure. We do have security audits and the like, but I have yet to be invited to one such event. I might benefit James’ tutorial if I focus on questioning instead of auditing. If the scope wouldn’t be so narrow as to concern only audits, I would find it more beneficial to my current work.

If I could combine questioning to other areas, like specific levels of testing (unit, module, etc.) I could be able to teach or coach other testers and programmers to question their work more efficiently. So long term benefits could outweigh those of short term. More so, I don’t know where my life takes me so having some skills in challenging auditors might be my thing in the future.

I have such limited knowledge on auditing as such, so I find it difficult to disagree with questioning. Usually the person being questioned could benefit from the questioning too. I have been in a situation where I learned more by being challenged than by acquiring book knowledge on the subject. Like I said earlier, I would like to see tracks on more general questioning, arguing and challenging. This tutorial might answer some questions I have, but I’m not sure at this point.

On my Birdy scale, James Christie’s “Questioning Auditors Questioning Testing” would scale as follows:

• Person-to-person: **
• Short time value: 
• Long time value: **
• Steal-ability: *
• Challenge-ability:  *
• Total: 6/15 stars

Pradeep Soundararajan & Dhanasekar Subramaniam’s “Context Driven Mind Mapping”

I know Pradeep from tweeting with him and reading his blog. I also have followed the progress of Moolya for some time and I’m really impressed in their success. I’m also looking forward meeting Pradeep and Dhanasekar in Gothenburg to talk about mind mapping and all testing related stuff. I’m glad that Pradeep is hosting two sessions at the conference so I can join at least one of them.

I’m a bit of a mind map enthusiast myself so this tutorial is almost tailored for me. I find a lot of things here that are almost exactly from my workshop year ago from Nordic Testing Days 2012. I do believe that I have a lot to learn on both using the mind maps and hosting workshops. In short term, I would like to learn the most effortless way to utilize mind mapping. I tend to procrastinate during the testing, so if mind map can keep me focused, I would be on cloud nine. I also see mind maps as the tool of the future for it utilizes the brain instead of some arbitrary tool.

This tutorial would be worth stealing in its entirety and then I would go on promoting the ideas and practices to my company and to my peers in the community. In the long term, mind maps could help make exploratory testing both understandable and credible to stakeholders with minimum effort on documentation. I have already played around with the thought of decompiling the mind map into coverage charts by scripts, so this might even further automate the documentation of exploratory testing.

As for challenging, I know where I was stumbling in my workshop so I might tap into those subjects. First would be the content-switching during testing from the mind map to the software under test. If the mind map requires another window in addition to database browser, Unix log screens, browsers, standalone tools, etc. the content-switching becomes a burdening factor in the long run. Second would be the “free form” of the maps, which could result in inconsistent ways to report. I’m curious to see how Panda and Commander can tackle these. :)

I would recommend taking the Monday tutorial by James Lyndsay and combining it with this to make an awesome combo of exploratory testing and modeling. I haven’t yet decided which to attend, but if you, dear readers, should consider this combo really hard.

On my Birdy scale, Pradeep Soundararajan & Dhanasekar Subramaniam’s “Context Driven Mind Mapping” would scale as follows:

• Person-to-person: ***
• Short time value: **
• Long time value: **
• Steal-ability: **
• Challenge-ability:  **
• Total: 11/15 stars

Afterword

Once again I have not yet decided which to attend. There seems to be 2 top dogs right now, but I cannot yet say which to attend. I might even change my mind right before the session if other community members talk me to join a session other than what I would have chosen.

Anyway, I have quite a task ahead of me to plow though the conference tracks one by one. But be assured, I will go through as much as I can.

Also, I got interviewed to EuroSTAR community spotlight. I thank Emma Connor for the interview, and wish her and every tester out there a great summer!

- Peksi

Conference at a Glance, part I – My glance on the Monday tutorials

The EuroSTAR 2013 is knocking on my door. It's about time get some scribbles to my blog about it.

Quick word about what I’m doing here: I’m trying to get my thought on paper both to ease my burden of choosing the talks which to attend and also to write out my thoughts before the conference (as after the conference I might have a bit of a "information hangover" and I'm not that keen to write on stuff I didn't attend). If you think I have mistaken, misinterpreted or been outright wrong, please comment and discuss about it. I also encourage all the speakers, who I mention here, to comment on my expectations. I might be off by a mile on what the true content of the tutorial is, but please correct me if I’m on the wrong track.

I will also try a heuristic grading system to determine what would be the best session for me. I will grade the stuff with Angry Birds ™ grade – 0-3 stars per area – on five areas:

• Person-to-person (How will the person and his/hers work affect/inspire me or the people I know?), 
• Session value – short time span (How much can I get out of the session tomorrow – next year?), 
• Session value – longs time span (How much can I implement o my work and teach to my colleagues, my community?), 
• Steal-ability (How much of it am I willing to borrow and further develop to make it better and, more importantly, mine?), and 
• Challenge-ability (My past knowledge on the topic and my willingness to challenge the session contents.)

This doesn’t mean that 0 stars would mean “I hate it”, instead I’m looking for personal value against my beliefs, biases, former knowledge and worldview. I’m not saying that I won’t attend a low scored session, no, I might change my mind after talking to people at the conference and to the speakers. So I encourage you to talk to me about your session and comment what I have written about you.

James Lyndsay’s “Insights Into Exploratory Testing”

As a first thought, James Lindsay isn’t the one person that I recognize as my personal idol. He ought to be, because I read his blog once in a while, but I haven’t really gotten into his stuff. I love the blog series in which he describes different ways to manage exploratory testing! Kudos for that! However, I think I need to see him in person to suck in the charisma that he might have. After that I might regard him as my Top-3 testers. Right now, he’s a bit of a mystery man to me. And I always mix him for James Whittaker! *grin*

I’ve done exploratory testing a while now, so to answer the question: “What’s in it for me for tomorrow/this year?” I think hands on testing practices could be a remedy for my “sohpomoristic” (I have knowledge, but not enough experience implementing my knowledge to practice) way of approaching testing. The analyzing nature of the workshop could help me be a better tester on a daily basis. I was struck by a realization on few pair testing sessions with people who I really look up to. They were so much better hands-on testers that I, but they had high expectations on my skills.

Because I need to bring something back the “Ye olde Sweatshop” at Helsinki, I need to look for stuff that are good for my colleagues. The nature of the workshop could sprout some lightning workshops or sessions with my testing fellows, so I’m keen to see how to run a successful workshops with different backrounds of people. I’m also keen to see how I can use my skills to use attacking and exploitation in my everyday testing, ‘cause I work at a security software corporation. So I might be able to bring that home with me. Over all, all the things in exploratory testing might be good for my colleagues. The focus in my company should continuously veer towards exploratory approach. This workshops could give a lot – that’s for sure.

As I always tend to look on the bright side of life, I do want to be able to challenge James
and his topic. So what areas might be prone for me disagreeing? Workshops usually bring new course to the testing buffet, so I would have to taste the dish before being able to make hard decisions about it. I can’t say what would be the areas where I would disagree but I would definitely ask about how you could implement exploratory approach on a company that uses mainly test case driven approach. Let’s say there’s a company that uses only outsourced testers and they have one test analyst whose job is to design proper tests for that group of testers. How could one use exploratory approach to make their testing more efficient, creative and manageable?

What could be the next step after having this workshop? Personally I would try this on a larger scale. Oh! That’s a good question to James: “How will I be able to scale this up? How do I manage a team of these explorers?” Personally James Bach workshop could answer some of these questions. If it was somehow possible, I would attend to both workshops and then combine then seamlessly together. The next step would be to combine Lyndsay’s techniques to a broader audience and to organizations where testing is still in baby shoes (i.e. über controlled, test case driven, hierarchial, to name some).

I have one more thing to say. Mr. Lyndsay, if you happen to be at Gothenburg on Sunday evening, I would like to have a pint or two with you and talk about testing and your workshop. I might not attend it as there’s so much to choose from, but I’d like to talk to you about this particular workshop. You can give me a shout at Skype or Twitter, if you want to. ?

As for James Lyndsay’s Monday tutorial “Insights Into Exploratory Testing”, I would score like this:

• Person-to-person: **
• Short time value: **
• Long time value: **
• Steal-ability: **
• Challenge-ability: **
• Total: 10/15 stars

James Bach’s “Rapid Test Management”

Personally, I know James Bach and I’ve spent some time with him face-to-face. I attended his RST class on 2012 and the Rapid Testing Intensive course. I was exhilarated to be invited to the course as a special guest and it was one of the most joyous events of my end-year of 2012. I had the opportunity to have dinner with James and to get face-to-face coaching. He’s a radiant person who some people misunderstand as angry or frightening – choosing to see behind the façade gives you the opportunity to see a multi-dimensional, empathic person who’s a power to Testing craft all around the globe. I know it sounds like I’m secretly in love with him, but trust me: he can turn your world upside down.


As for the Monday tutorial, I feel the topic is slightly too narrow for me. I did learn a lot on the RTI course last year and I think this tutorial might repeat some of the stuff that I learned that year. For short time value the tutorial might not be in the top-3. I have a lot to learn about test management, yes. I do think, however, that I can get more by starting finally to implement the stuff I learned year ago. The opportunities to do that have been scarce. I think I need more advices on how to implement the lessons to my work instead of repeating the theory myself.

What could I do to make my colleagues life better using the lessons learned from the tutorial? I have experience already from the RTI, but my situation after that session forced me to forgo the implementation of the stuff I learned. The job as a maintenance manager also did not fully support the further teaching of the methods. At the moment I am in a position where I could help my team and all Quality Engineer at F-secure to make their testing both better (using exploratory testing) and ways to manage it properly.

I am quite biased in challenging James’ tutorial because I wish I was the one with skills to pull off a tutorial like that. I have seen the traps and pitfalls at least partially already, but I think I need a different perspective to the tutorial altogether. I like to think about the scaling of the methods and also the managing of outsourced testing. How will the methods James purposes will scale across rigid and wide spread organization? How could I manage a team of tester in Kuala Lumpur according the principles? I would have to delve a bit deeper into his material. That would give me pointers on the terms and techniques he uses and possibly I could find some holes in his reasoning to sprout a fruitful conversation.

When I think about areas I’m willing to steal, the whole material could be worth stealing. The concept of managing exploratory testing using the Rapid approach gives me new tools and tricks to make our testing at F-secure more efficient and manageable.

The next step with this presentation would be almost the same as I described earlier. The approach needs to be implemented on a team to see where it might go wrong. In addition to this tutorial, one might benefit from different coaching and mentoring lessons so that the problems are found early and dealt with – almost like testing you test management.

On my Birdy scale, James’ “Rapid Test Management” would scale as follows:

• Person-to-person: ***
• Short time value: *
• Long time value: **
• Steal-ability: ***
• Challenge-ability: *
• Total: 10/15 stars

Paul Gerrand’s “How to Create a Test Strategy”

Paul is one of those people whose name pops up every here and there – he seems to be able to do everything. Personally I don’t know him or his work, but it seems it’s about time. The description at EuroSTAR page describes him to be attending lots of events so it’s clear why he’s a well-known person. It's hard for me to develop an understanding of Paul’s achievements on a personal level but after a few YouTube videos and some googling, I think he knows his way around testing. I may not agree with some of the stuff he speaks for, but I cannot put my finger on any specific topic. Hopefully I can get some challenging happening about hit EuroSTAR tutorial.

The first thing that comes to mind is a hint of worry – he’s using a template. I know from past experience that templates are seldom abused and used to hide incompetence and lack of interest. They look good, though. Is there something fór me in this tutorial that I could use in the near future? I wish I had taken this tutorial 5 years ago when I was struggling to tears with a testing strategy to cover a whole organization’s testing. I did do some short and efficient testing strategies on a project-by-project level, but the framing, all-encompassing strategy was a vague scribble that I loathed where other loved it – I knew I could do it better, but I didn’t have the motivation to really get into test strategies at that time (or the time to do it, I might add). As for “past” value, I would rate this really high, but now I don’t see too much value to my work. This could be one of those inspiring and challenging tutorials to attend but I don’t see short term value to me here.

How could I use test strategy workshop in my company? I’d have to say: in a punch of ways! The opportunities provided by honing the existing strategy and being able to make high and low level strategies would be helpful. The test strategy does push you towards organized testing and even give credence to ones testing if one has a strategy which to follow. On that aspect, I might even benefit from it.

I try to find something worth disagreeing or to challenge in all the tutorials, but in Paul’s case the answer is clear – templates. I loathe templates as base of documentation. They usually lose the meaning as a template and become “fill form and deliver” –documents. So I am eager to challenge template in every aspect I can. I would rather have a set of skills to enable me create my own template than a readymade one. Does Paul provide ways to create a skill set for that purpose?

I would like to be able to understand a bit more about the techniques Paul Gerrard uses to create the testing strategy. I guess that the only way is to attend the tutorial. For now I don’t see much else worth stealing than the concept of improving test strategy thinking. I’m not sure if this actually increases or decreases my willingness to join Paul’s tutorial, because I might be lacking in knowledge to actually make that decision.

Personally I would focus more on the skill set instead of the template. If he does encourage in a mindset change then that would be one of the topics I’d like to steal also. This tutorial would benefit from getting the test strategy to context and possibly the implementation to a testing project. I hope Paul has some concrete material on how the strategy implementation has worked in the past, if he has used it before.

On my Birdy scale, Paul Gerrard’s “How to Create A Test Strategy” would scale as follows:

• Person-to-person: *
• Short time value: **
• Long time value: *
• Steal-ability: **
• Challenge-ability: ***
• Total: 9/15 stars

Torbjörn Ryber’s “Boosting Your Test Design Powers”

I know Tobbe just a little. I have met him face-to-face, but we haven’t actually had a long conversation at that point. What lingers in my mind from our talk is a single phrase: “High-6” I hope I get a chance to talk to Tobbe at the conference at least once for he’s a great character. He has a tremendous knowledge in test design and critical thinking, besides he’s funny as hell! So I encourage all you, go and talk with him – you won’t get bored!

The tutorial seems, at a glance, to be run-of-the-mill presentation on test design. I have Tobbe’s book on test design and it is THE book for every tester! If you don’t have it, join the tutorial – you will receive a complimentary copy of the book “Essential Software Test Design”. Just like James Lyndsay’s tutorial, this one would be a fun thing to join as I know already something about the topic and to increase skills in test design cannot be harmful. Besides, ways to design testing with tools (i.e. charts, graphs, mind maps) would help me on my daily work. I do hope that we are able to get some hands-on experience on using the design techniques.

I have already promoted Tobbe’s book here at F-secure and I lend my book to one out Quality Engineers to get some advice on her work. I would like to have all our testers to join the tutorial as a wakeup call. As this is elementary for any tester out there, I encourage every pudding tester and developer to join the tutorial. For those that have been doing test planning for long time, this could act as a reminder of the good practices in test design.

Disagreeing with Tobbe’s topics can be difficult because I believe in the most parts. I do however see an opportunity to play devil’s advocate and challenge Tobbe and his claims. I do however see more beneficial to get into a debate on a separate occasion to let him bring out the most of the tutorial to people who need the knowledge. I’m not saying I don’t need more knowledge, but I will restrict myself from spoiling the fun from others.

I did a similar class few years ago where I taught test design on our testers and developers. My point of view was however a little more exploratory testing oriented and heuristic driven. I could try to steal some pointers from Tobbe to support my own material.  I could then arrange a workshop here, at out office, to spread the joy.

The next step after this tutorial would be a walk to the test lab and to use these skills in practice. In that sense the schedule of this tutorial is perfect. Later in the conference people could try their newly learned or reminded skills to test stuff. I think Tobbe would appreciate the feedback that practical use of those skills could bring. For example, what areas require more attention in the future?

On my Birdy scale, Torbjörn Ryber’s “Boosting Your Test Design Powers” would scale as follows:

• Person-to-person: **
• Short time value: **
• Long time value: **
• Steal-ability: **
• Challenge-ability: **
• Total: 10/15 stars

Matt Barcomb and Jim Holmes’ “Becoming A Testing Craftsman”

To be honest, I don’t remember any significant details about either of these guys. I seems that matt is a busy conference speaker (according to his blog), but I did not yet find the tone in which would be in harmony with my own thoughts about testing. I do see that he’s a coach so I could try to approach him via Skype and ask for some coaching and to chat about the conference topic. Jim also seems to be quite a veteran in conferences. I scrolled through his blog and the slides that he shares are excellent! I love the way to make things simple and bite-size. If the duo is anything like what I learned from them, I would definitely want to meet with them and talk about coaching, motivation, innovation and testing topics.

On short term, this tutorial is like candy to me. Little programming excersises? Sign me in! Building your tools? I’m there! I also see that these guys focus on skills to get things done – automation is the extension of human mind, not the only solution to testing. This is by far the most fitting tutorial for the first day for me and my development, short and medium term. I am getting into trouble at my work by some tedious manual repetition that could be made easier with a shell script or a python script. I hope these guys can brrring it!

When I think about how to educate and help my fellow testers, I’m not actually sure if I could bring enough to the table. I believe, though, that the attitude towards being a craftsman instead of rank-and-file- employee could be beneficial to both them individually and to the company as a whole.

I would like to learn more programming and tool building before I could confidently teach how to build them. There’s a hell of a group here who can make tools, apps, what-do-you-need to make testing easier. I’m willing to see the applications of tool building and the test data creation/management and possibly try to teach that to my fellows.

This is an interesting topic, because I have little experience in actually building something functional for others (for myself a few scratch-built scripts, but nothing serious). If I was to choose what I would like to hear, I would like to hear more about the attitude towards craftsmanship and how we can spread the word around. I believe that craftsmen aren’t supposed to hold the secrets to themselves but to share their wisdom, like Matt and Jim.

I’d like to see these teachings to be implemented on some practical hands-on session, test-lab or something, so we can really get our craft to shine. I would also promote testing to other than testers – I think managers, documenters and all software project stakeholders could benefit from craftsmanship attitude.

On my Birdy scale, Matt Barcomb and Jim Holmes’ “Becoming A Testing Craftsman” would scale as follows:

• Person-to-person: *
• Short time value: ***
• Long time value: **
• Steal-ability: *
• Challenge-ability: *
• Total: 8/15 stars

Afterword

I’m still teetering on which tutorial to join, but I will try my best to decide. I know I need to make decisions fast so I can book my seat before they run out. I will let you know which one I chose after I book the tutorial. As for now, I will keep on writing about the conference. I’ll try to cover every conference day at least on some level, but I do not guarantee anything.

As you might know I am speaking at EuroSTAR on a Wednesday. I would like to hear what you might get from that session if you are to attend it. I will also hold a preliminary practice talk here in Helsinki, so if you’re interested to join, give me a tweet. ?

You gotta fight for your right to test!

I am terribly sorry for the soon to be rant and biased output that is going to happen. I have did this once, but the conversation resulting from that was rather pleasant and constructive. This is a comment / response / rant about a blog page I stumbled upon today. You can find the original post here. The page may have been outdated since it was created in 2010 but it seems that it was updated half a year ago.

My first though was “not another test case” when I read the article. I’m currently trying to figure out what I will write to my next column on the “Testaus ja Laatu” –magazine, and since the theme is juxtaposing, I thought I’d try to write some blog-stuff first. One of the topics could be “test cases vs. no test cases” and I shy away from that strict way of thinking. Exploratory testing, to me, is utilizing all the available tools and gimmicks to get to the best results. Black and white –world vision narrows too much my take on testing.

Having said that, I will use my recently found taste to logic to point out what I found could be wrong in the article. I do not know the state of mind in which the article was written or the context in which it is supposed to be fitted, so I will project it to my own workplace where that is necessary.

You are warned…

The first sentence goes like this: “A significant factor in a successful testing effort is how well the tests are written.”  I have no idea how much significant is this case. I would guess that it is either the largest part or the one after that. When I do testing, the significance of the test cases are miniscule, and I tend to do successful testing. Not all the time, but I know a certain cases where properly written test cases did not contribute to the successful testing effort. Also I don’t exactly know what qualifies as successful testing effort. It could mean having run all the tests within the schedule (which doesn’t quality for successful for many reasons), having written all the test cases (do I need to say more about this), the product is shipped to end user (lots of products have been shipped to customers and later fixed due to poor quality), etc. So I would say that well written test cases /may/ be a factor in the successful testing effort just as well as poorly or not-at-all-written test cases.

“They have to be effective in verifying that approved requirements have been met AND the test cases themselves are understandable so the tester can run the test as intended by the test author.” 

So the test cases have to be effective in verifying requirements. Granted. Do the /have to be/ written. No. Even poorly written test cases can be effective at verifying requirements. I think the ability to verify something comes not from the writing of a test case but from the skill of a tester. If a tester is skilled there may not be need for any written test cases to get the job done. “--approved requirements have been met --” So anything that is not written to requirements doesn’t get tested? We also have expectations about the product that don’t really qualify as requirements. Testers still need be aware of anything that might threaten the quality of the product.

“The 3C’s: Clear, Concise, Complete. Test Cases are not open to multiple interpretations by qualified testers. Ambiguous or nonspecific Test Cases are difficult to manage.” 

What if we do not know about the product enough at the time we write the test case? Should we wait until the product is finished and then write the cases? Isn’t that just huge waste of time and money? And when it comes to managing test cases, I think lines of text are the easiest to manage, it’s the people that might require managing. It is true that estimating coverage and depth may be difficult if the test case is ambiguous. I also think that it is difficult to estimate those with good and precise test cases. People are best to estimate the depth of their testing and the confidence in their testing. ASK IF SOMETHING IS AMBIGUOUS. -> Fight ambiguity with openness and communication

“Test Cases are easily understood.  They provide just enough detail to enable testers familiar with the functional area under test to run the test case. Note:  Detailed click by click steps are only useful for automated tests.” 

This is something that I agree! Test case can be easily understood even if it’s like this: “Play around with the product and describe the person next to you what it does.” It is both easily to understand and you already have enough familiarity because the base requirement for that is none! This could be the baseline for any testing ever to be done at any context. FAMILIARISE YOURSELF WITH THE PRODUCT.-> Fight ignorance with eagerness.

“Test cases include setup requirements (environment & data), expected results, any dependencies on other tests and tracing to the requirements they validate. Are traced to the results of each test run, including defects (bugs) discovered, test platform run on, software build version used.” 

For the data and expected results, I recommend all you read Michel Bolton’s and James Bach’s conversation and decide yourself if the test case is complete with expected results. It is necessary to document stuff that mentioned in the text so to avoid unnecessary overlapping. But traceability to test cases? Is that possible for bugs that are found during the test case execution but outside the intended observation area? BE PREPARED FOR THE UNEXPETED RESULTS WITH UNEXPETED INPUTS. -> Fight patters with chaos and vice versa.

“Measurable: For each test case, there must be a way to objectively measure that the test case has either passed or failed. All test cases must be linked to the requirements that they verify to enable impact of changes made to the requirements or solution designs to be identified (measured).” 

OK. Let’s say that the test is executed perfectly and the result is exactly as expected. A minute after that the computer crashes. Does that constitute as failed? How much do we actually know about the product to say that a test has passed or finished. And if we want to find bugs, isn’t the test passed only if it finds bugs? I think test only fails if you don’t run the test, because it didn’t test anything. TEST IS ONLY FAILED IF IT IS NOT RUN. -> Fight unnecessary documentation with rightly timed planning. I.e. keep the time between planning and doing minimum. Preferably do them at the same time.

“The test case must have been approved, prior to being run, by the key stakeholders of the requirements that the test case verifies. Any changes made to test cases caused by requirements or solution designs changes must also be approved.” 

There is ABSOLUTELY NO POINT IN THIS! Why the hell do we need approving for our testing? Only thing that requires approving is the results of our testing. If the stakeholders do not trust us to test the software, we could record everything we do and ask them to audit the test material. I would think they are happier to audit actual results than worthless, constantly changing trivial documentation. IF WE NEED EVERY TEST APPROVED, WE LOSE MONEY AND TEST COVERAGE. -> Fight über control with proper* documentation.

“Realistic test cases DO: Verify how the product will be used or could be misused, eg positive and negative tests. Verify functions and features that implement approved product requirements. Can be run using the platforms and software configurations available in the test environment.
Realistic test cases DO NOT: Verify out of scope functions and features. Verify unapproved product requirements.”

I agree with the first sentence – a good test could test how the product could be used/misused. Also how it should/will be used. How much does the product actually differ from what the customer actually wants? I also agree with the second sentence, but it should be broader. It should test the features, platforms, integrations, data integrity, security, performance, etc. I don’t understand the third sentence for if we don’t have the proper environment or setup, we should acquire them in time. If we do have a scope (referring to the fourth sentence) then I agree that we should not spend too much time on out of scope elements. They could be mutated or broken due to changes made somewhere else so regression/smoke testing should be implemented to out of scope areas. If the fifth sentence I don’t understand what is meant by “unapproved”. Unapproved by who? By the testers, client, developers, managers? It really depends, so making a claim like that is just nonsense. A REALISTIC TESTCASE IS REALISTIC BUT FLEXIBLE. -> Fight rigid descriptions with autonomy, critical thinking and challenging.

“Test Cases must be able to be successfully completed within the project scope and schedule.” 

There is no point on writing test cases that are not run. But “must”? And why do they have to be “successfully run” and is a failed test a “successfully run” test? If a test finds three hundred bugs, is it successfully run if it doesn’t reach the expected result? WE CAN ONLY RUN AS MUCH TEST CASES. -> Fight the quantification of test cases with the amount of time spent doing testing. Count time, not test cases.



To summarize, I think the whole concept of SMART test cases is wrong excluding few things that I agreed with. I also encourage to keep the writing of test cases in minimum and the amount of testing done in maximum. Use the time wisely and appropriately! But if you do, you should consider these instead of the SMART way:

• Fight ambiguity with openness and communication
• Fight ignorance with eagerness.
• Fight patters with chaos and vice versa.
• Fight unnecessary documentation with rightly timed planning. I.e. keep the time between planning and doing minimum. Preferably do them at the same time.
• Fight über control with proper documentation.
• Fight rigid descriptions with autonomy, critical thinking and challenging.
• Fight the quantification of test cases with the amount of time spent doing testing. Count time, not test cases.

I will leave you to this. I promise I will be back with more about test cases (if I have the time). I’m not saying “don’t write test cases”, but use your head! It is not smart (bun intended) to follow some rigid procedure for all context, but to adapt to situation and to make most of the time you have to test.

- Peksi

* Proper: Appropriate as automated as possible. Including video recording, automated logs, scans from scribbles, session sheets, statements, etc. What is required to get enough information to the decision makers without hindering the job of a tester.

Testgasm

This is a blogpost to describe what was going on in the Rapid Software Testing class held by James Bach. This is not meant to be a comprehensive analysis on what happened or the lessons I learned but few highlights and hindsight analysis on what happened.

The dice game

...this is just a few of what we had...

Those that have been attending the dice game know that the initial exercise is hard. When one gets the idea on how the algorithm goes it’s easier to play variations on the game by group of friends or colleagues (not to say these are necessarily different). I (thanks to Michael Bolton’s course), Henri Hannuniemi and Sami Lehtonen (thanks to Antti Niittyviita) had played the game earlier and so we were grouped together and given a different algorithm than the others. I was so thrilled by the idea that I could use all the skills that I had learned to crack that nut open.

The game started and we were given a bunch of dice: a handful of regular, different colour/size dice; few D20 dice; few D10 hand drawn dice; a die that was in a transparent die; some “poker dice”, etc. We had so many variables that we became a bit confused. I started a list of different things that we could be analyzing. The size, the colour, dots or numbers, amount of dice, type of dice, arranging of dice, “zero is not zero” etc. We then arranged them so that there were different D6 coloured dice stacked together in different ways, five groups of five. James came to the table, we said “2” as a guess for every single dice pile and James replied “0”.

We had the miscellaneous dice lying in the stack nearby and I thought I ask a reply on that also. I said it “2” also and James replied “1”. A one? We had a huge number of different types of dice lying there. What could it be?

We proceeded to arrange the dice in a fashion that there were 2 or 3 special dice and regular dice bundled up so that all special dice were in use. All but one group ended up in a “0”. The one that was the “1” had the die where there was a dice within a dice. A frantic math calculation ensued. We tried to sum up the two dices together, but the result was always a “1”. So I took the die in hand and turned it so that James could see only “1” and a “2”, and we thought it was a “1”. The answer was “2”. Frustration!

Testgasm

We then analyzed the steps we took to get to that “2”. We had a couple of theories, but when the die was on the table, it was always a “1”. Heureka! We took five dice off the table and said “5”. And a five it was!

The thought process took 3 persons 20-25 minutes. The sparring between the team enabled us to try different even crazy-sounding ideas to good extent without exhausting our innovation. We simplified the data and made it more complex. We tried to look for changes in the output by varying the input. We used the different models of problem solving from our lives to figure out the pattern. We also solved a pattern composing of vocal input pattern (I save that for one of my exercise in the company) and one with a difficult mathematical pattern composing of differently grouped dice.

"Give me your hardest problem!"

Like James said, we were doing testing Kung-fu! Solving problems like matrial artists! Bonk! Blam! Thwaak! Zlott! Swoorsh! Phatam! Ka-pow! Like Batman fighting crime, we fought problems! (I’m getting a bit excited here, if you can’t read it from between the lines.) We all felt like we were invincible and we had the tools to crack every problem in the world! “Give me your hardest problem and I will solve it for you!”

The high after testgasm

After we got all the patterns solved we were all ecstatic about what we had done. We were in a problem-solving high! The word to describe the feeling would be “testgasm”. And truth be told, after a serious testing session where you find something awesome, something really important; you will get a testgasm. I’m not sure if other that testers understand the rush after a successful testing session; it is hard to explain. The joy of completing a hard task and feeling joyous about it in ways you never thought you could be. That feeling is something that testers all around are looking for when they test. Your face may light up and you yell “Yes! I got it!” and other people in the cubicle stare at you like you’re deranged.

Focus/Defocus

I had heard the concept of focusing/defocusing in Michael Bolton’s class before but I never truly understood the meaning behind it. ("If focusing is focusing, defocusing is the opposite of that") I tried to look for material online, but it was all vague and didn’t make an impact. The way that James described the method was simple: “When confused, focus; when frustrated, defocus.” Wow! When testing one sometimes loses momentum and struggles with a single piece of data for long time. Using input patterns that don’t find the problem may lead into frustration. I felt exactly that during an exercise about systematic testing.

James had us testing a piece of software that had a bug in it. We were asked to find the bug and then try to figure out why it fails. The input was a valid IP address. You know what I figured out? I have so strong built in mental models about meaningful number combinations that I was grinding my teeth and sweating to break that pattern. I had tried a pattern by testing the high and low numbers, duplicate numbers, you name it. Could I just be blind to the bug? Then James said “Look into your data. Can you see a pattern?” Yes I did. And lots of them! “Now try to find an input that is as different as possible from previous data.” And guess what IP address I used? “1.2.3.4” That’s right! I broke my pattern by trying “1.2.3.4”! What was I thinking?!?!

James was like “Dude! What the hell?” and I sat there frustrated and confused. Then I realized what he had meant. If I had drawn a line to represent my test data, it would have been like this:

I just needed to break the pattern and be more random and use data from between my clustered data pattern. And with first random IP address I put in, I found the bug. Later James explained why the program behaves like that, but I can’t remember the true root-cause of the bug, but I did learn a lot about focusing and defocusing.

Hung over (from learning)

It has now been two weeks since the course. After the class I was in a high that lasted through the weekend. I thought about giving a thorough analysis about the course but I see no need for that. I had the time to let the learning infuse to my spine and now I feel that the stuff I learned make even more difference. I did have an information hangover after the class and it was hard to get back to grips with non-testing work again. I was glad, however, that I could attend the Intensive course the next week so I wasn’t that bummed.

I recommend the class for everyone willing to improve their critical thinking, testing skills and/or argumentation skills. The exercises were good and to the point. The hot seat treatment James gave to some of us gave us experience to stand pressure and perform well when in a stressful situation. I also learned a lot about myself and the way I learn. It was also cool to hang out with great testers like Samuli Elomaa and Aleksis Tulonen.

I still have a long way to go and a lot of learning to do. (After James mentioned it) I began to think myself as a constant student, always learning. I promise James (and I will talk about this to Mr. Bolton) that I will try to compare the classes done by him and Michael. That way both of them could learn what could be done better or differently. That however will have to wait a few days (i.e. weeks) as I have a growing backlog of blog posts.

Are you part of the faceless masses?

I was on my way home the other day when I happened to talk to a woman who works in HR in some consultancy company. We were talking about how to get oneself hired to a company and how to be one step ahead of the other applicants. With recent discussions with James Bach in my pocket I became challenging her claims about employing oneself and best practices thereof.

What she suggested was that by certifying one can have an advantage in the job market. If one is not certified there is no knowing what the person can or can’t do, right? Certification is the first gateway to rule out incompetent testers, right? I had some thoughts about this prior to talking to James but some of the arguments that I had been using were lacking momentum. James gave me some golden thoughts in how to be one step ahead of the cattle of unskilled certified testers (I believe that if skilled they will see that the certification will not get them employed).

Certified terminology

Certification by name certifies a certain characteristic of a person, object or organization. It is not comprehensive but directed to a specific area. It can be achieved by passing an audit, an examination, a course of some kind, to name a few. Certification is not to certify the skill of doing something but the knowledge of some fashion. It can be of terminology, syntax, operating system details, etc. None of these, however, point toward the skill of doing something.

The certification where you get certified by defining a set of terms is quite usual. You get the information from books and then you take the test, usually multiple-choice examination to ease the assessment process. You can basically get a certification by guessing the right row of answers, having a list of those answers achieved by cheating, or by memorizing a set of terms. Because the test doesn’t aim to assess the understanding of those terms the value of it is quite minimal.

As terminology changes from company to company, from context to context, we do not need to memorize a set of terms and apply them by force. We need to understand the meaning behind the terms we use and to explain them. Then, if conflicting with the other terms within that context, the use of terms must be adjusted. If you say “testing” and mean “systematic evaluation of the quality of the system using various techniques applicable to the context”, and the organization uses “uggabugga” for that same meaning and “testing” for coffee tasting, it might be good to adjust the use of terminology for that context. If you just keep using the term taught to you by some book somewhere, you will drive yourself unto a cul-de-sac of misunderstanding.

Proving skills and standing out

The testing certification provided by ISTQB is a good example on how certify a tester without any skills to actually test. It is a book examination, essentially. More so, there is a course aiming to pass the test. I’m not going to talk about the syllabus at all, but about the skills taught at the course, or the lack thereof. The certification doesn’t aim in proving skills of the tester. A tester with no experience in testing can go to the certification examination and pass before doing any testing at all.

A recruiter going through applications for a job look for something that might look for certifications as a proof of skill. When confronted by a certification that by nature does not measure skills but the knowledge of terminology (which obviously is the wrong way to go) the recruiter might confuse the person having skills. This is not to say that the applicant doesn’t have skills, they might very well have a huge set of skills, but they are not the characteristics that get you to the interview.

“If you don’t get to the interview then you cannot show your skills. That is why I use the certification to get to interviews!” I hear you. Read more.

Rising over the masses

The recruiters have a hard decision in determining who to call to interviews and who not. They look for something that stands out! You might think that a certification gets you the interview. Stop there for a moment. How many other applicants you think do the same thing? How many of the 500 applicants to a Test Engineer job have the exact thought of having a certification as a ticket to interview? 50? 100? 450? This obviously depends, but think this: The certification makes you part of a mass of people. By using a generic way to stand out, you’re “massifying” yourself – you become part of a gray mass that doesn’t stand out in any way.

Like I asked in the beginning, “Certification is the first gateway to rule out incompetent testers, right?” it actually is so. But it not a gateway only to rule out incompetent testers, it is there to rule out incompetent recruiters. A person without proper testing knowledge, skills and passion does not know how to recruit a good tester. He may know a little and thus relies on the magic of certification, at least a little. Even the most unqualified recruiter is looking for SOMETHING to make the call who to call to the interview. By having 90% of the applications look the same, they have a hard time deciding. The gateway works so that it rules out the certified testers and leaves those that have the spark, the passion and the skill. Do you want to be the one getting the interview or be ruled out because “you don’t have the spark”?

First thing recruiters see is the application. We are force-fed the template from recruitment agencies and we use that. We are afraid to be different and difference is what you should be aiming for! Instead of creating a dull list of what you can do and schools you’ve been to, do something else! Write a bug report where you describe your behavior and characteristics. Do an interview of yourself and post it like a newspaper column. Send them a video of you testing a product while explaining what you’re doing. I could go on and on! Be outrageous, but professional. The point is to make a statement. Send a filled template and you’re doomed.

Conclusions

When I had explained myself to the HR person on my way home, she said: “Well you’re obviously a guru, so you don’t need a certification.” That is not correct, although I briefly basked in admiration. I’m not a guru any more than the next guy. I have passion and I’m not afraid to show it! I have goals and I’m not afraid to tell them! I have a hard built reputation as a tester and I'm not afraid to promote it! (Maaret might say “He’s cute when he rambles.”)

Every single tester can be a professional, and they should. Every single tester can apply to a place where they want to work and get employed, and they should. Be ambitious, be excited, be passionate. And show it. Don’t fall into marketing trap and be part of the mass – be you!

And get refining that CV right now!