Test Strategy in 10 minutes

This post covers both the workshop I did on test strategy at Solita as a part of the Testing Tuesday and the extempore workshop I arranged at the Nordic Testing Days in 2015. While both had the same agenda, they were vastly different. (Why this post is published now is because it had converted into a draft which I noticed only now.)

The one at Solita

The workshop was not supposed to be a slide show, like none of the workshops arranged at Testing Tuesday. Once again I drew stuff on the white board and let rip. I started by describing testing strategy quickly. Then we discussed about project elements, product elements and quality aspects. These were loosely based on the Rapid Software Testing models. The audience gave ideas on what kind of elements we need to take into account when choosing a test strategy. There is no comprehensive list on what we came up with, but we had a whole lot of ideas ranging from schedules to "pissed-offness" and expected user behavior.

After we covered most of the areas, I split the audience into three groups. They all were supposed to create a testing strategy in ten minutes. The product I chose was a web shop that sells knick-knacks and mails them to people. After ten minutes I had three completely different strategies.

The first one was focused solely on money. They mapped out every aspect that could hinder the income of revenue and prioritized them according to importance to the "owner" of the store.

The second one was a "software engineer approach". That group mapped out all the aspects that were important to different kind of engineers. There was an aspect of security testing, transaction testing, happy day testing of known processes and some complementary approaches. There was no prioritization but there was definite focus on tools and know practices.

The third one was focused on business processes. They mapped out as many user stories as they could and dissected them to steps. They then tried to figure out what kind of a prioritization would make sense to test the processes.

In 10 minutes we had three outstanding drafts of testing strategies. All of them had different approaches and they the strategy supported the focus they had chosen. They were not comprehensive, nor should they have been, but they were "good enough" to start testing the most important thing as soon as possible. We discussed the fact that when combined these three could actually complement each other. In a coffee break, we can create a draft of a strategy and even introduce that draft to a potential customer to explain our strategy to test their product.

All in all, the workshop was a success, since it spawned a new way to think testing strategy. I never thought I could choose a simple focus like "money" as the guideline of my testing. It does reveal the importance of talking to stakeholders to understand their values and needs. Without any templates or predetermined practice every team could conjure an awesome strategy which seemed even executable after they explained what they thought.

The one at Tallinn

Feeling confident on the outcomes of the workshop, I agreed with Helena Jeret-Mäe that I would do an extempore workshop during a coffee break of the Nordic Testing Days (see here). The workshop was supposed to be on the last day. I dragged a white board to the hallway near sofas and gathered people to do the exercise. I was able to gather a couple great minds of software testing on the sofa including Erik Brickarp and Santosh Tuppad. The workshop started by me explaining the idea and then I gave the same task to them: "Generate a test strategy in 10 minutes."

The problem was that the pro testers weren't too happy with the vagueness of the context. They wanted to know more about the product, more about the stakeholders, more about the project. In the end we didn't actually generate a context map for the product (plus some testing strategy elements).

The best outcome for me was (once again) to observe experts in their work and how the dynamics within a group actually work. Also the lack of structure was something they pointed out. Erik mentioned that we could have described four key aspects of the product and then figure out how to test those. Where the first three groups I had at my office all chose a focus, we had none. We could have spent a few minutes in the beginning to define where we would like to focus (even an arbitrary focus) and then create a strategy based on that. There were hints of focus surfacing when they interviewed me about the product, but none that was chosen as the key thread.

What next?

Having had two vastly different workshops on the same subject, I think it makes sense to arrange these even more. The original idea to this came from Fiona Charles at EuroSTAR 2013, but I think we approached it quite differently. I am planning to do these extempore workshops at every conference I attend, since it makes sense to give people a chance to try their skills at this. Every time I get a huge amount of pointers, ideas and lessons learned. I would think that it is the conversation that ensues rather than actually creating a strategy that counts.

The key lessons learned in both sessions were:
- The dynamics in the group determine a lot what kind of strategy is created
- Know your stakeholders!
- Choose a focus as a skeleton and then fatten it up
- Don't over-do it! Ten minutes might be enough to start testing the most important thing.
- Have some structure, but keep ideas flowing

When you see me at a conference, come ask when am I gonna pop out the white board. It might be the next coffee break. ;)

- Peksi

Nordic Testing Days 2015 – Conference at a glance

Just to keep the momentum going, I shall tackle the Nordic Testing Days 2015 in a similar manner I did the Let’s Test. I chose the two conferences for they’re close by and I wish I could attend either or both of them this year. The real reason is that I want to keep writing, now that I have momentum. This is

Nordic Testing Days 2015 – Conference at a glance

The Nordic Testing Days 2015 is the fourth of its kind. Having had the privilege to attend the first of its kind as a speaker, I have a special kind of attachment to it. I did a full blown evaluation of the sessions I attended while I was at the conference 2012, but I shan’t do it this time. The conference is a 3 day spectacle with tracks, tutorials, workshops and more. I’ll choose the sessions as follows:

• Choose one session from each day based on my familiarity of the speaker
• Choose one session from each day based on my interest in the title
• Choose one session from each day that I pick randomly

Those should total 9 sessions. I’ll choose 2 random key notes to accompany those.

I will use a heuristic grading system (introduced here) to determine what would be the best session for me. I will grade the stuff with Angry Birds ™ grade – 0-3 stars per area – on five areas:

• Person-to-person (How will the person and his/hers work affect/inspire me or the people I know?), 
• Session value – short time span (How much can I get out of the session tomorrow – next year?), 
• Session value – longs time span (How much can I implement to my work and teach to my colleagues, my community?), 
• Steal-ability (How much of it am I willing to borrow and further develop to make it better and/or mine?), and 
• Challenge-ability (My past knowledge on the topic and my willingness to challenge the session contents.)

Keynotes

Mart Noorma’s “ESTCube-1: Testing in Space”

Intergalactic journey ahead. I would pay money to contribute to something that eventually goes orbiting the Earth. Alas, I cannot yet. Soon, perhaps. I know next to nothing Estonia’s space program, but I think this keynote requires some background checks to be able to get everything out of the keynote. Since I followed Philae landing (they had some Finnish technology there also) I am keen on hearing more on the subject.

I must admit I haven’t heard of Mart Noorma, but I think he’s not that loud on the testing scene. What are the key values here might be the inspiration to reach the stars. The short time value might be high-ish but I cannot see too much long term value in this. I might be the wrong crowd for this session, but I’m expecting inspiration and insight from Mart Koorma.

Although this keynote might be hugely inspirational, I see very little in the light of challenging or stealing ideas. It’s a shame, actually, for I am an avid science follower. The thing is that I might be expecting more of a testing approach to the keynote and less of a technical story. Experience report working in difficult situations is always good, but I don’t see myself as the optimal audience for this.

• Person-to-person: 
• Short time value: **
• Long time value: *
• Steal-ability: 
• Challenge-ability: 
• Total: 3/15 stars

Rob Lambert’s “Why Remaining Relevant Is So Important”

Why is it important to stay relevant? I mean, Rob is obviously going to answer that, but why make a keynote of it? Don’t we all know, if we fossilize we are, out in the next round of layoffs? What is the big deal? What I feel Rob is trying to say is we need an attitude change. The relevance comes from want to thrive and be the best. If you’re the one who’s always on the cutting edge of technology, skills and thoughts, other people want to be like you! You become the beacon people look up to.

Rob Lambert is one of the most influential people in the testing scene. His blog was one of the first ones I started to read as a budding tester. I have met him once in person, and he’s a warm, easily approachable character. The problem is that should I have more time on my hands, I’d be more keen on approaching him with my ideas on managing testing. Alas, I have not.

Staying relevant has far-reaching influence. It brings high long time value to the company and to myself. The ideas sound like easily adaptable and with genuine examples the value might become even greater. Short time value might be in form of planning ahead my skillset. With this session and Alexandra Casapu’s “Examine Your Testing Skills” session at Let’s Test 2015, I see no reason why one couldn’t stay relevant to their company or their community.

• Person-to-person: **
• Short time value: **
• Long time value: ***
• Steal-ability: ***
• Challenge-ability: *
• Total: 11/15 stars

Wednesday

Kristoffer Nordström’s “Taming the Terminal-based Applications and Testing Them” (based on familiarity to speaker)

Kristoffer was the guy that inspired me into taking my Pythonian skills forward. His lessons in “Python for testers” and many more inspire many. He’s a great sport and I wish I could attend his tutorial at the conference. If the session has anything close to what I expect off of him, people will be having a hoot!

After reading the description, the tutorial seems quite useful to me. When I was doing testing at F-secure I ran into terminal-based application every once in a while, I might even have created some tooling with Python. This tutorial strikes that particular nail in my skill repertoire.

The values right now are mediocre, however, since my current job description doesn’t touch terminal based stuff. I would have rated this very differently a year ago, I must admit. Also being one of the few testing specialist at the office, I think I wouldn’t be the guy to be teaching this to developers. I could benefit from having a better understanding of testing frameworks and tools, better confidence in my skills, and have a good time at the session. Also, since I have some experience, I could challenge Kristoffer to make him hone his material to perfection. ;)

• Person-to-person: ***
• Short time value: **
• Long time value: **
• Steal-ability: *
• Challenge-ability: **
• Total: 10/15 stars

Kristjan Uba’s  “Let's Learn: Experience Learning through Gaming” (based on interest in the title)

Gaming. My favorite way of learning. If we’re going to play games and learn from testing, I think this session is worth its weight in gold! …at least to a procrastinator and a child-minded person like me. The values are both immediate and long lasting, if done properly: one starts to seek out the games mentioned to play with their friends and colleagues, besides those games can spawn entire new epiphanies on some other testing related area.

I don’t Kristjan Uba from before, but I bet he’s the kinda guy I would get along really well. He sounds enthusiastic, innovative and funny, the kind of a person I like spending time outside work with. Perhaps, should I miss the opportunity to join, I can badger him to play some of the games on some other occasion.

• Person-to-person: *
• Short time value: ***
• Long time value: **
• Steal-ability: ***
• Challenge-ability: *
• Total: 10/15 stars

Robert Sabourin’s “Just-In-Time Software Testing” (random pick)

Ok. Rob has been on my radar from the beginning of my testing career, yet I know next to nothing of him. He’s one of those “one-star-should-be-three-stars” kinda fellas. I am aware of this “just-in-time” method, from some blog post in the past.

The description made me hum in pleasure. That is something, not only I want, but I need. As a professional tester I need to be able to make snap decisions about prioritization, change of focus and moving people to test the right thing. With content like that on a tutorial, I see no point to sit this one out! The values of this sessions are far reaching and immediate! These are the things I must educate my colleagues with, my community should be aware of this, and my work would vastly benefit from the skills and knowledge this tutorial gives.

For I know quite little about the subject to begin with, I see a lack of challenge-ability for me. Maybe a quick 1-on-1 with Rob might get me into the mood. Perhaps a blog post or two to limber my mind...

• Person-to-person: * (I wanted it to be ***)
• Short time value: ***
• Long time value: ***
• Steal-ability: ***
• Challenge-ability: *
• Total: 11/15 stars

Thursday

Sami Söderblom’s “If James Bach and Mary Gorman had a baby, how would it test?” (based on familiarity to speaker)

Mr. Happy Monkey himself talking about… WHAT? Biology? Child birth? Intercourse? I have to say if he hadn’t been picked based on me knowing him, I would have chosen the session based on the title. Sami Söderblom is a good friend of mine. He’s a whiskey-junkey, a cat-photographer, a father, an explorer and a good friend. We have a history through my whole testing career, from my first big testing project to this day. He’s the “three-stars-should-be-the-milkyway” kinda fella.

I must say I’m on pins and needles what the session is about. The description says: “FITCODES”. I’m sold. SFDPOT has been my guideline through my recent testing career. At the Turku Agile Days 2012 I modeled the testing of speedos using the SFDPOT. This week we did testing exercise on testing whatever found in one’s pocket using SFDPOT. To advance the heuristic that has been the lifeline for me is something I really, REALLY, would like to see.

The values, for me, are huge. Should I be able to use the heuristic in my everyday work is a great benefit. In addition to this, to be able to help others test their software better, to design better tests, to manage testing in a better way, make the session even more valuable. To learn how Sami came up with the heuristic is a good steal-able. To refine it to suit my particular needs would be awesome. And, since I know quite a lot about the subject, challenging would be the cherry on the cake.

• Person-to-person: ***
• Short time value: ***
• Long time value: ***
• Steal-ability: ***
• Challenge-ability: ***
• Total: 15/15 stars

Beat that! No pressure, Sami. ;)

Stephen Janaway’s “Why I Lost My Job as a Test Manager and what I Learnt as a Result” (based on interest in the title)

Test coach, he says. I think I like him already. The transformation to Test Coach has been my goal during this year. Teaching people on how to test and make them better at what they do. If Stephen can help me achieve that, I’d be happy as a hippo.

I actually don’t know Stephen from the past, but with his attitude towards coaching, I bet we can hit it on. Value coming out of his “shift to coaching” session could become quite valuable for me with my ambitions and goals, but to all traditional test managers. When I came to my current work place, I told I wanted to be a coach, but I have yet to find my focus and methods in implementing it.

To more easily understand Stephen’s session, I feel I must read the blog post first. Maybe then I can be able to challenge him in a better way.

• Person-to-person: *
• Short time value: **
• Long time value: ***
• Steal-ability: **
• Challenge-ability: *
• Total: 9/15 stars

Erik Boelen’s “Acceptance Testing At Its Best” (random pick)

I’ve heard some experience reports on the subject of coaching end-users to test the acceptance of a product. They have all been educational, but seemed to be lacking some punch - the methods to implement the procedure to one’s own context. I have never heard of the speaker before, but I fresh blood to the Testing Arena is always a crowd pleaser. ;)

This seems like a good session for those battling with limited testing resources and acceptance testing stuff. I see a lot of material I would like clarification to and some areas where challenging might be in order. Values are unpredictable here. Since I work closely to projects and the project management nowadays, I see some intersections to my work. This might be a huge value to a test manager of any kind. I cannot say.

• Person-to-person: *
• Short time value: **
• Long time value: *
• Steal-ability: *
• Challenge-ability: **
• Total:  7/15 stars

Friday

Erik Brickarp’s “Going Exploratory” (based on familiarity to speaker)

Erik. My man! He’s been around for a while. I was introduced to him by a hint from James Bach few years ago for paying the RST course from his own pocket. Anyone to do that is a personal hero of mine. He’s a great thinker, a fine coachee – fine and dandy bloke on all fronts. I’m really excited to see him in person since he has evaded me the few times I’ve attended conferences in the past.

So, Erik’s gonna talk about how he switched from rigid testing process into an exploratory one, failed, learned, tried again, repeated, succeeded. This is something that I want to do! I think my key takeaway is the sandboxing. In my recent project I have drastically changed the process with constant deliverables. This means more freedom in the execution but rigid documentation. I think Erik can give me a couple of good tips how to make the process less painful and more appealing to… the client. (You wer thinking I was gonna say “opposite sex”, weren’t you?)

I definititely see value in this. Short time value comes mainly from the insights that I can implement as soon I hit the desk after the conference and long term effects can sprout an inspiration where I combine my learning to what Erik gives. Very valuable in deed.

• Person-to-person: ***
• Short time value: ***
• Long time value: **
• Steal-ability: ***
• Challenge-ability: **
• Total:  13/15 stars

Radomir Sebek’s "You don't need to be a musician to test music production software" (based on interest in the title)

Music is close to my heart. I compose various kinds of music, hence I have an interest in both the industry and the tools of trade. Combine music and testing – I’m hooked. Although I have never heard of Mr. Sebek, I am keen on hearing what he has to say.

The whole concept is intriguing, having to quickly learn a vast domain to better test it. I think that is the core of software testing in general – fast learning, adapting, moving focus and prioritization based on learning. I want to examine his methods of approaching the subject. The coaching aspect (as in using testers with various backgrounds and influencing them) is also interesting. I am intrigued what kind of methods have been used in the influencing. Experience reports like this are usually difficult to challenge, but usually highly steal-able.

• Person-to-person: *
• Short time value: **
• Long time value: **
• Steal-ability: ***
• Challenge-ability: **
• Total:  10/15 stars

Ilari Henrik Aegerter & Ben Kelly’s “Ben and Ilari's Spectacular Testing Circus” (random pick, honestly!)

Like I mentioned in my earlier post, I have spent some time chatting with Ilari. He has coached me on different things. Ben Kelly has been on my radar, like many testers, but I haven’t yet figured him out. It seems I am compelled to read his blog a few times before I go chat with him.

The session itself is a puzzle (pun intended) since it can contain many things. Interactive games, puzzles, cool problems, etc. are the salt of testing skills. I bet this is a session where I shall spend at least an hour or two, since I just like to challenge myself. I expect to be challenged and to be able to challenge other testers and maybe increase some skills while having fun.

• Person-to-person: ***
• Short time value: ***
• Long time value: **
• Steal-ability: ***
• Challenge-ability: **
• Total:  13/15 stars

Conclusion

There are many sessions that I didn’t tackle although they might be worth gold. If you feel (as a speaker) that I should tackle yours, drop me a line. I’m also eager to discuss my choices with the people I rated.

It seems that I am attending the Nordic Testing days. I shall be there with my beard flowing and throwing #high6’es to people. ;)


Varsti näeme!
- Peksi

Reading Practice

I’m reading currently a book called ”How to read a book?” by Mortimer J Adler and Charles van Doren. I have just started it but I talked about it in the office the other day. One of my colleagues said that one way to read a book is to first read the table of contents and the read the first page of every chapter. This post is an experience report on what can I learn by using a different reading strategy. This blog post will act also as platform for my thoughts when I read my target book. It was inspired by Erik Brickarp's blog post on taking notes.

The first pages of  “How to read a book?” book states that there are two ways to learn from books: to illicit information, and to understand the book.  My goal here is to run this exercise to illicit information. I choose a book called “Being logical” by D.Q. McInerny as my target book.

Preface

I decided to read the preface to be more in tune with what is going to be written about the subject. Forewords usually try to tell the reason behind why the book is written and to inspire the reader to go on. So when I got to the end of page 1, I decided to go on.

I had read the first paragraph of the book earlier (doesn’t count as cheating) and didn’t continue because of eldritch terms and words. Having read the preface, I the realized they will be explained. The book says it will concentrate to leave as little room for assumptions as possible thus trying to be explicit rather than implicit. Also it is not a text book for a class but a practical guide. This made me interested even more.

The preface addresses the structure of the book. It seems to be in five parts; each part building on the previous one. I’m interested if the parts contain what they were supposed to contain. The first part is told to be preparatory, the second one to lay the foundations, the third one to the meat and potatoes of thinking logically, fourth part discusses attitudes and fifth discusses fallacies. I know that I am fallacious, even more than regular people, so it is interesting to see what can I learn about myself while reading.

The table of contents

First thing that came to mind was that never before have I really read through the table of contents. This is the first time I really look into them. I usually skip the pages altogether to get cracking with the reading. Truth be told, I was quite excited to see how the table of contents was structured. It felt like a story was already told with the items on the list.

I pretty soon realized that every chapter had 8-30 sub chapters. Each chapter was about 1-5 pages long. I was starting to question my decision to read only first page from each chapter. For most of the sub chapters it would get the whole chapter done already. So I decided to read only the first *subchapter* of each chapter. They were, according to the table of content, almost always the introduction or foundation for the chapter.

Also I noticed that as for this book, it makes a great checklist for practicing critical thinking. By looking into the titles of each sub chapter, you would get a coherent guide to practice logical thinking. I could create a powerful mindmap out of those and print it on my cubicle wall. (And I will share it with you! Tadaa!)

Feel free to print this also!

Also this was a great stage to look for words and phrases you don’t understand that are essential to the understanding the content of the chapter. “Syllogistic”, “agnosticism”, “antecedent”, “equivocation”, “Post Hoc Ergo Propter Hoc” and “expediency” were new to me. Now that I checked them, I know better.

1st Part – Preparing the Mind for the Logic

I tackled it with my usual gusto; get cracking with it and read it again if I need to. And again I needed to read it twice for me to understand the message. I think I shouldn’t as this type of reading is just to get information. Sadly I didn’t /read/ it, I skimmed it. And after the second reading, I realized that the first sub chapter was exactly about that: Paying attention. And I had to pay, for it took my time and time is money. The book encourages you to listen instead of hearing, to see instead of looking. It gives great insight to the foundation of logical behavior.

The table of contents told that topics like “Effective Communication” and “Truth” were in latter parts of the chapter. This is the problem of this kind of quick reading, I see interesting stuff and I don’t want to stop reading. But I must refrain from reading more as I was to read this thing quickly, so on to the next chapter.

2nd Part – The Basic Principle of Logic

At this point, the technique I chose was beginning to bother me. I did read it fast, but this was only a glimpse of what it could have been. The principles could have been more thoroughly read but I chose to stick to my plan. The 1st subchapter was about the first principle of logic. Before the 1st subchapter was a brief piece of text regarding logic as three separate but linked entities; as science, as art and as skill. This did raise some questions, but I think they will be answered if I read the book with thought.

The first subchapter tries to focus on practical side of things. The theory will be discussed elsewhere, but I was intrigued by the the notion of reading even more about the theory of logic. This subchapter proved some examples which clarify the link between logic and human reasoning. Again I was tempted to read on, but managed to jump to the next chapter.

3rd part – Argument: The Language of Logic

The subchapter was about founding an argument. I did have some experience in the practical side of forming an argument, as I had been practicing that with Michael Bolton on Skype, so I knew the elements to build an argument. Those were “premise” and “conclusion”. With the help of examples the book showed some good uses of supporting and supported arguments, giving me a good ground to build upon – in fact just like the preface promised.

As this on itself might have been a good starting point to a person not knowing anything about logic and was just seeking a quick 15 minutes intense session on logic, this only made me hungrier to read through the whole chapter. I noticed that, if not the writing style or the content, my eagerness to read on was huge. I had to struggle every time to quit reading further.

4th part – The Sources of Illogical Thinking

I was a bit taken aback with the concept of writing about things that made you illogical. When writing about negative stuff some might absorb the “bad habits” instead of seeing them as negative. After reading the subchapter, I felt that it only made me stronger thinker and I was trying to spot those weaknesses – you might say - and try to find ways to mitigate them.

Skepticism was the topic of the first subchapter and it was really informative. It gave insight on selective skepticism and to behavioral skepticism – and gave some concrete examples. I began to think immediately about exercises I could to using skepticism in my coaching. I may have already done that but unintentionally. Not I could be even better at forming an argument. The book made some examples about damaging skepticism but tried to enforce the power of healthy doubt – doubt as a catalyst for learning.

5th part – The Principal Forms of Illogical Thinking

This 5th part was about fallacies. They were explained to be the typical patters people do/use to act illogically and to build illogical arguments. Usually fallacies appeal to emotions so they are more powerful than sound logic. Again by recognizing the patterns I might be better at building an argument that is based on sound logic.

The lure of using a fallacious argument is high, because it might be quicker and more effective than using sound logic. Even though you’re right and have a sound logic behind the conclusion, basing that on fallacious argument is almost never a good thing – it might bite you back when you least expect it. I had read something about the fallacies so this area in particular was really tempting.
Afterword
The afterword was a good reflection through the book. They stressed the concentration when arguing so you focus on using your logic in building an argument.  I rarely read the afterword as I see no input to the meat and potatoes of the text. I did realize that the afterword did provide a fresh view to the book as a whole. It might even be effective to read the afterword /before/ starting to read the book itself as it can provide a fresh view to the book before trying to tackle the reading.

About the exercise

After having done the exercise, I feel it was a good practice. I might use it again but with a twist – I will write up questions about the first page/subchapter to a piece of paper and try to answer those questions as I read on. In that way not only will I illicit information, I will begin to understand the meaning of things.

The exercise was good fun as the text was close to my liking – it wasn’t a textbook but a guide, the topic was near to me, and it was short enough – so the exercise was a bit easy. It was good for a first experiment to this kind of reading, but I need to make this process more efficient. I might even learn more about this type of “stub skimming” (if there is no word for this type of reading, let it be that).

So right now the process goes like this:

1. Choose a book to your liking (any book will do)
2. Read the table of content
3. Read the preface
4. Read the first page of every chapter
5. Read the afterword

On every step, make notes and questions about the content
If there are fundamental questions about the content, read on and find the answer.

Thoughts

This is just one way to read. This is more like giving an indication of what the book is about. This is not used to understand to book but to have some knowledge of what there might be if you’d read on. This is a good way to tackle a book on before you read it thoroughly, because you already have questions about the content. On the time you re will systematically read it through (or parts of it) might be significantly faster and make it easier to understand.

Now I need to continue reading the book about reading a book. I think I will use this technique to give me a head start.