Responce to "Oy you tester!" By ElizaF

I was strolling casually in the Twitter-land and I saw this link. It was a blog written by ElizaF (maybe that's her real name, but who knows). She had a post that was a bombardment of questions. It got me thinking and suddenly I started to write up the answers to those questions. Here they are:

What do you think when you see people talking about spending three weeks writing tightly worded test cases before they start testing a product?

• What would you think of a person refusing to do it? It there is pressure from organization is it worth your employment to refuse? Testing based on strict test cases forces you to forgo your creativity (unless exploration is allowed or better yet encouraged). But it’s not what I think when I see that kind of a thing (and boy do I see that!) but what I do. I speak up and try to my best ability steer the testing process towards enoughness, quickness, agility, exploration -> TESTING! Away from checking, towards testing.

What do you say when you read of people describing their work environments as Agile with elements of Waterfall?

• It can be agile-ish, but there no agility in waterfall! This just means that a PM or some high level dude doesn’t understand testing to any extent. I would stamp “TEST EARLY” (better yet: “TEST NOW!”) onto his/hers (from now on I speak genderless when I say “he”) forehead so he will remember it every morning he brushes his teeth. Even the most top level people should be aware of what testing is and why is it so important!

What do you comment on blogs where people write of successfully using a commercial tool time and time again project after project?

• Good question. I almost answered “I don’t.” but they may be posting things like that due to insecurity of trying things that FIT THE CONTEXT. There may be reasons behind the choise of tools that are unclear to me and they may be good ones. I however have not stumbled upon such blogs. Any hints?

How do you reply when people ask  on forums about where to get the answers to the ISEB exams?

• I say: “You don’t need the answers. Better yet you don’t need the exam!” And to be realistic the questions in ISEB tests are mostly obvious and you can answer to them without any knowledge about testing -> you just need to memorize a book. And that’s it. You will almost never use the information in the cert for anything (except you know to stay away from people that require you to). Instead of cert you need knowledge about your domain, procedures, tools, practices and make them work for you. Professionally (or CV-wise) you could take the test but relying it to bring you competence over doing testing is where the certifying goes wrong.

What impression do you have of other testers who profess to having no knowledge of who the leading speakers and thinkers in the world of QA are today?

• I know some. Not all. I know those who I think matter in the field.

How do you engage with other members of your profession with whom you disagree with or whose outlook you consider yourself to have evolved beyond years ago?

• This’ a tricky question. Be it the language barrier or whatever but this is how I understand this: “What if you think some other way to approach to something is better than the one who is suggesting an approach?” I need to know the reason why they support the chosen approach and try to understand the meaning behind it. If a valid reason is presented then it might be the best approach for the context. Or there may be some room to improve on which case the change must be found.

How do you write about QA people saying they took a long time to replicate and report a bug of which you have no knowledge but the amount of hours mentioned sound like a "long time" to you?

• Instrument (run in debugger, record GUI, kernel, etc) beforehand so less time consuming replication is needed. I would find out the context on which the system has been tested and bug found. If customer found the bug then we need to know the environment and such. We cannot say something took a long time unless we know how long it should take (mostly we don’t).

Do you talk to people who are not in perfect alignment with your way of thinking like they are below you? Do you think that evolution as tester can only be achieved by going on a certain course and anyone not following that path is below you?

• No reason to rate people by their current way of thinking. What should be done is rate them by the way they have improved their thinking. Some people do box themselves and never change POV. These people should be looked down upon. The self improvement may be just as effective as a course but sometimes preachy people make your heart turn into a certain direction which may seem the right direction for a while. Look into facts and whether the thing you are doing fits the context you’re in.

When I got to the finish of the blog I saw that this was a trap. "What do you think of yourself for thinking things like that?" I started to scroll what I wrote and came to the same conclusion as ElizaF:  

If we have the skills and the knowledge to consider ourselves as good testers we encourage others to be good testers also!  

There is no backdoor of which you can go. If you have knowledge and you don't share, your not a good tester. If you have skills that others require and it is possible to teach those, teach them or you're a bad tester. Be a good tester. Question your thinking. Think what you question!

Why top performers are top performers?

I made some suggestions in twitter about why does certain people have certain skills. Why is Mozart so skillful at composing symphonies? Or why has Bill Gates been able to build such a vast company? I got some interesting answers (“the lack of schooling”), but none of them was what I was looking for. After reading the book “Talent is overrated” (from now on referred at “the book”) by Geoff Colvin I got the answers I needed. The book was inspiration to this text and I encourage everyone to read the book even if you don’t want to be a world class “whatever” (surgeon, tester, manager). I tried to quote everything that was directly from the book


Why top performers are top performers? - They practice.

“Duh?!” you might say because you practice too. Most of us practice testing during some testing dojos or at courses. Some might even practice some during normal work. So how does practice make us top performers? Shouldn’t we all be the top of the game since we all practice to some extent? “I have tested for 20 years and I’m still not top performer. How do you explain that?”

Experience doesn’t count! Most of us work in the field we’re in and are just fine at it. We know the basic thing we need to know to get by with our daily work. The book explains that there is no difference between a new worker and experience worker unless the experienced worker has done practicing during ones career. One is not a master of a certain field with ONLY years of experience, but one needs to master the skills and it usually takes years to hone those skills. “Maybe the top performer was born with the gift. Maybe he was a testing star at birth!”

There are no gifts or talents or whatever. There is no innate talent, says the book and points our years of gene research. We create the talent with dedication and practice. “Ok, but how do you explain that great violinists come from musical families or lawyers come from lawyer families? If they have it in the family how come it’s not innate?” Family has some effect on the skills people accumulate. I will return to it later, but suffice to say now is that there are some environmental factors that help us (or hinder us) in our quest to be better. One might ask whether Michael Bolton or James Bach were testing gurus at birth. As I recall Mr. Bolton was a comedian and Bach was a software developer. Neither probably didn’t test software at the age of six (don’t know but I assume). What they did to be so prestigious was that they honed their skills and became better than most of us.

Nowadays we have higher standards and we need to be more successful than those before. The bar is higher for us to be great. Take science for instance. In the past people like Newton got the status of great physicists quite easily (for our standards) He just explained gravity. “I can explain gravity! Why aren’t I considered a great physicist?” The standard was different for people before us because they had not the access to information we have today. We need to better than those before to be considered great. “Do I need a brain the size of a mountain and memory capacity to match today’s super computers?”

Memory or intelligence has nothing to do with success. High IQ is not a perquisite for being top performer. IQ doesn’t measure critical thinking, emotions (and the ability to use them to our advantage), social skills, honesty, wisdom, etc. Even if you do have very high IQ you might not be a top performer. Look at the membership list for MENSA and count all great performers from the list, then compare it to those that are not members of MENSA. You can have high IQ and be a top performer but it is not required. Same rule applies to memory. Some geniuses (not all top performers are though) forget all kinds of things but are still considered great in their field. What the top performers have what the rest of us don’t is domain knowledge. I’ll return into that a bit later.
“Again: What makes us great?!” If we all practice and increase our knowledge, and only some of us become great, what is it with practice that excludes some of us from greatness? “Am I doing it wrong?”

Well… You probably are. The top performers practiced just like the rest of us but they did it with purpose! They practiced deliberately. And they accumulate domain knowledge in doing so. What we “regular” people see as practice is the things we do to entertain ourselves with domain related things. We might “practice bug hunting” and try a few things and say “Now I practiced the skill. I must be better at it.” You probably ain’t… In fact you just might be worse as you perhaps didn’t know what you were doing and thus did some essential thing the wrong way. You probably did some exercises that you knew you could do or played with software you had played with before.

Deliberate practice is practicing those areas that you are not good at. And doing that systematically! The thing with deliberate practice is planning what you want to achieve and knowing the road to get there. It may not be fun, but getting to the top of the game is not about fun but being the best. A deliberate practice needs to be designed to suit your needs and to hone skills that you’re not good at. Basketball players may shoot hoops in the evening as they know they can do a hoop from three point line. Pro basketball player shoots hoops from half court. If he/her hasn’t the strength he goes to gym and shoots again. And again. And again! Then he/she shoots hoops while moving, etc. In deliberate practice one moves away from comfort zone and practices skills he doesn’t yet master. You cannot learn in your comfort zone but you need to know not to push too hard or you’ll panic and there’s no learning. It is all about increasing the performance of a certain aspect of our domain.

As we all know we might not know what to practice, therefore we do what we know. So we need tutoring. The tutoring provides two things: guide and feedback. The tutor may or may not be a physical person but a goal we aspire to reach. For instance we may look at a video of a guy doing a yoyo trick and look at it closely and try to replicate the trick. Or we could have a coach that guides us during football practice. As in sports and music, we can get tutoring in testing and programming. Tutoring helps us knowing where we want to be and means to get there. We just need a person that is more qualified in some area that we seek to master and get him to coach us. Of course it’s not THAT simple, but we all know the basics of tutoring and coaching. Preferably your tutor is the current best in the field.

Just as much we need guidance we need feedback. At early stages of our lives our parents provide feedback. “That’s a beautiful drawing.” But when we start practicing deliberately we need constructive feedback. Again the source of the feedback need not be a person: you can watch yourself do the yoyo trick from a video also. You can even record yourself shooting hoops, but in the hoops case the ball going to the basket is feedback enough. Tutors provide feedback. We can do some amount of things without systematic feedback but we may learn to do things inefficiently or wrong. In testing the feedback can be bugs. The systematic and good feedback is the kind that makes us find those bugs more efficiently, easier, more, etc.

And when plan is made and feedback is given, you repeat. By repeating you can hone that skill. Masters make things they do look effortless and they may be. They have practiced that skill so much that they no longer have to put all their effort into the basic mechanics of things. They do not, however, automate themselves. The book presents an example of Tiger Woods teeing a golf ball. Someone coughs at the audience and he stops the swing in the mid-motion and starts again. The rest of us just let the swing go to the finish and hope it’s a good one. The masters are so aware of the things that they do that they don’t need to focus on what they do. They focus instead into HOW they do it. They seek flaws in their manners and techniques and practice those skills to do them better. “Where does all this practicing lead?”

The top performers are able to perceive more than the regular people. For example by looking at an X-ray picture a novice might see only the most prominent features and describe them. The top surgeon might see the little details that determine whether the person has cancer or pneumonia or a hidden heart disease. They see more with less looking. They see the subtleties and details possibly ignored or misinterpreted by the rest of us. They uncover hidden things with the same information we have to cover only visible or most prominent things. Top performers understand the significance (or insignificance) of subtleties and details. When bug hunting testers need see things they aren’t looking for. Seeing things you don’t look for and looking for things you don’t see. Deliberate practice helps you perceive more with less information.

It all sounds simple when it is said like this. The book has tons of information about the subtleties of deliberate practicing and many references into practical things. Music, sports, chess. They all have a their intersections with deliberate practicing, even some highly specialized learning techniques to practice in that area. Even business, programming and testing have their own special ways to build skills in domain.

We all still need things that motivate us to practice. The practice in itself may not be motivational so we need things to support the practicing for us to keep doing it. We need a supportive environment to practice. Mommy and daddy were sufficient enough in our childhood with their comment and encouraging, but in business world we need a different support. Some of us don’t have the possibility to have a tutor but still can use the basic methods of deliberate practicing. We can do practicing while we work. The working environment may or may not support you practicing so you need to be able to do deliberate practicing in the midst of the daily labor. “I just said before that I do practice while I work. What are you saying?”

The practicing at work can be done in three parts (and should, might I say): before work, at work, after work. This doesn’t exclude that you can’t stop to think the “after” part during the workday, but all require some thinking and time so before and after work you most likely have enough time to think these things. Before work practicing is setting goals for that day. It’s not about some high level goals like “I’m going to do my best today” but specific goals like “today I’m going to find out why my build fails the first time I commit it every time”. The goal is to produce a working build and to find out why it had failed the previous time. After goal is set you should plan how to get there and you need to be exact! Then during work the key to practicing at work is metacognition (this is also from the book) that is thinking that you think, knowledge about your knowledge. You can study yourself when you do certain activities at work and go through what you really are doing. Then spot any flaws and make corrective measures to succeed in the task. And think what you do! After the work analyze the performance. Use this analysis to increase performance the next time at work (or practice it before hand if possible).

Motivation is a tricky thing to maintain, so I'll delve into it in a separate blog post. What you need to know is that motivation may come from within or without. And it all supports the practice.

All in all: We’re not all going to be super professionals but we all can be! If we truly want to be extraordinary there is a way to be! It takes hard work and dedication, sometimes sacrifices, but a way is free for all of us to be top performers in our chosen field. Just practice deliberately and hone those skills!

The first impression bias

Hi y'all!

I just started in a new job in a new city so everything is new to me. I still live in Tampere (a midland city in Finland) but I commute to Helsinki every day (about 180 km). The job I got is a Quality Engineers position at F-secure Corporation in Maintenance team with (hopefully soon) some managerial tasks regarding Customer Care and whatnot. Most of all testing, testing, testing. Enough of that! My point was that when everything is new you get to look at things fresh eyed. I didn't say biased as I'm TOTALLY biased when it comes to all these new things as I compare them to my previous jobs. I just have a different perspective onto things some people have been looking at for years. I have longed to work at F-secure since I first started to dabble with testing and now that I have the chance I embrace this opportunity with every cell of my being. (This is to make my boss happy! ;) ) But nonetheless I have be critical as I am the new guy and I may have an impact onto those settled ways of doing things and maybe even improve them. We'll see how this goes...

In the Ohjelmistotestaus.fi -blog (which is in Finnish, sorry) Antti Niittyviita wrote about first impressions. He said (free quote) "The first impression is an opportunity to get new information of the quality of the product or service. It should not let get wasted!" When I got to the company I though "Wow, this is quite rigid stuff", for the security part was mandatory and quite formal. Actually on retrospective the event was less than formal, but I think I was so nervous.

Later this week I got to get to know the products from public website so that I get a basic knowledge of what this all is about. I thought about Antti's blog post and took a critical and learning attitude towards the browsing of the products. I was amasing how much defects, issues and such I picked up from public website! The new perspective really opened my eyes.

The testing I got to do the first week got my critical thinking up and I ended up making observations that might have been insignificant to others but I reported them nonetheless. As it turned out these thing teached me quite a bit about the behaviour of the product and the platform I ran on. They were no bugs but something that were not documented in a sense that I would have been able to learn from documentation or from tutoring. So the first impression and critizism thereof teached me something important.

The first impression also has a negative side. If you get a first impression about something you rarely change the attitude if you don't have to analyze the cause of the dissatisfaction. For instance a product that is thought of as slow and reduces performance on your computer (my sister-in-law said she hates a F-secure because it slows her computer down and eats the disk space) doesn't get to change the consumers opinion if they toss the product. If the bad image has gotten through to customer, the dent in the image will be hard to repair.

--

As testers we have the job of trying to test on the best ablity possible. We all get a first impression on something and we may guide our next move by the impression we get. This applies especially in exploratory testing. We guide our next step so that we take in account what we have learned in previous steps (in steps I don't mean scripted testing but actions we take during our testing). Because it is very power tool, we should use first impression as a first guidance tool for our testing. To remain critical we NEED the first impression. What is the first thing we feel might just be right.

As the first impression might be a inference instead of observation we must be careful not anchor ourselves onto the first attribute we percieve. Here the critical thinking of a tester comes to fore. We need to be able to recognise what we observed and afterward make assumptions from those observations instead of inference. By making judgement upon inference we lead ourselves into a trap and start thinking we something that is not there and miss things that are as irrelevant.

There are some basic tools to guide ourselves away from anchoring ourselves onto a inference, but I ain't gonna go through those here. Point being that we have to remain critical even though we have indication to start thinking something about the product/service/whatever. The first impression is a good tool just as long as you remain critical and don't let yourself be anchored onto an attribute that you have infered. (Here's the difference between inference and observation)

Hopefully I remain critical in my new job and start defending the quality of those products we offer. The first week is behind now and new challenges are coming. I hope all my colleges have the nerve to cope with my bad humor and loud voice. :)

Testers are rock stars!

Teemu Vesala wrote in his STC blog comparing testing to poetry. It began a thought process where I found myself comparing testing (noun) to classical music. "Testing is like composing classical music with all the nuances involved" was pretty much the message of my. This was the beginning of tweeting that went like this:

Teemu Vesala:
Poem writing and #testing has plenty of common. See my blog entry @ #stc http://ning.it/jF60aR

Pekka Marjamäki:
@teemuvesala #Testing is more like composing classical #music: even the tiniest sounds change the feel to it. And testing tells a story.

Teemu Vesala:
@pekkamarjamaki Acctually #testing is much like any art. No matter if it's poetry, music or painting. All have same properties.I love'em all

Testing is comparable to the art because they are basically built of same thing (both in their respective context). But which would not be comparable to art? What is art?

Wikipedia (ah! the an inexhaustible source of information!) describes art like this:

Art is the product or process of deliberately arranging items (often with symbolic significance) in a way that influences and affects one or more of the senses, emotions, and intellect.

Can testing be culture? Can there be testing without it being part of the project contextual (I made up the term. Wonder if there even is a term for a thing like that…) culture? Testing is THE element that offers information about the pretty much anything, because testing occurs on so many levels: from unconscious questioning and doubt into systematic examination of the whole system both dynamically and statically. So, yes, testing is a significant part of the culture (the context) in which the testing is done.

Testing is a set items (techniques, approaches, tools, strategies, etc.) that are deliberately arranged so that they form a basis to achieve a wanted result. As we implied before, the test can be performed adhoc, but as art, it does not always reach the goals that are sought. Testing is at least partially subjective, because it is the opinion and interpretation of the task at hand. By removing the subjectivity of art it is possible that ideals of beauty or a non-intelligent interpretations of things are forced to the spectator. The subjective nature of art is a force that makes art – well – art. The nature of the testing is subjective, which is definitely an asset to testing itself, because different people interpret different things in different ways. We can always say that "This thing is X. Period!", but it is not necessarily correct in context. The contexts makes subjectivity important.

We can not underestimate the importance of what feelings are to testing and testing to feelings. Both of these are factors that affect each other. In art, the composer, painter, choreographer, sculptor, you-name-it have feelings or emotions. These are driving forces that will create art. Of course there is art done without feelings, but is it art? Or is the lack of feelings a feeling itself? Is all that is art on some level? In addition, there is an art that does not affect the feelings (in a given context and subjectively). Art is trying to be bi-directional with the emotions. Testing tries that too: feelings (understanding the risks, prioritization, etc.) drive our testing forward and testing is guided by the feelings to help in our decision-making.

Art is expression, communication, statement and pleasure / annoyance. This is true for testing, word for word! Testing is expression: we express the current quality of the system by testing. Testing is communication: There is no other way to express and inform the quality to the stakeholders. Is there? Can quality be expressed by programming? It creates the quality which the testing expresses. Testing does not make quality. In addition, when it comes to pure communication, testing is a social event where communication between groups is important! How can we express the quality, if we do not communicate it to the stakeholders? And testing makes a statement: Michael Bolton said "Testing is defending the quality of the product." Testing comments the quality, testing defends the quality of the product. If testing doesn’t do that, who will?

What comes to pleasure, to me, every day of testing a great pleasure (my testers have undoubtedly noticed that). Every day that I can to promote the triumph of testing both organizationally and the genre produces pleasure. Every day that I learn more about the testing produces pleasure. Every time when I find a new bug system it produces pleasure (displeasure for the project manager ;) ).

To summarize: Testing is art. Testing will be appreciated as art. There are mathematicians, which state that mathematics are the greatest art (graphs based Fibonacci numbers, chaos theoretical curves and fractal art). Testers are therefore artists. We testers who respect ourselves and our skills can hold ourselves as the Rock stars of the IT world (or as Leonardo da Vincies). We bring the joys and the sorrows, we express feelings, even beauty. For me, the test is the biggest art.

"Why did this bug get through testing?"

Ahh the question that will come to every test manager at some point in their career. Mine such was today.

It went something like this:

An email came late yesterday night: "We need explanation for there bugs that went through your testing. And we need it by tomorrow." I loosened my collar and wipe sweat off of my forehead. What did my team miss? Have I focused my testing on wrong part of the system? Have I unintentionally lied to customer? The questions kept coming and I kinda lost my sleep.

Then the morning came and more self confidence with it. I gathered my team and asked them to replicate every bug the client reported. 3/4 were their error on executing their own production process. All things they had done were handled correctly and they just expected something to happen that was not meant to happen. Let's say they missed a step in their process and therefore the outcome was not what they expected.

The 4th one was a clear defect. It was not however gotten though our testing but was reported and scheduled to be fixed later. Also its seriousness was heightened by the fact that it did have an effect on production use. Of course we react to the re-prioritization and we'll get it as a hot-fix if need be.

This reaction to the "bug-slip" however got the client to bomb us with defects that are clearly wishes or new functionalities that interfere the current requirements. These should not be mistaken as bugs but as change requests and therefore require some changes to requirements, design, etc. But still they were something the testing did not catch.

My initial reaction was quite overrated as there was no hole in our "net", but it got me thinking: Does the testing team get the blame for all thing unwanted in production? Is it just communications that failed in this scenario or the testing? Are we supposed to know all what is said and not said? Wishes and hopes? Dreams?

If this scenario had unfolded differently, say "Should we discuss about our findings in pilot environment so that there is no miss-communications here", things could have gone differently. We might have not spent the best part of the day recreating defects and preparing for haranguing. We could have gotten testing done on the parts that are not that well covered yet.

The meeting with the client went well however when we walked though every step and finding. It all came down to communication, again!

Ifs & Buts, Couldas & Wouldas whatcha gonna do?