Managing Heuristic Exploratory Testing Based on MindMap

The FreeMind mind map template that I use.

The MindMap allows me to create a structured testing quickly and efficiently so that the testing is accountable and efficient. The mind map functions as requirements document, test plan, test report, bug report and heuristics cheat sheet for my testing. The mind map functions simultaneously as the tool and the product of testing thus supporting the values of agile and lean development. I’ll explain how.

Gathering Information

As the picture above states that there are multiple heuristics to choose from and when starting to test the best way to start is to gather information. The “Project” heuristic is a good way to start developing a sense of what to test, how and in what order. Using the applicable heuristics one can form the basic knowledge about the test object and the surrounding project.

This phase may be as quick or as time consuming as necessary. The key is that you’ll establish the needed knowledge about the test object. This may include (but not restricting to) referring to requirements documentation or other documents, talking to product/project/solution owners/architects/managers, requesting consulting from developer/other tester or the like; anything you see helpful to your task at hand.

At this stage you’ll start also forming you testing plan. This may include equipments and resource scouting or gathering, scheduling and possibly limiting the scope of testing. By limiting the scope into testing that can be done in 90-120 minutes sessions the testing is more manageable and less tedious. Forming the testing into specific missions also makes your testing more motivating as you’ll achieve lesser goals on your way to larger goal.

At this point you have a mind map that looks something like this:

Project heuristics during Information gathering

You should have sought answers to the questions or decided they do not need answering. This forms the basics of your test planning. At this point you’ll also have a good grasp of what is important and should be tested first. It is possible that the priorities change during testing (and it should) as does the scope. This is however the framework in which the testing starts.

The gathered information is context-dependent and varies across domains. The mind map may also be tweaked to suit the domains special needs. Also the items/tokens may be chosen to best suit the purpose either increasing or decreasing the amount. What I use are Question (“Someone needs to tell me something regarding this topic”), Exclamation (“There is something in this topic that needs to be looked into in detail”), Idea (“Possible solution, test idea or a thought”), Blocker (“This topic might cause a block to testing now or at some point”), OK (“This topic is tested and considered not needing further testing”) and NOK (“This area requires more testing at some point”). The most important thing is that the choice of items supports the cause instead becoming the cause itself.

How to measure coverage?

As we have now established the basic knowledge behind the testing task we may start to sketch the coverage and depth of the testing. This is done by modeling the test object. Modeling is a simple way to map or… hmmmm… model the test object. This is done by creating a representation of the test area using the modeling heuristics. The models may be as detailed as needed or as vague as they can be, again what bests suits the context.

I myself start the modeling by mapping the structure of the test object. In case of GUI testing I try to sketch the basic layout of the GUI forms and deconstruct the components within the GUI. For example some text fields can be tested with same inputs (possibly). This also gives a good representation of the depth of the GUI (as if there are any forms /behind/ the previous). Menus are a good guide in deconstructing the GUI as they also may hint onto what functions the system has to offer.

The structure may consist of anything “physical” or fairly easily recognizable objects of the test object. The operations of a web service are easily modeled as are the fields within the operation. There are possibly even tools (spiders) that will form the structure model for you by minimal work. This is the first coverage model that I use and frankly the most simple. The coverage may even be derived into X/Y –coverage to sate the managers’ need for figures, but this is also to remind you of what has been done and what needs to be done.

Functional coverage is the list or the model of what functions the test object has to offer. This may also be somewhat easily derived from visual/physical things but also the hidden/hard-to-come-by functions should be stated. The function may however be used in a different way that it was intended so user/admin/automatic operations should be stated also either as their own or where they differ from the intended functionality.

The data coverage is important to some and should be deconstructed into a heuristic list. Easiest way is to find out what kind of data the structure requires and what is in the interface requirements or in database. Also analysis of what kind of data is it possible to insert into system and HOW! There may be data security issues in public interfaces or GUIs that need to be tested with different approach or depth.

Platform may affect all tests and if there are any platform dependencies they should be stated. Sometimes it is not seasonable to test everything in every platform so the criticality of the testing should be estimated on each platform. Sometimes test automation and changing the test approach may come handy if testing multiple platforms (i.e. mobile phone testing on different phone models).

Time-related testing is also a way to measure coverage of the test object. If there are time-related testing to do (performance testing, stress testing, concurrency testing, “24/7 testing”, etc.) these are also a way to measure coverage. Performance coverage may be important in a SOA world where critical components require consistent and good performance.

These form into a map of models of which we can inspect coverage and progress of testing.

The coverage break down using Models heurisrics

Heuristics support testing

The use of heuristics enables you to think of multiple ways to test something. One can apply many different techniques onto ones testing to dig up different kinds of information about the product. They can be test techniques (remember that exploratory testing is an approach not a technique), quick tests, tours, you name it. The point is that the heuristics mind map functions as a checklist (if that is the way you want to use it, for example coverage reasons) or as a heuristic in itself.

Whatever the approach to testing is the heuristics are there to help you in your thought process. Using the mind map to record your thoughts about some aspect of testing is advisable and easy, as you can connect techniques to models and parts thereof. For example using quick tests (“clicking frenzy”, “banging the keyboard”, etc.) to stroll through a GUI or flow testing to do some E2E testing, the mind map is a good tool to design testing on the go and to report any findings into the map.

The mind map can be used to report bugs using the video clips generated from test sessions in addition to possible description in the mind map. These help communication to developers, managers and clients, and writing the defect reports. The issues and questions that testing uncovers can be linked to oracle heuristics to make more case to the defect.

For example a basic stress test may indeed pass within the NFR description but the memory usage or the amount of handles may not even be described in the NFR. Of course it is up to the tester to choose the best way of describing his/hers findings. The rule of thumb is that there should be enough information to help the developers to fix the problem AND to make an appealing case to fix it! (No bug report is good enough if the bug doesn’t get fixed!)

Heuristic testing skills (for example Rapid Software Testing skills) are also important here as one can rely too much on the check list and forget to do the testing in their heads (where most testing is done). Heuristics should be applied (if applicable) not followed. There’s no harm trying to apply some heuristic in a context, but using the heuristic as a hammer and trying to smack everything is not advisable (it might even be stupid). They are there to help you reach your testing goal and provide tools to your brain.

If stuck, the heuristic mind map can give you a good boost into proper direction: just look at the heuristics (if you can’t remember them) and think of what to do next and how. Heuristics work also as guides in tweaking your plan. You may have gotten false information during your info-gathering or bugs prevent you from doing something. Heuristics can provide a solution to a changed situation when a new mission needs to be formed.

The testing heuristics applied

A product and a tool of testing

The agile principle describes unnecessary documents as bad (or that is how I have interpreted the manifesto). Michael Bolton talked about using “memos” in testing to help the brain to interpret all info pushed into it. Instead of being a document (a product which you produce) the memo is used as a tool to help testing. The mind map has a similar value to testing: it helps make notes about things you see, feel, and observe, but also enables you to use these notes as a report of what you have done (if someone asks).

The mind map serves as a tool to facilitate the testing but also as a product of testing. It serves as a platform for communication when describing what went wrong and how. It serves as a report of the testing to increase accountability. It serves as a session report of which the Test manager or other testers can fathom what you did, how and what important knowledge there is that they need to consider.

A finished mind map looks something like this:

A finished mind map with references to other heuristics and links to screen caps and video clips.

The questions should be answered, if there were any. The bugs, issues, ideas, etc. should be discussed and possibly closed. New testing sessions should be organized to cover areas that were not covered.

I used FreeMind 0.9.0 in my example and in my work daily. The example mind map was done to illustrate the possibilities of using mind maps. Normally my mind maps are a bit more restricted to certain type of testing.

The web is full of mind map software from browsed based (like mindmeister) into downloadable tools (like XMind). Just choose the one best suit you (or ask someone for a recommendation) and create a mind map that suits your style of testing. And get testing!

If you have had good experiences using mind maps, please share them. :)

Motivation 3.0

After reading the Talent-book by Jeoff Colvin I had a hole in my mind that needed filling. Why does a person embark on a grueling adventure of deliberate practice? What drives you to excel? Poof! (Somehow the poofing got into my writing, but I can’t fathom why…) In came the Drive, a book by Daniel H. Pink about motivation. Mr. Colvin was quite vague about motivation (or the focus was more on what and how instead of why) so I made a decision to skip the motivation part and go deeper in a future post.

Now that I have read (and somewhat understood) the Drive I feel like I can make a contribution and share some of the secrets about “what motivates us”. The deliberate practice (of which mr. Pink also talks in his book) is fuelled by motivation: If we really want to do something, we can do it. “But why?” Hold on, I’ll share some of what I have learned. (This is not a book review but interpretation about motivation using the book as guide.)

A little history about motivation: When we were cavemen, Neanderthals or Finnish ski-jumpers, we had a drive (the book calls it Motivation 1.0). That was the need to feed, drink and f…. reproduce – the drive to survive. “Hey look! we survived!” And that motivation served us to get by, to fight cougars and wolverines, to eat nutritious food. But as humans developed the Motivation 1.0 did not serve us properly; we needed something more to motivate us. We needed upgrade to the Motivation 1.0 to motivate us to do our job. The second basic drive that motivates us is the extrinsic motivators: rewards and punishment, carrot and stick, etc. The book calls it Motivation 2.0; the upgraded way to operate and motivate us. It served us well with basic-straightforward –repetitive tasks like assembly line or the like. The system awarded us from good behavior and punished us from bad behavior. The managers of that time thought they could use incentives like bonuses to motivate people to do their bidding. And people did… when their jobs were “basic-straightforward –repetitive”. Nowadays the menial tasks have been automated and people are moving towards problem solving, innovative, creative jobs that are hard if not impossible to automate. The Motivation 2.0 doesn’t support that kind of work. In fact it may result in unwanted behavior and to diminish the wanted behavior. And how many managers and companies have seen this?

The DRIVE introduces a third drive that is called Motivation 3.0. It is the intrinsic motivation that we all have. A person that responds to intrinsic motivation may not respond well to extrinsic motivation. The book uses example from children’s play.

If a group of children are asked to paint or draw, they all draw. Kids love painting! But then half of the children are told that they can get a reward from drawing. What happens is that they all draw for a moment but the rewarded children are less likely to continue drawing after they get the reward as the children that were not awarded continue drawing like they used to. The award turns play into work. And that de-motivates us.

There are however some ways to get “carrot and stick” working to increase out motivation. The “if-then” reward may cause a moment’s burst in motivation and efficiency but they are prone to do down very quickly. “If-then” reward may lead into addiction (as in “I won’t do the same task for the same money anymore” or “I want TWO candies to take out the trash”) or suppress the wanted behavior. When used as a “now that” award it can sometimes encourage a person to create an intrinsic motivation to reach the possible award. For example an Olympic runner has a set of goals (to beat personal best, country top ten, etc.) instead of just one; the Olympic gold medal. They may try to reach that goal but they aren’t looking the medals as “If-then” but “now that” reward. The same drive, well, drives us forward in our work and personal life. We tend to seek goals that are fuelled by intrinsic motivation and steer away from extrinsic motivation (or at least most of us do). In most of us there is both kind of motivation, extrinsic and intrinsic, but what makes us most tick is the inner drive to accomplish something, not what others want or tell us to be.

Daniel H. Pink has studied the field of human motivation and cites numerous studies from psychology and economics in his book. He has narrowed down the intrinsic motivation into three main elements that make the intrinsic motivation work: Autonomy, mastery and purpose. Those are not elements that Motivation2.0 willingly supports or acknowledges.

Autonomy

We people have the need for autonomy, need to be free. Nobody WANTS to be… ermmm….. not-free. The managerial style that supports extrinsic motivation focuses on compliance (“Do as you are told”). Instead the intrinsic motivation focuses on engagement. With autonomy people are able to engage into their job and be more satisfied in it. The old style world says “People need to be managed” but in truth (on a number of studies show) that people need autonomy to shine. Autonomy over Tasks, Time, Technique and Team (4T) invokes the inner motivation and boosts our performance, get us into FLOW! Companies like Google, Atlassian, etc. are great examples of autonomy.

Even though autonomy motivates us it needs to be well thought of. For example autonomy should be a perk that encourages people to CHOOSE some of the things they do, CHOOSE the time they do it (at least on some level), CHOOSE the way they do it (in the frame of what is accepted or tolerated) and CHOOSE who they work with.

Atlassian shows a good example of autonomy over tasks. They hold a “FedEx day” every quarter. The “FedEx day” comes from “delivery overnight” and it basically means that people can choose ANY task they like (a problem waiting to be solved, an innovative idea) and they have one day to make it work. In the noon of the next day they all have to present their product, solution, whatever to other co-workers. This sprouted some of the most innovative things they’ve done. The problem was that they only had 24 hours in a quarter to do it. So they changed it into 20% of all working time. This “20% time” is now where innovation happens.

Where Atlassian excels at Task autonomy, companies like Best Buy, Meddius, F-secure, etc. excel at autonomy over time. “When people don’t just show up and grind through their day they’ll be more productive.” Scott Adams, the creator of Dilbert, says: “Nothing is more important to my success than controlling my own schedule.” Autonomy over time gives you the freedom to do things when you are on your most productive state, in the FLOW. Monitoring is again needed but trust also motivates people; trust that they will show up and do their work.

Autonomy over technique is the corner stone of craftsmen. Take art for example where new techniques are found every day. There would not be Monet’s pointillism technique if he had not the autonomy over technique he uses. If people are forced to obey the rules by which method they work, they tend to be unhappy. If exploratory testing would be forbidden in where I work it rules out so much testing done and so many techniques I use, I couldn’t do my work efficiently. The choice over HOW I do the things I do is for me the most important aspect (as you can see from “Best Practices (NOT!)”).

Autonomy over team is probably the most difficult one to achieve. Some companies have done their share in team-wise autonomy by letting the workers into the recruiting process. For example, F-secure Corp. has an additional step on in the recruitment process which includes a team interview. All candidates get a chance to meet the team and mingle a bit. Then the team says their opinion about the guy and a decision is made. Other companies (like Whole Foods) have removed their executives from the hiring process and let the team members do the hiring. Then after 30 days trial time they can decide to “keep ‘em” or “lose ‘em”.

Other form of autonomy over teams is the free time activity with the co-workers. The autonomy over team presents itself with off-time clubs and sports teams that the workers put together. After being made the social connection to people it is easier to communicate even with harder issues in the work place. After you have a choice of the people you work (or play) with the work has more meaning and is more fluent.

Even though autonomy is a hard biscuit to swallow, companies eventually move towards autonomy. Though some managers still think people are from the same mold as they came in the 1910’s we all strive towards freedom. If we weren’t there would be no companies like Google or Zappo’s. And by giving people more choice over the four T’s (Tasks, Time, Technique and Team) they just might perform better and be more creative than their counterparts that are forced to comply instead of engaging.

Mastery

Motivation 3.0 allows us to be more engaged in what we do. Instead of complying other people strict rules we seek to accomplish goals that are closer to us and thus being more close to the subject we work with. Engagement has also this weird way of encouraging towards mastery. By getting closer to the subject we find a need to better ourselves in doing the thing we do. E.g. by being allowed to program without the boss breathing to your neck, one just might find an innovative solution to a problem. Mastering our jobs motivates us to do our jobs better.

I already had a few words about getting better at what you do (in my post "Why top performers are top performers") but mastering is a motivating thing as well. There might be no need to motivate you separately to master a field as the prospect of mastery provides the motivation needed.

When people reach a flow in what they do (check out books about the Flow) they usually are more prone to searching mastery. They notice that they can achieve something they do with ease and start increasing the performance and the difficulty little by little. This is what the book calls “Goldilocking”; not doing things that are too easy (they get you bored) or things that are too hard (they get you anxious), but doing things that are “just right”. Jeoff Colvin describes this in three circles inside the other. The outer layer resulted in boredom and the center resulted in anxiety. By staying between boredom and anxiety is the optimal place for achieving mastery (and learning). But it seems that mastery is a fickle thing…

Mastery is a mindset! If you don’t think you can achieve mastery then you probably won’t get it. In the book there was a good example about intelligence and development thereof. If you think about your intelligence as fixed amount of knowledge then your intelligence might not increase from what it is currently. If you think your intelligence as a muscle that you can train it most probably grows. As I described in the performance post, there is nothing preventing you mastering the domain you want except yourself.

Mastery is PAIN! Every single athlete knows that without pain there is no result. Only by repeating the exercises, training programs, etc. can mastery be achieved. There are no shortcuts to mastering a domain only hard work. And to top it all mastery is an asymptote. One can never truly and fully master something as there is always more to learn. That is one of the most appealing things that drive us towards mastery: it is almost impossible to achieve. The fact that full mastery cannot be achieved is both frustrating and alluring and it seems that it is the process of mastering an art that drives us towards it.

Purpose

The third element of intrinsic motivation seems to be a sense of purpose. We rarely do things (and enjoy them) if we don’t know answer to the question “why”. In companies that have large engines of bureaucracy and command chains the purpose of doing something is easily lost. Doing something that “makes no sense” is a serious de-motivator to us using the Motivation 3.0. Companies seek purpose to make their product/service more appealing. Companies support developing countries and thus make a purpose for business; it’s more fun to work if you know that your work has a meaning.

In business world the purpose is something of a nicety; you can live without realizing the full purpose of what you are doing. In the TV-show Friends this guy Chandler is a data analyst who has no idea why he does he does. By realizing the fact that there was no purpose for him to do the job he lost his motivation. If you feel you don’t know why you do something, start asking “Whose purpose is it anyway and what have I got to do with it?” We need to do our work/live our lives to accomplish something and Purpose is “the something”. There are three basic pillars to support the purpose motivator: Goals, Words and Policies.

Goals work as a milestones to purpose. By setting different goals you feel you are accomplishing something even though you have some path to tread until you achieve some bigger goal. By setting both performance goals (getting an A in a math test, reaching your sales quota) and learning goals (being able to understand and apply the math, being able to fully understand the need of a customer) their overall impact on your motivation is optimal. Performance goals support the extrinsic motivation and learning goals build intrinsic motivation. The goals we set pave the way to the purpose and as we strive to reach the goals we move towards our purpose.

Words are fickle. Words can change our way of behavior. Think about vows. If you vow something (like a “I speak the whole truth, etc.”) it sets you a standard, a purpose, to strive to. When things are said out loud they are stamped into our brain as a code of conduct. When a group of Harvard graduates in 2009 saw that the economy is going downhill because of economists and business people cheated or cut corners rather than practice ethical economics, they saw that they could make a chance. They made “the MBA oath” (just like a Hippocratic Oath) to “strive to create sustainable economic, social, and environmental prosperity worldwide.”

Words are also used to measure our behavior. Listen to your co-workers talk about your company. How many times they use the word “they” to refer to the company instead of “we”? The “we” signifies that the company is coherent and everyone thinks they are in the same ship sailing towards common goal.

Policies are also an aspect of purpose. They overlap a bit with both Goals and Words as they are a part of both. Policies refer to ethics and good manners. Wrong policies can push towards unethical behavior or even discrimination. Let’s say we have a HR policy that goes something like this: “All members of the company should treat each other with respect and kindness.” The atmosphere in the office might just be open and friendly and all conflicts are resolved with a friendly talk. Let’s say if they change the policy into: “Abusing, harassing, bullying, physically or mentally assaulting a co-worker is strictly not allowed with a penalty of a fine. Racist or sexual comments, behavior, writings…” Now that most likely sets the environment not possibly to encourage bad behavior but at least make it more controlled and closed. Policies allow people to pursue the Purpose with their own terms.

Motivation 3.0

The science has known the third, intrinsic drive for many decades and it is researched by many. The business world just hasn’t grasped the Motivation 3.0 yet. The “carrots and sticks” are for cavemen; the new trio is what motivates us today: Autonomy, Mastery and Purpose.

See more about the Drive from http://www.danpink.com/drive.

Have you been spotting mythical creatures lately?

Right... This is no after-the-weekend delirium post but a testing post instead. At least I think so... The other day I was browsing the magick land of Interweb and stumbled into a word "unicorn". I stuck into my mind and today as I was travelling to work I started thinking what mythical creature I would be (I have a soft spot for fantasy literature and all sorts a coo coo things). I thought about the unicorn and poof - a testicorn came into mind.

What a testicorn is? Here a list that I came up:
- mythical creature no-one really understands
- avoids human contact
- has a protruding thing coming from the forehead
- a long face like that of a horse
- its blood will cure the sick

When put into context the testicorn is much like a tester in some bad apple companies (no pun intended). At some companies a testicorn is a lonely creature that lurks in dark corners of the Developland trying to do its magick. He is rarely fully understood except by other testicorns who sometimes try to rise against the tyranny of the Developland, shy creature that rarely comes out in the open anymore. These creatures are forced to behave like they are expected to; work as to detect wrongness in the universe and report the findings. Sometimes they are forced to present a magick scroll into which they must write the spells they use to get rid of the evil but are rarely encouraged to cast spells from their memory or develop new spells to get rid of a certain kind of evil entity. These spells are then stored into a library by the head wizard and the spells are given to new testicorns when old ones are expended. "Testicorns are of plenty. Why try preserving them? We have new testicorns coming from the School of Magick as much as we want!" And testicorns retreat into their magical forests to do the things they are forced to do.

At first the testicorns didn't have a "horn" in their forehead. They developed that during overtime conjuring and after getting their mythical rights trampled upon. They also developed a long face with dark spots under their eyes. They do what they are forced to do and the horn grows. A sad thing actually as at some point (and in some benevolent magical kingdoms) they are regarded as beautiful and wise creatures. They used to be highly regarded for their expertise in evil ridding spells and a mind for the common good. Now they are the grease that lubricates the machine of tyranny.

Testicorns were the healers of the magic kingdoms once. They taught the ways of healing to people. The testicorns still have the ability to heal (and do it with plethora of ways known as Schools of healing. To mention one the School of Exploratory Healing...) with their innate ability, but the blood of the testicorns is known for their incredible healing abilities. The Chief Magickmen use the blood of the testicorn to heal the sickness without regards of the testicorns health. They are bled to heal the hurt. This consumes the testicorns power and at some point they have no more strength and they wither away.

In some kingdoms you see  these mythical creatures laboring their tailed behinds off to get from day to day. At some point they are replaced by a hornless, careless little testicorn and the old withered one gets laid off. A sad story actually...

The kennings above were used as stark image of the way some companies use testers as a way to increase the quality. They are put into a mold and told what to do instead of doing what should be done. They are exhausted with stupidifying tasks and are kept as a scapegoat for bad quality and bugs in the system. Instead of being a mythical creature a tester should be intelligent and self aware. Testers are not “the final gate of quality" but “the defender of the quality”. They do what they can and sometimes must to defend the quality and provide as much information about the current quality. That may include healing (fixing) of some sort and other gimmicks that improve the quality, but I think it’s not the essence of testing.

Now some people may recognize themselves as testicorn (at least I did - I was one in my early testing career). Are you the beautiful and wise testicorn, who uses its abilities for good, or the long faced, horny (no pun intended) testicorn who has been greasing the machine of tyranny?

Responce to "Oy you tester!" By ElizaF

I was strolling casually in the Twitter-land and I saw this link. It was a blog written by ElizaF (maybe that's her real name, but who knows). She had a post that was a bombardment of questions. It got me thinking and suddenly I started to write up the answers to those questions. Here they are:

What do you think when you see people talking about spending three weeks writing tightly worded test cases before they start testing a product?

• What would you think of a person refusing to do it? It there is pressure from organization is it worth your employment to refuse? Testing based on strict test cases forces you to forgo your creativity (unless exploration is allowed or better yet encouraged). But it’s not what I think when I see that kind of a thing (and boy do I see that!) but what I do. I speak up and try to my best ability steer the testing process towards enoughness, quickness, agility, exploration -> TESTING! Away from checking, towards testing.

What do you say when you read of people describing their work environments as Agile with elements of Waterfall?

• It can be agile-ish, but there no agility in waterfall! This just means that a PM or some high level dude doesn’t understand testing to any extent. I would stamp “TEST EARLY” (better yet: “TEST NOW!”) onto his/hers (from now on I speak genderless when I say “he”) forehead so he will remember it every morning he brushes his teeth. Even the most top level people should be aware of what testing is and why is it so important!

What do you comment on blogs where people write of successfully using a commercial tool time and time again project after project?

• Good question. I almost answered “I don’t.” but they may be posting things like that due to insecurity of trying things that FIT THE CONTEXT. There may be reasons behind the choise of tools that are unclear to me and they may be good ones. I however have not stumbled upon such blogs. Any hints?

How do you reply when people ask  on forums about where to get the answers to the ISEB exams?

• I say: “You don’t need the answers. Better yet you don’t need the exam!” And to be realistic the questions in ISEB tests are mostly obvious and you can answer to them without any knowledge about testing -> you just need to memorize a book. And that’s it. You will almost never use the information in the cert for anything (except you know to stay away from people that require you to). Instead of cert you need knowledge about your domain, procedures, tools, practices and make them work for you. Professionally (or CV-wise) you could take the test but relying it to bring you competence over doing testing is where the certifying goes wrong.

What impression do you have of other testers who profess to having no knowledge of who the leading speakers and thinkers in the world of QA are today?

• I know some. Not all. I know those who I think matter in the field.

How do you engage with other members of your profession with whom you disagree with or whose outlook you consider yourself to have evolved beyond years ago?

• This’ a tricky question. Be it the language barrier or whatever but this is how I understand this: “What if you think some other way to approach to something is better than the one who is suggesting an approach?” I need to know the reason why they support the chosen approach and try to understand the meaning behind it. If a valid reason is presented then it might be the best approach for the context. Or there may be some room to improve on which case the change must be found.

How do you write about QA people saying they took a long time to replicate and report a bug of which you have no knowledge but the amount of hours mentioned sound like a "long time" to you?

• Instrument (run in debugger, record GUI, kernel, etc) beforehand so less time consuming replication is needed. I would find out the context on which the system has been tested and bug found. If customer found the bug then we need to know the environment and such. We cannot say something took a long time unless we know how long it should take (mostly we don’t).

Do you talk to people who are not in perfect alignment with your way of thinking like they are below you? Do you think that evolution as tester can only be achieved by going on a certain course and anyone not following that path is below you?

• No reason to rate people by their current way of thinking. What should be done is rate them by the way they have improved their thinking. Some people do box themselves and never change POV. These people should be looked down upon. The self improvement may be just as effective as a course but sometimes preachy people make your heart turn into a certain direction which may seem the right direction for a while. Look into facts and whether the thing you are doing fits the context you’re in.

When I got to the finish of the blog I saw that this was a trap. "What do you think of yourself for thinking things like that?" I started to scroll what I wrote and came to the same conclusion as ElizaF:  

If we have the skills and the knowledge to consider ourselves as good testers we encourage others to be good testers also!  

There is no backdoor of which you can go. If you have knowledge and you don't share, your not a good tester. If you have skills that others require and it is possible to teach those, teach them or you're a bad tester. Be a good tester. Question your thinking. Think what you question!

Why top performers are top performers?

I made some suggestions in twitter about why does certain people have certain skills. Why is Mozart so skillful at composing symphonies? Or why has Bill Gates been able to build such a vast company? I got some interesting answers (“the lack of schooling”), but none of them was what I was looking for. After reading the book “Talent is overrated” (from now on referred at “the book”) by Geoff Colvin I got the answers I needed. The book was inspiration to this text and I encourage everyone to read the book even if you don’t want to be a world class “whatever” (surgeon, tester, manager). I tried to quote everything that was directly from the book


Why top performers are top performers? - They practice.

“Duh?!” you might say because you practice too. Most of us practice testing during some testing dojos or at courses. Some might even practice some during normal work. So how does practice make us top performers? Shouldn’t we all be the top of the game since we all practice to some extent? “I have tested for 20 years and I’m still not top performer. How do you explain that?”

Experience doesn’t count! Most of us work in the field we’re in and are just fine at it. We know the basic thing we need to know to get by with our daily work. The book explains that there is no difference between a new worker and experience worker unless the experienced worker has done practicing during ones career. One is not a master of a certain field with ONLY years of experience, but one needs to master the skills and it usually takes years to hone those skills. “Maybe the top performer was born with the gift. Maybe he was a testing star at birth!”

There are no gifts or talents or whatever. There is no innate talent, says the book and points our years of gene research. We create the talent with dedication and practice. “Ok, but how do you explain that great violinists come from musical families or lawyers come from lawyer families? If they have it in the family how come it’s not innate?” Family has some effect on the skills people accumulate. I will return to it later, but suffice to say now is that there are some environmental factors that help us (or hinder us) in our quest to be better. One might ask whether Michael Bolton or James Bach were testing gurus at birth. As I recall Mr. Bolton was a comedian and Bach was a software developer. Neither probably didn’t test software at the age of six (don’t know but I assume). What they did to be so prestigious was that they honed their skills and became better than most of us.

Nowadays we have higher standards and we need to be more successful than those before. The bar is higher for us to be great. Take science for instance. In the past people like Newton got the status of great physicists quite easily (for our standards) He just explained gravity. “I can explain gravity! Why aren’t I considered a great physicist?” The standard was different for people before us because they had not the access to information we have today. We need to better than those before to be considered great. “Do I need a brain the size of a mountain and memory capacity to match today’s super computers?”

Memory or intelligence has nothing to do with success. High IQ is not a perquisite for being top performer. IQ doesn’t measure critical thinking, emotions (and the ability to use them to our advantage), social skills, honesty, wisdom, etc. Even if you do have very high IQ you might not be a top performer. Look at the membership list for MENSA and count all great performers from the list, then compare it to those that are not members of MENSA. You can have high IQ and be a top performer but it is not required. Same rule applies to memory. Some geniuses (not all top performers are though) forget all kinds of things but are still considered great in their field. What the top performers have what the rest of us don’t is domain knowledge. I’ll return into that a bit later.
“Again: What makes us great?!” If we all practice and increase our knowledge, and only some of us become great, what is it with practice that excludes some of us from greatness? “Am I doing it wrong?”

Well… You probably are. The top performers practiced just like the rest of us but they did it with purpose! They practiced deliberately. And they accumulate domain knowledge in doing so. What we “regular” people see as practice is the things we do to entertain ourselves with domain related things. We might “practice bug hunting” and try a few things and say “Now I practiced the skill. I must be better at it.” You probably ain’t… In fact you just might be worse as you perhaps didn’t know what you were doing and thus did some essential thing the wrong way. You probably did some exercises that you knew you could do or played with software you had played with before.

Deliberate practice is practicing those areas that you are not good at. And doing that systematically! The thing with deliberate practice is planning what you want to achieve and knowing the road to get there. It may not be fun, but getting to the top of the game is not about fun but being the best. A deliberate practice needs to be designed to suit your needs and to hone skills that you’re not good at. Basketball players may shoot hoops in the evening as they know they can do a hoop from three point line. Pro basketball player shoots hoops from half court. If he/her hasn’t the strength he goes to gym and shoots again. And again. And again! Then he/she shoots hoops while moving, etc. In deliberate practice one moves away from comfort zone and practices skills he doesn’t yet master. You cannot learn in your comfort zone but you need to know not to push too hard or you’ll panic and there’s no learning. It is all about increasing the performance of a certain aspect of our domain.

As we all know we might not know what to practice, therefore we do what we know. So we need tutoring. The tutoring provides two things: guide and feedback. The tutor may or may not be a physical person but a goal we aspire to reach. For instance we may look at a video of a guy doing a yoyo trick and look at it closely and try to replicate the trick. Or we could have a coach that guides us during football practice. As in sports and music, we can get tutoring in testing and programming. Tutoring helps us knowing where we want to be and means to get there. We just need a person that is more qualified in some area that we seek to master and get him to coach us. Of course it’s not THAT simple, but we all know the basics of tutoring and coaching. Preferably your tutor is the current best in the field.

Just as much we need guidance we need feedback. At early stages of our lives our parents provide feedback. “That’s a beautiful drawing.” But when we start practicing deliberately we need constructive feedback. Again the source of the feedback need not be a person: you can watch yourself do the yoyo trick from a video also. You can even record yourself shooting hoops, but in the hoops case the ball going to the basket is feedback enough. Tutors provide feedback. We can do some amount of things without systematic feedback but we may learn to do things inefficiently or wrong. In testing the feedback can be bugs. The systematic and good feedback is the kind that makes us find those bugs more efficiently, easier, more, etc.

And when plan is made and feedback is given, you repeat. By repeating you can hone that skill. Masters make things they do look effortless and they may be. They have practiced that skill so much that they no longer have to put all their effort into the basic mechanics of things. They do not, however, automate themselves. The book presents an example of Tiger Woods teeing a golf ball. Someone coughs at the audience and he stops the swing in the mid-motion and starts again. The rest of us just let the swing go to the finish and hope it’s a good one. The masters are so aware of the things that they do that they don’t need to focus on what they do. They focus instead into HOW they do it. They seek flaws in their manners and techniques and practice those skills to do them better. “Where does all this practicing lead?”

The top performers are able to perceive more than the regular people. For example by looking at an X-ray picture a novice might see only the most prominent features and describe them. The top surgeon might see the little details that determine whether the person has cancer or pneumonia or a hidden heart disease. They see more with less looking. They see the subtleties and details possibly ignored or misinterpreted by the rest of us. They uncover hidden things with the same information we have to cover only visible or most prominent things. Top performers understand the significance (or insignificance) of subtleties and details. When bug hunting testers need see things they aren’t looking for. Seeing things you don’t look for and looking for things you don’t see. Deliberate practice helps you perceive more with less information.

It all sounds simple when it is said like this. The book has tons of information about the subtleties of deliberate practicing and many references into practical things. Music, sports, chess. They all have a their intersections with deliberate practicing, even some highly specialized learning techniques to practice in that area. Even business, programming and testing have their own special ways to build skills in domain.

We all still need things that motivate us to practice. The practice in itself may not be motivational so we need things to support the practicing for us to keep doing it. We need a supportive environment to practice. Mommy and daddy were sufficient enough in our childhood with their comment and encouraging, but in business world we need a different support. Some of us don’t have the possibility to have a tutor but still can use the basic methods of deliberate practicing. We can do practicing while we work. The working environment may or may not support you practicing so you need to be able to do deliberate practicing in the midst of the daily labor. “I just said before that I do practice while I work. What are you saying?”

The practicing at work can be done in three parts (and should, might I say): before work, at work, after work. This doesn’t exclude that you can’t stop to think the “after” part during the workday, but all require some thinking and time so before and after work you most likely have enough time to think these things. Before work practicing is setting goals for that day. It’s not about some high level goals like “I’m going to do my best today” but specific goals like “today I’m going to find out why my build fails the first time I commit it every time”. The goal is to produce a working build and to find out why it had failed the previous time. After goal is set you should plan how to get there and you need to be exact! Then during work the key to practicing at work is metacognition (this is also from the book) that is thinking that you think, knowledge about your knowledge. You can study yourself when you do certain activities at work and go through what you really are doing. Then spot any flaws and make corrective measures to succeed in the task. And think what you do! After the work analyze the performance. Use this analysis to increase performance the next time at work (or practice it before hand if possible).

Motivation is a tricky thing to maintain, so I'll delve into it in a separate blog post. What you need to know is that motivation may come from within or without. And it all supports the practice.

All in all: We’re not all going to be super professionals but we all can be! If we truly want to be extraordinary there is a way to be! It takes hard work and dedication, sometimes sacrifices, but a way is free for all of us to be top performers in our chosen field. Just practice deliberately and hone those skills!

The first impression bias

Hi y'all!

I just started in a new job in a new city so everything is new to me. I still live in Tampere (a midland city in Finland) but I commute to Helsinki every day (about 180 km). The job I got is a Quality Engineers position at F-secure Corporation in Maintenance team with (hopefully soon) some managerial tasks regarding Customer Care and whatnot. Most of all testing, testing, testing. Enough of that! My point was that when everything is new you get to look at things fresh eyed. I didn't say biased as I'm TOTALLY biased when it comes to all these new things as I compare them to my previous jobs. I just have a different perspective onto things some people have been looking at for years. I have longed to work at F-secure since I first started to dabble with testing and now that I have the chance I embrace this opportunity with every cell of my being. (This is to make my boss happy! ;) ) But nonetheless I have be critical as I am the new guy and I may have an impact onto those settled ways of doing things and maybe even improve them. We'll see how this goes...

In the Ohjelmistotestaus.fi -blog (which is in Finnish, sorry) Antti Niittyviita wrote about first impressions. He said (free quote) "The first impression is an opportunity to get new information of the quality of the product or service. It should not let get wasted!" When I got to the company I though "Wow, this is quite rigid stuff", for the security part was mandatory and quite formal. Actually on retrospective the event was less than formal, but I think I was so nervous.

Later this week I got to get to know the products from public website so that I get a basic knowledge of what this all is about. I thought about Antti's blog post and took a critical and learning attitude towards the browsing of the products. I was amasing how much defects, issues and such I picked up from public website! The new perspective really opened my eyes.

The testing I got to do the first week got my critical thinking up and I ended up making observations that might have been insignificant to others but I reported them nonetheless. As it turned out these thing teached me quite a bit about the behaviour of the product and the platform I ran on. They were no bugs but something that were not documented in a sense that I would have been able to learn from documentation or from tutoring. So the first impression and critizism thereof teached me something important.

The first impression also has a negative side. If you get a first impression about something you rarely change the attitude if you don't have to analyze the cause of the dissatisfaction. For instance a product that is thought of as slow and reduces performance on your computer (my sister-in-law said she hates a F-secure because it slows her computer down and eats the disk space) doesn't get to change the consumers opinion if they toss the product. If the bad image has gotten through to customer, the dent in the image will be hard to repair.

--

As testers we have the job of trying to test on the best ablity possible. We all get a first impression on something and we may guide our next move by the impression we get. This applies especially in exploratory testing. We guide our next step so that we take in account what we have learned in previous steps (in steps I don't mean scripted testing but actions we take during our testing). Because it is very power tool, we should use first impression as a first guidance tool for our testing. To remain critical we NEED the first impression. What is the first thing we feel might just be right.

As the first impression might be a inference instead of observation we must be careful not anchor ourselves onto the first attribute we percieve. Here the critical thinking of a tester comes to fore. We need to be able to recognise what we observed and afterward make assumptions from those observations instead of inference. By making judgement upon inference we lead ourselves into a trap and start thinking we something that is not there and miss things that are as irrelevant.

There are some basic tools to guide ourselves away from anchoring ourselves onto a inference, but I ain't gonna go through those here. Point being that we have to remain critical even though we have indication to start thinking something about the product/service/whatever. The first impression is a good tool just as long as you remain critical and don't let yourself be anchored onto an attribute that you have infered. (Here's the difference between inference and observation)

Hopefully I remain critical in my new job and start defending the quality of those products we offer. The first week is behind now and new challenges are coming. I hope all my colleges have the nerve to cope with my bad humor and loud voice. :)

Testers are rock stars!

Teemu Vesala wrote in his STC blog comparing testing to poetry. It began a thought process where I found myself comparing testing (noun) to classical music. "Testing is like composing classical music with all the nuances involved" was pretty much the message of my. This was the beginning of tweeting that went like this:

Teemu Vesala:
Poem writing and #testing has plenty of common. See my blog entry @ #stc http://ning.it/jF60aR

Pekka Marjamäki:
@teemuvesala #Testing is more like composing classical #music: even the tiniest sounds change the feel to it. And testing tells a story.

Teemu Vesala:
@pekkamarjamaki Acctually #testing is much like any art. No matter if it's poetry, music or painting. All have same properties.I love'em all

Testing is comparable to the art because they are basically built of same thing (both in their respective context). But which would not be comparable to art? What is art?

Wikipedia (ah! the an inexhaustible source of information!) describes art like this:

Art is the product or process of deliberately arranging items (often with symbolic significance) in a way that influences and affects one or more of the senses, emotions, and intellect.

Can testing be culture? Can there be testing without it being part of the project contextual (I made up the term. Wonder if there even is a term for a thing like that…) culture? Testing is THE element that offers information about the pretty much anything, because testing occurs on so many levels: from unconscious questioning and doubt into systematic examination of the whole system both dynamically and statically. So, yes, testing is a significant part of the culture (the context) in which the testing is done.

Testing is a set items (techniques, approaches, tools, strategies, etc.) that are deliberately arranged so that they form a basis to achieve a wanted result. As we implied before, the test can be performed adhoc, but as art, it does not always reach the goals that are sought. Testing is at least partially subjective, because it is the opinion and interpretation of the task at hand. By removing the subjectivity of art it is possible that ideals of beauty or a non-intelligent interpretations of things are forced to the spectator. The subjective nature of art is a force that makes art – well – art. The nature of the testing is subjective, which is definitely an asset to testing itself, because different people interpret different things in different ways. We can always say that "This thing is X. Period!", but it is not necessarily correct in context. The contexts makes subjectivity important.

We can not underestimate the importance of what feelings are to testing and testing to feelings. Both of these are factors that affect each other. In art, the composer, painter, choreographer, sculptor, you-name-it have feelings or emotions. These are driving forces that will create art. Of course there is art done without feelings, but is it art? Or is the lack of feelings a feeling itself? Is all that is art on some level? In addition, there is an art that does not affect the feelings (in a given context and subjectively). Art is trying to be bi-directional with the emotions. Testing tries that too: feelings (understanding the risks, prioritization, etc.) drive our testing forward and testing is guided by the feelings to help in our decision-making.

Art is expression, communication, statement and pleasure / annoyance. This is true for testing, word for word! Testing is expression: we express the current quality of the system by testing. Testing is communication: There is no other way to express and inform the quality to the stakeholders. Is there? Can quality be expressed by programming? It creates the quality which the testing expresses. Testing does not make quality. In addition, when it comes to pure communication, testing is a social event where communication between groups is important! How can we express the quality, if we do not communicate it to the stakeholders? And testing makes a statement: Michael Bolton said "Testing is defending the quality of the product." Testing comments the quality, testing defends the quality of the product. If testing doesn’t do that, who will?

What comes to pleasure, to me, every day of testing a great pleasure (my testers have undoubtedly noticed that). Every day that I can to promote the triumph of testing both organizationally and the genre produces pleasure. Every day that I learn more about the testing produces pleasure. Every time when I find a new bug system it produces pleasure (displeasure for the project manager ;) ).

To summarize: Testing is art. Testing will be appreciated as art. There are mathematicians, which state that mathematics are the greatest art (graphs based Fibonacci numbers, chaos theoretical curves and fractal art). Testers are therefore artists. We testers who respect ourselves and our skills can hold ourselves as the Rock stars of the IT world (or as Leonardo da Vincies). We bring the joys and the sorrows, we express feelings, even beauty. For me, the test is the biggest art.