Managing Heuristic Exploratory Testing Based on MindMap

The FreeMind mind map template that I use.

The MindMap allows me to create a structured testing quickly and efficiently so that the testing is accountable and efficient. The mind map functions as requirements document, test plan, test report, bug report and heuristics cheat sheet for my testing. The mind map functions simultaneously as the tool and the product of testing thus supporting the values of agile and lean development. I’ll explain how.

Gathering Information

As the picture above states that there are multiple heuristics to choose from and when starting to test the best way to start is to gather information. The “Project” heuristic is a good way to start developing a sense of what to test, how and in what order. Using the applicable heuristics one can form the basic knowledge about the test object and the surrounding project.

This phase may be as quick or as time consuming as necessary. The key is that you’ll establish the needed knowledge about the test object. This may include (but not restricting to) referring to requirements documentation or other documents, talking to product/project/solution owners/architects/managers, requesting consulting from developer/other tester or the like; anything you see helpful to your task at hand.

At this stage you’ll start also forming you testing plan. This may include equipments and resource scouting or gathering, scheduling and possibly limiting the scope of testing. By limiting the scope into testing that can be done in 90-120 minutes sessions the testing is more manageable and less tedious. Forming the testing into specific missions also makes your testing more motivating as you’ll achieve lesser goals on your way to larger goal.

At this point you have a mind map that looks something like this:

Project heuristics during Information gathering

You should have sought answers to the questions or decided they do not need answering. This forms the basics of your test planning. At this point you’ll also have a good grasp of what is important and should be tested first. It is possible that the priorities change during testing (and it should) as does the scope. This is however the framework in which the testing starts.

The gathered information is context-dependent and varies across domains. The mind map may also be tweaked to suit the domains special needs. Also the items/tokens may be chosen to best suit the purpose either increasing or decreasing the amount. What I use are Question (“Someone needs to tell me something regarding this topic”), Exclamation (“There is something in this topic that needs to be looked into in detail”), Idea (“Possible solution, test idea or a thought”), Blocker (“This topic might cause a block to testing now or at some point”), OK (“This topic is tested and considered not needing further testing”) and NOK (“This area requires more testing at some point”). The most important thing is that the choice of items supports the cause instead becoming the cause itself.

How to measure coverage?

As we have now established the basic knowledge behind the testing task we may start to sketch the coverage and depth of the testing. This is done by modeling the test object. Modeling is a simple way to map or… hmmmm… model the test object. This is done by creating a representation of the test area using the modeling heuristics. The models may be as detailed as needed or as vague as they can be, again what bests suits the context.

I myself start the modeling by mapping the structure of the test object. In case of GUI testing I try to sketch the basic layout of the GUI forms and deconstruct the components within the GUI. For example some text fields can be tested with same inputs (possibly). This also gives a good representation of the depth of the GUI (as if there are any forms /behind/ the previous). Menus are a good guide in deconstructing the GUI as they also may hint onto what functions the system has to offer.

The structure may consist of anything “physical” or fairly easily recognizable objects of the test object. The operations of a web service are easily modeled as are the fields within the operation. There are possibly even tools (spiders) that will form the structure model for you by minimal work. This is the first coverage model that I use and frankly the most simple. The coverage may even be derived into X/Y –coverage to sate the managers’ need for figures, but this is also to remind you of what has been done and what needs to be done.

Functional coverage is the list or the model of what functions the test object has to offer. This may also be somewhat easily derived from visual/physical things but also the hidden/hard-to-come-by functions should be stated. The function may however be used in a different way that it was intended so user/admin/automatic operations should be stated also either as their own or where they differ from the intended functionality.

The data coverage is important to some and should be deconstructed into a heuristic list. Easiest way is to find out what kind of data the structure requires and what is in the interface requirements or in database. Also analysis of what kind of data is it possible to insert into system and HOW! There may be data security issues in public interfaces or GUIs that need to be tested with different approach or depth.

Platform may affect all tests and if there are any platform dependencies they should be stated. Sometimes it is not seasonable to test everything in every platform so the criticality of the testing should be estimated on each platform. Sometimes test automation and changing the test approach may come handy if testing multiple platforms (i.e. mobile phone testing on different phone models).

Time-related testing is also a way to measure coverage of the test object. If there are time-related testing to do (performance testing, stress testing, concurrency testing, “24/7 testing”, etc.) these are also a way to measure coverage. Performance coverage may be important in a SOA world where critical components require consistent and good performance.

These form into a map of models of which we can inspect coverage and progress of testing.

The coverage break down using Models heurisrics

Heuristics support testing

The use of heuristics enables you to think of multiple ways to test something. One can apply many different techniques onto ones testing to dig up different kinds of information about the product. They can be test techniques (remember that exploratory testing is an approach not a technique), quick tests, tours, you name it. The point is that the heuristics mind map functions as a checklist (if that is the way you want to use it, for example coverage reasons) or as a heuristic in itself.

Whatever the approach to testing is the heuristics are there to help you in your thought process. Using the mind map to record your thoughts about some aspect of testing is advisable and easy, as you can connect techniques to models and parts thereof. For example using quick tests (“clicking frenzy”, “banging the keyboard”, etc.) to stroll through a GUI or flow testing to do some E2E testing, the mind map is a good tool to design testing on the go and to report any findings into the map.

The mind map can be used to report bugs using the video clips generated from test sessions in addition to possible description in the mind map. These help communication to developers, managers and clients, and writing the defect reports. The issues and questions that testing uncovers can be linked to oracle heuristics to make more case to the defect.

For example a basic stress test may indeed pass within the NFR description but the memory usage or the amount of handles may not even be described in the NFR. Of course it is up to the tester to choose the best way of describing his/hers findings. The rule of thumb is that there should be enough information to help the developers to fix the problem AND to make an appealing case to fix it! (No bug report is good enough if the bug doesn’t get fixed!)

Heuristic testing skills (for example Rapid Software Testing skills) are also important here as one can rely too much on the check list and forget to do the testing in their heads (where most testing is done). Heuristics should be applied (if applicable) not followed. There’s no harm trying to apply some heuristic in a context, but using the heuristic as a hammer and trying to smack everything is not advisable (it might even be stupid). They are there to help you reach your testing goal and provide tools to your brain.

If stuck, the heuristic mind map can give you a good boost into proper direction: just look at the heuristics (if you can’t remember them) and think of what to do next and how. Heuristics work also as guides in tweaking your plan. You may have gotten false information during your info-gathering or bugs prevent you from doing something. Heuristics can provide a solution to a changed situation when a new mission needs to be formed.

The testing heuristics applied

A product and a tool of testing

The agile principle describes unnecessary documents as bad (or that is how I have interpreted the manifesto). Michael Bolton talked about using “memos” in testing to help the brain to interpret all info pushed into it. Instead of being a document (a product which you produce) the memo is used as a tool to help testing. The mind map has a similar value to testing: it helps make notes about things you see, feel, and observe, but also enables you to use these notes as a report of what you have done (if someone asks).

The mind map serves as a tool to facilitate the testing but also as a product of testing. It serves as a platform for communication when describing what went wrong and how. It serves as a report of the testing to increase accountability. It serves as a session report of which the Test manager or other testers can fathom what you did, how and what important knowledge there is that they need to consider.

A finished mind map looks something like this:

A finished mind map with references to other heuristics and links to screen caps and video clips.

The questions should be answered, if there were any. The bugs, issues, ideas, etc. should be discussed and possibly closed. New testing sessions should be organized to cover areas that were not covered.

I used FreeMind 0.9.0 in my example and in my work daily. The example mind map was done to illustrate the possibilities of using mind maps. Normally my mind maps are a bit more restricted to certain type of testing.

The web is full of mind map software from browsed based (like mindmeister) into downloadable tools (like XMind). Just choose the one best suit you (or ask someone for a recommendation) and create a mind map that suits your style of testing. And get testing!

If you have had good experiences using mind maps, please share them. :)

Motivation 3.0

After reading the Talent-book by Jeoff Colvin I had a hole in my mind that needed filling. Why does a person embark on a grueling adventure of deliberate practice? What drives you to excel? Poof! (Somehow the poofing got into my writing, but I can’t fathom why…) In came the Drive, a book by Daniel H. Pink about motivation. Mr. Colvin was quite vague about motivation (or the focus was more on what and how instead of why) so I made a decision to skip the motivation part and go deeper in a future post.

Now that I have read (and somewhat understood) the Drive I feel like I can make a contribution and share some of the secrets about “what motivates us”. The deliberate practice (of which mr. Pink also talks in his book) is fuelled by motivation: If we really want to do something, we can do it. “But why?” Hold on, I’ll share some of what I have learned. (This is not a book review but interpretation about motivation using the book as guide.)

A little history about motivation: When we were cavemen, Neanderthals or Finnish ski-jumpers, we had a drive (the book calls it Motivation 1.0). That was the need to feed, drink and f…. reproduce – the drive to survive. “Hey look! we survived!” And that motivation served us to get by, to fight cougars and wolverines, to eat nutritious food. But as humans developed the Motivation 1.0 did not serve us properly; we needed something more to motivate us. We needed upgrade to the Motivation 1.0 to motivate us to do our job. The second basic drive that motivates us is the extrinsic motivators: rewards and punishment, carrot and stick, etc. The book calls it Motivation 2.0; the upgraded way to operate and motivate us. It served us well with basic-straightforward –repetitive tasks like assembly line or the like. The system awarded us from good behavior and punished us from bad behavior. The managers of that time thought they could use incentives like bonuses to motivate people to do their bidding. And people did… when their jobs were “basic-straightforward –repetitive”. Nowadays the menial tasks have been automated and people are moving towards problem solving, innovative, creative jobs that are hard if not impossible to automate. The Motivation 2.0 doesn’t support that kind of work. In fact it may result in unwanted behavior and to diminish the wanted behavior. And how many managers and companies have seen this?

The DRIVE introduces a third drive that is called Motivation 3.0. It is the intrinsic motivation that we all have. A person that responds to intrinsic motivation may not respond well to extrinsic motivation. The book uses example from children’s play.

If a group of children are asked to paint or draw, they all draw. Kids love painting! But then half of the children are told that they can get a reward from drawing. What happens is that they all draw for a moment but the rewarded children are less likely to continue drawing after they get the reward as the children that were not awarded continue drawing like they used to. The award turns play into work. And that de-motivates us.

There are however some ways to get “carrot and stick” working to increase out motivation. The “if-then” reward may cause a moment’s burst in motivation and efficiency but they are prone to do down very quickly. “If-then” reward may lead into addiction (as in “I won’t do the same task for the same money anymore” or “I want TWO candies to take out the trash”) or suppress the wanted behavior. When used as a “now that” award it can sometimes encourage a person to create an intrinsic motivation to reach the possible award. For example an Olympic runner has a set of goals (to beat personal best, country top ten, etc.) instead of just one; the Olympic gold medal. They may try to reach that goal but they aren’t looking the medals as “If-then” but “now that” reward. The same drive, well, drives us forward in our work and personal life. We tend to seek goals that are fuelled by intrinsic motivation and steer away from extrinsic motivation (or at least most of us do). In most of us there is both kind of motivation, extrinsic and intrinsic, but what makes us most tick is the inner drive to accomplish something, not what others want or tell us to be.

Daniel H. Pink has studied the field of human motivation and cites numerous studies from psychology and economics in his book. He has narrowed down the intrinsic motivation into three main elements that make the intrinsic motivation work: Autonomy, mastery and purpose. Those are not elements that Motivation2.0 willingly supports or acknowledges.

Autonomy

We people have the need for autonomy, need to be free. Nobody WANTS to be… ermmm….. not-free. The managerial style that supports extrinsic motivation focuses on compliance (“Do as you are told”). Instead the intrinsic motivation focuses on engagement. With autonomy people are able to engage into their job and be more satisfied in it. The old style world says “People need to be managed” but in truth (on a number of studies show) that people need autonomy to shine. Autonomy over Tasks, Time, Technique and Team (4T) invokes the inner motivation and boosts our performance, get us into FLOW! Companies like Google, Atlassian, etc. are great examples of autonomy.

Even though autonomy motivates us it needs to be well thought of. For example autonomy should be a perk that encourages people to CHOOSE some of the things they do, CHOOSE the time they do it (at least on some level), CHOOSE the way they do it (in the frame of what is accepted or tolerated) and CHOOSE who they work with.

Atlassian shows a good example of autonomy over tasks. They hold a “FedEx day” every quarter. The “FedEx day” comes from “delivery overnight” and it basically means that people can choose ANY task they like (a problem waiting to be solved, an innovative idea) and they have one day to make it work. In the noon of the next day they all have to present their product, solution, whatever to other co-workers. This sprouted some of the most innovative things they’ve done. The problem was that they only had 24 hours in a quarter to do it. So they changed it into 20% of all working time. This “20% time” is now where innovation happens.

Where Atlassian excels at Task autonomy, companies like Best Buy, Meddius, F-secure, etc. excel at autonomy over time. “When people don’t just show up and grind through their day they’ll be more productive.” Scott Adams, the creator of Dilbert, says: “Nothing is more important to my success than controlling my own schedule.” Autonomy over time gives you the freedom to do things when you are on your most productive state, in the FLOW. Monitoring is again needed but trust also motivates people; trust that they will show up and do their work.

Autonomy over technique is the corner stone of craftsmen. Take art for example where new techniques are found every day. There would not be Monet’s pointillism technique if he had not the autonomy over technique he uses. If people are forced to obey the rules by which method they work, they tend to be unhappy. If exploratory testing would be forbidden in where I work it rules out so much testing done and so many techniques I use, I couldn’t do my work efficiently. The choice over HOW I do the things I do is for me the most important aspect (as you can see from “Best Practices (NOT!)”).

Autonomy over team is probably the most difficult one to achieve. Some companies have done their share in team-wise autonomy by letting the workers into the recruiting process. For example, F-secure Corp. has an additional step on in the recruitment process which includes a team interview. All candidates get a chance to meet the team and mingle a bit. Then the team says their opinion about the guy and a decision is made. Other companies (like Whole Foods) have removed their executives from the hiring process and let the team members do the hiring. Then after 30 days trial time they can decide to “keep ‘em” or “lose ‘em”.

Other form of autonomy over teams is the free time activity with the co-workers. The autonomy over team presents itself with off-time clubs and sports teams that the workers put together. After being made the social connection to people it is easier to communicate even with harder issues in the work place. After you have a choice of the people you work (or play) with the work has more meaning and is more fluent.

Even though autonomy is a hard biscuit to swallow, companies eventually move towards autonomy. Though some managers still think people are from the same mold as they came in the 1910’s we all strive towards freedom. If we weren’t there would be no companies like Google or Zappo’s. And by giving people more choice over the four T’s (Tasks, Time, Technique and Team) they just might perform better and be more creative than their counterparts that are forced to comply instead of engaging.

Mastery

Motivation 3.0 allows us to be more engaged in what we do. Instead of complying other people strict rules we seek to accomplish goals that are closer to us and thus being more close to the subject we work with. Engagement has also this weird way of encouraging towards mastery. By getting closer to the subject we find a need to better ourselves in doing the thing we do. E.g. by being allowed to program without the boss breathing to your neck, one just might find an innovative solution to a problem. Mastering our jobs motivates us to do our jobs better.

I already had a few words about getting better at what you do (in my post "Why top performers are top performers") but mastering is a motivating thing as well. There might be no need to motivate you separately to master a field as the prospect of mastery provides the motivation needed.

When people reach a flow in what they do (check out books about the Flow) they usually are more prone to searching mastery. They notice that they can achieve something they do with ease and start increasing the performance and the difficulty little by little. This is what the book calls “Goldilocking”; not doing things that are too easy (they get you bored) or things that are too hard (they get you anxious), but doing things that are “just right”. Jeoff Colvin describes this in three circles inside the other. The outer layer resulted in boredom and the center resulted in anxiety. By staying between boredom and anxiety is the optimal place for achieving mastery (and learning). But it seems that mastery is a fickle thing…

Mastery is a mindset! If you don’t think you can achieve mastery then you probably won’t get it. In the book there was a good example about intelligence and development thereof. If you think about your intelligence as fixed amount of knowledge then your intelligence might not increase from what it is currently. If you think your intelligence as a muscle that you can train it most probably grows. As I described in the performance post, there is nothing preventing you mastering the domain you want except yourself.

Mastery is PAIN! Every single athlete knows that without pain there is no result. Only by repeating the exercises, training programs, etc. can mastery be achieved. There are no shortcuts to mastering a domain only hard work. And to top it all mastery is an asymptote. One can never truly and fully master something as there is always more to learn. That is one of the most appealing things that drive us towards mastery: it is almost impossible to achieve. The fact that full mastery cannot be achieved is both frustrating and alluring and it seems that it is the process of mastering an art that drives us towards it.

Purpose

The third element of intrinsic motivation seems to be a sense of purpose. We rarely do things (and enjoy them) if we don’t know answer to the question “why”. In companies that have large engines of bureaucracy and command chains the purpose of doing something is easily lost. Doing something that “makes no sense” is a serious de-motivator to us using the Motivation 3.0. Companies seek purpose to make their product/service more appealing. Companies support developing countries and thus make a purpose for business; it’s more fun to work if you know that your work has a meaning.

In business world the purpose is something of a nicety; you can live without realizing the full purpose of what you are doing. In the TV-show Friends this guy Chandler is a data analyst who has no idea why he does he does. By realizing the fact that there was no purpose for him to do the job he lost his motivation. If you feel you don’t know why you do something, start asking “Whose purpose is it anyway and what have I got to do with it?” We need to do our work/live our lives to accomplish something and Purpose is “the something”. There are three basic pillars to support the purpose motivator: Goals, Words and Policies.

Goals work as a milestones to purpose. By setting different goals you feel you are accomplishing something even though you have some path to tread until you achieve some bigger goal. By setting both performance goals (getting an A in a math test, reaching your sales quota) and learning goals (being able to understand and apply the math, being able to fully understand the need of a customer) their overall impact on your motivation is optimal. Performance goals support the extrinsic motivation and learning goals build intrinsic motivation. The goals we set pave the way to the purpose and as we strive to reach the goals we move towards our purpose.

Words are fickle. Words can change our way of behavior. Think about vows. If you vow something (like a “I speak the whole truth, etc.”) it sets you a standard, a purpose, to strive to. When things are said out loud they are stamped into our brain as a code of conduct. When a group of Harvard graduates in 2009 saw that the economy is going downhill because of economists and business people cheated or cut corners rather than practice ethical economics, they saw that they could make a chance. They made “the MBA oath” (just like a Hippocratic Oath) to “strive to create sustainable economic, social, and environmental prosperity worldwide.”

Words are also used to measure our behavior. Listen to your co-workers talk about your company. How many times they use the word “they” to refer to the company instead of “we”? The “we” signifies that the company is coherent and everyone thinks they are in the same ship sailing towards common goal.

Policies are also an aspect of purpose. They overlap a bit with both Goals and Words as they are a part of both. Policies refer to ethics and good manners. Wrong policies can push towards unethical behavior or even discrimination. Let’s say we have a HR policy that goes something like this: “All members of the company should treat each other with respect and kindness.” The atmosphere in the office might just be open and friendly and all conflicts are resolved with a friendly talk. Let’s say if they change the policy into: “Abusing, harassing, bullying, physically or mentally assaulting a co-worker is strictly not allowed with a penalty of a fine. Racist or sexual comments, behavior, writings…” Now that most likely sets the environment not possibly to encourage bad behavior but at least make it more controlled and closed. Policies allow people to pursue the Purpose with their own terms.

Motivation 3.0

The science has known the third, intrinsic drive for many decades and it is researched by many. The business world just hasn’t grasped the Motivation 3.0 yet. The “carrots and sticks” are for cavemen; the new trio is what motivates us today: Autonomy, Mastery and Purpose.

See more about the Drive from http://www.danpink.com/drive.

Have you been spotting mythical creatures lately?

Right... This is no after-the-weekend delirium post but a testing post instead. At least I think so... The other day I was browsing the magick land of Interweb and stumbled into a word "unicorn". I stuck into my mind and today as I was travelling to work I started thinking what mythical creature I would be (I have a soft spot for fantasy literature and all sorts a coo coo things). I thought about the unicorn and poof - a testicorn came into mind.

What a testicorn is? Here a list that I came up:
- mythical creature no-one really understands
- avoids human contact
- has a protruding thing coming from the forehead
- a long face like that of a horse
- its blood will cure the sick

When put into context the testicorn is much like a tester in some bad apple companies (no pun intended). At some companies a testicorn is a lonely creature that lurks in dark corners of the Developland trying to do its magick. He is rarely fully understood except by other testicorns who sometimes try to rise against the tyranny of the Developland, shy creature that rarely comes out in the open anymore. These creatures are forced to behave like they are expected to; work as to detect wrongness in the universe and report the findings. Sometimes they are forced to present a magick scroll into which they must write the spells they use to get rid of the evil but are rarely encouraged to cast spells from their memory or develop new spells to get rid of a certain kind of evil entity. These spells are then stored into a library by the head wizard and the spells are given to new testicorns when old ones are expended. "Testicorns are of plenty. Why try preserving them? We have new testicorns coming from the School of Magick as much as we want!" And testicorns retreat into their magical forests to do the things they are forced to do.

At first the testicorns didn't have a "horn" in their forehead. They developed that during overtime conjuring and after getting their mythical rights trampled upon. They also developed a long face with dark spots under their eyes. They do what they are forced to do and the horn grows. A sad thing actually as at some point (and in some benevolent magical kingdoms) they are regarded as beautiful and wise creatures. They used to be highly regarded for their expertise in evil ridding spells and a mind for the common good. Now they are the grease that lubricates the machine of tyranny.

Testicorns were the healers of the magic kingdoms once. They taught the ways of healing to people. The testicorns still have the ability to heal (and do it with plethora of ways known as Schools of healing. To mention one the School of Exploratory Healing...) with their innate ability, but the blood of the testicorns is known for their incredible healing abilities. The Chief Magickmen use the blood of the testicorn to heal the sickness without regards of the testicorns health. They are bled to heal the hurt. This consumes the testicorns power and at some point they have no more strength and they wither away.

In some kingdoms you see  these mythical creatures laboring their tailed behinds off to get from day to day. At some point they are replaced by a hornless, careless little testicorn and the old withered one gets laid off. A sad story actually...

The kennings above were used as stark image of the way some companies use testers as a way to increase the quality. They are put into a mold and told what to do instead of doing what should be done. They are exhausted with stupidifying tasks and are kept as a scapegoat for bad quality and bugs in the system. Instead of being a mythical creature a tester should be intelligent and self aware. Testers are not “the final gate of quality" but “the defender of the quality”. They do what they can and sometimes must to defend the quality and provide as much information about the current quality. That may include healing (fixing) of some sort and other gimmicks that improve the quality, but I think it’s not the essence of testing.

Now some people may recognize themselves as testicorn (at least I did - I was one in my early testing career). Are you the beautiful and wise testicorn, who uses its abilities for good, or the long faced, horny (no pun intended) testicorn who has been greasing the machine of tyranny?