Managing Heuristic Exploratory Testing Based on MindMap

The FreeMind mind map template that I use.

The MindMap allows me to create a structured testing quickly and efficiently so that the testing is accountable and efficient. The mind map functions as requirements document, test plan, test report, bug report and heuristics cheat sheet for my testing. The mind map functions simultaneously as the tool and the product of testing thus supporting the values of agile and lean development. I’ll explain how.

Gathering Information

As the picture above states that there are multiple heuristics to choose from and when starting to test the best way to start is to gather information. The “Project” heuristic is a good way to start developing a sense of what to test, how and in what order. Using the applicable heuristics one can form the basic knowledge about the test object and the surrounding project.

This phase may be as quick or as time consuming as necessary. The key is that you’ll establish the needed knowledge about the test object. This may include (but not restricting to) referring to requirements documentation or other documents, talking to product/project/solution owners/architects/managers, requesting consulting from developer/other tester or the like; anything you see helpful to your task at hand.

At this stage you’ll start also forming you testing plan. This may include equipments and resource scouting or gathering, scheduling and possibly limiting the scope of testing. By limiting the scope into testing that can be done in 90-120 minutes sessions the testing is more manageable and less tedious. Forming the testing into specific missions also makes your testing more motivating as you’ll achieve lesser goals on your way to larger goal.

At this point you have a mind map that looks something like this:

Project heuristics during Information gathering

You should have sought answers to the questions or decided they do not need answering. This forms the basics of your test planning. At this point you’ll also have a good grasp of what is important and should be tested first. It is possible that the priorities change during testing (and it should) as does the scope. This is however the framework in which the testing starts.

The gathered information is context-dependent and varies across domains. The mind map may also be tweaked to suit the domains special needs. Also the items/tokens may be chosen to best suit the purpose either increasing or decreasing the amount. What I use are Question (“Someone needs to tell me something regarding this topic”), Exclamation (“There is something in this topic that needs to be looked into in detail”), Idea (“Possible solution, test idea or a thought”), Blocker (“This topic might cause a block to testing now or at some point”), OK (“This topic is tested and considered not needing further testing”) and NOK (“This area requires more testing at some point”). The most important thing is that the choice of items supports the cause instead becoming the cause itself.

How to measure coverage?

As we have now established the basic knowledge behind the testing task we may start to sketch the coverage and depth of the testing. This is done by modeling the test object. Modeling is a simple way to map or… hmmmm… model the test object. This is done by creating a representation of the test area using the modeling heuristics. The models may be as detailed as needed or as vague as they can be, again what bests suits the context.

I myself start the modeling by mapping the structure of the test object. In case of GUI testing I try to sketch the basic layout of the GUI forms and deconstruct the components within the GUI. For example some text fields can be tested with same inputs (possibly). This also gives a good representation of the depth of the GUI (as if there are any forms /behind/ the previous). Menus are a good guide in deconstructing the GUI as they also may hint onto what functions the system has to offer.

The structure may consist of anything “physical” or fairly easily recognizable objects of the test object. The operations of a web service are easily modeled as are the fields within the operation. There are possibly even tools (spiders) that will form the structure model for you by minimal work. This is the first coverage model that I use and frankly the most simple. The coverage may even be derived into X/Y –coverage to sate the managers’ need for figures, but this is also to remind you of what has been done and what needs to be done.

Functional coverage is the list or the model of what functions the test object has to offer. This may also be somewhat easily derived from visual/physical things but also the hidden/hard-to-come-by functions should be stated. The function may however be used in a different way that it was intended so user/admin/automatic operations should be stated also either as their own or where they differ from the intended functionality.

The data coverage is important to some and should be deconstructed into a heuristic list. Easiest way is to find out what kind of data the structure requires and what is in the interface requirements or in database. Also analysis of what kind of data is it possible to insert into system and HOW! There may be data security issues in public interfaces or GUIs that need to be tested with different approach or depth.

Platform may affect all tests and if there are any platform dependencies they should be stated. Sometimes it is not seasonable to test everything in every platform so the criticality of the testing should be estimated on each platform. Sometimes test automation and changing the test approach may come handy if testing multiple platforms (i.e. mobile phone testing on different phone models).

Time-related testing is also a way to measure coverage of the test object. If there are time-related testing to do (performance testing, stress testing, concurrency testing, “24/7 testing”, etc.) these are also a way to measure coverage. Performance coverage may be important in a SOA world where critical components require consistent and good performance.

These form into a map of models of which we can inspect coverage and progress of testing.

The coverage break down using Models heurisrics

Heuristics support testing

The use of heuristics enables you to think of multiple ways to test something. One can apply many different techniques onto ones testing to dig up different kinds of information about the product. They can be test techniques (remember that exploratory testing is an approach not a technique), quick tests, tours, you name it. The point is that the heuristics mind map functions as a checklist (if that is the way you want to use it, for example coverage reasons) or as a heuristic in itself.

Whatever the approach to testing is the heuristics are there to help you in your thought process. Using the mind map to record your thoughts about some aspect of testing is advisable and easy, as you can connect techniques to models and parts thereof. For example using quick tests (“clicking frenzy”, “banging the keyboard”, etc.) to stroll through a GUI or flow testing to do some E2E testing, the mind map is a good tool to design testing on the go and to report any findings into the map.

The mind map can be used to report bugs using the video clips generated from test sessions in addition to possible description in the mind map. These help communication to developers, managers and clients, and writing the defect reports. The issues and questions that testing uncovers can be linked to oracle heuristics to make more case to the defect.

For example a basic stress test may indeed pass within the NFR description but the memory usage or the amount of handles may not even be described in the NFR. Of course it is up to the tester to choose the best way of describing his/hers findings. The rule of thumb is that there should be enough information to help the developers to fix the problem AND to make an appealing case to fix it! (No bug report is good enough if the bug doesn’t get fixed!)

Heuristic testing skills (for example Rapid Software Testing skills) are also important here as one can rely too much on the check list and forget to do the testing in their heads (where most testing is done). Heuristics should be applied (if applicable) not followed. There’s no harm trying to apply some heuristic in a context, but using the heuristic as a hammer and trying to smack everything is not advisable (it might even be stupid). They are there to help you reach your testing goal and provide tools to your brain.

If stuck, the heuristic mind map can give you a good boost into proper direction: just look at the heuristics (if you can’t remember them) and think of what to do next and how. Heuristics work also as guides in tweaking your plan. You may have gotten false information during your info-gathering or bugs prevent you from doing something. Heuristics can provide a solution to a changed situation when a new mission needs to be formed.

The testing heuristics applied

A product and a tool of testing

The agile principle describes unnecessary documents as bad (or that is how I have interpreted the manifesto). Michael Bolton talked about using “memos” in testing to help the brain to interpret all info pushed into it. Instead of being a document (a product which you produce) the memo is used as a tool to help testing. The mind map has a similar value to testing: it helps make notes about things you see, feel, and observe, but also enables you to use these notes as a report of what you have done (if someone asks).

The mind map serves as a tool to facilitate the testing but also as a product of testing. It serves as a platform for communication when describing what went wrong and how. It serves as a report of the testing to increase accountability. It serves as a session report of which the Test manager or other testers can fathom what you did, how and what important knowledge there is that they need to consider.

A finished mind map looks something like this:

A finished mind map with references to other heuristics and links to screen caps and video clips.

The questions should be answered, if there were any. The bugs, issues, ideas, etc. should be discussed and possibly closed. New testing sessions should be organized to cover areas that were not covered.

I used FreeMind 0.9.0 in my example and in my work daily. The example mind map was done to illustrate the possibilities of using mind maps. Normally my mind maps are a bit more restricted to certain type of testing.

The web is full of mind map software from browsed based (like mindmeister) into downloadable tools (like XMind). Just choose the one best suit you (or ask someone for a recommendation) and create a mind map that suits your style of testing. And get testing!

If you have had good experiences using mind maps, please share them. :)