Mushroom-picking heuristic

Imagine this: You are in a forest and you are trying to find juicy mushrooms. You'd like to find some porcini, black chanterelle, maybe some yellow bearded milk-cap. Edible mushrooms nonetheless. Before you go, do you need something? Perhaps the following:

• the calendar which shows different mushroom appearance into your local forest (not too close to habitat because mushrooms ingest heavy metals)
• maybe some research on what the mushrooms look like that you are trying to find (also pictures thereof so you don't pick something poisonous that looks vaguely like edible mushrooms)
• some knowledge on how to pick mushrooms (pick them up intact and one piece either by twisting or pulling)
• maybe choose the weather when you go mushroom-picking (the mushrooms should be rather dry when being picked up)
• prepare the mushrooms as soon possible (remove sand, pine needles, moss, etc)
• use proper tools (a knife, a brush, a basket instead of a plastic bag)

These are things that you might want to consider when going out for mushrooms. Now, these are all preparatory things, bits of knowledge you might want to have before actually going to the forest. Obviously you can go without knowing these things, but you may end up with no mushrooms or worst case poisonous bastards like amanita or cortinars. The trip to the woods might have been productive nonetheless - you got fresh air and some exercise, maybe you had a good chat with a fellow mushroom-picker. Not all unprepared trips are totally worthless, but you need some preparations to achieve good results.

How does this translate to testing? You prepare yourself to the testing task by doing almost same things you do when you go mushroom-picking. Like so:

• You check the schedule for the most convenient time to do specific kind of testing. (In January you might want to ice-skating instead of mushroom-picking, which might still be fun.) If you're doing usability testing, you might want to choose a time when there is something someone can actually use. When doing penetration testing you might want to pick a time when there is something to penetrate. Cuz if you choose the timing badly you might not achieve the best results. Also checking the schedule gives you clues on how to time-box the testing.
• You might want to research the subject of testing. What should you be looking for, what things you expect to discover, what are the risks that are already known. Perhaps you want to learn more by exploring the product, by trying things, playing and clicking around, banging the keyboard with a shoe. You might have pictures of the GUI or architecture schematics, maybe a person to help you go about the product. Like mushroom-picking, you can stumble on interesting things, but to recognize the important, the critical, the alarming stuff you might need to do some research.
• You might need knowledge on how to do software testing. By clicking around without a purpose might not be good testing. A good tester has a skillset that she utilizes to perform good testing. It is also important to know the domain, how to perform testing in that particular product/domain/service.
• Choose the weather when you go testing... Urhm... Testability and configuration, choose the best starting conditions, datasets, timeslots, loads, etc. to achieve the best testing performance and to find interesting things. You might want to do testing when the backend is performing poorly or the network connections are bad. Maybe choose a dataset that is production like or maybe it should be a fuzzing test to generate weird data to the APIs.
• Prepare your mushrooms. Deliverables! You should take notes on your testing performance. This is to help you steer your testing, generate ideas that couldn't yet be executed, dot down risks and bugs you find. You can then explain to other people what you did, why you chose to do specific tests and checks, what did you find. If needed, prepare a useful report on the testing you did.
• Use proper tools. Testing is about using tools, obviously. Most important tool is your brain! Use it. Also use tools to help do things that are difficult or time-consuming, keep your concentration while testing. Use scripts when they are useful, record your screen, have quick note taking equipment. Use other people! Two brains are sometimes better than just one... A person can be a tool!

(This is not comprehensive list in any case. You might want to take other context variables into account to achieve the best result when you actually go testing.)

I'm now in the woods with my wellies on, my trusty 'shroom-basket, a J. Marttiini mushroom knife, and a backpack willed with sandwiches, a “Book of Mushroom and Black Magick”, coffee and some chocolate. Maybe a map, a compass, some survival gear if I get lost in the woods. I want to be prepared and tooled up. How the hell do I find those mushrooms?

This is where a Lévy Flight heuristic kicks in (a heuristic mentioned by James Bach at CAST 2014). It is an algorithm by which animals (and humans) go foraging. "When defined as a walk in a space of dimension greater than one, the steps made are in isotropic random directions." says Wikipedia. Essentially doing something in one spot until you move to other area to do something there. So, I stand on the road and head to the woods. I try to look for sweet spots in the woods, like decaying fallen trees, dry mossy areas under pines or furs. If I have dome my preparations correctly I look for these areas and I might know where to find them. So I start roaming in the woods and stumble on an area that has something interesting. A fallen tree! Yay! Maybe I'll find some black chanterelle there. So I look closely at the area and spend time there, perhaps picking some mushrooms or just scouting the area for clues where I might find some. After a while I head to a new location keeping in mind where I am in the forest.

So I move around the wooded area in a pattern that tries to achieve the best coverage of the important areas. The pattern might seem random, but I have a mission which I am trying to fulfill with the choices of direction to wander. When I am halfway on my walk, I might want to go towards the road so I don't end up too far when the sun goes down. I spend time on areas that are either rich in mushrooms or interesting places in themselves. Maybe I learn something while scouting for porcini, something about birds or types of moss I tread on. Maybe I note down areas that have some other mushroom that I am not intending to pick (you don't want to mix mushrooms in the same basket - I don't know why...). I take mental notes, maybe write stuff down, mark places on my map. All the while trying to achieve a goal I set for myself before I went foraging. On my way back I stumble on an abandoned building. Cool! I might take a look inside and maybe I find something interesting. Maybe an old newspaper or a book? I might spend time there even though I was set out to pick mushrooms. This is interesting new place and I want to know more of it. So I deviate from my initial mission and investigate the building. It's apparently someone's old home and its walls have sunken in to the ground. Maybe the architecture of the pre-WW2 era interests me, maybe the newspaper has some information from the "ye olde times". Maybe the book has a letter tucked in between the pages. I allow myself to deviate because this might be more important than mushroom-picking.

Back to the testing world. The woods turn into APIs, GUIs and code. The moss becomes the date on which I tread on. The mushrooms... They're not bugs, if that's what you thought. Mushrooms are information, relevant information. It can be about a behavior that is annoying the user, an error message in the wrong place, a risk that needs to be communicated to the stakeholders. There might be bugs, but there is so much more. The you go foraging in the software you can apply the Lévy Flight  heuristic either by accident or purposefully. It is called focusing and de-focusing. You focus on some area to find interesting things for some time. Then you move to other area. If the area you first stumble upon is hugely rich in things waiting to be discovered, you might spend most of your time there. Or you might just stop there briefly and look for more important things to discover.

You start with a mission and you head out to accomplish that mission. You take notes and notice things. You investigate things that look and feel important. You forage information on the product under test.

Here's a scenario that might give clues to choosing mission for your foraging.  On Monday you wonder if you should go picking mushrooms. It's a fine day, but instead of going head first into woods you barely know, you go investigate. You take your dog with you and go scouting the forest. You find a batch of blueberries and eat a few. Oh, they're so nice! The dog, Rover, eats some also. On Tuesday it rains. Bummer! So you decide to go to a shop and buy some equipment for the trip as soon as the rain stops. You decide to get a basket that has compartments for different mushrooms. You didn't even think of it before talking to the shop keeper who's apparently an expert on the matter of picking mushrooms. Great find! On Wednesday you are called to the office for an important meeting. A nice, sunny day wasted in meeting.  No time for foraging today. On Thursday you get your gear, your dog and head out to woods. You check the sweet spots you discovered on Monday and pick delicious mushrooms and even some blueberries. On Friday you make delicious mushroom stew with some potatoes and carrots. Then, to top it off, you make a blueberry pie. All the recipes for these you checked online but used your own twist to make them your signature dishes. A perfect way to start a weekend.

To sum it up, Mushroom-picking heuristic is a two-fold heuristic:
- First it is a preparation heuristic. It helps you create a mind model that allows you to plan for the up-and-coming testing session, testing phases, etc. To an acceptance testing session one might prepare differently than to a security testing phase. Nonetheless testing needs preparation and a good way to tackle the preparation is the Mushroom-picking heuristic. You should make the preparing heuristics to match your own context. Think of tools, background knowledge (oracles), time-frames and constraints, people attending the sessions, bug reporting procedures, facilities, etc.
- Second it is a testing management - a steering heuristic. It helps you move from one focus area to another. Focusing and de-focusing is one aspect. With note-taking you can keep track on the areas you have covered and to remind if there's need to return that area. Keep in mind the Stopping-heuristics so you won't get stuck too much in one area.

The Mushroom-picking heuristic is not comprehensive nor should it be. It is a model that might work or you may find it useless. Perhaps you might try it and give me feedback on how it worked and how I should improve it. It is a work in progress.

Have a tasty spring!

- Pastori