How to cope with few end-user-testers in a testing project?

This is a summary about the 2nd track of the first day at Nordic Testing Days 2012, how I interpreted the process presented and what learning from that track. The track focused on building skills for the Business end-users to be able to test the system under test properly and as quickly as possible.

The track was performed by Maili Markvardt and was called “Testing: saved by the end-users”. I wasn’t expecting much out of it (I wonder why? Was I biased somehow?), but what eventually went down was very inspiring. Personally I have been in situations where the testing budget is so small that all guns must be brought to bearing. I thought this was one of those rants that “we didn’t have enough time” or “the client was behaving like child” kind of situations. Luckily it was not.



She started the track by presenting the situation: a total makeover of all information systems in a company, 7 large systems with ~90 interfaces, integrated in a “big bang” fashion. Immediately she got my attention, I’d been in a situation like that in my previous company (although not that bad, but similar on many levels). The system was done by many contractors and had about ~250 people involved. No professional testers and 1 test manager, Maili.

The problem was being solved by giving her a bunch of business people – people with no testing skills apart from some IT-skills acquired from previous life. So there was the lack of skills that needed to be taken care of.

How she handled the situation? Well, she trained the testers. The truth is that you can’t train people to become testers without them having enough time and motivation to be trained, or it can be very difficult. What she did was tackle the source of the problem: get a written mandate from the manager of these business people to be available as tester the promised amount of time. And holding them to that promise was (if I interpreted it correctly) an uphill battle. The business people obviously had their own work to take care of.

The would-be testers had little to none experience in testing. What Maili trained to them was the basic testing skills; mindset, test analysis and bug reporting. The approach was test case driven and governed by the test manager. Some challenges started to immerge at early points of testing; the business process was still unclear and under constant change, documentation was too complex, test cases too generalized, and people lacked the base knowledge to execute the test cases.

This is where I started thinking the possibilities of doing the testing the exploratory way. Could they have utilized the Rapid software testing approach to maximize learning and be more responsive to change? As the testers face these kinds of problems, strict and over-documented processes tend to fail due to lack of flexibility. I immediately reflected this to my own work and saw that we had the advantage to use RST to our advantage and benefit from the learning. Sure we also had problems, but the way we communicated with the stakeholders enabled us to make our testing more “to the point” and we uncovered lots of information that the stakeholders didn’t know they needed.

The ugly world of constant change took its toll on testing. Lots of bugs were found and not all of them were reporting real issues. The bug tracking process required the test manager to be a filter that would channel the bugs to correct stakeholders. Due to ambiguous documentation and knowledge about the interfaces (the testing for some of them was postponed to very late stages of testing), testing was difficult and the systems suffered from poor quality, and thus led into postponing the “Go LIVE” three times. The testing might have been the best quality available in that situation, but the amount of work forced the prioritization to ignore some areas until the very last days of testing.

All this put aside, Maili was able to create a process which involved described the process of training business people as testers. The process feels very sturdy and practical, but simple enough to be implemented by all test managers trying to cope with little resources. I hope people got as much out of the track as I did even though I didn’t expect much. Eventually I learned a lot about using what resources you have and to make most out of them. It’s the context driven way, I think: “Throw in whatever you have and the kitchen sink” to get the work done. To my view, Maili succeeded in the task that was very difficult to begin with.

Kudos!