The branch that you’re sitting on is getting cut

The third track at Nordic Testing Days 2012 was a multiple case study or an experiment report from Kristjan Karmo from ASA Quality Services Oü. There were 5 cases that Kristjan introduced in his track, all real scenarios with some data changed to protect the privacy (also some figures exaggerated to make them more dramatic). They were all entertaining but each of them had a punch-line that summarized the case to a lesson learned. In the end all projects had one fundamental flaw that affected the results. Read through and see what it was.

The process is a tool - a fool with a tool is still a fool. Communication is the key!

The first case study one was a company called Wolf Inc., a company that wanted to change name and become Strawberry Inc. The task was supposed to simple: Find-and- replace and go to production. Simple, fast, infallible. No testing required. What happened was exactly the opposite. “Wolf” as a word changes in different situations: wolves, wolf’s, and even more if in a different language. Also all the logos, brochures, documents, templates, etc. were affected. All this was to be avoided by asking questions about the task, by listening to the people making the change. It was all about communication.

The second case was a “CRM on steroids” by a company called Fields & Co. There were lots of people involved, lots of locations, lots of documentation and interpretation thereof. The most catastrophic mistake was a misinterpretation of a date. The date was THOUGHT to be the “ready for testing” –date but it turned out to be the “ready for LIVE” –date. 1 hour session of critical exploratory testing on the product by all parties involved. Group effort and commitment to the end product saved the day this time.

The third case was a company called Forever Ltd. where testers were doing the best they could and producing bug reports. These reports ended up on the developer’s desk and were returned with a stamp “works on my machine”. The system consisted of multiple integrations of different products so the all the contractors said that the problem was on the other contractor’s court. The solution was to meet with all the parties and talk it through. People were negotiating about their defects and issues and then he proper stakeholder would claim the bug and fix it.

The fourth case was Pepper plc., a company that had a testing team of end-users. When the testers were introduced to the project, all testing done by the developers stopped. The problem was solved introducing a few experienced experts to the team and mandating the train testers to make more effective bug reports and testing was done using the exploratory approach. The lesson here would have been the experience and the structure the testers brought to the team.

The fifth case was a company called BadCom Corporation. The requirements for performance had been made early because the system was performance dependent. The product that came to testing was 200 times slower than the requirements stated. So the team did an effort to increase the performance and they managed to get it to be 10 times faster. That was still not enough, as the system still took lots of time to respond. An assumption was made that the 20 times slower would still be good enough. The reality was far worse at the performance rendered the system inoperable and useless.

So here’s what I think:

• In the Wolf-case the work group didn’t know what they were doing or they were blindly taking orders. No-one questioned the orders by looking at the id-tag hanging from their necks, etc. So questions that needed to be asked were not asked, thus creating a situation where a consultant firm was called to douse the fire.
• In the second case there were project management and consensus problems. Floor-level did not know enough about the scheduling thus making interpretations about them. What lacked was communication between stakeholders. The last minute exploratory testing show might have been a confidence boost, but feels like a show more than productive effort. Don’t know, but feels like it.
• Third case is a typical multi contractor problem. Everyone’s interested on part made by them and will not think of others because they’re not paid to think about them. The more bugs the other contractor has, the better we look. So there was no big picture or team in this case and everybody tried to cover their own asses. Again communication and challenging could have done the trick and much earlier.
• The fourth case was a basic bottleneck-situation, where the developers thought that they can now concentrate on their job and leave the testing to the test team. The problem was solved by amping up the test team, which was a good solution, but not what I’d had done. I would have talked to the developers and convinced them to do more testing and would have shown them the results if they did the testing.
• The fifth one was burned even though a consulting agency did their best to get things better. Here the issue was realized so late in the product life cycle that the fix was almost impossible. The performance would have been the top priority regarding testing if it was a performance ciritcal system.

So, all in all, all cases were doomed because of the lack of conversation and the critical thinking of a professional tester. By outsourcing testing you outsource thinking, and might lead into these situations all over again. So think before ending in a situation where the branch that you’re sitting on is getting cut. Use challenging, questioning, feedback, workshops, to raise communication in within the project.

Communication is the key to success.