"Why did this bug get through testing?"

Ahh the question that will come to every test manager at some point in their career. Mine such was today.

It went something like this:

An email came late yesterday night: "We need explanation for there bugs that went through your testing. And we need it by tomorrow." I loosened my collar and wipe sweat off of my forehead. What did my team miss? Have I focused my testing on wrong part of the system? Have I unintentionally lied to customer? The questions kept coming and I kinda lost my sleep.

Then the morning came and more self confidence with it. I gathered my team and asked them to replicate every bug the client reported. 3/4 were their error on executing their own production process. All things they had done were handled correctly and they just expected something to happen that was not meant to happen. Let's say they missed a step in their process and therefore the outcome was not what they expected.

The 4th one was a clear defect. It was not however gotten though our testing but was reported and scheduled to be fixed later. Also its seriousness was heightened by the fact that it did have an effect on production use. Of course we react to the re-prioritization and we'll get it as a hot-fix if need be.

This reaction to the "bug-slip" however got the client to bomb us with defects that are clearly wishes or new functionalities that interfere the current requirements. These should not be mistaken as bugs but as change requests and therefore require some changes to requirements, design, etc. But still they were something the testing did not catch.

My initial reaction was quite overrated as there was no hole in our "net", but it got me thinking: Does the testing team get the blame for all thing unwanted in production? Is it just communications that failed in this scenario or the testing? Are we supposed to know all what is said and not said? Wishes and hopes? Dreams?

If this scenario had unfolded differently, say "Should we discuss about our findings in pilot environment so that there is no miss-communications here", things could have gone differently. We might have not spent the best part of the day recreating defects and preparing for haranguing. We could have gotten testing done on the parts that are not that well covered yet.

The meeting with the client went well however when we walked though every step and finding. It all came down to communication, again!

Ifs & Buts, Couldas & Wouldas whatcha gonna do?