Porridge is bad for you!

"What?! What is he talking about?" This blog post is about challenging one's criticism and trying to make it more effective using simple methods. This post is about moving away from unhealthy criticism into constructive critique and feedback. This is about finding the right mind-set.

Why do people criticize?

"You're doing it all wrong!" "Your tie is hideous." You've seen it. You've heard it. Probably you've even done it. I know I have. And in most occasions I have let my self-criticism be clouded by someone else’s opinion about the same subject. By not having a concrete opinion of one's own it's easy to adapt to criticism of someone else.

Let's take an example of a guru that has strict criticism against one subject - a porridge (seems like a neutral enough). We all know that porridge is good for us, right? What if a nutrition guru says that porridge is bad? What do we do? If we have huge respect on that guru's thoughts and doings, might we blind our own judgment with the upward gaze towards the guru? We might just take the guru's opinion granted and start proclaiming that porridge is bad for you.

One way out of this bad equation is to step back and challenge one’s own critique. Is it justified? Are you making an assumption? (Great blog post about defeating assumptions by Ilari Henrik Aegerter) What if the guru was wrong? After making sense of what do YOU think, then you should back the critique up with facts, not opinions. By finding the facts, you might be looking for biased facts, but at least you have something to back it up. It's obvious (to some) that a biased opinion should stay as an opinion, but they rarely do. Instead of being opinions they become statements supported by chosen facts.

Doing basic critique, source checking, challenging, context projecting, you can find the root cause of the guru's opinion about the porridge. Does he (our guru is a he today) have an agenda of his own? Are there hidden meanings in the critique itself? Does it provoke thinking instead of criticizing the product?

How do people criticize?

"When giving feedback, do it like so: Always give good feedback in public and be precise about what was done well. Always give negative feedback in private and be precise. Try to find the solution instead of the one to blame." This was said by my father who has decades of experience in management and leading people. I have always thought this as the fundamental guideline of critique. I think most people know this and agree with this, but how come most people don't act accordingly?

Let's say that the guru had discovered some facts that "Ye olde bran porridge" has all sort of chemicals in it that disable some growth hormones on a child. Obviously that's a statement to be told to the public, right? And as we hold the guru in high regard, he is mandated to present his opinion (possibly supported by facts) in some public media. There are channels in which you can present a complaint about food (health inspector or some kind of an agency) and they will take the necessary precautions to tell the public that "porridge is bad for you". Possibly they have first discussed with the porridge company, who might have taken the product off the market.

The guru might give criticism about porridge in public and have the wrath of the porridge company on his shoulders. He might not care as he's a nutrition guru and has an agenda of his own (hoes he?). Is the guru doing the right thing expressing his opinion so loudly in public? Is the guru promoting himself instead of giving critique?  Was the bad thing in the porridge, in the chemical, or in the company making the porridge?

Where's the difference in the approach between the two models of critique? Was the guru able to achieve the goal of his criticism through a public channel (which ever the goal might have been)? Was the "behind closed doors" critique more efficient than the "in your face" critique? They all depend on the context, obviously. What was the goal of the critique?

Feed-forward

Some people think critique is feedback. Well it kinda is in some extent. Feedback however can be constructive even when the feedback is negative. Feedback is given when someone asks for it; critique is given when ever. Feedback is not trying to make one feel happy/sad but to make them improve; critique is about making a statement. When giving feedback don't sugarcoat it, instead say what YOU like and you'd like to see improved. "I liked the taste of porridge and how my stomach feels afterwards. To make it even more healthy I would not put in the chemicals that prohibit my growth."

As the feedback is a kind of a thing to be asked for, critique is the kind of a thing you just blurb out. Feedback has a purpose and it is meant to improve the one receiving the feedback. Critique has the tendency to provoke something. Conversation, debate, hatred, etc. Challenging can be more effective a way than critique. Challenging the critique itself can become the most valuable feedback there is!

Is the content self-justifying or do we need to empathize to support the critique?

There are tons of guides in how to give feedback without being critical. I know a dozen occasions where I have let my judgment be clouded by numerous things that have lead into bad critique and undesired results. Here's one:

I try to promote intelligent testing and intelligent approach to quality in general. I also believe that certifications that focus on the certificate itself are no good. A certificate that focuses on skills in a field that requires skills is a good thing; artificial certificate concentrating on a narrow view about best-practices (and only knowledge thereof) is a bad thing. This is what I thought and still think. I was having a discussion with people I think highly of about “what is your opinion about ISTQB-provided series of certificates”. By making a comment that the certificate looks good on paper (with some unflattering spices), I provoked a series of questions about “how do I back up my statement” and "do you even know what you're speaking of".

The questions stuck home and I started to think of how I really thought about the issue. The fundamental thought behind the issue remains the same, but as I have not delved deep enough into the syllabus, the history thereof, the initial goal behind the syllabus. Am I eligible to make claims about the issue? Was I repeating what the other people were saying and them making myself feel important about myself by making a rash claim? Does my opinion really matter in this case and could I do some good without being so loud about it? Is there a possibility to raise conversation about the issue within the certificate organization without sounding like a zealot?

With the comment I made (which was criticism at its worst) I thought the content of the comment was self-justifying. "Obviously all the people were thinking the same so I just said it out loud." Even though some of them were, they rightfully challenged my comment and forced me to think about it. What was my goal when stating something like that? What was the desired outcome? Praises to me? More Twitter followers? To raise conversation? To sound like a dumb-ass?

What I did achieve with the comment was for me to be able to criticize my own behavior and claims. I once claimed (in Finnish) that one way to achieve the best quality of an end-product is to "Murder you darlings" - by finding the most direct route from the current point into the desired point. By removing all the excess and self-promotion from the content. To go directly towards results. In making the comment I a was focusing on "sounding cool" instead of trying to use the words as a tool to achieve a goal (which apparently was shrouded).

Did I hurt someone in the process? Can't tell. Not directly, I assume (Pekka, you're assuming things).

Did I achieve the goal? Can't tell. I wasn't aware that there was a goal.

Did I learn something? Oh boy, did I!? ;)

In the face of failure - part 1

Sometimes we fail to achieve what we have set our minds to achieve. That is not always a bad thing as we might learn something about the process of failing. The failure itself may require us to change our perspective or approach to the task at hand. in the first part I cover what happens when a test fails. In the second part I'll look into people's failure in effort to do something. All along I try to offer insight and learning possibilities to the subjects.

When a test fails (or doesn’t fail)…

As testers we do tests (DUH!). This may be a mission, a scenario, a flow, a check, basically of any proportion of work effort done in order to achieve some testing goal. This may also be a check conducted by a machine of some kind (test automation script, you name it). The point is that there is a test and we have or might not have some assertions (assumptions, expectations) regarding the outcome of the test. There may be tests that we have no idea what the outcome is (“What happens if I press this blank button at GUI?”) but it may result in a future assertion regarding the same test object.

Test may fail or they may pass. That is the binary nature of a test. They may however trigger a whole another result, for example “indefinable”, “false negative”, “false positive”, "what the bloody hell is that?". What happens if a test results in a “test passed”? What does it mean? Can it still fail? Can it mean something more?

Test fails because the test object doesn’t pass the assertion

This is what we want to happen, when a test fails: The test fails because there is something broken in the test object. By wanting this to be the case we might close our eyes from something important. The case simply states that the test object does not pass the assertions for reason unknown.

We think that the problem lies in the test object but are we sure? We need to eliminate the "think" and go towards the "know". We must analyse the test itself to determine whether the result is infact correct. The result may be a false-negative as where we need to determine if the test itself is not faulty. We may be testing the wrong thing or asserting the wrong things.

After we have determined that the test object infact is not built to match the test, we must ask: "have we built our test incorrectly?" This may result in information about the behavior of the test object just as well as the behavior of the TEST! The behaviour may be incorrect in both situations, but the newly found behaviour may be the thing customer/user/stakeholder (that is important enought) wanted or truly needed. The current behavior may also be a better solution than the intended/planned. In any case, this information must be revealed with more testing and analysing.

The test result may also uncover a risk that was not taken into account or was ignored before. This rises important questions about the the test object and the processes of development and testing. If there is a hole in our risk mitigation strategy could there be a need to revise the process or processes? Could there be more areas that we have not yet covered?

Lastly we may require to re-test some parts of the test object as the unwanted behavior mey be required to be fixed (or the test if chosen so). In any case the test requires revising, possibly some fixing and definitely more analysing. Do we need more tests to cover the area where the behaviour was found? Do we need to revise MORE tests? Was the test extensive enough to be feasible after the fix?

False negatives and positives

In a false positive/positive case we have already done some analysing to reach the conclusion that the test gave faulty information. Both cases may reveal something important about the test object but more than that they tell that there is something wrong with the testing. Do we have enough information about the test object to be making statements by the test results? Do we need to do more research on the test object to make our tests better? Have we missed something important in the process? These results always require analysis on why the tests give false results in the first place.

False positive is basically a situtation, where we think that the test object is behaving the way we think it is. This result is corrupted by the defects in the test itself and thus the test gives a "pass" result. it could be that the test is concentrating on the wrong area or function, thus giving false information. Therein could lie a risk that we have not covered a critical part of the product with sufficient testing. The assertion in the test itself could be missing or not strict enough ("Check if there is a response of any kind." -> Soap error message.) To get this situation corrected, usually both the test and the teset object need to be analysed and possibly fixed.

False negative is a situation where the test gives a falsely prompted negative result to a test. Almost all the situations apply to this as in the false positive case. We may need to back up the result with additional tests to rule out the possibility that the test object is behaving incorrectly. We may be lacking in skill, we may have ignored something or simply there is a defect in the assertions.

Tests always reveal something, and it may be important

By doing testing we uncover information. The goal of testing is to give enough knowledge for the deciding-people to make the right desicions. Even a poor test can reveal important information. If critical information is revealed at this stage (early or late), have we done something wrong in the previous stages? Is the poorly constructed test a waste? Do we need to construct better tests or are we aiming for fast results at this stage?

It could also be that because of some flaw in some process we have stumbled upon the wrong area to test. It could be that there are communication problems, ignorance or some other reason that we do testing to an area we are not supported to do that. It could be that the feature is under developments still, the environment is not ready, etc. Does the testing we did vale any value or was it waste? Do we value learning? Did we learn anything about the test object?

Even if we have the most beautifully constructed tests and the test object is in good shape, we may encounter problems if we do not know how to interpret the results of the tests. Results contain sometimes false results (positive/negative) that must be uncovered and examined so that we have correct information about the test object at all times. The one interpreting the results (tester during testing, test automation specialist, etc.) should have enough competence and criticism to question the results. We may ignore all "pass" results and just focus on the "fail" leading to false information.

We may think we know enough about defect. We did find it, didn't we? By analyzing the failure itself and the root cause we can uncover more information about the test object and its surroundings. By questioning the test results just as we question the test object, we may reveal some information about the testing methods, processes, tools, etc. and be able improve our testing.

Read also the part 2.