Making a requirements pie

This is a summary of the track by Ainars Galvans at Nordic Testing Days 2012 and how I viewed the subject. The topic was “Care for quality, not for bugs! Do you know the difference?“.

I met Ainars the day before the conference at the hotel and we discussed about bugs and quality. We are on the same line with the overall quality but I disagree with some points he made. So here is what happened and how I saw it and some thought about challenging the claims.

“The best tester isn’t the one who finds the most bugs or embarrasses the most programmers. The best tester is the one who gets the most bugs fixed.” –Cem Kaner

See the difference?

True and true. Ainars had great thoughts about increasing the value of the bugs. By focusing on the bugs that had a story that compelled to the developer (and the triage of people deciding which to fix), you will get more out of the bugs that you find. When I think of myself as a tester, I tend to get as much information about the product and thus uncover as much bugs as I can. I know that by doing so I learn about the product, the risks, the critical areas, the things that people making decisions value the most. When Ainars said “don’t report the bugs that aren’t getting fixed”, I was shocked. Am I doing it the wrong way? I may have a huge amount of rejected bugs, but it’s my way of learning. What if I uncover some minor looking manifestation of a bug that turns out to be a showstopper? I don’t know if I don’t ask and communicate.

Usually when you start communicating with the developer about a bug that seems like a show stopper, he will immediately see that it should be fixed immediately. If he can pinpoint the problem, you won’t need to make a bug report if the problem is sufficiently communicated onwards. So this is where I disagree with Ainars; I will make bug reports but I get the important ones fixed.

Ainars had great thoughts about prioritizing testing. He said that he’s not going to spend his time on testing something he KNOWS is going to pass, so he tests something he doesn’t know that will pass. He saves time doing so and can focus on the important areas of testing. So you should always do the most important testing at the every moment.

How do you know what is important? Ask! I ask if there is some area that is valued by the product owner and I focus on that. Then I ask what the next valuable thing is and focus on that. By time I get some testing done I can start formulate my own view about the importance of different things. So Ainars had a good thing going on there, as he knew the product so good he was able to guide his own testing prioritization using his knowledge of the product. The knowledge acquired by learning and testing the product.

Picking up requirements.

Ainars also introduced a cool way to do exploratory testing. He called it “Requirements driven exploratory testing”. They were almost like test cases but more like missions or topics, which were then tested. They were measured using traffic lights: Green (good enough testing AND good enough quality),Yellow (not started) and Red (not good enough testing OR not good enough quality). I challenged the concept as the testing team was focusing on only documented requirements, and I still do! There is a need to test also things that the client takes for granted and things that are discussed but not documented. I also challenged the fact that requirements are usually ambiguous, and that “requirements are not items to be gathered”. They're not like apples that you can get into your basket and bake a testing pie out of them. Ainars could clarify that if he can. From the presentation I did not get an answer to that.

As a summary, I got a lot out of the track as it provoked me to think about how I do my testing, but also I got a chance to challenge the speaker. I feel that Ainars has a lot to say about this and I will be checking his blog as often as I can. I hope he has a way to address my concerns about the requirements and “not-important” bugs.

Preacher man Marjamäki

I was reading own blog entries and I started to truly think about my own writings. Those writings have clearly the idea and insights - some of the insights derived from others, but still. However, part of my text is “porridge” - it lacks the red wire. I am a novice blogger, so it is certainly not a great disadvantage here, but when I read more and more testing blogs of other, I see they write long and coherent entities, where as mine are short and fragmentary.

So I have found the opportunity to develop myself from my own errors, which I did not even know were there at the time of writing. Someone may disagree on whether the writing style in itself flawed, but perhaps "lacking" is the word that describes it best. Whet perhaps led to the creation of this writing was a text about challenging oneself by James Bach. So I found a questionable approach in my posts, and it cries for improvement.

So how do we improve the concept, which is not really wrong - or at least not harmful? How to motivate to repair, which in itself is not necessary to be repaired?

-

When the tester examines an application (i.e., exploratory tester) he finds bugs. He also finds deviations. Fault in this case would probably be writing off topic and / or misspellings within the text. Potential errors nonetheless.

Typographical errors in this case are the lowest of severity, and the repair improves (only) the cosmetics. They make it more compromise usability (readability) when the user (reader) has to constantly think about whether a typo is a typo or deliberately wrong (or even a different word). Repair is easy in these cases, so despite the low severity the correction should be made to find out the “bug” is noticed.

Factual errors are more severe errors. It may mislead the reader, which is not usually the purpose of the blog. In addition, it reduces the confidence enjoyed by the writer, so his texts published as “true” are not considered true but false. Reviewing context is crucial! Factual errors, severe errors! Correction in these cases results in a new version of the text or freezing the text. If a bug can be found in an application and it prevents the use of the application to full extent or to which it is designed, it will be deactivated and make the necessary corrections in order to function as desired (as expected) in a way.

Deviation in this case is a deviation from the actual or supposed truth. If something does not match the reader's prior expectations, he considers it of a deviation. A deviation in itself is not a fault, it has been just implemented different from expectations. If someone publishes the text, which is different from the mainstream, he must to be able to justify his choice to gain of user satisfaction (in which case it becomes a feature), or the deviation turns to be a fault (i.e., a defect in the text). The same applies to applications where a feature is the done in violation of the requirements (prejudices, assumptions, expectations).

Derived from that: Is there a previous blog post that is defective? Does my unintentionally preaching-like blogging result in faulty, incorrect or different writings? My bias is that they are both defective and deviant. Let’s see whether it is true...

My intention in the text "How do I interview a new tester?" was to write down ideas and to create guidelines for myself and other similarly interested, who wrestle with the same thing. The text is incomplete on many parts, due to a lack of perspective. References to existing texts and comparing them would have added depth to the text. For example, Software Testing Club, funny book, "The Ridiculously Simple Guide Test Building A Team" would have been good reference material ... Ashamed that I made probably most of the "best proven" things mentioned in the eBook and I probably asked exactly the wrong things. (As it turned out, I wrote the whole shebang again. But this refers to the original, Finnish version of the text.)

"Performance Testing Integration Project" was designed to provide a clear frame on performance testing – area where I hadn’t gone before. The post studied the technical point of view and may even provide a reference material for those interested. As a result, the post was quite successful individual, in my opinion. It was based on existing performance testing of the model. This could have been mentioned in the text, because I make it seem like I invented the whole testing model.

So that this blogging does not change into defensively, I want to keep it in the critical path. As I told earlier, these were the assumptions and outcomes, and comparing them. The outcomes were somewhat different from the assumptions, but in some cases (as in "Performance Testing Integration Project"), the deviations were very small. Although they were not well-founded, they were such that their correction is not priority number one. Au contraire, "Testing Jin and Jang!" fails to meet expectations by a long shot. As I say in the text "My thoughts began to roll like crazy at the time -", which was true at the time, the red wire in my head is not conveyed to the text as I would have liked it to. In retrospect, I could filter the preaching part out of the text, and focus on the essential and the cold facts. Would I be able to submit my own experiences or to refer to other’s experiences on the subject? Could that material be gathered and refined in to a good and sententious and incite other to think "like mad"? The text is lacking the touch with reality, which prevents the reader from taking the text seriously. This therefore calls for a re-write due to the defects and deviations of the post.

-

So, how to motivate the repair, if the repair itself is not necessary? Self-improvement! If you have the opportunity to develop yourselves, it should most certainly be done. You never know where reviewing your own shortcomings may lead.

In order that this blogging does not turns into preaching on behalf of self-improvement, I would like to stress that all faults, errors and deviations should be identified and investigated. The correction of these may happen spontaneously by studying and searching for them. This could be like examination of the desert: the examination of the sand reveals an anomaly in the smooth surface od the desert (the tip of the pyramid), examination that the finding (digging) is revealed to be mightiest tomb monument in the World containing divine treasure and riches beyond imagination.