validation | Debi'Z Blog

I must say that I had fun giving this presentation, and I guess that the audience felt it too.

Requirements Review – it sounds dreadfully boring. Months ago, when I was asked to speak, I thought “what can I do to make it a bit more interesting?”. I came up with the idea of involving the audience, making it interactive. How so interactive? Well, lots of questions.

And then, an exercise at the end, for which the answers can be found – guess where? On this blog !!!! During the 1st week of July, the best answers will be up on the blog, with the names of their authors. (I must admit that the idea of putting the answers on the blog was something I thought of the day before the presentation, but I think it was a cool way to get people to come and see what I’m writing about, and hopefully turn it into a great forum for discussions 🙂 ).

I also wanted to have the audience actually come away with something that they could use straight away.

So I started with a story about one of my favourite bugs – actually it was 5 bugs that I found by just clicking on a button that wasn’t supposed to be there. The button said “clear database”, and believe me, it did a lot more than clear!
What were the bugs I found?

“clear database” button was not supposed to be there (not in the requirements)
there were no mechanisms to check whether the button was clicked on purpose or by mistake
the “cancel” button, that was supposed to stop the “clear” didn’t work
the whole database was erased, both data and tables
there was no backup of the database ….

The system analyst had to sit for 3 days and recreate the database by hand. OUCH! But at least it was the day after the demo to the CEO, and not the day before.

Why am I bothering you with this story? Well, I should have felt great, I just found this really serious bug on the test system, before someone completely wiped out a live system with 10 million records on it.

But I didn’t.

I was frustrated that I hadn’t been working on the project from day 1, that I hadn’t found this bug in the requirements documents, like the fact that there was no requirement to backup the database once a day.

I didn’t want the audience (both at the presentation, and those of you who are reading this) to feel the same sense of frustration, so my presentation was actually an interactive lesson about 2 tools that can help find bugs in requirements … pretty easily.

The first tool is used for validation of requirements. What is validation? Validation answers the question “Are we building the right thing?”.
In this case, should we give our customer (the baby) a bottle or a new nappy/diaper?
By validating his requirements, we know that he wants a bottle, so that is what we will give him.

The second tool is used for verification of requirements. What is verification? Verification answers the question “Are we building the thing right?”
In this case, which bottle are we going to build for our customer, the baby?
Using verification, we can clarify whether we are building the thing (the bottle) right?

The Validation Tool: “Problem Space” vs. “Solution Space”

Normally, one would expect customer requirements to be written by customers. But in the world of hi-tech, they very often are written by “techies” – system architects, project managers and the like. These “pseudo-customers” already know how they can build a system that answers the customer’s needs, and so they can all too easily write the requirements with a particular solution in mind. This solution may not be the best answer to the real need – the real requirement. It should be no surprise, then, if the final product does not satisfy the customer.

How should you, as a test engineer or test manager, go about finding these types of problems?

You need to look for solutions masquerading as requirements.
Then you need to find out what the REAL requirement is.

How do you do that? You use a technique called “the 5 why’s” (a technique originally developed by Sakichi Toyoda, and often used for “root cause analysis” of defects; but just as useful for finding the “root requirement”).

The Verification Tool: “Ambiguity Review”

Many test professionals are aware of the list of words and phrases that make requirements ambiguous. Things like “user-friendly”, “etc.”, “and/or”, “efficient”, “state-of-the-art”. In fact there are any number of ambiguity lists.

The idea behind the ambiguity review is to find:

All words and expressions that are ambiguous
Anything else that can be interpreted in more than one way, for example: “If this happens, then do that“. If there is no requirement regarding what to do if “this” does NOT happen, then we are leaving it up to the developer to decide what to develop (not a very desirable situation!!!).
The thing we really don’t like doing : GO AND BUG SOMEONE with Domain knowledge, who can give you an answer as to what the ambiguous or unclear requirement is REALLY meant to say.
Go back and fix the requirements, getting rid of the ambiguities

Why bother with all this work?

Because you probably don’t want to end up with this ….

baby-drinking-beer