Monday, August 29, 2016

Complete testability - analysis quality essentials for the confused

Requirements are usually described as having an intertwined bunch of qualities we aspire to have. In reality, it ends up like a person admonishing you to lead a good life. Engineers took a lot of time listing these qualities out in documents like the IEEE Standard 830-1998 superseded by the more impressive 29148-2011 defining qualities like

  • measurable
  • testable
  • consistent
  • correct
  • complete
  • unambiguous
  • ranked
  • modifiable
  • traceable
  • feasible
  • independent
  • necessary
  • non-redundant
  • terse
  • understandable
Most people don't want to take the time to assess a list of qualities like those on a set of a thousand requirement statements, nor do they think it has any value. They're right but they probably don't know why. They want their requirements to lead a good life, but there's so many options it's hard to focus. Here's some advice to help people focus.

The two most important qualities of requirements that are useful to keep in your head as you write documents for projects are easy. All the others are dependent most likely when these two are followed. All the important documents for modeling requirements are covered when these two are followed. The ability for the documents to be understandable and communicate well will be enforced when the two qualities are paid attention to.

The qualities we seek to promote to the top of our analysis minds are testability and completeness. Let's call it complete testability for short. They have the most value for peer reviews because they are easily remembered. And they handle the different levels of granularity well addressing the needs of the single requirement task as well as the larger requirements in scenarios and entire projects.

Testability per requirement

To be testable we must be able to write a test for a requirement. If we say we are going to arrive at work, we can't test that because we don't have a metric to measure by. Business people have seen that kind of task irresponsibility led by vague promises and the need for follow-up at a specific time or day. An assigned task requires a test to see if it was completed.  Those metrics have to be known ahead of time.

If we say we are going to arrive at work and our only metric is that of time we have to give some sort of unit and measurement to the test. That unit is in minutes and that measurement will be at 8 am. Now I can send someone to my office and have them check to see if within one minute I show up at the office at 8 am. Test done.

Tests can not be formed from a requirement that is negative unless there is a constraint involved. If you said you will not be at work today, you have a negative requirement to test but limited by a time constraint. Still, a person would have to spend the entire day waiting for the event to happen from 8 am to 5 pm just in case you showed up for one minute to invalidate the test. There would be no end to the test if there were no constraint so no final test result would occur.

So a test requires both a positive event and a way to measure it with a metric. The metric involves measuring something with a unit. Writing the requirement this way creates a formal approach that eliminates most bad requirement writing.

Completeness per project

The other type of requirement quality that works to create great requirements is really not about the individual requirement. The completeness check is about traceability and dependencies throughout the project. If you know you have to be at work today, you might want to think about all the other days you want to be at work. This is looking at the requirement as a specific and making a generality out of it. You can find more requirements for days you should be at work this way.

The other type of completeness check involves the going from generality to specific. In this case of going to work, when you go to work, should you stay at work all day? Can you leave for lunch? How long should you be gone before it's not seen as being at work a full day? If you drill down into just the lunch period for more detail, then you can ask if taking a longer lunch but talking about work counts as working?

Another type of completeness check might involve switching the role or actor of that scenario or looking for alternate scenarios. Is the work day the same for all job titles? How about if we talk about a different type of work activity like being on the road? Now being at work today takes on a different approach. Also in the scenario of a use case, we can ask if we captured all of the sequential tasks of achieving our goal.

Even looking at one specific quality for completeness can be rolled up into asking whether we have captured all the qualities that should be checked for all of our requirements. Now we are asking as an analyst about what we have forgotten. Sometimes, it an implicit requirement that no one has said but everyone expects. Or maybe it was just forgotten. Don't let this lack of completeness analysis turn into scope creep you blame on the users.

Complete testability

I do like completely understanding and testing the requirements so much that the documents I write can be used in their entirety of content and structure as test documents for successful test cases. It's also in my mind the reason that programmers have taken to Test Driven Development which is just making requirements testable in code editors because they didn't get good requirements in the first place and feeling that a giant methodology is an administrative waste of time.

If the developers can change how they write code, surely the analysts can change to a more structured approach that keeps emphasizing the testability and completeness of the requirements document. It does mean that an analyst now can't just document what they hear. They will have to apply a judgment using some rules about what is complete and testable for their environment.

But the end result will be that the business or individual will end up with a better and cleaner implementation in code or business process, that will be easily tested and create more feedback to improve the quality of the end product.

No comments:

Post a Comment