Risk based testing - talking the talk, or walking the walk.

Over the years I have read a lot of master test plans and test strategies. And very often, there is some variation of the topic "we are doing risk-based testing". But when I subsequently dive into the test project, I find it hard to see it happen in practice. There is no red thread from what is stated in the test strategy for the test that has been specified and executed, and it is often not based on a product risk analysis.

In order to be able to do a risk-based test, you first need to know what risks are related to the product / system you are going to test. That is, some kind of product risk analysis must be carried out.

But one thing is to make a product risk analysis, something else is to plan, specify and run tests that support the risk image you have identified. The activities carried out in the test and quality assurance project must all support a focus on mitigating the identified product risks - illustrated in the following.  

That is, as a minimum:

• The entire project is familiar with the output of the product risk analysis
•  When test is planned, it must be based on the identified risk picture. That is, the intensity of the individual test tasks with reflects the risk it addresses
• The test techniques used for test design must be selected based on the product risk they address.
• The prioritization of the test run must reflect the risk picture - the highest risk is tested first

Now I can imagine at least two objections:

1.     My testers cannot use test techniques so it does not make any value to make the product risk analysis

2.     We are agile so we do not do product risk analysis.

Let's start with the test techniques; A product risk analysis identifies a common picture of what the test assignment is, it is important no matter whether the testers can use test design techniques or not. You can still give them a direction for how much test should be done, based on some rules of thumb, e.g;

The list above is just meant as an example of how you can use the output from your product risk analysis to provide a set of rules of thumb for what to test, define those that fit your context.

Now the testers have a basis for their test specification, but you also have a basis for making your test plan - it must also reflect the above.

When the test is executed, follow up on your product risk analysis - so you can report whether the identified product risks are actually addressed in the test - and with what result. This of course requires that you have some form of traceability between risks and tests.

But we are agile....

Regarding being in an agile project - product risk analysis still adds value! Make it as part of your team's sprint planning, perhaps integrate it directly so that product features are identified as part of the clarification of features and stories. Perhaps playing risk poker like playing planning poker. The discussion that will take place with Product owner and team when product risks are identified gives great value to the team as a whole. When the team has a common picture of the risks they face, they are also more aware of their share in mitigating it.

Final comments

Remember, whether you are in an agile or traditional project; It's not enough to do a product risk analysis once - it's a natural follow-up as more clarifies in the project, which can pose new risks, we can mitigate risks on a continuous basis, etc. And then you might consider whether you should actually Have more levels of product risk analysis; one for the project as a whole and one more detailed as the tester does on single features to get deeper insight into a more specific level.

And remember - risk based testing is not just to identify the product risks, you need to make them known and mitigated.

Stop thinking about HOW - start thinking about WHY

This article was originally written on my linkedin profile, but I want to give it a more personal "home", så here we go :-)

A whil ago I saw the TED talk from Simon Sinek about how great leaders inspire action. In the talk he introduces the golden circle, explaining the different perspectives WHY, WHAT, HOW - and how we should always communicate starting with WHY. Getting introduced to his golden circle got me thinking; Maybe some of the challenges we see in agile transformations could be prevented if we used that mindset?

We are so good at defining processes, describing swim lanes, creating templates etc. We train people in the method - the mechanics, but very often I think we forget to explain why. Do you know why your company is in the middle of an agile transformation? Has anyone explained to you why that journey was started, what value it will bring? have you read, and do you understand the 12 principles behind the agile manifesto?

If we don't understand the WHY, how can we then be good at the how and the what? A couple of examples;

I see companies that doesn't prioritize the continuous or at least the early focus on quality within their projects – on building quality in. Why do I want them to focus more on that?

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

This is the first of the agile principles. How can you create early and continuous delivery of valuable software if you don't have the focus on ensuring that you are building the right thing... and building it right? How can you have that focus if you don’t’ focus on getting your foundation right? The user stories and accept criteria? How can you have a continuous focus on this if testing is something you do as a "finishing touch" done by the business?

Or another aspect; how can you have focus on value if the product owners chooses to define the user stories and acceptance criteria deciding the what and how, rather then explaining the value needed? The user story is a placeholder for a conversation – a conversation between PO and team, and the acceptance criteria is the input to the confirmation of whether we have created what the PO need. It is not a specification of what to create in detail, it is not the design.

Working software is the primary measure of progress.

Another of the agile principles. How do you know if your software is working? you do that by executing the software - by testing it. Whether this is manual or automatic is another discussion, but you cannot say that your user story or feature is "done" if you haven't tried using it, focusing on both whether it works or not and whether it fulfils the needs of the users. Unit test, system test, acceptance test is still relevant in agile, just not as a phase but as a continuous activity across the lifecycle.

Build projects around motivated individuals. Give them the environment and support they need and trust them to get the job done.

A major part of motivation is to understand the difference you are making, the value you bring to the team. This is relevant, no matter whether you are developer, tester, scrum master, product owner or something else – we need to understand the value we bring to the table, why we are there.

Sometimes we are challenged in creating that environment and support. Sometimes we just give the “title” of Scrum Master to the project manager and say “now you are no longer a product manager, you are a scrum master”. And the will then continue their merry way doing more or less what they used to do, just with another hat on. They will still manage, not being servant leaders for their team – supporting them in their agile journey. The stand up will be a daily reporting activity to the project manager rather than something we do to and for the team to ensure that we are on the right track and get the support needed.

And examples could be given for all 12 principles I’m afraid – but I hope you get the point from these three.

I wish we could start all transformation journeys with a discussion based on the golden circle, getting an understanding of:

• Why do we want to do this?
• Why have we chosen approach X?
• Why can this support our business?
• How does this support our vision and strategy for the organization?
• How do we communicate the answer to the WHYs to the organization?
• How do we ensure that our employees see the value and see their part in the transformation

Do you know why your company is in the middle of an agile transformation?