onsdag den 18. juli 2018

Risk based testing - talking the talk, or walking the walk.

Over the years I have read a lot of master test plans and test strategies. And very often, there is some variation of the topic "we are doing risk-based testing". But when I subsequently dive into the test project, I find it hard to see it happen in practice. There is no red thread from what is stated in the test strategy for the test that has been specified and executed, and it is often not based on a product risk analysis.
In order to be able to do a risk-based test, you first need to know what risks are related to the product / system you are going to test. That is, some kind of product risk analysis must be carried out.
But one thing is to make a product risk analysis, something else is to plan, specify and run tests that support the risk image you have identified. The activities carried out in the test and quality assurance project must all support a focus on mitigating the identified product risks - illustrated in the following.  
That is, as a minimum:
  • The entire project is familiar with the output of the product risk analysis
  •  When test is planned, it must be based on the identified risk picture. That is, the intensity of the individual test tasks with reflects the risk it addresses
  • The test techniques used for test design must be selected based on the product risk they address.
  • The prioritization of the test run must reflect the risk picture - the highest risk is tested first

Now I can imagine at least two objections:
1.     My testers cannot use test techniques so it does not make any value to make the product risk analysis
2.     We are agile so we do not do product risk analysis.
Let's start with the test techniques; A product risk analysis identifies a common picture of what the test assignment is, it is important no matter whether the testers can use test design techniques or not. You can still give them a direction for how much test should be done, based on some rules of thumb, e.g;

The list above is just meant as an example of how you can use the output from your product risk analysis to provide a set of rules of thumb for what to test, define those that fit your context.
Now the testers have a basis for their test specification, but you also have a basis for making your test plan - it must also reflect the above.
When the test is executed, follow up on your product risk analysis - so you can report whether the identified product risks are actually addressed in the test - and with what result. This of course requires that you have some form of traceability between risks and tests.
But we are agile....
Regarding being in an agile project - product risk analysis still adds value! Make it as part of your team's sprint planning, perhaps integrate it directly so that product features are identified as part of the clarification of features and stories. Perhaps playing risk poker like playing planning poker. The discussion that will take place with Product owner and team when product risks are identified gives great value to the team as a whole. When the team has a common picture of the risks they face, they are also more aware of their share in mitigating it.
Final comments
Remember, whether you are in an agile or traditional project; It's not enough to do a product risk analysis once - it's a natural follow-up as more clarifies in the project, which can pose new risks, we can mitigate risks on a continuous basis, etc. And then you might consider whether you should actually Have more levels of product risk analysis; one for the project as a whole and one more detailed as the tester does on single features to get deeper insight into a more specific level.

And remember - risk based testing is not just to identify the product risks, you need to make them known and mitigated.

onsdag den 11. juli 2018

Stop thinking about HOW - start thinking about WHY

This article was originally written on my linkedin profile, but I want to give it a more personal "home", så here we go :-)

A whil ago I saw the TED talk from Simon Sinek about how great leaders inspire action. In the talk he introduces the golden circle, explaining the different perspectives WHY, WHAT, HOW - and how we should always communicate starting with WHY. Getting introduced to his golden circle got me thinking; Maybe some of the challenges we see in agile transformations could be prevented if we used that mindset?

We are so good at defining processes, describing swim lanes, creating templates etc. We train people in the method - the mechanics, but very often I think we forget to explain why. Do you know why your company is in the middle of an agile transformation? Has anyone explained to you why that journey was started, what value it will bring? have you read, and do you understand the 12 principles behind the agile manifesto?

If we don't understand the WHY, how can we then be good at the how and the what? A couple of examples;

I see companies that doesn't prioritize the continuous or at least the early focus on quality within their projects – on building quality in. Why do I want them to focus more on that?

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

This is the first of the agile principles. How can you create early and continuous delivery of valuable software if you don't have the focus on ensuring that you are building the right thing... and building it right? How can you have that focus if you don’t’ focus on getting your foundation right? The user stories and accept criteria? How can you have a continuous focus on this if testing is something you do as a "finishing touch" done by the business?
Or another aspect; how can you have focus on value if the product owners chooses to define the user stories and acceptance criteria deciding the what and how, rather then explaining the value needed? The user story is a placeholder for a conversation – a conversation between PO and team, and the acceptance criteria is the input to the confirmation of whether we have created what the PO need. It is not a specification of what to create in detail, it is not the design.

Working software is the primary measure of progress.

Another of the agile principles. How do you know if your software is working? you do that by executing the software - by testing it. Whether this is manual or automatic is another discussion, but you cannot say that your user story or feature is "done" if you haven't tried using it, focusing on both whether it works or not and whether it fulfils the needs of the users. Unit test, system test, acceptance test is still relevant in agile, just not as a phase but as a continuous activity across the lifecycle.

Build projects around motivated individuals. Give them the environment and support they need and trust them to get the job done.

A major part of motivation is to understand the difference you are making, the value you bring to the team. This is relevant, no matter whether you are developer, tester, scrum master, product owner or something else – we need to understand the value we bring to the table, why we are there.
Sometimes we are challenged in creating that environment and support. Sometimes we just give the “title” of Scrum Master to the project manager and say “now you are no longer a product manager, you are a scrum master”. And the will then continue their merry way doing more or less what they used to do, just with another hat on. They will still manage, not being servant leaders for their team – supporting them in their agile journey. The stand up will be a daily reporting activity to the project manager rather than something we do to and for the team to ensure that we are on the right track and get the support needed.

And examples could be given for all 12 principles I’m afraid – but I hope you get the point from these three.
I wish we could start all transformation journeys with a discussion based on the golden circle, getting an understanding of:
  • Why do we want to do this?
  • Why have we chosen approach X?
  • Why can this support our business?
  • How does this support our vision and strategy for the organization?
  • How do we communicate the answer to the WHYs to the organization?
  • How do we ensure that our employees see the value and see their part in the transformation
Do you know why your company is in the middle of an agile transformation?

fredag den 3. marts 2017

Thinking about agile these days....

I don’t blog often, and when I do I tend to stay within my field of expertise – testing. But I need to get something of my chest today, so here we go! (sorry about the long blog post J)
When I read books or blogs about agile, or hear presentations or tutorials at conferences I hear about all these great methods and tools for building quality in, for testing first; test driven development, behaviour driven development, continuous integration/delivery, business as an integrated part of the development team, tightly knitted teams – high performance you know, tribes, squads, a/b testing etc.
All that stuff is really great; and hearing the stories of how to embark on an agile journey is a dream – but can you see that dream happen just like that in your reality? I mean if you start an agile transition when starting the creation of a new system it surely makes sense, and you can start “the right way”. And maybe then it is “just” something you can do.
But what if your context is different? When I look around at the companies I know of, or hear about from fellow testers, and I can’t help thinking: there is a HUGE journey to take from that reality and to the Nirvana described above;
Huge system landscapes with large dependencies between systems, old legacy systems mixed with new, rather bad test environments, only manual test cases above unit testing – huge manual regression test suites – or maybe the opposite; no regression test at all. Unit test coverage around 2-4% if any at all, product owners totally stressed out because they have their normal day job besides the product owner role, projects with people assigned 30-60% because they have a couple of other assignments... oh yes and lets top this with a desire to outsource because that seems to be the thing too.
So with the risk of awakening the unicorn world vs. real world from Agile Testing Days past – could we move our inspirational talks and tutorials a bit from the perfect world scenario to the one that many people are in today? In a context like that how do you get started with agile? What is needed before you embark on that journey? If you have an old organization used to work within a waterfall approach with strict roles and responsibilities – the shift is not that simple. How do you eat that elephant? Let me hear presentations about the struggles, the challenges and how you overcame them – not just the fairytale where all just went well. I learn so much from mistakes; mine or yours doesn’t really matter.
I don’t have the answer – but I think there are some fundamental things you need to start with, before starting to talk about all the different methods and tools.
First of all, understand that agile is indeed a Journey – it is not a switch you can hit and then presto - you are agile! Maybe you will one day reach that Nirvana you’ve read about – but maybe not, and that might actually be okay. Understand what is possible in the context you’re in.
Before you embark on the journey; understand what agile is – get an understanding of the agile mindset, the principles and the values. Agile was not created for the companies to save money – it was created to strive to give the maximum customer value, to create a better understanding and faster feedback between business and IT, to deliver quality – of course these things may save money too, but that was not the end goal in my humble opinion.
Get a clear picture of your own context. And if you are an external agile coach reading this – PLEASE get a good understanding of the context you are in – coming from the outside, start with using some time understanding! If you have several hundred systems entangled in a big pile of spaghetti of dependencies,  and no test automation at all - maybe shouting “let’s do continuous deployment” is not the right place to start J. Understand the people you are going to help – where are they, competences, values, organization, culture.
The next thing to do is not to find the right tools – it is to find the right people, and bring them together. It is hard to be agile if you are sitting in different locations, if all communication is via phone, Skype or video. We need to meet, learn, and share – if every day is not an option, then at least often. And remember it is not for everyone to take that journey to an agile way of working, some will chose not to and that is okay too J
And then I haven't even mentioned the business yet - how to integrate them, under stand their world and needs. But that is for a whole other blogpost. Well I am a tester, so I need to jump to that area of the journey. The decision is made – we want to move towards an agile way of working... How do we get this show on the road when it comes to testing? Imagine that:
  1. Your system is 5-6 years old maybe older - and it is BIG and complex
  2. You have a huge manual regression test suite
  3. The system integrates to a number of other systems
  4. There are limited to no unit test and no other automation at all
  5. Developers are doing a minimum of manual testing before it is handed over to business to test
  6. There are no experienced testers – business only and they differ from time to time
  7. There is a test manager – he is responsible for getting the business resources for testing, but have little influence on how the development team tests.
Where to start, what to do? What do you think?
Thank you for your patience... I will stop now ;-) but I need to write a part two I think :-)
/Gitte aka godtesen

mandag den 22. juni 2015

Thoughts on test process improvment - Ownership and Commitment

Remember my list from a couple of weeks ago?

  • Know where you are (Current situation)
  • Know where you are going (Target situation)
  • How to eat an elephant
  • Learning
  • Ownership and commitment
  • Process first, then tools

Today: Ownership and commitment

Any successful process improvement is as I see it almost 100% depending on whether the people who has to “live with” the process  takes ownership, participates in the definition and implementation of the process and acts as ambassadors of the process for the remaining part of the organization

Here I would like to point you in the direction of the ADKAR  model (Awareness, Desire, Knowledge, Ability, Reinforcement), created by Jeff Hiatt. The model is a goal-oriented change management model which was developed to guide activities during the change process.

(ref. http://www.change-management.com/tutorial-adkar-overview.htm)

  • Awareness of the need to change (culture, process)
  • Desire to participate in the change and support it
  • Knowledge of how to change
  • Ability to implement the change within the day to day work
  • Reinforcement to ensure that the change is kept in place

I like to use a combination of Ambassadors who can champion the process, super users who can support the implementation of process and tools, guilds and knowledge networks to create a forum for knowledge sharing and functioning as sparring partners and early feedback, and of course retrospectives on a regular basis to ensure that feedback from the projects are gathered and used to the continuous improvement of the process.

So here it is; my keywords and focus areas when I am a part of test process improvements... or process improvement in general. Focus on the people and on doing this as an iterative process.

mandag den 15. juni 2015

Thoughts on test process improvement - Tools

Remember my list from last week?

  • Know where you are (Current situation)
  • Know where you are going (Target situation)
  • How to eat an elephant
  • Learning
  • Ownership and commitment
  • Process first, then tools

Today: Process first, then tools

We often identify tools to support the improvements we make, but the key here is that the process identifies needs for tools – not the opposite. Maybe a certain tool is given e.g. a certain test management tool is mandatory within the project. But we can still customize the tool to fit the process – not the opposite.

Involvement and ownership is again important here - involve the team and relevant stakeholders when identifying the needs for a tool; what value do we want from the tool, what requirements do we have. E.g. when evaluating a test management tool; Do we have requirements for traceability to requirements/user stories?  is it easy to access (yes I know this is not a testable requirement :-)), can we make the overview we need for our testcases? I don't in any way think you should write a requirement specification, but at least write down some checkpoints which are crucial for you (I think we had about 20 things) so that we evaluate the tools based on the same questions.

Evaluate several tools against the list, don't just take the one that comes first in a google search ;-) Talk to other testers, get inspiration from them - what tools do they use. I prefer getting my recommendations from people using the tools rather than people selling them :-)

At my current assignment we evaluated 5 or 6 different lightweight test management tools and ended up with two that we presented to the test team, from that we did our final selection.

This is one of the places where I would recommend the use of pilots; don't start introducing a tool using a big bang approach "as of tomorrow the entire organization will be using tool X to ensure test case management." We had a small project we could try it out with first, setting it up for them, introducing it and of course supporting the use during the first release. We also created a sandbox project that everybody had access to, thereby giving the people a playing ground where they could learn without being scared of ruining stuff :-)
We of course did some tweaking during the first project to make it fit the project, but after that we could reuse most of the lessons learned to create a setup to the main project and thereafter new projects to come. This is not "one size fits all", we created a basic setup to start up with, so that new projects shouldn't start from a blank canvas, but of course the configurations were customized after their individual needs.

When implementing the tool as a part of introducing the part of the process it supports  I think we need to focus on several things:

  • Make the value of the tool visible - what will change and be easier?
  • Ensure that sufficient introduction/training/information about the tool is given.
  • Find ambassadors in the project, someone who can help advocate the tool if necessary, someone who is always present within the project rather than a distant support.
  • Find super users in the project, again it is crucial that it is someone who is always present in the project so that help is close by.
  • If possible - make your tools talk together! find tools that can do more than one thing :-) I our case we chose TestRail as a test management tool, and Jira for taskhandling and defect management - and for people it felt natural to work with the two together because of a good integration.

mandag den 8. juni 2015

Thoughts on Test Process Improvment - Learning

Remember my list from last week?

  • Know where you are (Current situation)
  • Know where you are going (Target situation)
  • How to eat an elephant
  • Learning
  • Ownership and commitment
  • Process first, then tools

Today: Learning

Learning is both on an individual level and in the organization as a whole. As individuals we need to learn about the new process, we need new skills to follow the process e.g. learn about test design techniques because we now have a strategy about structured test and wants to ensure the most efficient testing to be done. About tools if new tools appear because of the changed processes and of course also changes in culture and responsibilities in the team and for the individual team members.

But also on organizational level we need to learn. We may take inspiration from the Deming cycle or  PDCA – Plan, do, check, act.
I am a big fan of pilot projects, let’s not implement the process improvements big bang in the entire organization, but rather create pilot projects where we implement and learn from the experiences we gain.

We plan the test process improvement project by creating the roadmap, breaking the project down into smaller tasks, creating a backlog. As described above we need to prioritize the backlog and identify quick wins. We plan how to implement it – identifying the pilot projects to involve. The pilots should also be a part of the prioritization of the backlog, they know the current situation better than anyone and can supplement our knowledge from the assessment.

Based on the backlog we do; create the necessary process to support the individual backlog items and we implement them in the pilot projects. Again I recommend including relevant members of the pilot projects and also other projects in the creative process of creating the process parts.  We also ensure that the necessary training is conducted as a part of the “do”, we need to ensure that the people in the pilot projects are “dressed for the job”.

When the pilot is running we need to move to check, supporting the pilot projects and follow up on whether the changes have the anticipated effect. In some instances we can define formal KPI’s for this check, but often interviews and retrospectives are more valuable – you tend to get more information and feedback during that than from a KPI.

Based on the feedback from the pilot projects we can now act, we need to improve what we have already created and make it support the feedback we got from the “check”.  And round and round and round it goes J now this is a cycle that runs continuously, an iterative process of test process improvement.