Thoughts on Process Improvement

A week ago I posted a small article with thought on sucessful facilitation of test process improvement at techwell.com. In that article I focused on one my favorite key points within process improvement - how to eat an elephant! the fact that big bang process improvement projects rarely have success because people can't keep up and digest all the new activities, workproducts etc. In stead I recommend an approach where you do it just like you would eat that elephant - one bite at the time.

But there are several key things to focus on when you are involved in improving a process, at least;

• Know where you are (Current situation)
• Know where you are going (Target situation)
• How to eat an elephant
• Learning
• Ownership and commitment
• Process first, then tools

In the following weeks I will do a bit of blogging on the other parts than the elephant :-).
  

Today: Know where you are - and know where you are going!

If you are going on a road trip and you want to draw a route (or get the route in your GPS) you need two key points: the position you are at now, and the target position where you want to go to. 

But that doesn’t just go for road trips and travelling – it goes for process improvement as well.

How can we identify what we need to improve if we don’t know how we are working today, and even more important – what works and what don’t? So you need to start with creating the baseline – identify current state of the process in your team/project/organization. 

There are many ways to do this, two of the formal ways of measuring the process maturity within testing is the TMMI and TPI NEXT assessment. Both of these are formal processes for identifying, analyzing and evaluating the maturity of a given test process – no matter whether we are talking about a single project or an entire organization. With these formal methods you can either do the assessment yourself or you can ask someone outside the organization to do it, and the result of the assessment should be both a report on current state but also a recommended roadmap for improvement.

But you can also do that in an informal manner. The important thing here is; Involve the right people from the beginning! In my current assignment we started with a brown paper exercise involving the different stakeholders in the program. Together we drew the current process using post-its and identified unknowns, questions and problems with post-its of another color.

 

We discussed every step on the way agreeing how the process looked at that time, and after our workshop the drawing was presented to the rest of the team to ensure that we were agreeing on a common picture of the as-is process. The drawing stayed on a wall for a period allowing people to think and comment, a new color of post-its were available making it visible what was added afterwards :-)

With the drawing you have an illustration of where you are – what is the current state of your organization within testing, where your strengths are and also your weaknesses. 

With that knowledge it is now time to discus where you want to go. Together identify potential improvements, prioritize them and create an improvement backlog. Since I love illustrating things in swim lanes I took the brownpaper and created a process drawing in Vision that made the current responsibilities and flow visible. One of the first things in the backlog was then to create the to-be process drawing - the "dream target" and discuss what low hanging fruits you could pick to make the first visible improvements and get the sense of progress from an early point.

We of course agreed that this was not a process goal that could not change, we learn as we go and improve the dream target as we do - it is a moving target but at least we had a basic idea about where we were going.

At this point in time we didn't talk about tools, we talked a bit about what basic internal training we could do - but the main thing was to get a common picture of the journey we were going to take.

Product risk, are we talking the talk or walking the walk.

Product risks is something we talk a lot about as testers... but do we only talk the talk or are we also walking the walk?

Recently I visited a project where they had written in their test strategy that they were doing risk based testing. They had completed a PRA (Product risk analysis), had a beautiful table in the strategy document showing all identified product risks and weighted them according to damage and change of failure. They also identified on which test levels they should test with what intensity to mitigate those risks… even identified test techniques to use to get the best test done….completely by the book… I felt happy… ;-) so beautiful. 

But then I started to take a look at the test being done, the test designs and test cases and I started to wonder. Nowhere was it visible to me that the identified test strategy was addressed, that any kind of test design techniques were used etc. So I asked the testers in that project: have you considered the PRA which was conducted for the system? Have you used the test techniques, have you even looked at the risk table when you designed and implemented the test for this system… and sadly the answer was NO. They hadn’t had time to use test design techniques they said, and they had forgotten about the test strategy document. 

Since then I have stumbled upon that a couple of times. One of my friends who’s also a test manager had conducted a PRA together with the business and the testers to get a picture of how the business saw the system. But when the result was presented to the testers and test lead the answer were; nice table but we don’t use it anyway.

So how do we change this? How do we go from talking the talk to actually walking the walk? Or should we maybe just accept that we don’t?

I actually think that we should walk the walk, the process of identifying and classifying product risk as a foundation for a test strategy and for testing is the right thing to do, but maybe we could do it in another way? Maybe we shouldn’t just hide the result of risk analysis in spreadsheets and tools? Maybe we should focus more on the conversation we have when we do the risk analysis – the knowledge we share and less about the formalities?

For example I am a great fan Product Risk Analysis as described in TMap, but I have my own lightweight version of how to do it – have taken a lot of the formality away and primarily focus on getting people to talk about risk. Getting the right mix of people together around a whiteboard, getting them to talk about what THEY see as product risks, and even more important getting them to discuss both damage and chance of failure – explaining to each other why they see the risk as that high (or low). 

The table that comes out of it is just like in TMap, but we have made it together, we have discussed, shared knowledge and even clarified potential misunderstandings with the scope during the workshop.

I even do the test strategy table (maybe not the test techniques… that depends on the testers), but rather than just putting it in a test strategy document I make it visible just next to the task board. And when someone starts a new task/story we talk about how that fits into the risk picture.  And when a tester starts on a new feature we take a look at the product risks identified and break it down to more detail for the given feature, ensuring the right focus and weighting of the test.

The main thing in my humble opinion is that we talk about risk to ensure that we have a common picture, and that we actually address them when we test - what form, shape or name we give it is less important..

3 Pillars of Agile Quality

I am currently at STARWEST in Anaheim CA, and have participated in a couple of interesting tutorials the last two days.
Monday was "a dozen keys to agile testing maturity" with Bob Galen and Mary Thorn. Two good take aways; the walk through of 15 agile testing myths and realities, and the presentation of the three pillars of agile quality. The two presenters are currently writing a book about the subject and we got a walk through of the concept.
Roughly it covers the fact that you can split agile quality up in three major areas; "development and test automation", "software testing" and "cross-functional team practices". The three pillars stands on a foundation consisting of amongst other whole-team ownership of quality, building it "right"; building the "right" thing and healthy agile centric metrics.
A sound focus on quality in an agile context builds on a strategic balance between the three pillars, and all three must be considered.
We'll I just got a quick intro but I am looking forward to read much more about it, and I could imagine the contents of the three pillars would form an excellent basis for a conversation in the team about agile quality.

Take care,
Gitte - aka Godtesen

The missing tweets from #STARCanada

I normally try to tweet a bit when I participates in conferences, but at STARCanada I simply couldn’t – my phone did not agree with the wifi, couldn’t find any in the rooms where track sessions and keynotes where held. 

So I wrote down my tweets as they popped up, so here is my STARCanada collection of tweets.

Actually a bit funny to see them all together in one page J

K1 Michael Bolton – why software drives us crazy

• From quality assurance to quality assistance
• Expectations vs desires
• I keep finding myself shaving a yak!! (I think it meant: a chain of activities you want to do in order to reach a goal making it so difficult that one end up forgetting the original goal)
• Too often a system presents what it can do as a presentation of the internal datastructure rather thatn actually supporting what the user needs to do
• Geeks drives cars with stick shift, because they’re interested in the process of driving
• We report: are there problems in the product that represent risks to the use.
• Organisations don’t like hearing about problems
• Testing is viewed as cost centers in many companies rather thatn the nerve system of the organization.
• Think of testing not as “test cases” but as learning about product through experimentation.

T5 Nancy Kelln, Are your test reports a death sentences?

• Comparing the five states you may go through when receiving a death sentence with reactions to getting bad news (test reports)
• Anger – deneil – bargening – depression – acceptance
• Every time you have a problem, then look at it and figure out how it can be a “people problem”.
• Depression: due to the nature of testing we cannot just disconnect and move on
• An emotional response to bad news may mean your message was heard

T9 Paul Holland, Agile test management and reporting – even in a non-agile project

• Using bad metrics – promot bad behavior
• Measure/compare elements that are consistent in size or composition. Test cases are not! (can take 5 minutes or 5 days). Bugs aren’t  either (complexity, probability etc).
• Bad metrics risk creating unhealthy competition between teams (Test cases per tester, bugs per tester)
• Bad metrics contain misleading information or gives a false sense of completeness 8e.g. coverage, pass rate, number of test cases)
• Use white board for test execution

K3 Ray Arell, the art of complex system testing

• Move from “what did you do yesterday” to “what did you LEARN yesterday”
• The magic roundabout (see here)
• The complex adaptive system model
• Youd cannot connect the dots going forward, you can only connect them looking backwards (Steve Jobs)
• Cognitive-edge.com got to check out!!
• Don’t waste your crisis – this is where innovation happens
• Simple systems: Mind numbing bureaucracy
• Complex systems: Fluffy bunnies and tree huggers
• Complicated systems: Tyranny of the experts
• Chaos: True catastrophe

W8 Scott Barber: How metrics programs can destroy your soul

• Qualitative vs quantitative metrics
• Metrics without context doesn’t tell you anything
• Build me software that makes me money – the “bring me a rock” game
• “measurement dysfunction”. People tend to optimize metrics => invites “bad stuff”
• Quantitative metrics are invitations for conversations
• Inconsistent units: test/test cases, Size/importance of defects
• Quality is subjective: How good is “good enough”? What does pass/fail mean?
• What do you really want to know?

The right way to learn!

How do you learn?

A discussion that often arises during conferences, on twitter and different blogs is the approach to learning - what is the "right way" to learn, and what is the wrong way? Is certification the evil of all times, or is it okay, should we learn about testing by reading books about flying hot air ballons or should we do a TMap or ISTQB training?

I can't answer that question - i don't have "best practice", but I can tell you a bit about how I learn. In order to do that a few words about my background:

• highschool (I survivied - didn't exactly excell :-))
• 9 years in the airforce as corporal
• 17 years in Systematic as tester, test manager, test architect and last program test manager
• 2 years as consultant in Sogeti Denmark
• scout (not currently active)
• creative, love using my hands
• love interacting with other people with the same interests as me
• read books about testing... or fantasy... sometimes

So not a classic IT background, but ever since I started my carreer as tester I had a need/urge to learn - I wanted to know more - move further on. I have learned through many different channels during my life as a tester. I have completed a bit of formal training:

• ISEB foundation
• ISEB practitioner
• Certified scrum master
• Certified CAT trainer
• TPI foundation
• TMap Test engineer
• BUsiness driven test management
• Test management training with SQE
• and probably others over the years ;-)

I have participated in conferences every year since 2004. And I still go to participate in tutorials every time - there are so much great stuff to learn from that, from skilled practitioners who have a lot of experiences to share.

I participate in networks in Denmark when I have time. Sharing knowledge, giving some to others and getting a lot back. And that is also what I love about the conferences, not just the track sessions, but all the discussions in the breaks and in the evenings - I grow and learn from every meeting with new practitioners.

I read books. That is; I attempt to read books but lack time. My library is always growing and I always bring home an extended list of books when I have been together with other testers. I don't always read the entire book, there might be parts that catches my eyes and others that doesn't sound too interesting - some books I buy but never get to read! (an example; how to test SAP... I thought I really needed to learn but I never got around to read it, it's just standing there).

I read blogs. Not as many and as often as I would love to - time is still a factor here. But I try to read regularly. And from twitter I get pointed in the direction of interesting blogs all the time.

I do other things than testing! yep I said it, my life is so many other things than testing - and I prioritize to have a full life with people I love, doing other things that I also love and more than anything else... have fun.

A long story but what is it really I am trying to say: I honestly don't think there is a right way or a wrong way to learn, I will not fuss about certifications being the evil of all time or the best of all - ask your self -what is the best way for you to learn. If you cannot stand the thought of going to formal training and get certificed... then don't. If you think that is the best way for you to get started, then by all means go. The main thing is that you find out what works for you, no one else should tell you how to learn - you know best. But don't justt just do ONE thing - there are many channels of learning and I think we should use as many as possible to get as diverse an approach to testing as possible.

/Gitte
 

An updated reading list after agile testing days 2013

I have amended my reading list a bit, some are now done and a some new have been added. 



"Thinking fast and Slow" by Daniel Kahneman

"Explore It" by Elisabeth Hendrickson

"Agile Management 3.0" by Jurgen Appelo

"Tab into Mobile application testing" by Jonathan Kohl

"Are your lights on" by Gerald M Weinberg and Donald C . Gause

"Trust and betrayal in the workplace"by Dennis Reina and Michelle Reina

"The black swan" by Nassim Nicholas Taleb

"How to solve it" G. Polya

"Discover to deliver" ?

"How to measure everything"

"Sparks of genius" by ROber and Michele Root-Bernstein

"Gut feelings" Gerd Gigerenzer

"Adaptive thinking" Gerd Gigerenzer



Done:

"Impact Mapping" by Gojko Adzic 

"Agile Test" by Lisa Crispin and Janet Gregory



A couple of blogs and websites worth visiting:



satisfice.com by James Bach

developsense.com/ by Michael Bolton

http://googletesting.blogspot.dk/

http://happytesting.wordpress.com/ by Sigurdur Birgisson

http://theadventuresofaspacemonkey.blogspot.com by Sami Söderblom

http://www.huibschoots.nl/wordpress/ by Huib Schoots

Being a Pragmatic Tester?

Okay everybody – this is the first time I take a chance and raise my voice… but I think someone has to start this discussion!

Last week I participated in Agile Testing Days in Potsdam, a great conference where I got a lot of new ideas and inspiration, and even more important got to know some great people!

But one thing made me wonder… and sort of made me sad; I got the feeling that fundamentalism has entered our community - Both within the agile community and the context driven community. You are with them, or you are against them, there is only one universal truth and way of testing for parts of that community…. Does that ring a bell anyone?

I don’t really like fundamentalism, not in religion, not in politics and not in testing.

I consider myself a context driven tester, and an agile tester – that is how I work and that is how I want testing to be done... if possible. I know the 12 principles of agile and the 7 principles of context driven testing… and I try to use what they stand for to the best of my abilities and the CONTEXT I am in. 

Yesterday I talked with a very smart man on the phone, and we discussed the fundamentalism in the testing community. He said something like ”I sometimes feel that the context driven community has lost the context” – and to be honest sometimes I feel that he is right.

Now this is where I just don’t get it; the context driven approach is all about CONTEXT – why is it then that I often feel that parts of the context driven community tells me that there is only one universal truth and that is exploration, session based testing, heuristics, algorithms, no formal training and certification, rapid software testing etc. Why is it that I feel that it is not ”allowed” and recognized that someone could actually find themself in another CONTEXT – a place where some things can be applied and others cannot, where artifacts and ways of testing are dictated by standards (safety, military etc) or by contracts.

"Context-driven testers choose their testing objectives, techniques, and deliverables (including test documentation) by looking first to the details of the specific situation, including the desires of the stakeholders who commissioned the testing. The essence of context-driven testing is project-appropriate application of skill and judgment. The Context-Driven School of testing places this approach to testing within a humanistic social and ethical framework."

The quote above is from context-driven-testing.com written by James Bach and Cem Kaner, probably quite a while ago, but I think it in very simple language that we all can understand and comprehend tells us the essence of context driven testing – or at least how I would love context driven testing to be.

If my customer asks me to deliver artifacts or requires testing done in a certain way, I will surely try to convince him otherwise if I do not agree – but if he chooses to continue with his way (which is often the standard-driven-approach) then I will have to accept that and try to do the best of my abilities within that CONTEXT.

Okay I can already hear the first comment; yeah right but you are Sogeti – you have to say that because you are all in to that TMap stuff and you do ISTQB certifications too. But honestly guys; I have been in testing since 1995, standards, agile, exploratory and all the other stuff is a part of my luggage. 

I feel like I am the audience at a boxing game: In the green corner; the context driven community and in the yellow corner TMap/ISQTB/CMMI/TMI/TPI and all the other abbreviations – the “standards test approach” you might say. And now I am forced to cheer for one of the corners, green or yellow. But sorry guys I will not!!

What about another solution, why not be a PRAGMATIC TESTER. What about taking what we can use from the context driven approach, and what we can use from the standards test approach– and then make our own test soup and make it work in the context we are working within. What about the concept of peaceful co-existence? could that work for our community too?


Call it real world call it unicorn world…. Honestly I don’t really care… it would be great if it was our world.