3 Pillars of Agile Quality

I am currently at STARWEST in Anaheim CA, and have participated in a couple of interesting tutorials the last two days.
Monday was "a dozen keys to agile testing maturity" with Bob Galen and Mary Thorn. Two good take aways; the walk through of 15 agile testing myths and realities, and the presentation of the three pillars of agile quality. The two presenters are currently writing a book about the subject and we got a walk through of the concept.
Roughly it covers the fact that you can split agile quality up in three major areas; "development and test automation", "software testing" and "cross-functional team practices". The three pillars stands on a foundation consisting of amongst other whole-team ownership of quality, building it "right"; building the "right" thing and healthy agile centric metrics.
A sound focus on quality in an agile context builds on a strategic balance between the three pillars, and all three must be considered.
We'll I just got a quick intro but I am looking forward to read much more about it, and I could imagine the contents of the three pillars would form an excellent basis for a conversation in the team about agile quality.

Take care,
Gitte - aka Godtesen

The missing tweets from #STARCanada

I normally try to tweet a bit when I participates in conferences, but at STARCanada I simply couldn’t – my phone did not agree with the wifi, couldn’t find any in the rooms where track sessions and keynotes where held. 

So I wrote down my tweets as they popped up, so here is my STARCanada collection of tweets.

Actually a bit funny to see them all together in one page J

K1 Michael Bolton – why software drives us crazy

• From quality assurance to quality assistance
• Expectations vs desires
• I keep finding myself shaving a yak!! (I think it meant: a chain of activities you want to do in order to reach a goal making it so difficult that one end up forgetting the original goal)
• Too often a system presents what it can do as a presentation of the internal datastructure rather thatn actually supporting what the user needs to do
• Geeks drives cars with stick shift, because they’re interested in the process of driving
• We report: are there problems in the product that represent risks to the use.
• Organisations don’t like hearing about problems
• Testing is viewed as cost centers in many companies rather thatn the nerve system of the organization.
• Think of testing not as “test cases” but as learning about product through experimentation.

T5 Nancy Kelln, Are your test reports a death sentences?

• Comparing the five states you may go through when receiving a death sentence with reactions to getting bad news (test reports)
• Anger – deneil – bargening – depression – acceptance
• Every time you have a problem, then look at it and figure out how it can be a “people problem”.
• Depression: due to the nature of testing we cannot just disconnect and move on
• An emotional response to bad news may mean your message was heard

T9 Paul Holland, Agile test management and reporting – even in a non-agile project

• Using bad metrics – promot bad behavior
• Measure/compare elements that are consistent in size or composition. Test cases are not! (can take 5 minutes or 5 days). Bugs aren’t  either (complexity, probability etc).
• Bad metrics risk creating unhealthy competition between teams (Test cases per tester, bugs per tester)
• Bad metrics contain misleading information or gives a false sense of completeness 8e.g. coverage, pass rate, number of test cases)
• Use white board for test execution

K3 Ray Arell, the art of complex system testing

• Move from “what did you do yesterday” to “what did you LEARN yesterday”
• The magic roundabout (see here)
• The complex adaptive system model
• Youd cannot connect the dots going forward, you can only connect them looking backwards (Steve Jobs)
• Cognitive-edge.com got to check out!!
• Don’t waste your crisis – this is where innovation happens
• Simple systems: Mind numbing bureaucracy
• Complex systems: Fluffy bunnies and tree huggers
• Complicated systems: Tyranny of the experts
• Chaos: True catastrophe

W8 Scott Barber: How metrics programs can destroy your soul

• Qualitative vs quantitative metrics
• Metrics without context doesn’t tell you anything
• Build me software that makes me money – the “bring me a rock” game
• “measurement dysfunction”. People tend to optimize metrics => invites “bad stuff”
• Quantitative metrics are invitations for conversations
• Inconsistent units: test/test cases, Size/importance of defects
• Quality is subjective: How good is “good enough”? What does pass/fail mean?
• What do you really want to know?

The right way to learn!

How do you learn?

A discussion that often arises during conferences, on twitter and different blogs is the approach to learning - what is the "right way" to learn, and what is the wrong way? Is certification the evil of all times, or is it okay, should we learn about testing by reading books about flying hot air ballons or should we do a TMap or ISTQB training?

I can't answer that question - i don't have "best practice", but I can tell you a bit about how I learn. In order to do that a few words about my background:

• highschool (I survivied - didn't exactly excell :-))
• 9 years in the airforce as corporal
• 17 years in Systematic as tester, test manager, test architect and last program test manager
• 2 years as consultant in Sogeti Denmark
• scout (not currently active)
• creative, love using my hands
• love interacting with other people with the same interests as me
• read books about testing... or fantasy... sometimes

So not a classic IT background, but ever since I started my carreer as tester I had a need/urge to learn - I wanted to know more - move further on. I have learned through many different channels during my life as a tester. I have completed a bit of formal training:

• ISEB foundation
• ISEB practitioner
• Certified scrum master
• Certified CAT trainer
• TPI foundation
• TMap Test engineer
• BUsiness driven test management
• Test management training with SQE
• and probably others over the years ;-)

I have participated in conferences every year since 2004. And I still go to participate in tutorials every time - there are so much great stuff to learn from that, from skilled practitioners who have a lot of experiences to share.

I participate in networks in Denmark when I have time. Sharing knowledge, giving some to others and getting a lot back. And that is also what I love about the conferences, not just the track sessions, but all the discussions in the breaks and in the evenings - I grow and learn from every meeting with new practitioners.

I read books. That is; I attempt to read books but lack time. My library is always growing and I always bring home an extended list of books when I have been together with other testers. I don't always read the entire book, there might be parts that catches my eyes and others that doesn't sound too interesting - some books I buy but never get to read! (an example; how to test SAP... I thought I really needed to learn but I never got around to read it, it's just standing there).

I read blogs. Not as many and as often as I would love to - time is still a factor here. But I try to read regularly. And from twitter I get pointed in the direction of interesting blogs all the time.

I do other things than testing! yep I said it, my life is so many other things than testing - and I prioritize to have a full life with people I love, doing other things that I also love and more than anything else... have fun.

A long story but what is it really I am trying to say: I honestly don't think there is a right way or a wrong way to learn, I will not fuss about certifications being the evil of all time or the best of all - ask your self -what is the best way for you to learn. If you cannot stand the thought of going to formal training and get certificed... then don't. If you think that is the best way for you to get started, then by all means go. The main thing is that you find out what works for you, no one else should tell you how to learn - you know best. But don't justt just do ONE thing - there are many channels of learning and I think we should use as many as possible to get as diverse an approach to testing as possible.

/Gitte
 

An updated reading list after agile testing days 2013

I have amended my reading list a bit, some are now done and a some new have been added. 



"Thinking fast and Slow" by Daniel Kahneman

"Explore It" by Elisabeth Hendrickson

"Agile Management 3.0" by Jurgen Appelo

"Tab into Mobile application testing" by Jonathan Kohl

"Are your lights on" by Gerald M Weinberg and Donald C . Gause

"Trust and betrayal in the workplace"by Dennis Reina and Michelle Reina

"The black swan" by Nassim Nicholas Taleb

"How to solve it" G. Polya

"Discover to deliver" ?

"How to measure everything"

"Sparks of genius" by ROber and Michele Root-Bernstein

"Gut feelings" Gerd Gigerenzer

"Adaptive thinking" Gerd Gigerenzer



Done:

"Impact Mapping" by Gojko Adzic 

"Agile Test" by Lisa Crispin and Janet Gregory



A couple of blogs and websites worth visiting:



satisfice.com by James Bach

developsense.com/ by Michael Bolton

http://googletesting.blogspot.dk/

http://happytesting.wordpress.com/ by Sigurdur Birgisson

http://theadventuresofaspacemonkey.blogspot.com by Sami Söderblom

http://www.huibschoots.nl/wordpress/ by Huib Schoots

Being a Pragmatic Tester?

Okay everybody – this is the first time I take a chance and raise my voice… but I think someone has to start this discussion!

Last week I participated in Agile Testing Days in Potsdam, a great conference where I got a lot of new ideas and inspiration, and even more important got to know some great people!

But one thing made me wonder… and sort of made me sad; I got the feeling that fundamentalism has entered our community - Both within the agile community and the context driven community. You are with them, or you are against them, there is only one universal truth and way of testing for parts of that community…. Does that ring a bell anyone?

I don’t really like fundamentalism, not in religion, not in politics and not in testing.

I consider myself a context driven tester, and an agile tester – that is how I work and that is how I want testing to be done... if possible. I know the 12 principles of agile and the 7 principles of context driven testing… and I try to use what they stand for to the best of my abilities and the CONTEXT I am in. 

Yesterday I talked with a very smart man on the phone, and we discussed the fundamentalism in the testing community. He said something like ”I sometimes feel that the context driven community has lost the context” – and to be honest sometimes I feel that he is right.

Now this is where I just don’t get it; the context driven approach is all about CONTEXT – why is it then that I often feel that parts of the context driven community tells me that there is only one universal truth and that is exploration, session based testing, heuristics, algorithms, no formal training and certification, rapid software testing etc. Why is it that I feel that it is not ”allowed” and recognized that someone could actually find themself in another CONTEXT – a place where some things can be applied and others cannot, where artifacts and ways of testing are dictated by standards (safety, military etc) or by contracts.

"Context-driven testers choose their testing objectives, techniques, and deliverables (including test documentation) by looking first to the details of the specific situation, including the desires of the stakeholders who commissioned the testing. The essence of context-driven testing is project-appropriate application of skill and judgment. The Context-Driven School of testing places this approach to testing within a humanistic social and ethical framework."

The quote above is from context-driven-testing.com written by James Bach and Cem Kaner, probably quite a while ago, but I think it in very simple language that we all can understand and comprehend tells us the essence of context driven testing – or at least how I would love context driven testing to be.

If my customer asks me to deliver artifacts or requires testing done in a certain way, I will surely try to convince him otherwise if I do not agree – but if he chooses to continue with his way (which is often the standard-driven-approach) then I will have to accept that and try to do the best of my abilities within that CONTEXT.

Okay I can already hear the first comment; yeah right but you are Sogeti – you have to say that because you are all in to that TMap stuff and you do ISTQB certifications too. But honestly guys; I have been in testing since 1995, standards, agile, exploratory and all the other stuff is a part of my luggage. 

I feel like I am the audience at a boxing game: In the green corner; the context driven community and in the yellow corner TMap/ISQTB/CMMI/TMI/TPI and all the other abbreviations – the “standards test approach” you might say. And now I am forced to cheer for one of the corners, green or yellow. But sorry guys I will not!!

What about another solution, why not be a PRAGMATIC TESTER. What about taking what we can use from the context driven approach, and what we can use from the standards test approach– and then make our own test soup and make it work in the context we are working within. What about the concept of peaceful co-existence? could that work for our community too?


Call it real world call it unicorn world…. Honestly I don’t really care… it would be great if it was our world.

All bugs are great bugs.... or

When it comes to bugs I used to have a clear approach when it came to bugs; "the only good bug is a dead bug" sort of .... :-) I thought that all bugs were great, that we needed to focus on attacking the system as aggressive as possible, that only our own limit when it came to imagination were setting the boundaries for what crazy test we could do in order to find these bugs.

Eg.for our C2 system; "let's make an area over the north pole with 274 position points, then change map format between lat-long, mgr and georef and for each map type zoom and pane like crazy. I bet that will kill it".
And it did - the system crashed eventually and I was happy - I had found a severity 1 bug - a crash.

Or for a recent system I tested within the energy sector "lets put a lot of hardcore html code into all the data entry fields, paste HUGE amounts of data into  fields that should have field length limitations".

And this is all fine - these are great and aggessive test scenarios... but what about the defects I found, how realistics are they? would a user (normal or hacker type) ever do this? will these defects ever be prioritized and fixed? Do my test case add value?

And here I really have a discussion with myself :-) because:
On one hand I think they do, they give us knowledge about weaknesses in the system, scenarios that we might use as basis for more realistic scenarios for other areas of the system. The one with the 274 points area was actually realistic, turns out the army actually uses a lot of position points when the create areas - it is just the part with the north pole and the crazy use of map types and zoom/pane I question.

On the other hand some of the really aggressive and to some extend crazy test cases I have executed in my time really are totally unrealistic, they are so far away from reality (see not real world - just reality) and from the users way of using the system that I feel that I am to some extend wasting my time doing those tests - and wasting developer time in getting them fixed.

So in the future I think I will do one thing for sure, I will ask my self "is this in any way realistic, will a user ever do anything like that - and if one in a million does...is it worth while?". Of course there is always the matter of security for systems that are open to the outside world - that makes the boundaries for what is realistic a bit different since hackers have a somewhat "crazier" imagination than most other users... but still.

What do you think - do you think we should attack all kinds of test situations, that ALL bugs needs to be addressed?

My Reading List

Do you ever have the problem - too many things to read too little time? I am trying to compile a list of stuff I either have to read or re-read! on the bookshelf I have:

"Explore It" by Elisabeth Hendrickson
"Impact Mapping" by Gojko Adzic
"Agile Management 3.0" by Jurgen Appolo
"Tab into mobile applkcation testing" by Jonathan Kohl
"Agile Test" by Lisa Crispin and Janet Gregory

And then I have to visit some of my favorite blogs and homepages:

satisfice.com by James Bach
developsense.com/ by Michael Bolton
http://googletesting.blogspot.dk/
http://happytesting.wordpress.com/ by Sigurdur Birgisson

But what else do I need to read???? where to go on the web to find inspiration? Write a comment with  suggestions please - would love to hear what you are reading right now.