All bugs are great bugs.... or

When it comes to bugs I used to have a clear approach when it came to bugs; "the only good bug is a dead bug" sort of .... :-) I thought that all bugs were great, that we needed to focus on attacking the system as aggressive as possible, that only our own limit when it came to imagination were setting the boundaries for what crazy test we could do in order to find these bugs.

Eg.for our C2 system; "let's make an area over the north pole with 274 position points, then change map format between lat-long, mgr and georef and for each map type zoom and pane like crazy. I bet that will kill it".
And it did - the system crashed eventually and I was happy - I had found a severity 1 bug - a crash.

Or for a recent system I tested within the energy sector "lets put a lot of hardcore html code into all the data entry fields, paste HUGE amounts of data into  fields that should have field length limitations".

And this is all fine - these are great and aggessive test scenarios... but what about the defects I found, how realistics are they? would a user (normal or hacker type) ever do this? will these defects ever be prioritized and fixed? Do my test case add value?

And here I really have a discussion with myself :-) because:
On one hand I think they do, they give us knowledge about weaknesses in the system, scenarios that we might use as basis for more realistic scenarios for other areas of the system. The one with the 274 points area was actually realistic, turns out the army actually uses a lot of position points when the create areas - it is just the part with the north pole and the crazy use of map types and zoom/pane I question.

On the other hand some of the really aggressive and to some extend crazy test cases I have executed in my time really are totally unrealistic, they are so far away from reality (see not real world - just reality) and from the users way of using the system that I feel that I am to some extend wasting my time doing those tests - and wasting developer time in getting them fixed.

So in the future I think I will do one thing for sure, I will ask my self "is this in any way realistic, will a user ever do anything like that - and if one in a million does...is it worth while?". Of course there is always the matter of security for systems that are open to the outside world - that makes the boundaries for what is realistic a bit different since hackers have a somewhat "crazier" imagination than most other users... but still.

What do you think - do you think we should attack all kinds of test situations, that ALL bugs needs to be addressed?

My Reading List

Do you ever have the problem - too many things to read too little time? I am trying to compile a list of stuff I either have to read or re-read! on the bookshelf I have:

"Explore It" by Elisabeth Hendrickson
"Impact Mapping" by Gojko Adzic
"Agile Management 3.0" by Jurgen Appolo
"Tab into mobile applkcation testing" by Jonathan Kohl
"Agile Test" by Lisa Crispin and Janet Gregory

And then I have to visit some of my favorite blogs and homepages:

satisfice.com by James Bach
developsense.com/ by Michael Bolton
http://googletesting.blogspot.dk/
http://happytesting.wordpress.com/ by Sigurdur Birgisson

But what else do I need to read???? where to go on the web to find inspiration? Write a comment with  suggestions please - would love to hear what you are reading right now.