Testing Quadrants As A Communication Tool

 

Screenshot at toukokuuta 15 23-13-23

 

Here’s a modification of Agile Testing Quadrants by Lisa Crispin & Janet Gregory (originally by Brian Marick). I’ve used this as a communication tool in our current project. As it might raise few eyebrows, let me explain couple things.

(Also want to mention that I’ve modified the Quadrants a bit from how they are actually on the project – i.e. what we are focusing on & what we are not focusing on)

Context & Background

I explained very briefly my current context on my previous blog post: Striving For Early Feedback

Besides that, I’ve had challenges with explaining what we (our test team of 2) is testing, with respect to all others on the overall system. Then I was listening Paul Gerrard’s talk (Agile Test Strategy) a while ago. There he mentioned Agile Testing Quadrants. Of course I knew them and had seen them several times before. I just couldn’t come up with the idea of them being useful in my current context.

Agile-Testing-Quadrants

I first used Crispin & Gregory Quadrants while describing our approach to one vendor. There was though many things that I didn’t personally see useful (e.g. Automated & Manual or ET being on Q3). Then I had a discussion with Jari Laakso on Twitter. After that discussion I realized – with the help of Jari – that I can modify the Quadrants in any way I want. And that I did.

Things I want to emphasize

With my modified Quadrants I want to emphasize basically 4 things:

  1. Our testing being a division between testing the expected vs. unexpected – Or in other words ‘Focusing on Expectations’ vs. ‘Focusing on Risks’
  2.  We are intentionally not focusing on testing certain things (e.g. unit testing) – but are aware of how that’s being done by certain people
  3. All testing is based on exploration – in other words: Learning. We don’t test for the sake of testing, but to learn more about our product with every test
  4. Testing mentioned on the top is about People and not only about Business. It’s about people whose lives our product touches (Marc McNeill – according to Dan North)

What was challenging

Perhaps the biggest challenge with Quadrants is that it’s quite challenging to fit different forms of testing there. For example in performance testing we might be testing the expected (e.g. explicit documented requirements), but also regardless of those. That would mean that performance related testing goes to many places.

In the end I found that that particular challenge was something I can live with. Considering I can emphasize those other things with the model.

What next?

I don’t know yet if I will continue to modify and improve the model to make it more useful while describing our testing approach. Or if I will use this model later on the following projects or companies. I might. Or I might not. What matters now, is that it makes it a bit easier (from my perspective) to explain what our testing looks like.

7 thoughts on “Testing Quadrants As A Communication Tool

  1. I like the quadrants as a model for strategy and communication (and have used it myself). In your case, what happens with the bottom half of the quadrant. Is it done by the programmers, ignored (a little of both)?

    It’s interesting to note that on my team, programmers generally own everything on /the left side/ of the quadrant, while testers generally own everything on /the right side/.

    • Hi Alan,

      Bottom half of the quadrant is something we’ve put thought into, but under the current constraints – aren’t going to be testing by ourselves. It’s though something that we know that certain people are going to test, at several locations of overall system.

      My previous post (this one has link at the beginning) explains a bit our context, which is quite typical multivendor project where one vendor is building “main solution” and other vendors integrations related to that.

      Thanks for sharing your approach. While thinking our current project – that would most likely be a better situation. There’s though quite much challenges on getting several vendors considering especially top-left corner on their testing.

      Aleksis

  2. Nice Aleksis, I like how you’ve made the model fit for you and your context.

    How has the model been received by the people you’re using it communicate your testing? Who’s in the audience? Any iterations of the model after trialling it out with the audience?

    I have a couple of questions about the model:
    1. Is there any significance to the numbering of your quadrants? I see you’ve numbered them differently to Janet and Lisa.
    2. By “we” do you mean Testers?
    3. You mention you are aware of unit testing – are you aware of the coverage there is in the unit & integration tests? How do you know if there are any gaps or unnecessary duplication between your testing and those doing the unit testing?
    4. Are you not focussing at all on risks in q1 or q4? Likewise, do you not have any expectations in q2 & q3?

    Some great food for thought Aleksis, thanks for posting!

    Duncs

    • Hey Duncan,

      So far I’ve showed the model for couple vendors while describing our testing. Not so surprisingly there hasn’t been any feedback related to that. Mainly just people nodding. I’ll probably know more in few weeks, when I’ll introduce it to business people.

      I’ve iterated it, but only based on my own satisfaction on it. I modified it to begin with only for helping me with explaining our testing.

      1. Not significance in numbering. It could’ve been same way as your (& James) modification. I was also troubled with people seeing those as numbers to follow. So I mixed the numbers so it’s less linear.

      2. Yes. ‘We’ refers to me & one other tester.

      3. We know this because we’re reviewing their tests & also have a visibility to how they’re progressing. Doing that mainly because of the duplication you mention. But we also let other vendors review our tests, so they have chance to give feedback related to that.

      4. That’s a good question & I think it’s as you hint. We do pay attention to risks on Q1 & Q4 and to expectations on Q2 & Q3. For example use cases might have exceptions that have came from identifying risks. Or our test session might be focusing on seeing how disabling user’s rights from identity management solution works. I’d say there are expectations related to how we expect it to work – but then there’s also a lot of room for exploring and coming up with test ideas that are not directly linked to anything we’ve explicitly specified on documented requirements.

      In my opinion the biggest difference is that tests on Q1 & Q4 are created based on e.g. specifications and tests on Q2 & Q3 are created based on recognizing possible risks. Many of our test ideas that we try out on sessions are basically risks that we have identified.

      I guess I could also mention that it’s been quite common that testing on current context is based highly on seeing if software behaves as it’s specified in documented requirements. Therefore we want to emphasize that we keep that in mind while focusing on expectations, but we also don’t forget that users don’t care if product behaves as specified. That’s we put effort on recognizing those risks and testing them.

      Aleksis

  3. This is an excellent thought Aleksis. I have been using the Testing Quadrants a lot to devise my testing strategy for a particular solution but using it as a communication tool is a brilliant idea. I’ll try this :)

  4. I love how you have used the Quadrants, and that is exactly what we (and I think Brian as well) hoped – that people will use them to think about what kinds of testing they need to do, where to focus their efforts next, what they might need to learn or create in order to accomplish the testing along with coding. I particularly like how you’ve put exploration in the middle – it is so true that we are always able to explore, no matter what testing we’re doing.

    Janet and I have made some modifications to our original model for our new book _More Agile Testing_ which will be published in a few months, when I find time I will post it and explain how it has evolved for us. The use and outcomes of the Quadrants model are unique to each team, but we’ve heard from many teams who found them useful.

  5. We have found more and more people challenging / adapting the quadrants – which is excellent. It means people are thinking and applying the ideas to their context. No matter how you adapt, I think 2 of the most important things that come out of it are:
    1. it can be used as a communication / conversation starter to talk about testing and what needs to be done for your application or solution
    2. it emphasizes that testing really is an activity that all team members (and others like customers) must participate in.
    I also like to say the left side is about preventing defects, and the right side is about finding them, which is similar to your idea, just expressed slightly less eloquently :-).
    We left the numbering as Brian Marick had originally had it since we couldn’t find a better way… not for lack of trying.
    Thanks for sharing.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s