Frustrated by how attached to detailed static Test Cases my teammates seem to be, I recently presented to them on Session-Based Test Management. I wanted to show that test cases are not the only way to track and report on testing, and even give management those metrics that they love.
Talking about SBTM was also a jumping-off point for introducing other topics including "Testing vs Checking"; how exploratory testing is understood in Context-Driven Testing; and approaches to test design in ET eg. heuristics, personas.
There was nothing special about the presentation and I'm slightly hesitant to include it here. Anyone who has worked in an organisation that takes ET and SBTM seriously will probably find it lacking.
To try and illustrate SBTM in action I also showed examples from a "home-brew" version of it that I had used in a previous job. (I was a sole tester in my first testing job so, with no-one to guide me, I put a SBTM process together from blog posts and my own trial-and-error experiments.)
To be clear: although I am generally a fan, I claim neither to be a member of the Context-Driven Testing community, nor qualified to accurately represent their views.
My Exploratory Testing is better than your Exploratory Testing
My teammates certainly would argue that they do exploratory testing already but my impression is that it tends to be closer to the "ad hoc" approach that got ET an unfair reputation for being vague - and which SBTM was designed to overcome.
Also, in my team ET is definitely seen as a poor relation to test cases. And I have genuinely heard comments like "if there's time left after running all the tests in the spreadsheet we can do some exploratory testing".
Failing to create debate
Talking about SBTM was also a jumping-off point for introducing other topics including "Testing vs Checking"; how exploratory testing is understood in Context-Driven Testing; and approaches to test design in ET eg. heuristics, personas.
There was nothing special about the presentation and I'm slightly hesitant to include it here. Anyone who has worked in an organisation that takes ET and SBTM seriously will probably find it lacking.
To try and illustrate SBTM in action I also showed examples from a "home-brew" version of it that I had used in a previous job. (I was a sole tester in my first testing job so, with no-one to guide me, I put a SBTM process together from blog posts and my own trial-and-error experiments.)
To be clear: although I am generally a fan, I claim neither to be a member of the Context-Driven Testing community, nor qualified to accurately represent their views.
My Exploratory Testing is better than your Exploratory Testing
My teammates certainly would argue that they do exploratory testing already but my impression is that it tends to be closer to the "ad hoc" approach that got ET an unfair reputation for being vague - and which SBTM was designed to overcome.
Also, in my team ET is definitely seen as a poor relation to test cases. And I have genuinely heard comments like "if there's time left after running all the tests in the spreadsheet we can do some exploratory testing".
Failing to create debate
I'm not delusional enough to think that one presentation from me would start a revolution, but I did hope we might at least have some debate about why they see so much value in test cases. And I hoped to hear some opinions/feedback on SBTM as a possible approach to delivering value through more thoughtful and skilled Exploratory Testing - with proper note-taking!
Sadly that just didn't happen. And what I thought would make a good blog post ended up being this one. Sorry.
It was almost as if what I was describing was a kind of interesting curiosity, but not relevant to them. Like when we watch a documentary about how people live in another culture.
Because they already do exploratory testing (as they see it). After all, exploratory testing is one of the things they use to come up with ideas for the real stuff - the Test Cases!
Now, of course, you'll be thinking that the failure to engage with my audience was down to my poor presentation skills. Maybe so.
Aware of that possibility, and of the fact that I was only skimming the surface of some big topics, I followed up the presentation by circulating links to articles by better communicators than me. (Reproduced below.)
I also offered to demo Rapid Reporter later to anyone was interested; and even emailed them all their own copy of the Test Heuristics Cheat Sheet.
A couple of weeks later, not only has no-one shown any interest, I find myself being asked to write even more test cases (because apparently our problem is that we don't have enough), and even attending Test Case Peer Review meetings.
Not giving up
It has to be said that after moaning about my presentation's lack of impact on Slack, I got some encouragement and a couple of good suggestions there.
Damian Meydac Jean pointed out that in situations like these there's often more to be gained by working on newbies than on people who've worked there for a while.
And Vernon Richards made a good point about how it can be more powerful to relate new ideas like these to specific issues or pain-points facing the team.
Seeking the case for test cases
But maybe I'm the one who's wrong. Why should the rest of the team/business change what they do to fit in with a lot of fancy-Dan ideas I'm repeating from the Twitter-bubble?
If I try to ask colleagues why they find test cases so valuable it does seem to strike them as an odd question to ask. There is just a sense in which there have to be test cases because, you know, we're doing testing. (Although they tend to say we're doing "QA".)
But more specific reasons include:
There is definitely a possibility that I am against them simply because I am too lazy to write them, and I find executing them boring. (In which case, maybe I should just get out of testing - God knows I think about that a lot.)
So, if there's anyone reading this with a view on the value to be found in test cases please add a comment. Despite my overall jokey tone here I really would like to be challenged on my cynical attitude.
-------------------------
For reference, below are the "further info" links I circulated, and added to our internal Wiki, after my presentation
-------------------------
Sadly that just didn't happen. And what I thought would make a good blog post ended up being this one. Sorry.
It was almost as if what I was describing was a kind of interesting curiosity, but not relevant to them. Like when we watch a documentary about how people live in another culture.
Because they already do exploratory testing (as they see it). After all, exploratory testing is one of the things they use to come up with ideas for the real stuff - the Test Cases!
Now, of course, you'll be thinking that the failure to engage with my audience was down to my poor presentation skills. Maybe so.
Aware of that possibility, and of the fact that I was only skimming the surface of some big topics, I followed up the presentation by circulating links to articles by better communicators than me. (Reproduced below.)
I also offered to demo Rapid Reporter later to anyone was interested; and even emailed them all their own copy of the Test Heuristics Cheat Sheet.
A couple of weeks later, not only has no-one shown any interest, I find myself being asked to write even more test cases (because apparently our problem is that we don't have enough), and even attending Test Case Peer Review meetings.
Not giving up
It has to be said that after moaning about my presentation's lack of impact on Slack, I got some encouragement and a couple of good suggestions there.
Damian Meydac Jean pointed out that in situations like these there's often more to be gained by working on newbies than on people who've worked there for a while.
And Vernon Richards made a good point about how it can be more powerful to relate new ideas like these to specific issues or pain-points facing the team.
Seeking the case for test cases
But maybe I'm the one who's wrong. Why should the rest of the team/business change what they do to fit in with a lot of fancy-Dan ideas I'm repeating from the Twitter-bubble?
If I try to ask colleagues why they find test cases so valuable it does seem to strike them as an odd question to ask. There is just a sense in which there have to be test cases because, you know, we're doing testing. (Although they tend to say we're doing "QA".)
But more specific reasons include:
- they provide scripts which can then be automated
- this does strike me as a pretty good reason although the reality is we have little automation and only a fraction of test cases written will ever be automated
- they suggest test ideas
- by this it's meant that team members can read the test cases to get ideas for what tests to carry out on a feature/product
- they serve as documentation
- they can be useful as a way for testers to familiarise themselves with products they haven't worked on before
- we have a situation where test cases are sometimes viewed as an oracle on "correct" product behaviour. Yikes.
- they tell us when we're finished regression testing
Let's have the debate right here!
Now I couldn't resist unfairly undermining some of the reasons I just listed, but I would genuinely be interested to hear more about the positives regarding the test case approach and why it might provide more value than exploratory testing in certain contexts.There is definitely a possibility that I am against them simply because I am too lazy to write them, and I find executing them boring. (In which case, maybe I should just get out of testing - God knows I think about that a lot.)
So, if there's anyone reading this with a view on the value to be found in test cases please add a comment. Despite my overall jokey tone here I really would like to be challenged on my cynical attitude.
-------------------------
For reference, below are the "further info" links I circulated, and added to our internal Wiki, after my presentation
-------------------------
Exploratory Testing
Why Scripted Testing Sucks, Steve Green
Exploratory Testing in a Regulated Environment, Josh Gibbs
Exploratory Testing, Michael Bolton
(Describes what ET is, and what it isn’t, according to Context-Driven
Testing. Also has a long list of links
to further resources on ET)
Session-Based Test Management
Session-Based Test Management (overview), James Bach
Session-Based Test Management (practical details of using it to manage
testing), Jonathan Bach
Managing Exploratory Testing, Rob Lambert
Learning to use Exploratory Testing in Your Organisation, Mike Talks
Benefits of session-based test management, Katrina Clokie
Exploratory Testing Frameworks
Generic Testing Personas, Katrina Clokie
Testing Tours, Mike Kelly
James Whittaker’s Exploratory
Testing Tours
https://msdn.microsoft.com/en-us/library/jj620911.aspx
