How to share what you discover on your next exploratory testing session
I grew up listening to amazing artists such as Prince and Michael Jackson. I didn't know much about them other than their incredible music and mind-boggling videos.
Back then they had no ability to share what they had for breakfast or who they were hanging out with last night. They had an air of mystery about them and I liked it. They focused on their craft and shared material at a reasonable rate — particularly Prince. This satisfied my musical cravings.
Too much information unrelated to the essence of their artistry and I'd lose the joy of their output.
Too little information and I'm confused and disillusioned by the lack of content. Remember Chinese Democracy?
Simplify your exploratory testing output
Exploratory testing sessions create frustration if too much or too little information is shared.
Simplify your exploratory testing output and reduce frustration for you and your team.
I'll contribute some thoughts on how to share what you discover on your next exploratory testing session. Consider trying the following:
1. Avoid using pass and fail
What does pass and fail tell you about your product? If a test passes it indicates that something specified might have been met. If a test fails it suggests something is different to what was desired.
What if you have further thoughts beyond pass or fail? What if you'd like to provide context to what you've discovered?
Learnings aren't binary so it feels odd to group your test output with pass and fail.
Using pass and fail closes the door on product discovery – the opportunity to find out what the product actually does instead of what it was intended to do.
2. Capture problems and discussions
It's difficult to group the outcome of your exploratory testing discoveries. Output is often refererred to as bugs, defects, issues, enhancements, ideas, feature requests, tasks, notes, thoughts and more.
Try using two categories:
- Problem: a discovery that threatens the value of the product e.g. a checkout that doesn't accept new credit card details.
- Discuss: a discovery that warrants a discussion e.g. it's unclear how to add a new credit card.
Using Problem and Discuss alleviates non-value adding comments such as:
- "That's not a bug it's a feature!"
- "Our defect count is 16."
- "You've grouped the issues into bugs, enhancements and feature requests. That's not how I see it."
3. Get to the point
A while ago I took pride in configuring JIRA to ask for information such as a summary, steps to recreate, actual result, expected result, known workaround, number of users affected and more. This info didn't add much value.
Consider your target audience and capture the essence of your exploratory testing discoveries.
Instead of capturing so much information, describe how you discovered the problem using bullets. And only do that.
Softer language might help create better discussions. For example:
- "I have a hunch users might feel confused by the red plus sign ..."
- "We have an icon for all other parts of the page. Maybe we could try ..."
- "Might it work to prompt users to ..."
4. Invite people to the conversation
You've learnt many things during your exploratory testing session. Invite others to engage with your discoveries and create a conversation.
Centre your team around your product and what you've discovered by asking this question: "What do you think?".
Over to you
I'd love to hear your thoughts. How do you capture and share exploratory testing discoveries? What challenges do you face sharing with others? Have you used some of the suggested techniques? How did you get on?