Exploratory Software Testing with James Whittaker

I just finished listening to the uTest Webinar with James Whittaker about Exploratory Testing.  James is the Director of Test Engineering at Google and has written “Exploratory Software Testing: Tips Tricks, Tours, and Techniques to Guide Test Design.”

So this was interesting stuff!  I always thought of “exploratory testing” as “adhoc” testing, but as James pointed out, it’s much more.  He started by giving several examples of some classic anecdotal bugs…  One such bug is demonstrated in the map… rather roundabout way of getting from Start to Finish!  James noted that there are techniques that testers use to figure out what the most important tests are and how to go about executing them.  Often this comes from experience. These can fall into patterns. James calls these “tours.”  In his book, James describes a variety of different types of “tours” used as metaphors, that capture the methodology of discovering bugs that are likely to occur in certain situations.

“The Intellectual Tour” is the “tour” where the hard questions are asked…the most complex data or complicated task is done with this tour. The “Rained Out Tour” is about doing things that you didn’t originally plan to do (ie. when your tour is rained out, you take a different route than usual) so you would “cancel” prematurely with this kind of testing.

Another example is the “Landmark” tour.  This is comparable to using a compass in the woods in order to locate landmarks.  With the Landmark tour, you identify a set of software capabilities (the landmarks) and then visit those landmarks, but randomizing their order.  Sometimes, by changing the sequence of events, an unexpected error will occur.

James described how to use Attribute / Component / Capability  matrix (using a Google spreadsheet) to help do a risk analysis and prioritize test cases, testing the most important things first.

Many of us find bugs that are the same kind of bugs that are being found all over the world, based on tester’s creativity and their test strategy.  What if we could capture these “strategies”  (aka “tours” or “patterns”) in a global repository  so that other testers could use those same strategies to find similar bugs? James has begun to do that with his book, and he has challenged others to do more than test…  When you find a bug, document your methodology.  Did you use one of his tours? Or did you come up with a new tour? Let James know on his blog!

After the presentation, James  said he’d give out books to those that asked the best 5 questions.  I frantically struggled for two minutes trying to figure out how to get the question screen up. (I really like to win this kind of stuff, especially being an unemployed test manager who’s trying to sell herself as an expert!)   I finally got the screen up (phew!) and asked if there was an online repository available to share our bugs/tours (patterns, strategy).   James read questions and then he came to the same question that I’d asked (but, unfortunately, asked by a different person) and said, “Great question! Give that person a book!” I’m going to try and make a case that I should get a book, too, especially since  I’m even gonna do one better than just ask the question. I’m going to set up such a repository in the Beyond Certification network.  Maybe being the KUQ (Kiss Up Queen) that I am, I can even get James to join!