jump to navigation

Exploratory Software Testing with James Whittaker December 10, 2009

Posted by yvettefrancino in Uncategorized.
trackback

I just finished listening to the uTest Webinar with James Whittaker about Exploratory Testing.  James is the Director of Test Engineering at Google and has written “Exploratory Software Testing: Tips Tricks, Tours, and Techniques to Guide Test Design.”

So this was interesting stuff!  I always thought of “exploratory testing” as “adhoc” testing, but as James pointed out, it’s much more.  He started by giving several examples of some classic anecdotal bugs…  One such bug is demonstrated in the map… rather roundabout way of getting from Start to Finish!  James noted that there are techniques that testers use to figure out what the most important tests are and how to go about executing them.  Often this comes from experience. These can fall into patterns. James calls these “tours.”  In his book, James describes a variety of different types of “tours” used as metaphors, that capture the methodology of discovering bugs that are likely to occur in certain situations.

“The Intellectual Tour” is the “tour” where the hard questions are asked…the most complex data or complicated task is done with this tour. The “Rained Out Tour” is about doing things that you didn’t originally plan to do (ie. when your tour is rained out, you take a different route than usual) so you would “cancel” prematurely with this kind of testing.

Another example is the “Landmark” tour.  This is comparable to using a compass in the woods in order to locate landmarks.  With the Landmark tour, you identify a set of software capabilities (the landmarks) and then visit those landmarks, but randomizing their order.  Sometimes, by changing the sequence of events, an unexpected error will occur.

James described how to use Attribute / Component / Capability  matrix (using a Google spreadsheet) to help do a risk analysis and prioritize test cases, testing the most important things first.

Many of us find bugs that are the same kind of bugs that are being found all over the world, based on tester’s creativity and their test strategy.  What if we could capture these “strategies”  (aka “tours” or “patterns”) in a global repository  so that other testers could use those same strategies to find similar bugs? James has begun to do that with his book, and he has challenged others to do more than test…  When you find a bug, document your methodology.  Did you use one of his tours? Or did you come up with a new tour? Let James know on his blog!

After the presentation, James  said he’d give out books to those that asked the best 5 questions.  I frantically struggled for two minutes trying to figure out how to get the question screen up. (I really like to win this kind of stuff, especially being an unemployed test manager who’s trying to sell herself as an expert!)   I finally got the screen up (phew!) and asked if there was an online repository available to share our bugs/tours (patterns, strategy).   James read questions and then he came to the same question that I’d asked (but, unfortunately, asked by a different person) and said, “Great question! Give that person a book!” I’m going to try and make a case that I should get a book, too, especially since  I’m even gonna do one better than just ask the question. I’m going to set up such a repository in the Beyond Certification network.  Maybe being the KUQ (Kiss Up Queen) that I am, I can even get James to join!

Advertisements

Comments»

1. Cap'n Dave - December 10, 2009

I would never call you a KUQ!

Well not to your face anyway…lol

yvettefrancino - December 10, 2009

Aw go ahead and call me a KUQ to my face! I call myself that all the time! (I need a little kissy-face icon to go with this.)

2. Jim Hazen - December 10, 2009

So you were there too eh? Interesting talk, but I equate his “tours” to be more like models & oracles in their construction and usage. It seems each of his tours consist of states and transitions along with the data to drive them. So based on organization/execution pattern you vary the tour (model), or can come up with new tours. Some of the things in the tours remind me of the different types/groupings of testing you can do like: happy path, negative data inputs, fault injection & fault tolerance/handling, end-to-end usage scenarios, data tracing & transposition, etc. Seems to be the renaming/repackaging of some standard methods/patterns that we all have used over the years.

Again interesting talk and his idea of the repository is something he has been pushing for for a couple of years now. I will be interesting to see what you come up with regarding format and content in this respect.

Finally, I do agree with his points on certification and formal education. This discipline of software testing is very much built on OJT and experience.

yvettefrancino - December 10, 2009

Hey Jim, Yeah, I agree… Some of the tours were new names for methods that have been used in the past, but I really like the idea of structuring exploratory testing and capturing it in a repository for sharing with others. I also think that the strategies (tours) should be grouped by the different types of testing. He touched on this when he was talking about the question someone asked about Mobile devices. There are different categories of applications that are prone to certain types of bugs and it would be good if that were all cataloged somewhere…

I just sent an email to James and Peter Shih from uTest asking them to join Beyond Certification and making my case to get the book. It would be fun to get some challenges from Google!

3. Michael Bolton - December 14, 2009

I really like the idea of structuring exploratory testing and capturing it in a repository for sharing with others.

Me too. I don’t understand why James Whittaker (a Ph.D!) didn’t refer to the existing literature, because there’s plenty of it out there and we’ve been talking about it for a long time now.

So, in an effort to be helpful, I offer:

http://www.developsense.com/2009/12/structures-of-exploratory-testing.html

—Michael B.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: