During usability test sessions, we usually ask the participants to say out loud what they are thinking. What they say ends up being important data to justify what is right or wrong about a software page design. We trust that what they say as veridical, at least as long as it is consistent with our own perceptions of the events.
But what is happening when participants talk? Are they reporting events as they occur? Are they filtering what they perceive and reporting their interpretation of it? Is their interpretation biased by what they expect to see or what they think we expect them to see?
This issue reminds me of a book, The Electric Kool-Aid Acid Test, by Tom Wolfe published in 1968. In this work of new journalism, Wolfe describes the adventures of Ken Kesey. Kesey, the author of One Flew Over the Cuckoo’s Nest, was an early advocate of the use of LSD, before it became illegal. He believed that when you take LSD your brain chemistry changes and you are the closest to “being in the moment” that a human can be. He believed in something called “the gap.” When we perceive an event, is has already happened, hence the gap. There are gaps between the event and when we sense it; between the event and when we perceive it and between the event and our feelings about it. He and the people who gathered around him, called the Merry Pranksters, would take LSD to try to get as close as they could to being in the moment, that is to minimizing the gap between when an event occurs and their perception of it. They lived for living in the moment.
So if there is a gap between events and their perceptions, what is a participant in a usability test talking about? It can’t be what is happening. At best it is their memory of what just happened. Also we know that people can think many times faster than they can talk. So there is plenty of time to interpret what is perceived, especially if you slow down your talking. This gap is why cognitive psychology researchers don’t trust what people say as reflecting what is going on in their heads when they are allowed to report/interpret whatever they choose.
So what’s up with thinking aloud?
Love the philosophical post! Or at least that is my post-experience, filtered-then-translated-into-language impression from across the gap. Guess you’ll never know what I really experienced as I read it. And perhaps that is a good thing! It probably would have a lot of noise (some colors perhaps?) and be mostly useless until processed and translated.
As for the relevance to usability testing, the addition of mind altering drugs to the protocol could introduce a positive(?) biasing effect. And how would we separate the effects of the drug experience from the effects of experiencing the interface? Perhaps if our product targets the ‘perpetually medicated’, you’re onto something here…
Good stuff!
Well Vel, your either on the bus or your off the bus.
BTW the Merry Pranksters loved to videotape themselves. They enjoyed watching themselves experiencing what had just happened – being two steps removed from reality.
[...] there are still/always good questions to be asked about the way we pursue this goal, I am always interested in seeing other examples of how this kind of work happens both inside and [...]
