So a good moderator of a usability test is able to do several things at once, correct? Aside from keeping the participant in mind first, Jared Spool and other experts in the field talk about the various roles we need to play- the communicator, the listener, the observer, the note-taker, the time-keeper, the facilitator, etc. If we are to wear these several hats, how would the moderator’s observation of the participant data differ from other session observers (project/design team members) who are listening in remotely? Would it mean that the observers are listening more effectively than the moderator who has their hands full? Would they have a better grasp of the details that are happening in the test than the moderator? If so, by how much? Would the differences in observations be vast or just come down to the minor details and nuances?

I just came off a usability test project and ran into a couple of these incidents where the design and project team’s notes sometimes differed from my own as the moderator. If we’re all a part of the same event in real time, shouldn’t every individual walk away with similar perspectives and experiences? Some might say the differences are there because the session observers may not be user experience professionals or are part of the design team and too connected to the product, therefore, unable to observe objectively. Another thing to think about is the power of non-verbal communication and emotional response. Here’s what could happen in the testing room (and is very likely)- Though a lot is going on, part of interacting with a participant is to look out for what’s happening in front of you (them) and not just on the screen or your note-taking tool. A good moderator always puts the participant’s comfort and ease first. If a participant is frustrated, stressed, or giving an emotional response of some kind (whether positive or negative), only the moderator in the room will be able to see this detail and what issues about the UI are causing it. Unless explicitly expressed, the test session observers wouldn’t know what’d be going on there. Side note: This is tricky to do in remote sessions- you have to find participants really fit for giving feedback online. So what about the multitasking?

So much research is out there now about how multitasking is bad for the brain and could adversely affect short-term memory. Did I walk away from my test sessions with fragmented notes? No. I got really good data from all of my participants. Do I remember what happened? Yes. Did I capture every single detail that happened? No, but I got a good summary of it and it was more than enough to help put together the results I needed to find and fix problems. Would my results have been different if I weren’t multitasking and only focused on interviewing and delegated the note taking, time-keeping, and recording duties to other project team members? If I look at the research out there on multitasking, I want to say “yes”, but somehow I don’t believe the results of the study would’ve changed. So what do you think? If there are differences in perspectives and experiences among you and your project team on how a session went, where do you think the differences come from?