When a movie critic reviews a movie, he will often provide a score. 4 out of 5 stars! This score gives us an idea of how the critic experienced the movie.

Often, we base our own movie-going decisions on these reviews. However, are these reviews really giving us a complete and accurate assessment of the movie? Certainly, there are other factors that influence how the critic experienced the movie. Was it better than he expected? Did he read the book? Does he tend to like action movies? What was his mood before watching the movie?

Granted, most moviegoers aren’t interested in all of this information, but a researcher? Definitely.

These are variables for which most researchers would want to control. By controlling for these factors, we can account for any influence they may have had on the critic’s movie experience, giving us a clearer idea of how the movie itself was experienced.

When I was doing my postdoctoral fellowship at Stanford University I studied student outcomes and what I found was that, while students’ class grades and test scores are telling statistics, they don’t tell the whole story. Researchers have discovered that to get a fuller sense of a student’s experience, it’s important to gather information on their personal and motivational beliefs (e.g., their interest in the content, their perceived capability to pass the test).

Chances are you’ve heard of psychological constructs like grit and growth mindset, these types of motivational beliefs are the factors that most strongly predict student outcomes.

Controlling for these beliefs allows us to account for factors that we know impact student outcomes and to help answer questions like: Does she like chemistry? Did she study for the test? Does she tend to persist when reaching obstacles?

If we collect sufficient information on the students’ beliefs we can predict with statistical accuracy who is more likely to perform well on a test, even before the test is developed.

What does this have to do with the user experience? Well, just like how one experiences a movie has more to do with the movie itself. How a student does on a test has more to do with the test itself. How a user experiences a product has more to do with the product itself.

If a test designer asked me to evaluate students’ experience with a test, I’d be doing a disservice to the designer if I simply reported how students performed, because the quality of the test isn’t the only factor that determines how the students did.

Similarly, a user’s experience is more than just an interaction with a product. It’s a complex phenomenon colored by personal beliefs, past experiences, established thought patterns, and the specific circumstances within which the experience takes place.

Getting a sense of a user's experience.

Commonly, UX researchers will deploy usability tests to assess users’ experiences with a product.

Results from these tests can be informative, however, I’m always surprised when results are hyper-focused on the product, with pass/fail or time on task outcomes, without taking into account how the participants’ psychology impacted their product use. As mentioned earlier, there are factors other than ‘usability’ that influence a user’s experience with a product. The reality is, researchers get a more complete and accurate assessment of the product experience when they account for these confounding variables.

You may be thinking, “okay, that makes sense, but where do I start?”

There are several psychological constructs that may influence a user’s experience. Here, I will discuss one that many researchers believe is the most impactful motivational factor: self-efficacy.

Self-efficacy.

Self-efficacy is a psychological construct that comes from social cognitive theory.

Essentially, it is defined as one’s perceived capability to achieve a task; or, one’s confidence to do something. In educational, social, and motivational psychology, self-efficacy is a strong predictor of outcomes.

It makes sense, if you feel capable to do something, you’re more likely to try it and do it successfully. The impact of self-efficacy cannot be understated, it influences every aspect of our lives; whether it’s shooting a basketball or baking cookies.

For users, someone with high self-efficacy to achieve a task is more likely to do just that, compared with a user with low self-efficacy. For example, we all know that one person who doesn’t even try to troubleshoot when they’re faced with a technology issue.

It’s safe to assume that this person has low self-efficacy to overcome that barrier, whereas someone with high self-efficacy is more likely to attempt and fix the issue.

This is why it’s important to account for this psychological factor. In usability tests, we’d expect someone with low self-efficacy to have difficulty completing certain tasks.

However, if users who report high self-efficacy have difficulty with a task, it signals that something is fundamentally awry. In other words, when users who feel capable to use the product run into issues, it’s worth taking a closer look at what’s going on there.

Let’s take another example, Jeff Sauro conducted a study with 1100 participants who completed tasks with one of 62 websites (e.g., rental cars, airlines, retailers and government websites).

He found that users with prior experience with the website were significantly more likely to rate it as “more usable” (higher SUS score) than users with no experience with the website.

This may not seem surprising, however, how often do we actually take participants’ personal factors into account? Findings like these suggest that measuring psychological factors like self-efficacy can be beneficial, allowing us to account for those underlying forces that influence the perceived usability of a product.

Similarly, UX researchers will often collect participants’ level of expertise, which can be helpful when it’s specific enough to the product being tested. As social cognitive theory demonstrates, level of expertise is inherently linked to self-efficacy, whereby someone with a high level of expertise is more likely to be self-efficacious.

Measuring participants self-efficacy gives the researcher flexibility as they can use the data quantitatively or they can create qualitative groups (i.e., low, medium, or high self-efficacy).

Further, if researchers are able to conduct statistical analyses like multiple regression, self-efficacy can be included in the model as a control variable. This allows the researcher to statistically discount the impact that self-efficacy may have had on the dependent variables (e.g., task completion), giving them a purer sense of how the product was experienced.

How do you measure self-efficacy? Well, it turns out that Bandura, the father of social cognitive theory, published a guide on how to develop a self-efficacy survey.

As with any survey, I recommend keeping it as short as possible and ask participants to complete the survey before the usability test. While self-efficacy is best assessed with respect to specific tasks, I created a short technology-specific self-efficacy survey.

Preliminary results with this scale show that its structure is valid (one strong factor) and reliable (α = .89).

More research is needed, however, I encourage you to use it and see how much you can learn about how your participants’ self-efficacy impacts their user experience.

Final thoughts.

An experience, by nature, is an internal phenomenon. It’s subjective. It may seem counterintuitive, but if we shift the research focus from product to user, we actually get a richer sense of how our product is experienced.

The more we learn about how users’ psychology impacts their experience with a product, the better we can design products that enhance or even transform those experiences.