User testing is an integral part in an iterative design process. Blind spots can be seen early on and users’ insights can be gained to keep improving the design. While the whole design team benefits from the engagement of the participants, the participants seem benefit very little from the session. Why do they still invest time and effort with so little in return?
There must be something in their mind before the testing that prompt their decision, which I called the “pre-testing motivation”:
1. They want to maintain their relationship with you
They are your friends, coworkers, neighbors and family. And you need “help” from them to finish the using testing as soon as possible. Of course, they don’t want to reject you when you are in need (even they feel indifferent to your research topic).
2. They come for the incentives
Currently, the average incentives for consumers for user testing session are around $100/hr in the US. It can go higher for specialized professionals to $250/hr or more. Payment can be in the form of cash, checks or gift cards. This doesn’t sound like a bad way to supplement the income if necessary, does it?
3. They genuinely want to improve the product or service
They are currently the unsatisfied users of your product or service. They would appreciate if their feedback can be heard and the problems can be solved. Or, they are so enthusiast about your product or service that they can’t wait to share their insights with you.
To avoid research bias caused by participants with different types of pre-testing motivation, UX researchers/designers can filter out the inappropriate participants during the stage of participant recruitment. For example, don’t pack a study with friends and family unless it’s in a pinch; Don’t provide too much incentive that would attract “professional” participants. On a side note, a study screener is a good starting point to figure out who are the suitable participants for your research and how to find them.
Next to the motivations before testings, there are some other factors during testings that could potentially skew the result of the research. Psychologists classified participants of experiments into four roles (the good, the negativistic, the faithful, the apprehensive) whose motivation I call the “in-testing motivation”:
1. They want to help
The “good” participants highly regard the utility of their performance. They tend to provide positive feedback while reserving doubts, giving responses that they believe are right and helpful for producing favorable outcomes.
2. They try to resist
The “negativistic” participants, as opposed to the good participant, attempt to offer feedback that doesn’t support researchers’ hypothesis or hold an apathetic attitude in the session. They might be displeased because they feel controlled or treated as a subordinate.
3. They listen and follow
The “faithful” participants scrupulously comply with the instruction of the session and ask very few questions. They believe that a high level of compliance is required in a research session.
4. They disguise themselves
The “apprehensive” participants are anxious about their performance and afraid of being evaluated. As a result, they tend to present their favorable side in front of the researcher who they believe are experts of evaluation in terms of intelligence and social adaption.
Many reasons can possibly explain why a participant adopts a certain role, such as personality differences, past experience in user testing sessions, the relationship with the facilitator or the simple awareness of the applied research method. Since a role’s motivation is mostly triggered by internal factors, it is really difficult for researchers to spot participants with propensity to any of these roles before the session begins.
Nevertheless, what researchers can do is to to avoid further reinforcement of participants’ natural dispositions. In other words, don’t give specific cues when participants are making desirable or undesirable progress, such as saying “you are doing well” or wearing a overly radiant smile at a particular moment. Instead, simply verbalize participants’ thoughts (e.g. thinking out loud), or ask questions back (e.g. the echo, Boomerang, Columbo techniques).
It’s noteworthy that the type of motivation a participant has does not necessarily correlate with the validity of the research. It’s totally possible that a friend comes to the user testing session out of the kindness to help, but he/she still provides relevant and useful feedback; or an negativistic participant tries to blindly criticize the test product, but some of his/her points actually make sense.
In the end, it’s empathy that matters. After all, participants are not just experimental subjects. They are humans. Personally, I believe that the best way to understand how a participant thinks, is to become one (not in your own session of course). By being a participant, I got chances to experience firsthand how my pre-testing and in-testing motivation guide my thoughts and influence my interaction with the facilitators in different user testing settings.
(This article was also published on Medium)