In UX Republic’s UX Research training, trainees are invited to put their new knowledge into practice by conducting a user interview session themselves. I played the role of a mock interview participant in one of these training sessions, and the way I answered the questions profoundly changed my understanding of the participants and the research method as a whole.
The participant's state of mind in UX research
As researchers, we are aware of the biases induced by panels and the risk of recruiting participants who do not fit the target and who may not be completely honest because they are required to play a role. However, the vast majority of participants are well-intentioned, committed to being helpful and to telling the truth.
The topic of the interview was “our eating habits, both in our personal lives and at work.” As a fictional participant, I had no incentive to tell the truth and could play any role I liked. But seeing no particular reason to invent a character, I decided to answer in my own name and be as truthful as possible, as most participants do in real exploratory interviews.
The weight of cognitive biases
We understand that everyone is likely to have cognitive biases – such as wanting to give socially acceptable answers or having imperfect memories – that can affect the reliability of responses. With this in mind, we carefully prepare our interviews to minimize these effects and gather proven information. Yet our knowledge of these biases is theoretical and not always felt internally, and we can treat them as little things to keep in mind rather than as the way our brains work, coloring everything we think and say.
My behavior as a participant in UX research
I wanted to be myself, honest and helpful, to really put myself in the participant's shoes and try to forget the fictional context. But right from the start, when I was asked about the frequency of lunches at restaurants with colleagues, I greatly overestimated the number and did not correct myself. I gave a false figure, I realized it when I said it, and I continued the interview as if it were the truth. Then, to keep the story coherent and the rest of the interview to go as smoothly as possible, I maintained the lie with each follow-up question. I cannot be 100% sure that I would have acted in the same way in a real interview. But it is in fact entirely possible, even knowing how crucial it is for the researcher. I am also convinced that this is not an atypical mistake on my part and that this behavior is not rare at all in reality.
But why? How could this happen?
The first wrong answer made sense. Covid had passed and the frequency of lunches had since decreased significantly, but I still had the pre-Covid era in my mind. The question is difficult when there is no clear routine and these meals are infrequent, and my mind was busy recalling real meals at restaurants and times spent with colleagues rather than doing calculations. Since I didn’t want to spend too much time thinking before answering and the interaction lasted only a few seconds at most, I made a quick guess, and a very bad guess. It is actually very difficult to say a few moments later: “I apologize, my previous answer was incorrect. The real answer is actually very, very different. I don’t know why I said that in the first place.” Oddly enough, it is easier to answer the next question as if the first answer were true. And it becomes impossible to correct yourself once you have deliberately lied. In the hundreds of interviews and tests I have conducted, it is certain that participants have sometimes mistakenly said something wrong and realized it. However, no one has ever said to me, "My previous answer was wrong, let me correct myself."
What I learned
As a trained psychologist, I know that interviewees can easily say a lot of false things about themselves. But what I didn’t have was an intimate knowledge of the mechanisms at play. I know that memory is unreliable. I know a lot of biases, like social desirability bias, that affect declarative responses. But it’s another thing to experience it, as an interviewee, and realize that it doesn’t just mean that we should take declarative responses with a grain of salt, but that we should potentially not trust them at all. It doesn’t mean that we can’t do exploratory interviews, but the questions have to be very well designed and we have to approach them with a clear understanding of the limits of what we can learn from them.
TIPS FOR UX RESEARCH INTERVIEW QUESTIONS
- Avoid exploratory research that relies solely on statements, and focus on observations whenever possible.
- Avoid questions about the past or future, focus on the present.
- Make sure all questions are very easy to answer straight away (avoid questions that require the participant to do calculations, for example).
- Try to discreetly collect the same information with different questions to cross-reference the answers.
- In the analysis phase, give more weight to participants' current goals/motivations/emotions rather than to stated behaviors (especially past or projected).
FOCUS ON THE RESTAURANT EXAMPLE
Instead of asking up front about the frequency of eating out with colleagues, it would have been better to obtain the same information by asking more questions:
- First, the frequency of teleworking at the moment.
- Then, the proportion of meals taken with colleagues compared to meals taken alone/with clients/with other people…
- And finally, among these meals, the proportion of take-out meals compared to meals taken in restaurants.
The first question is easier to answer than the initial question because it is about the present and the information is easy for the participant to know. Subsequent questions should then be consistent with the first, meaning that the final figure will have a reduced margin of error. The same level of care should then be applied throughout the interview guide. Ensuring that each question is easy to answer quickly and does not require a complex assessment will greatly improve accuracy. With a series of easy questions, you will be able to go into depth on any topic.
Marie Euzen, UX Researcher/Designer at UX-Republic