Pass an unmoderated remote user test

Remote user testing is being practiced more and more: with the Covid, we have gone beyond the automation of face-to-face, and this type of test makes it possible to question profiles in several countries without traveling or recruiting teams in these countries.

The unmoderated test is a remote test, rather reserved for a B2C context, where the user is alone in front of his screen and will follow a set of instructions consisting of tasks and oral or written questions, in total autonomy. In this article, we will discuss the qualitative unmoderated test, but some platforms allow quantitative tests to be carried out on several hundred people.

 

What benefits?

Bandwidth gain...

An unmoderated test does not mobilize a UX Researcher for moderation, saving time on test sessions as well as on upstream preparation: no need to learn the protocol or train to accompany participants on this protocol.

…So sometimes a time saver

The bandwidth gain for the UX Researcher can translate into time savings and faster delivery. But if a session doesn't go as planned – the participant doesn't fit the target or didn't complete the tasks – a replacement has to be found, which lengthens the test time.

No cancellations or no-shows

Generally during an unmoderated test, the participant is not observed in real time and does not have to register for a specific time slot. They can start the test when they want and where they want. This flexibility can be useful when recruiting complicated profiles, who lack time and risk canceling at the last moment.

Close to the real context of use

As for all remote tests, the participant is generally in his environment and uses his own device: this eliminates the biases of the lab and the test devices. During an unmoderated test, the participant also chooses their moment very freely, is not observed during the test and, if they are often asked to think aloud, it is not in an exchange with an unknown person. but alone with the interface. The test scenario remains imposed but we are as close as possible to real use in a test context.

When should you choose to conduct an unmoderated user test?

The tested interface is simple

No one will be there to guide the participant, explain the interface, clarify a point or answer their questions. Of course, user misunderstandings are part of the data collected during any test and are not to be absolutely avoided, but it must be ensured that the user can carry out the entire protocol without being blocked at one step.
A simple interface or course to be tested, that is to say comprising few screens and usual interactions for the user, will also make it possible to favor short sessions of 20-30 minutes, ideal for this type of test.

The protocol is very linear

During a moderated test, the protocol can be very open, the tasks carried out in a different order for each participant, the questions asked at the moment when they make sense in the participant's interaction with the interface. The protocol can have different branches depending on the user's previous actions. However, during an unmoderated test, the sequence of tasks and questions is fixed, it will not be possible to adapt the test along the way. The protocol must be able to operate in a specific order regardless of the actions of the user and his path on the interface.

The tested interface is not a prototype

It is better to avoid prototype testing when you are at a distance, especially on too static prototypes such as InVision, Marvel… The user can find himself stuck, it is difficult to eliminate all the dead ends. The user will often think he is on a real interface, even if we explain to him in the introduction. The limits of the prototype thus generate major interference: the user risks spending time commenting on them, without distinguishing them from what concerns the UX of the product.

We don't have a UX Researcher on the team

This is a very bad reason to choose the unmoderated test. The expertise of the UX Researcher is not limited to the moderation of the test but to everything that happens upstream and downstream: recruitment of quality participants corresponding to the target, creation of a protocol adapted to the non-moderated and analysis of results. Building a protocol for a non-moderated test is more difficult than for a moderated test because it must be self-sufficient: The expertise of a UX Researcher is essential to carry out this step.

Do well on your unmoderated test

Write clear instructions

For the writing of statements, scenarios and questions, we find the same constraints as for the creation of a questionnaire: the instructions must be explicit, precise, unambiguous. The BRUSO model is a good tool to evaluate his statements.
Source: https://opentext.wsu.edu/carriecuttler/chapter/7-2-constructing-surveys/

Guide the user on the right elements

In addition to this questionnaire approach, it is necessary to think about the interface tested to ensure that the task or the question cannot relate to another element of the interface. For this, it is necessary to know the interface well as a whole and to test its protocol upstream.

 

Test his test

Nothing better than testing your protocol on several people before starting the test: it will preferably be done with relatives far from the subject, rather than colleagues who know the interface. The pilot tests must be close to the test situation: we leave people on their own. If corrections are made, we re-test with new people to validate.

Do not give too much importance to your introductory message

Testers tend to read diagonally. This is true for the instructions and the questions which, as we have seen, must be clear and concise, and it is all the more true for the introduction which will be skimmed over at best. Avoid including important information, especially if it is not repeated afterwards.

Take away

The unmoderated distance test is a valuable method for testing simple courses quickly and efficiently. It is worth including in its panoply of user research methods and activating at the right time.
Challenging in terms of its implementation, it must be piloted by an experienced UX Researcher.

 

 

 

 

Marie EUZEN, UX Designer @UX-Republic

Our next trainings