With the mainstream adoption of ChatGPT in 2023, a belief quickly spread: AI would be responsible for the great replacement.
And for the past two years, articles and social media posts have been constantly talking about the AI revolution. It has become LE an unmissable topic that is on everyone's lips and on all platforms.
No matter the field, everyone is talking about artificial intelligence because it's revolutionizing every profession and every field. No one is immune.
So I see tons of articles and posts on LinkedIn, my colleagues and clients are discussing it, the French Prime Minister is talking about it on TV. Even my mother and nephews are talking about it.
But as a UX professional, I think the questions we need to ask ourselves are: will artificial intelligence actually replace us? How does it work and how can we use it? What solutions exist for UX methodologies that actually work (no deceptive magic wand bullshit)? Will they protect the data I inject into it? …
So many essential questions that need to be explored to understand the impact of AI on our profession.
But today I would like to focus this article on one particular topic in the UX profession: user testing.
To be able to establish a status on what exists and how AI positions itself in this methodology.
First of all, why do we do user testing?
User testing is essential for a company to meet, observe, and understand its customers.
And I think that all the UX designers, service designers, product designers, UX researchers, UX/UI designers… who are reading us today will agree with me: this is one of the most enriching and necessary steps.
Either to uncover needs and refine our understanding, or to test our designs against real users. The ultimate crash test to ensure what we've designed works.
Because yes, we, UX and UI designers, are human too. We design to the best of our ability and with maximum expertise and minimal bias... but we are not the end users.
During tests, we will always be surprised by certain behaviors that we had not anticipated, discover unexpected things, sometimes notice that the paths taken are not at all those we thought, we can also see that the route is more complex in a real situation... In short, lessons galore that make us think.
This is why we, UX, will build a whole testing process to obtain the information we need and with the most accuracy:
- Define the test objectives, recruit participants and establish the protocol (scenario, hypotheses, questions, interview methodologies, etc.)
- Conduct tests to observe user behavior, collect their verbatim statements and notes, take notes and ask questions.
- Analyze the results in depth, extract quantitative and qualitative data and adjust/improve the tested product.
This process requires rigor, time and human expertise to be constructed correctly in order to obtain the information that is sought.
Because just like an AI, if you don't ask the right question, the answers can be completely distorted.
Okay, but what about AI? How does it help us with user testing?
In reality, it can be present at each of the stages I mentioned above:
- Development of test protocols
- Creating questions that avoid cognitive bias to avoid influencing users' responses
- Additional questions to explore an answer given by a tester
- Recruitment of participants
- Simulation of user behaviors
- Analyze user behavior, identify friction points and submit improvement ideas
We should see it as a super assistant that saves us time and avoids pitfalls.
But spoiler alert, no, AI is not at all ready to replace us.
And spoiler again: it's not infallible and requires human verification! Because in reality, its name is misleading. It's not "intelligent" in the sense that it can think, express ideas... it's a robot, a machine that performs a task assigned to it.
AI tools that might interest you
I would like to point out that my goal in this section is not to promote a particular tool, but rather to present existing options on well-known (or slightly less so) UX platforms.
There are a huge number of different tools on the market, and unfortunately, I can't provide an exhaustive list. However, I wanted to give you a quick overview of some of the tools that exist and integrate AI.
user Testing et Maze
These are two competing platforms that allow you to organize tests moderate (in live call with the tester) or unmoderated (carried out independently by the tester).
Both tools allow you to collect real-time feedback from real users on websites, apps, or prototypes. They help visualize paths and clicks, while providing videos, graphs, annotations, and detailed analyses of testers' interactions with your product.
It is also at the level of analyses that their IA (Maze / user Testing) will be very interesting. They will allow us to:
- Automatically extract insights from users' audio responses and their responses in questionnaires, analyzing a large volume of data and providing a summary.
- Create an automatic transcript of videos
- Detect points of friction
- Analyze the emotions felt by users for you.
In short, they help to quickly and easily extract valuable insights from user testing.
Maze also has (it must be said) additional specificities compared to User Testing.
Their AI will also help you in:
- The construction of the test protocol. By offering you reformulations of questions, including recommendations to avoid cognitive biases. Which is particularly interesting to avoid influenced responses.
- During testing, because if you are in a situation where the user is performing them independently, their AI can ask additional questions if it detects a need to go further. That is to say, let's imagine the tester gives a less detailed and vague answer, the AI can detect that it is necessary to dig deeper and ask one to three other questions to obtain more details.
Hotjar
Owned by Contentsquare, it's a tool based on analytics and reporting. It allows you to analyze user behavior on the site, helping you understand how users interact with your website or application. It offers features like heatmaps, session recordings, surveys, and user feedback.
Their IA is instead solely focused on investigations. It will allow:
- Build a complete survey all by yourself. You just need to tell it what the purpose of your survey is, and it will suggest all the questions you need to ask your users.
- Then, once the data is collected, it can generate an automatic summary report with: the main insights, verbatim comments and even suggestions for improvements.
Kameleoon
Allows you to do A/B testing powered by theIA. It allows you to adapt the customer journey in real time to optimize engagement and conversion. With Kameleoon, you can create page variations based on AI recommendations, use predictive insights to optimize campaigns, and AI also provides data reports to analyze the impact of tests on different audiences.
Their AI analyzes visitors' behavioral and contextual data to segment them in real time, identifying their conversion intent. This dynamic segmentation allows for highly precise audience targeting.
uizard
It's a bit like Figma or Sketch, but with automation thanks to the integration of AI. This allows you to create designs and prototypes in seconds. Editable based on feedback and iterations.
Basically theIA will allow to:
- Transform wireframe sketches into screens.
- Or describe what you want as screens from a prompt in a chatbot and it takes care of creating the screens and prototypes, without having to write a single line of code. This is particularly useful during design sprints, allowing you to quickly test a feature or an MVP.
- We can also inject a screenshot and an image that it will deconstruct into components that we can move and arrange. Pretty crazy.
This platform is more related to UI, but I wanted to introduce it to you because its construction is based on user testing and now allows you to optimize interfaces before going into testing.
Basically, their IA has been fueled by more than 20 years of research in neuroscience and eye tracking, which provides solid foundations on human behavior and how we analyze screens, make decisions...
So that now, this same AI is able to analyze your screens and predict user behavior based on what you've designed. Without user testing necessary at this stage, Neurons will be able to tell you:
- which parts of the content will attract the most user attention via heatmaps.
- the cognitive load required by the screen
- focus and commitment
- but also memorization.
And all this while showing you where your screens are positioned in relation to market standards.
Additionally, the platform provides AI-powered recommendations to improve the performance of visual media. Yes, because it's not just about interfaces; it can also be about marketing and advertising content.
But let's be clear, AI is still very FAR from replacing us
As you have seen, many tools already rely on AI.
But there's still no revolutionary platform that can do everything for us when it comes to user testing.
First, because current solutions lack nuance and depth in analyzing and understanding behaviors. Yes, AI can spot clicks, movements, and even micro-interactions, but when it comes to understanding why a user chooses to click in one place rather than another... that's when AI starts to scratch its head. It would be great if AI could capture the intention behind each gesture, a bit like becoming our psychologist assistant. But that's not happening anytime soon; AI is still a machine.
And above all, for the moment she is still incapable of being told: this is the site to test. Do it.
She is unable to understand the sector, the objectives of the users and their differences... and to be able to pretend to be a human being to simulate hundreds of variations and subtleties, differences in our behaviors in order to test in total autonomy.
Then, in terms of personalization, AI still has a long way to go. For now, it's excellent at processing large amounts of data, but when it comes to adapting to the specific context of each user, we can still hear misinterpretations.
And, let's be honest, there's also this creativity thing. AI can generate data-driven insights, but it often lacks that creative touch, that idea that emerges after a conversation with a user or an unexpected observation.
Conclusion... This AI that they sell us as a magic wand, that does everything for us: it doesn't exist. It's a myth.
AI is already a superb assistant that makes certain tasks easier, analyzes masses of data, can predict based on models and offers another point of view on certain elements.
Whether it's helping to create journeys during design sprint workshops, or analyzing surveys, videos, and questionnaires, AI is a solid support.
It's a real asset, especially useful for those on a budget, and it will save many of us time.
But it does not replace human beings. Business experts or users.
We still need to be there to frame, imagine, supervise, correct and, sometimes, explain to my AI that humans are not just lines of code.
However, maybe one day in the future, this will change. With all the investments they're making in these technologies, we'll end up with robots capable of thinking, feeling emotions, and being creative. Like iRobot (for the younger generation, this is a must-watch to put on your list).
Erwan Nisas, UX designer at UX-Republic
