Product innovations can be driven by technology. To be meaningful, users need to be at the center of any innovative solutioning – in this case the users are user researchers and designers. The User Research field saw online surveys and remote user research platforms integrating some form of Artificial Intelligence in their products. TypeForm came up with Formless.ai (Beta, access on request) – a conversational interface for people to take surveys.
Various user research platforms like Maze and usertesting.com have come up with beta versions of their AI to generate questions or analyze text. Almost all these tools we reviewed have their AI addons to make alternative text suggestions, create summaries from transcriptions, etc. While building such solutions, the Compliance aspects and Security of PII and other confidential data remains a concern which needs to be addressed.
Researchers we spoke with are open to adopting AI in their work. Our study of these tools proved beyond doubt, the accuracy of the output needs to be thoroughly examined before putting those to direct use in research deliverables. Almost all recent feature announcements on user research platforms have been based on generative AI powered by Large Language Models (LLMs). The output provided by these features serve as a machine companion to initiate thinking on research aspects which new researchers may accidentally miss. The use case which AI might be able to address in the future is quite a long list:
When we asked ChatGPT3 “Please create some Usability tasks for Shopping Cart”, the response it gave was exhaustive and rather generic. Some tasks were as generic as “Continue Shopping’, ‘Proceed to Checkout’, ‘Error Handling’ and ‘Contacting Support’. It missed out on ‘Partial checkout’, ‘multiple Carts’ and filling Cart from Wishlist, all of which are important use cases in eCommerce retail these days.
When a researcher asked about Personas for a business, ChatGPT generated 10 of them leaving them confused. While the generated personas helped to think of all possible groups in the population, that answer itself could not be applied to define a target audience.
There are things which AI on a computer alone would not be able to do e.g. accurate eye-tracking. Companies like VisualEyes (now closed) had claimed 93% accuracy in providing eye-tracking without need to use external Eye-tracking hardware.
UX teams would take different approaches to AI based tools for user research. The richness of user insights available within an organization would determine the usage level of AI based tools. Discussions around Synthetic users, generic LLM models would be for the less UX mature organizations or while doing desk research. Sense-making of large amounts of data collected from Longitudinal studies would call for AI tools in the User research space. AI could also help the researchers by generating test materials like variations of Design prototypes using Plugins to existing tools like Figma.
In our view, current AI applications in User research are at a starting point. AI is far more capable than conversational interfaces or fast processing of large amounts of text information. There is cautious optimism among the researchers to use the power of AI because many of them know that in the current state, AI could harm their research due to lack of its understanding of context.
At UXArmy we are exploring meaningful applications of AI which can help scale user research and create impactful outcomes. We are working on creating beyond what a ChatGPT prompt can simply generate. Security and compliance remains on top of our minds as we build a compelling solution for researchers. Video analysis is one of the areas we are working on.