LLMs as a Tool for User Experience Research: A Comparison of synthetic and real-world data
Organization
Sandbox
Abstract
This thesis will explore how Large Language Models (LLMs) could be used as a tool to create synthetic user data through ‘mock interviews’, ‘user scenarios’, and ‘personas’.
The research question to be answered is: How useful is artificially generated data compared to real-world data for understanding user needs.
The study will begin with an introduction and literature review to understand the current landscape of LLMs in User Experience Research. Next, real-world data will be gathered regarding user needs for a Bike Sharing App. This will include conducting between 5-10 interviews. From this data, user scenarios and personas will be created. After, LLMs such as ChatGPT, Gemini, Perplexity, and Claude will be used to generate synthetic user data. First, with mock interviews using the same interview questions, and then from these deriving artificial user scenarios and personas.
Both the real-world and synthetic scenarios and personas will be systematically compared based upon findings from each study. The thesis will conclude with a discussion of current limitations, potential uses in its current state, recommendations and further work.
The research question to be answered is: How useful is artificially generated data compared to real-world data for understanding user needs.
The study will begin with an introduction and literature review to understand the current landscape of LLMs in User Experience Research. Next, real-world data will be gathered regarding user needs for a Bike Sharing App. This will include conducting between 5-10 interviews. From this data, user scenarios and personas will be created. After, LLMs such as ChatGPT, Gemini, Perplexity, and Claude will be used to generate synthetic user data. First, with mock interviews using the same interview questions, and then from these deriving artificial user scenarios and personas.
Both the real-world and synthetic scenarios and personas will be systematically compared based upon findings from each study. The thesis will conclude with a discussion of current limitations, potential uses in its current state, recommendations and further work.
Graduation Theses defence year
2024-2025
Supervisor
Grace Eden, Yana Halas
Spoken language (s)
English
Requirements for candidates
Level
Masters
Application of contact
Name
Yana Halas
Phone
E-mail