Testing Emotional Support Features' Effect on Job Seeker's Well-being
CONCEPT TESTING TO UNDERSTAND THE PRODUCT'S VALUE AND ITS USER RECEPTION
I used moderated and unmoderated user testing to evaluate the value of product functions and assess if users were willing to use them for emotional support. The insights helped improve interface design, refine user stories, and update the product requirements document.
Context
My research team was tasked with evaluating two critical features for the product, with the results carrying significant weight in determining the final product direction. Given the high stakes involved, I adopted a two-stage concept testing approach to ensure the generation of comprehensive findings.
The first stage involved moderated testing with think-aloud tasks and interviews for feedback. The second stage tested refined wireframes through an unmoderated usability test. This mixed-methods approach assessed perceived product value and user interaction patterns to guide the final product direction.
MY ROLE
As the lead UX researcher, I planned and conducted stakeholder interviews, designed protocols, moderated interviews, analyzed data, and presented findings. I mentored interns to recruit participants, moderate interviews, clean and analyze data, and write the report.
Research
Study Goals:
-
Understand the potential value and impact of the features, and how it relates to prior apps on the market.
-
Understand if and where the features fit into their job seeking journey, the devices they would use it and frequency of use.
-
Identify likes, dislikes, confusions and changes to the user interface.
Methods:
-
Concept testing (moderated and unmoderated)
-
Moderated: Participants hosted on 1 hour Zoom session, prompted to test 2 concept prototypes on Maze, followed by a mini survey (5 questions), and structured user interview
-
Unmoderated: Screened participants were emailed a link to a Qualtrics survey, prompted to answer questions, then tested 2 concept prototypes on Maze independently.
User Demographics:
-
Moderated: 10 people, 25-44 years old, 8/10 female, looking for jobs in UX design and UX research, identified as Asian, Caucasian, and Hispanic/Latino
-
Unmoderated: 52 people, 24-48 years old, 40/52 female,
-
Aspiring field: UX Design, UX Research, Product Management
-
Ethnicity: Asian, Caucasian, and Hispanic/Latino
Outcomes:
-
2 User Personas & Stories
-
Feature prioritization & heatmaps of users navigating mid-fidelity wireframes
-
Insights into potential value of POC
Limitations:
-
users found self support function very hard to understand and use.
Outcome
The team was able to develop a proto user persona which could be used as a starting point for persona development.
Findings
We are 95% confident that 53-100% of our target user will find value in the social connections (feature 1) feature. And we can be 95% confident that 40-100% of our users will not find value in community letters feature (feature 2). [PMF question from unmoderated testing]
User segmentation:
-
There were two types of users: active (prefer 1:1 chats) and passive (hesitant to share or engage with other posts or users, prefer consuming/reading content).
Concept feedback:
-
Users found the concept of Social Connections very appealing, especially the real-time aspect and quick response time and exhibited willingness to adopt it ( moderated: 9/10 users, unmoderated: 47/52 users).
-
Most (>70%) didn’t have an understanding about browser extensions, but some were open to the idea of the product being used as an extension.
User Interface:
-
Majority (>80%) found the interface simple and easy to use, information well organized, ample users available to chat with, and the user profile relevant.
-
Majority (>90%) found number of clicks to connect with a person to be exhausting.
RECOMMENDATIONS
-
Scrap or redevelop Community Letters feature (feature 2).
-
Develop Social Connections feature as it has a great product market fit.
-
Define how Social Connections features are intended to be used on a browser extensions.
NEXT STEPS
-
Update interfaces based on user feedback and concept test once again.
-
Investigate browser extension common use cases and evaluate the suitability of the Social Connection feature for the platform.
Reflection
-
Using mixed-methods approach allowed me to evaluate how a person would use a feature, why they would use it, and if the findings were generalizable to the broader population.
-
If I had more time, I would have coordinated with the design team to ensure that the prototypes for each feature followed a similar design language.
