January-May 2024

Challenge: With the growing usage of generative AI, there is an increasing risk of encountering algorithmic biases within these tools. How can we design an experience that effectively educates everyday users about these biases, while also encouraging them to report any biases they encounter, specifically within the SnapChat generative AI system?

Project Overview

This project was for a course in CMU’s HCII focusing on User-Centered Research and Evaluation (UCRE). Our team started off with a general project prompt:

Harnessing Everyday Users’ Power to Detect Harmful Behaviors in Generative AI

In order to begin answering the project prompt, we went through many phases, including but not limited to, background research, usability testing, forming a research plan, speed dating, and prototype testing.

Research

For our initial research, we needed to better understand the context of this project prompt, and we were given the platform of TAIGA WeAudit to begin doing that.

We performed various research methods, such as data analysis using a data set from the TAIGA WeAudit platform users; heuristic evaluation of the platform; and usability testing to identify the knowledge gaps that users might have.

After gathering research data, we used the method of affinity diagramming to synthesize important findings.

We also conducted an activity called “Walking the Wall”, which allowed us to group findings/observations into breakdowns/needs, and then allowed us to form some important questions to further push the project direction.

Another activity we did was “Reverse Assumptions”, which helped us to reframe the problem and think about the assumptions we had about user needs and what motivated them to uncover and report AI biases.

These activities helped us to better think about the various factors that might play into a user’s experience on certain platforms. Because of all the research and synthesizing we had done before this activity, especially with the WeAudit Taiga platform, we were able to get a clearer perspective of users’ needs and wants. Our team decided to take the insights we gathered from the initial research phase and narrow our project scope to this research question:

How might we provide a more engaging and comfortable experience for users to detect and report bias found in Snapchat’s generative AI? 

Empathizing

The next phase in our project was empathizing with our target users. We wanted to better understand the problems that they might face when using generative AI systems, how they feel, and what we can do to make their overall experience better. We interviewed our target audience, Snapchat users, and used affinity diagramming to group together general user needs and concerns.

We also created an empathy map to understand our users’ pain points and how they are experiencing algorithmic biases from Generative AI content.

Finally, we were able to narrow down to the most important user needs and take those into consideration when ideating a solution to our research question.

Ideation

Here is an overview of the methods we used throughout our ideation phase:

The Crazy 8’s method helped us to rapidly brainstorm ideas, some of which were more risky and interesting concepts. These initial ideas were then created into storyboards to better conceptualize the context of how users would actually use these solutions. We presented these storyboards to users through speed dating in order to understand how these ideas would be received. Below shows some of the storyboards we created (15 total):

After testing out more risky ideas through methods of storyboarding and speed dating, we were able to narrow down our solution idea through some valuable key insights:

Prototyping

Our research and testing revealed crucial insights into the challenges users face when detecting and reporting bias in Snapchat’s generative AI content. Users expressed the need for a more interactive and easier experience, emphasizing the importance of streamlined reporting processes and gamified incentives to boost motivation. Additionally, the misunderstanding regarding community features and real person interactions highlighted the user’s desire for validation and impact through their reports, without the need for extensive human interaction. We sought to balance education about biases while maintaining the app’s fun and social nature.

Our final idea consisted of gamifying the bias reporting process in order to better incentivize users to make actual reports on the platform. We then created some lo-fi screen to showcase how our idea would work for the users:

Final Deliverable

Our solution revolves around a semi-interactive prototype, aimed at gamifying the reporting process for bias detection in Snapchat’s generative AI content. Through simulated interactions, users can experience unlocking incentives such as Bitmoji costumes and leaderboard progression upon reporting biases. This approach not only motivates users to actively engage in bias reporting but also fosters a sense of community and competition.

Reflection

Working with an interdisciplinary team was a really great experience! Each team member brought a unique set of skills and perspectives to the table, which not only made the collaboration more dynamic but also offered me the opportunity to learn from others.

This project, in particular, deepened my understanding of what a UX researcher’s role entails. I learned how important it is to apply the right research methods at each stage of the process to gain valuable insights. It’s not just about collecting data, but about interpreting it in a way that can guide effective problem-solving and design decisions that truly meet users’ needs. Through hands-on experience, I came to appreciate the importance of user-centered thinking and the impact it has on delivering solutions that are both functional and meaningful for the end user.

Overall, this experience emphasized the importance of thoughtful, methodical research to ensure we’re solving the design/research problems in the most impactful way.