Emotion detection AI is an emerging tool with applications across the fields of advertising, the automotive industry, education, human resources, and mental health. This innovative technology enables researchers to process large batches of images of human faces and obtain estimates of the emotions of individuals present within images.
Despite the benefits that these technologies offer, there remains a critical challenge in ensuring their accuracy and reliability. At the 2025 International Society for Data Science and Analytics (ISDSA) Conference, Lucy Graduate Scholar Austin Wyman introduced a vexing problem – how the presence of multiple detected faces can reduce the accuracy of measuring the emotions of the intended subject of an image – potentially introducing errors in the analysis data that incorrectly train emotion detection AI tools.
His presentation, “Emotion Detection AI Deceived by Two-Faced Images” received the Computation Best Paper Award by the ISDSA meeting committee.
Using over 1,400 labeled images from the RAVDESS dataset, Wyman’s study tested how variables such as the number, size, opacity, and emotional similarity of extraneous faces affected the accuracy of emotion detection tools like Py-Feat. In particular, the research identified a significant decrease in AI performance when extraneous faces were larger than the intended subject and when the faces displayed a different emotion than the primary face. Additionally, happiness, anger and fear in extraneous faces generated the most biased results for emotion detection tools.
The findings underscore the importance of removing additional unnecessary faces from images to improve the precision of AI emotion analysis. Emotion detection AI is a key facet of affective computing, which is an emerging direction in social and behavioral research with applications in government and public service. Bias limits the models’ ability to respond intelligently to human emotions and may even cause harm.
Reflecting on his time at ISDSA, Wyman said, “Attending ISDSA was a unique opportunity to present my research in a format that fosters interdisciplinary dialogue. I received insightful feedback from researchers in emotion detection AI that sharpened this study and deepened my perspective on responsible AI development. These interactions are building both the confidence and the expertise I need for a career in academia.”
Wyman’s participation in ISDSA was supported by the Lucy Graduate Scholars program, which empowers graduate students to serve as ambassadors who help drive intellectual engagement and shape Institute programming.
To learn more about the Lucy Graduate Scholars Program, please visit the Lucy Family Institute website.
Contact:
Christine Grashorn, Program Director, Engagement and Strategic Storytelling
Lucy Family Institute for Data & Society / University of Notre Dame
cgrashor@nd.edu / 574.631.4856
lucyinstitute.nd.edu / @lucy_institute
About the Lucy Family Institute for Data & Society
Guided by Notre Dame’s Mission, the Lucy Family Institute adventurously collaborates on advancing data-driven and artificial intelligence (AI) convergence research, translational solutions, and education to ethically address society’s vexing problems. As an innovative nexus of academia, industry, and the public, the Institute also fosters data science and AI access to strengthen diverse and inclusive capacity building within communities.