Emonyai: Contextual conversation guidance leveraging microexpression and body language interpretation

Goh, Chun Shing (2024) Emonyai: Contextual conversation guidance leveraging microexpression and body language interpretation. Final Year Project, UTAR.

Preview

PDF Download (5Mb) | Preview

Abstract

Social anxiety and introversion can create barriers to effective communication, limiting both personal and professional interactions. In this project, we develop EmonyAI, a system designed to understand situational contexts such as user facial emotions and conversation content, to leverage that information and provide suggestions for more appropriate response in real time. EmonyAI is an advanced AI platform designed to enhance human interaction by integrating multiple functions, including facial recognition, speech-to-text processing, emotion detection, and text summarization, all aimed at improving communication skills. Rather than relying solely on traditional verbal cues, EmonyAI incorporates CNN for processing facial recognition and emotion detection, enabling the system to capture the subtleties of nonverbal communication. The contextual functionality is built based on the MetaGPT framework, which stores each individual as character and continuously evaluates character profiles, allowing for dynamic adaptation and improvement over time. With each interaction, the profiles grow richer, enabling the system to better assess the characteristics and preferences of the user, leading to more accurate predictions and suggestions. By leveraging this comprehensive approach, EmonyAI generates contextual conversation suggestions tailored to each user's emotional state and character, promising to enhance social experiences and foster more effective communication across various digital platforms.

Actions (login required)