Izvorni znanstveni članak
Affective Interactive Virtual Agent that can Occupy Different Environments
Tomislav Stipančić
; Sveučilište u Zagrebu Fakultet strojarstva i brodogradnje
Leon Koren
; Sveučilište u Zagrebu Fakultet strojarstva i brodogradnje
Sažetak
This paper introduces a virtual being and a robot capable of nuanced non-verbal communication in virtual, mixed, and physical environments. The prototype of such an advanced system is PLEA, an “emotion-aware” virtual being. PLEA assesses a person’s emotional state during interaction based on a multimodal approach and then uses that information during non-verbal communication. PLEA leverages AI-driven sensory systems to interpret visual and auditory data, enabling it to understand and respond appropriately to human emotions and actions in various realistic scenarios. The research focuses on endowing this entity with the ability to convey and interpret non-verbal cues such as body language, facial expressions, and contextual gestures. The main goal of PLEA is to maintain human-robot interaction by building mutual understanding and common ground. We address the challenges in AI algorithms related to emotion and gesture recognition, 3D modeling, information visualization, and sensor integration, which are crucial for creating expressive, environment-appropriate non-verbal behaviors. We propose a multi-layered framework that enables virtual beings to adapt their non-verbal communication strategies to each specific environment. In virtual spaces, the emphasis is on creating expressive avatars. For mixed reality, integration into real-world contexts is crucial, while in physical environments, the challenge involves translating digital expressions into physical forms.
Ključne riječi
virtual beings, creative AI, cognitive robotics, multimodal interaction, mixed reality
Hrčak ID:
327426
URI
Datum izdavanja:
3.2.2025.
Posjeta: 0 *