TELIN – The Laughing Infant
Laughter, one of the earliest forms of communication in infants, emerges as early as three months of age, well before language, gestures, or walking. Recent studies have highlighted the close link between the acquisition of laughter and advanced cognitive skills, particularly related to understanding negation, offering an intriguing perspective on the evolution of human communication.
The TELIN project draws on a synthesis of diverse research areas including language acquisition, semantics and pragmatics of laughter, Socially Interactive Agents (SIAs), as well as laughter analysis and synthesis, combined with advancements in machine learning. Its aim is to develop an SIA capable of mimicking an infant during laughter acquisition, and to use this SIA to evaluate multiple learning algorithms. These algorithms take into account different input modalities such as audio, facial expression, and language, as well as various contexts such as playing with toys and family interactions, to generate laughter responses.
The project is supported by the Mission for Transversal and Interdisciplinary Initiatives (MITI) of CNRS, which funds interdisciplinary research projects by financing doctoral allocations for a period of three years, coupled with research budgets for the first two years. Led by Université Paris-Cité, the project also involves ISIR at Sorbonne University with Catherine Pelachaud, CNRS research director at ISIR, and the Sciences and Technologies of Music and Sound (STMS) laboratory at Sorbonne University.
Description of the TELIN project by Catherine Pelachaud, project member.
Developing a robotic agent that mimics infant laughter
The TELIN project focuses on developing a socially interactive robotic agent (SIA) capable of replicating the behavior of an infant during laughter acquisition. Its main challenges lie in modeling the robot’s laughter and in real-time decision-making on when to laugh, taking into account the cognitive state of infants, thus surpassing currently assumed cognitive capacities. To address these challenges, TELIN compiles a vast corpus of recordings of infants laughing in various contexts and develops methods for manual and automatic annotation. The project then analyzes the production of laughter in infants to create a formal model. Based on this model, TELIN develops and evaluates a computational model that enables the robot to decide and generate laughter in real-time during interactions. This initiative requires interdisciplinary collaboration between formal linguistics, artificial intelligence, and audio signal processing.
Modeling infant laughter in robotic agents
The thesis topic of the TELIN project is to develop formal and computational models that calculate when and how a robotic infant (using the Furhat robot with a baby mask) responds to human participant expression and activity. The focus is on infant laughter production. This entails:
- Analysis of a corpus of infant laughter,
- Development of a rigorous theoretical analysis of laughter during interaction between a parent and infants,
- Development of a computational model based on deep learning approaches that simulates when laughter should be triggered.
The models will be objectively evaluated as well as through experimental studies.
Contributions to AI, behavior and multimodal analysis, and communication development research
The integration of language and non-verbal communication is a crucial goal for AI. TELIN advances this field by studying it in a simpler yet ecologically valid environment, regarding natural language understanding, interaction, and world knowledge.
Modeling when laughter should occur in an interaction between humans and AI is still in its infancy. Research within the TELIN framework will address this question and contribute to refocusing efforts in this direction. Additionally, the development of a computer model of a laughing virtual agent (integrated into the Greta platform) will benefit the AI community by providing a new sequence-to-sequence architecture.
Finally, TELIN will provide a platform for a more ecologically valid study of communication development, given the emphasis on multimodal interaction. It will provide detailed empirical and formal reports on the emergence of laughter, a relatively unexplored domain. The SIA platform stemming from TELIN will be available for conducting human-agent studies.
Contact scientist at ISIR: Catherine Pelachaud, CNRS Research Director
Published March 26, 2024.
