Project PANORAMA: adaPtive Artificial iNtelligence fOR humAn coMputer interAction
The key concept of this project is “user adaptive AI in the context of human-computer interaction”. This project addresses two aspects for this concept. First, we will conduct research on user adaptivity of artificial intelligence embodied as a conversational agent. When people talk to other people, they change their verbal and nonverbal communication behaviors according to those of the partner. Therefore, user adaptivity is an essential issue in improving human-agent interaction.
Context
Communication style is also different depending on the culture, and adapting the agent behaviors to a target culture is useful. We will tackle this problem by employing a machine learning approach. However, a bottleneck of this approach is that annotating users’ multi-modal behaviors to create training data is time consuming. We will offer semi-automated annotations and provide visual feedback to inspect and correct machine-generated labels by incorporating eXplainable AI (XAI) techniques. Thus, the concept for user adaptive AI is used to support users in creating multimodal corpus as well as improve the human-agent interaction. Moreover, the concept of user adaptivity is also focused on the psychological studies in this project, in which user motivation will be investigated in one relevant use case (personalised motivational coaching for physical activity). Therefore, this project envisions a new research methodology for machine-learning-based conversational agents by focusing on the concept of user adaptivity.
Objectives
The PANORAMA project aims to accomplish the following 5 research goals:
- propose a user adaptive multimodal annotation tool based on XAI technique,
- exploit this tool to collect annotated multimodal corpora in three countries (France, Germany, and Japan),
- propose models and methods for developing conversational agents with multi-level adaptation functionality, where nonverbal signals of the agent as well as the content of the dialogue are adapted to the user,
- provide multitask learning and transfer learning techniques to learn models using the multi-cultural corpus obtained and adapt the conversational agent to each culture,
- and propose the design basis of adaptive AI systems grounded in psychological theories and evaluation studies.
Results
Adaptive AI interface impacts the economy and the future society.
First, user adaptive technology enhances the quality of human-computer interaction and this contributes to improve task performance and productivity of the users in industries.
Moreover, user adaptive AI will effectively motivate the users to adopt a healthy lifestyle in the long term, and this may change people’s lives in the future society.
Partnerships and collaborations
- Coordinator: Intelligent User Interface Lab (IUI-Lab), Seikei University, Japan,
- Human-Centered Multimedia (HCM), Augsburg University, Germany,
- Institut des Systèmes Intelligents et de Robotique (ISIR), Sorbonne University France,
- Laboratoire d’Informatique pour la Mécanique et les Sciences de l’Ingénieur (LIMSI), group CPU, France,
- Social Signal Interaction Lab, Japan Advanced Institution of Science and Technology, JAIST, Japan.
