NeuroHCI project – Multi-scale decision making with interactive systems
Neuroscience studies phenomena involving both decision-making and learning in humans, but has received little attention in HCI.
The NeuroHCI project is a human-computer interaction (HCI) project that aims to design interactive systems that develop user expertise by establishing a human-machine partnership. Interaction with these systems can be seen as a multi-scale decision-making problem:
- A task, e.g. choosing the right medical treatment on the basis of AI-based recommendations;
- A method, e.g. choosing between different devices or modalities to perform a task;
- An object, e.g. which physical or virtual object users will interact with;
- A movement, e.g. which trajectory to take to reach the target object.
Led by Gilles Bailly, CNRS Research Director at ISIR, the ANR NeuroHCI project is a multi-team project at ISIR, involving several members of the laboratory.
Description of the NeuroHCI project by Gilles Bailly.
Exploring Computational Neuroscience to optimize decision-making in human-machine interaction
This multi-disciplinary project uses Computational Neuroscience to develop HMI models of user behavior. The aim is to study the extent to which Computational Neuroscience theories, models and methods can be transposed to HMI.
The NeuroHCI project aims to improve human decision-making in the physical and digital worlds in interactive contexts. The situations in which a human makes a decision with an interactive system are varied:
Do I use my experience or Google Maps to choose my route? Do I reply to this e-mail on my smartphone or PC? Do I use menus or shortcuts to select this frequent command? Do I use the Da Vinci surgical robot to operate on my patient, or traditional laparoscopic instruments? How can I reach this object with my robotic prosthesis?
The decision may concern a complex computer-aided real-world choice (e.g., a medical treatment) or the choice of a method for carrying out a digital task (e.g., editing a photo with the preferred tool).
Understanding the evolution of user decisions for optimal interactive systems
The scientific objective is to understand how users make decisions with interactive systems, and how these decisions evolve over time. Indeed, users gradually develop expertise over the course of repeated use of interactive systems. This expertise influences the way users make decisions. This requires the simultaneous study of the learning and decision-making phenomena underlying the use of interactive systems.
The application goal is to design and implement better interactive and adaptive systems. Human beings adapt and develop their expertise by using an interactive system. The aim here is for the system, for its part, to also evolve to adapt to its users, i.e. to become accustomed to user behavior and in particular to their expertise. The aim is to establish a man-machine partnership in which both actors (human and machine) adapt to each other.
Scientific challenges and benefits of computational models in human-machine interaction
To achieve these objectives, we demonstrate the benefits of our approach through 3 applications, for which platforms already exist and are maintained by the partners, but where scientific challenges remain for their adoption in the real world. These three applications are:
- intelligent graphical interfaces such as AI-based recommendation systems;
- immersive simulation systems offering rich haptic feedback;
- and medical cobotic interfaces, which aim to restore or enhance the ability of humans to interact with objects in the real world.
Our research hypothesis is that it is necessary to develop robust computational models of learning and decision-making in HMIs. Computational models make it possible to explain and predict human behavior by synthesizing complex phenomena in a testable and refutable way. In HMI, they can be used to assess the quality of an interface without the need for costly and time-consuming user studies. When these models are robust, they can be integrated into interactive systems to optimize interaction and adapt the interface to users’ expertise and/or actions.
The ISIR laboratory is particularly well-suited to carry out this project, as it brings together researchers in robotics, HCI and neuroscience, all keen to work at the crossroads of these disciplines. Today, several teams are involved in the NeuroHCI project.
ISIR scientific contact: Gilles Bailly, CNRS research director
Follow the news on the project website.
Published 9th February 2024.
