While digital interfaces offer increasing quality in the visual and auditory domains, the tactile dimension is still largely under-exploited. The MAPTICS project is in line with this observation, with the aim of rethinking the role of touch in our interactions with screens.
Current human-machine interfaces provide information to users almost exclusively via visual and auditory stimulation. Technological efforts have focused on optimising technologies based on vision and hearing, and on the interaction between these two senses.
The introduction of haptic feedback adapted to auditory and visual capabilities could improve the acceptability of new digital devices and applications by providing a more convincing rendering. For example, educational and cultural applications (in schools, museums, libraries, etc.) need to interact emotionally with the public and capture their attention, which is currently made difficult by the ambient sensory hyperstimulation.
This is the background to the MAPTICS project, supported by the French National Research Agency (ANR). It is led by David Gueorguiev, affiliated researcher at ISIR, in partnership with Betty Lemaire-Semail from the L2EP laboratory (University of Lille). The project aims to develop a new multimodal haptic framework by studying how users integrate the various sensory signals into a unified experience.
Development of a multisensory framework for haptics
The MAPTICS project aims to create a new multimodal haptics framework for systems that combine several types of sensation, such as touch, sight and hearing. It seeks to understand how users perceive and combine these different sensory signals into a coherent experience, and to identify the most important aspects of touch to reproduce in digital and virtual environments.
To do this, the project will study the potential of techniques that combine vibrations and friction effects (known as vibro-frictional techniques), as well as the effect of multi-sensory reinforcement, i.e. the coordinated combination of touch, sound and image, to create the illusion of manipulating 3D objects on a screen.
MAPTICS is also exploring how these tactile sensations can be integrated into future digital interfaces, to make interaction more realistic and immersive. Ultimately, this research will enable new human-computer interfaces to be designed and tested, both for the general public and for people with sensory impairments.
The project is based on three main areas of development:
– The first is to design an interactive screen incorporating both haptic feedback (vibration, friction) and high-resolution audiovisual capabilities.
– The second is to establish a versatile framework for multisensory rendering and use it to understand the cognitive processes involved in processing multisensory data.
– Finally, based on these two strands, MAPTICS will develop a user interface with enhanced multisensory haptic feedback at high resolution. This will enable the study of scenarios linked to navigation in multisensory interactive maps or the use of virtual buttons.
Exploring the complementary nature of haptic technologies
One of the original features of MAPTICS is the combination of two techniques that are rarely combined: vibrotactile stimulation and ultrasonic lubrication. Each has its limitations – mechanical vibrations struggle to simulate precise shapes, ultrasound is less effective when the finger is not moving – but combining them could remove these obstacles.
This combined approach has never before been studied in depth. It could make it easier to reproduce textures, shapes or resistances on a flat screen, and so make tactile interfaces more expressive. In the longer term, MAPTICS hopes to promote convergence between these two fields of research, which are still too compartmentalised, to encourage the development of more integrated tactile devices.
ISIR offers a particularly favourable environment for this type of research: the laboratory has strong expertise in robotics, human-machine interaction and neuroscience, and brings together scientists with a wide range of backgrounds. This interdisciplinary framework means that the challenges of the project can be approached from a number of angles, both technical and cognitive.
Scientific contact: David Gueorguiev, affiliated researcher at ISIR
Published on 22/04/2025.
