Source : Sorbonne University
Launched in 2017 by ISIR, the CoVR project integrates the sense of touch into virtual reality through a robotic interface. An innovative device at the crossroads of robotics and human-machine interaction and which allows to propose an even more immersive experience.
It was possible to see and hear. From now on, in virtual reality (VR), it will also be possible to touch. This is the promise on which several research teams at ISIR are working. […] At the beginning, a simple idea: if, once the VR helmet is adjusted, the user’s sight and hearing are mobilized, the sensation of touch remains unexplored. “The big limit of virtual reality is to be able to feel the objects we manipulate as well as we can see or hear them”, says Gilles Bailly, CNRS research director. This researcher has made human-computer interaction (HCI), a discipline that focuses on ways for humans to interact with a machine and vice versa, his specialty.
Having arrived at ISIR in 2016, Gilles Bailly contributed with Sinan Haliyo to the development of a new trend in the laboratory within the Multi-Scale Interactions team, focused on research around haptics, the discipline that explores the sense of touch, and robotics for HMI. A field that whets the appetite of digital giants, such as Meta, the parent company of Facebook, with the goal of making future virtual worlds more realistic. “When you want to touch a wall in VR, your hand will go through. You don’t feel the contact between the hand and the wall,” explains Gilles Bailly.
“A robot in the right place at the right time“
If many systems to simulate the sensation of touch are developed around the world, the originality of the CoVR project lies in its robotic interface, which anticipates the user’s movements, and which brings “props”, real and cheap everyday objects, into contact with it.
In concrete terms, a column mounted on a robot moves through the middle of an arena and interacts with the user. “When the user approaches the virtual wall with his hand, our robotic column will move and arrive at the right place at the right time. Thus, when the virtual hand touches the virtual wall, the real hand will touch the column at exactly the same time” and thus have the impression to touch a real wall.
The result is simply amazing. Once the VR headset is on, it becomes impossible to guess the robot’s movements, or even imagine how many objects are in the room. “The user really has the impression of interacting with several objects,” smiles Gilles Bailly. With a fishing rod in front of a shark in the water or in front of an elevator going up, the user will really feel the strength of the animal or have to physically pull a sliding door to enter the cabin.
“The sense of touch is super complex”
At the origin of this project is a former doctoral student of the laboratory, Elodie Bouzbib, who after her first prototype won the prize for the best multidisciplinary thesis in 2022. Now a post-doctoral fellow at the Inria center at the University of Rennes, this lover of the sense of touch continues to work on creating robotic interfaces to improve haptics in VR. “I wanted to have a better understanding of touch. This sense is super complex, it gives a lot of information. But it is not dominant compared to sight. We control the environment through vision,” notes Elodie Bouzbib.
If for the moment, CoVR is intended for arcades or industrialists for learning and skills transfer, the technology developed could open the field of possibilities. Gilles Bailly evokes the multiplication of robots in the arena to propose a collaborative aspect, or even “lots of small robots, with different properties and see how they could cooperate together”. The development of haptics could eventually allow certain feats. Elodie Bouzbib wonders: “Could research succeed in simulating touch if we have prostheses? Let’s touch wood.
Contact: Gilles Bailly, CNRS research director.
