The CoVR project aims to improve haptic feedback, i.e. the sense of touch, in virtual reality. CoVR is a virtual reality arena augmented by a robotic system to make virtual objects tangible. This platform is at the crossroads of human-computer interaction (HCI), virtual reality, robotics and haptics, and thus illustrates a wide range of ISIR’s expertise.
The context
Numerous systems for stimulating touch are being developed around the world. The originality of the CoVR project lies in its robotic interface, which anticipates the user’s movements, and moves “props” – real, inexpensive everyday objects (a ball, a door, a wall, etc.) – into contact with the user.
In concrete terms, a column attached to a Cartesian robot moves through an arena and interacts with the user. When the user approaches the virtual wall with his or her hand, our robotic column will move to the right place at the right time. So when the virtual hand touches the virtual wall, the real hand will touch the column at exactly the same moment, giving the impression of touching a real, solid wall.
Objectives
The objectives of the CoVR project are :
- To be positioned as a VR experience demonstrator to showcase the laboratory’s expertise to academic and industrial partners ;
- An educational platform so that future engineers in virtual reality can have an experience on a platform implementing cutting-edge techniques and tools ;
- A space for popularizing advances in the field of robotics and human-machine interaction, and awakening the passion of the youth.
The results
The CoVR platform was published at the ACM UIST conference: https://hal.science/hal-02931830
It’s also a working platform, with several demonstrators already on show at the Fête des Sciences. Work is continuing to improve immersion and offer new, even more impressive demonstrations.
Partnerships and collaborations
ISIR is currently collaborating with ISART Digital, a school of video games, 3D animation and special effects, which incorporates scientific aspects into its curriculum. Students were able to contribute their own expertise to the project, enabling the development of scenes highlighting the platform’s strong points.
This project aims at developing a flexible manipulator for picking. Fruit or vegetable picking requires a compliant and dexterous grip to guarantee the integrity of the object as well as a flexible carrying structure to be able to navigate between the branches without damaging them. The objective is to develop a continuous and jointless proboscis robot operated by remote motors and cables, similar to the tendons in a human hand. The project is divided into 3 main issues well known in the field of robotics, namely design, modeling and control. These 3 aspects are actually very intertwined for this kind of complex systems. A good design with a good integration of actuators and sensors facilitates enormously the control by means of an adequate modeling which answers at the same time to a certain realism and an efficiency of calculation.
The context
In the context of the fight against global warming and unprecedented technological convergence (digital, artificial intelligence, etc.), agriculture must reinvent itself in order to produce better, while respecting the environment and human health. Agricultural robotics is one of the levers of this transformation. It is a promising solution for meeting the environmental and health challenges currently facing France and all other developed countries. The urgency of climate change will require an ever greater reduction in the use of inputs (pesticides, water, etc.) and fossil fuels. In addition, the agricultural sector suffers from a lack of attractiveness and a negative image among many young people (rurality, repetitive tasks, exhaustion, economic difficulties, isolation, etc.). Digitization and robotization will allow the valorization of this profession and will relieve the farmer of repetitive and tiring tasks such as weeding, carrying, picking and regular maintenance of crops.
The objectives
The primary scientific objective is to address the three main challenges of design, modeling, and control, in a simultaneous and integrated approach, sometimes referred to as co-design (hardware and software).
The targeted robotic system exhibits characteristics akin to a trunk, approximately one meter in length, with the capability to assume multiple stable configurations. Within the laboratory, two prototypes of continuous flexible manipulators have been developed to explore diverse solutions. The initial prototype features a backbone composed of braided metallic wires and low-friction material disks facilitating cable passage. The actuation involves nylon cables and servo motors with force control to ensure cable tension within predefined limits. Conversely, the second prototype, crafted from printed TPU, incorporates an antagonistic cables structure where a single motor actuates two antagonistic cables. This design inherently maintains minimal tension in the cables throughout its operation.
Two main approaches are studied regarding the control of these manipulators. The manipulator with braided metallic wires currently implements an open-loop control based on data. Its workspace has been explored to learn the inverse geometric model (cable length to pull as a function of the 3D position of the end effector) using a neural network. Conversely, the TPU manipulator implements perception-based control. The concept is to reconstruct its shape in space using stereo cameras to control the end effector.
The results
Recent findings have highlighted the viability of employing data-driven control in the context of cable-driven flexible manipulators. Whether for static tasks (reaching tasks) or dynamic tasks (trajectory tracking), the achieved precision is sufficient. Furthermore, the study of the system’s robustness and repeatability has showcased its ability to perform various tasks in its working environment repetitively with millimetric variability.
Partnerships and collaborations
The members of the Roboterrium – Equipex Tirrex network are partners in the project.
Project “Language and its semantics”
The context
This working group is interested in the different forms of language (written text and oral language, speech and social signals, gesture, face, etc.) as well as in the notion of semantics that derives from it. At the intersection between automatic language processing, perception, cognitive sciences and robotics, language raises many issues from analysis to generation, whether in an individual or interactive context.
Here is a non-exhaustive list of application examples from our research areas:
- Taking into account hesitations, laughter and other social signals ;
- Linking speech and non-verbal behaviors ;
- Contextualized dialogues (history, task, interaction) / Question and answer systems ;
- Textual synthesis of information and interaction ;
- Recommendation and information retrieval ;
- Semantic analysis and representation ;
- Style or content variation.
The objectives
The objective of this group is to bring together researchers with different expertise around language.
To date, the activities set up are essentially discussion groups or scientific presentations with the objective of bringing out common interests.
In the long term, one of the challenges will be to set up co-supervision of trainees and/or PhD students around this theme or scientific mini-projects.
Partnerships and collaborations
The project “Language and its semantics” is a federative project, internal to the ISIR, which does not involve any collaboration outside the laboratory.
Project contact: projet-federateur-langage(at)listes.isir.upmc.fr
Presentation
Robotics represents a challenge for learning methods because it combines difficulties: large and continuous state and action spaces, scarce rewards, dynamic, open and partially observable world with noisy perceptions and actions. Their implementation is therefore delicate and requires a thorough analysis of the tasks to be performed, which reduces their potential for application. In the European DREAM project, we have defined the basis of a developmental approach that allows us to combine different methods to reduce these constraints and thus increase the adaptation capabilities of robots through learning.
Context
The design of robots requires anticipating all the conditions they may face and predicting the appropriate behaviour. An unforeseen situation can therefore cause a malfunction that may recur if the same conditions occur again. This lack of adaptation is a hindrance to many robotics applications, especially when they target an uncontrolled environment such as our daily environment (for companion robots, for example) or more generally for collaborative robots, i.e. those acting in contact with humans. Artificial learning methods could help to make robots more adaptive, if they can overcome the multiple difficulties linked to the robotics context. It is these specific difficulties that this project aims to address.
Objectives
The objective of the project is to help design robots interacting with an uncontrolled environment, on tasks for which the desired behaviour is partially known or even totally unknown.
In this context, learning allows the robot to explore its environment autonomously, in order to extract relevant sensory, sensory-motor or purely motor representations. For example, learning to recognise objects, identifying which ones are manipulable, learning to pick them up, push them, throw them, etc. In this context, exploring the vast sensory-motor space in a relevant way is central, especially as many interactions are rare (the probability of catching an object with a purely random movement is almost zero).
We are therefore interested in the construction of these representations and rely on a modular and iterative approach aiming at exploring the robot’s capabilities and deducing representations that will facilitate the resolution of the tasks that arise, either with planning or learning methods.
Results
The creation of state and action representations that can be used later requires first of all the generation of behaviours that are relevant to the robot’s capabilities. A behaviour is relevant if it highlights the robot’s ability to achieve a particular effect by interacting with its environment. Knowing that many of the robot’s movements do not create any effect, discovering the effects that the robot is likely to generate is difficult. This is compounded by the difficulty of exploring to learn behaviours without appropriate representations.
We therefore rely on exploration algorithms based on novelty search and Quality-Diversity algorithms to generate a large number of exploration behaviours and to deduce appropriate state and action spaces for further learning.
Partnerships ans collaborations
The European project DREAM, coordinated by Sorbonne University (FET H2020 2015-2018), launched this research theme in the laboratory (http://dream.isir.upmc.fr/).
- ENSTA-ParisTech, in France,
- Sorbonne University, in France,
- the University of Coruna, in Spain,
- the University of Edinburgh in the United Kingdom,
- the Vrije Universiteit Amsterdam in the Netherlands.
This was an academic project, with no industrial partner.
It is being pursued in several projects to apply this work to an industrial context. The adaptive learning capability is intended to help engineers in the design phase and in updating the behaviour of a robot. The European SoftManBot project (http://softmanbot.eu) aims at applications to the manipulation of deformable objects. It has a consortium of 11 partners, including SIGMA in Clermont-Ferrand, IIT in Genoa and companies such as Decathlon and Michelin. The VeriDREAM project, in collaboration with DLR, ENSTA-Paristech, Magazino GmbH, Synesis and GoodAI, aims to facilitate the transfer of these methods to a wider industrial context, including in particular small and medium-sized enterprises with a focus on the logistics and video game sectors.
