The European research project SOFTMANBOT
SOFTMANBOT is an industrial-end-user driven project that will provide an innovative and holistic robotic system for the handling of flexible and deformable materials within labor-intensive production processes.
The robotic system will consist of three main elements:
- a generic robotic perception system (perception of the product and the human operator),
- a multi-sensor control and planning platform (advanced control algorithms for shape and contact servoing, AI-based task generalisation)
- and smart dexterous grippers (smart mechanical design which will embody grasping/manipulation skills and integrate sensors – mainly tactile – for identifying precisely the contact state between the product and the gripper) able to handle soft components with high-levels of robustness and flexibility.
In order to facilitate the evaluation of the performance, transferability, scalability and large-scale deployment of these solutions, demonstrations will be carried out in real industrial environments in four pilot demonstrators involving four key manufacturing sectors: toys, textiles, shoes and tyres. The project aims to support a profound change in the world of industrial production, where the handling of soft materials with the involvement of robots becomes a feasible and widespread alternative for European factories, especially SMEs.
The overall objective of the project is to develop and demonstrate a universal approach to robotic handling of flexible and deformable materials. This approach is based on the integration of advanced manufacturing technologies for the automation of tasks based on :
- the use of contact forces
- an intelligent robotic perception system,
- a multi-sensor planning and control platform,
- the deployment of intelligent grippers capable of handling soft components with high levels of robustness and flexibility.
Two ISIR teams are working on several themes within the European SOFTMANBOT project. On the one hand, the members of the SYROCO team are working on the modelling and control of deformations by visual and tactile feedback of the various objects or parts handled in the industrial processes considered. On the other hand, the members of the AMAC team are working on the generation of manipulator arm trajectories using evolutionary algorithms, while implementing generalisation methods allowing the adaptation and transfer of skills to all the industrial tasks to be solved.
Deux équipes de l’ISIR travaillent sur plusieurs thèmes au sein du projet européen SOFTMANBOT. D’une part, les membres de l’équipe SYROCO travaillent sur la modélisation et le contrôle des déformations par retour visuel et tactile des différents objets ou pièces manipulés dans les processus industriels considérés. D’autre part, les membres de l’équipe AMAC travaillent sur la génération de trajectoires de bras manipulateurs à l’aide d’algorithmes évolutionnaires, tout en mettant en œuvre des méthodes de généralisation permettant l’adaptation et le transfert de compétences sur l’ensemble des tâches industrielles à résoudre.
The SOFTMANBOT project is now at the halfway point. Discussions and collaborations within the consortium are very intensive thanks to many virtual meetings. The partners were nevertheless very happy to meet again in Paris on 13 and 14 October 2021, as the health crisis had not allowed any face-to-face consortium meetings since the project was launched.
Project progress at the halfway point
The teams began by investigating the possibility of learning trajectories for manipulator arms with evolutionary algorithms using an existing simulator that supports deformable objects using finite element modelling methods. It became apparent that this is not a viable solution as there is no simulator that is both sufficiently faithful and fast enough to run for the needs of the methods employed. The decision was therefore to move towards approaches based on learning a model of the transition dynamics of the system under consideration.
The idea is to learn a simulator specific to the system with which the robot is interacting, so that it is possible to use this model to learn the robot’s trajectories. Thus, this model learning is currently being tested directly on a simplified real system of the industrial problem considered (the demoulding of shoe soles manufactured by plastic injection).
On the other hand, the teams are concentrating on two main areas of research: the determination of the contact points between the robotic gripper and the deformable object and the control of the object’s deformations. Contrary to rigid objects where the stability of the grip is the main objective, a new problem emerges when trying to grip a deformable object: does the grip allow to perform the desired deformation? It was therefore necessary to take a geometrical approach to solve this problem. The initial shape of the object is compared with the final shape to determine the points of application of the forces. Although this method offers interesting preliminary results, it may lack precision. It is indeed necessary to combine it with a model of the object and to use optimisation methods in order to obtain the optimal contact points.
The second axis deals with the control of the shape of a deformable object, which implies moving towards physically realistic models (finite elements, Cosserat), in order to represent the deformable object. The contact points are considered as actuators and the forces allowing to deform the object towards the desired configuration are determined from the inverse model. This approach was tested in open loop to control the position of two points of a deformable torus.
The next steps of the project
On the one hand, the next steps of the project are the finalisation of a learning method aiming at learning the model of the considered system as well as possible during the interaction, while looking for trajectories for the robot allowing it to reach its desired states. The use of evolutionary methods to solve this problem seems to be relevant, on the one hand because the diversity of the trajectories obtained allows a better transfer of the trajectories of the learned model to the real system and on the other hand because this same diversity can help to improve the current model of the system. The objective over the next few months is therefore to implement this new method, aiming firstly to learn a model of the system that is as faithful as possible while having as few real interactions as possible, and then to use this model to find a set of diverse trajectories allowing the problem to be solved.
On the other hand, it is a question of finalising the model-based approach in order to determine the optimal contact points for deforming the object. This will make it possible to combine the two parts in order to determine the contacts as well as the forces in one go. The results can then be validated by experimenting on a real robot. So far, the final configurations of the target object have been obtained by manipulating the object manually. Stability is therefore assured, which is not necessarily the case in an industrial context. An area for improvement is to use other results to ensure the stability of the grip during manipulation.
The last objective is to loop the control loop in order to control the shape of the object in a closed loop.
Link to the project page: http://softmanbot.eu
Project contact on the ISIR/Sorbonne University side: Véronique Perdereau, University Professor and Vice President Europe of Sorbonne University
