Home » Join us » Opportunities

Opportunities

Job offers

Postdoc “Multimodal Machine Learning for User Modeling and Profiling”

Objectives:

Personalized Human-Machine Interaction systems aim to provide tailored experiences that cater to the individual needs and preferences of human users. To achieve this, these systems rely on user models derived from user profiles and observations of human actions. However, adapting to changing contexts or individuals presents numerous challenges, including multimodal data collection and interpretation, privacy concerns, and transparency issues. There is a pressing need to develop new representations of human behavior that can capture the diversity among users while safeguarding their privacy.

This post-doctoral position is centered on the development of human-centered machine learning techniques aimed at personalized adaptation in interactive applications, with an emphasis on human-robot interaction. Recent strides in artificial intelligence, especially in the domain of deep learning, have unlocked advanced methodologies for user profiling and adaptation. Notably, representation learning (deep learning) and reasoning (Large Language Models, LLMs) have emerged as influential approaches, offering promising avenues for comprehending user behavior and providing tailored experiences in interactive applications. Rooted in a human-centered approach, the position will address ethical issues inherent in both the modeling process (e.g., biases, privacy concerns) and experimental design (e.g., working with vulnerable participants).

This position is for 24 months contract, but there is a possibility to be extended depending on the performance and circumstances.

Responsibilities:

  • Develop advanced user modeling techniques to accurately represent user preferences, behaviors, and characteristics based on interaction data with AI systems.
  • Investigate methods for integrating various types of data, including user interactions, feedback, and contextual information, to build comprehensive user profiles.
  • Explore innovative approaches for dynamic user modeling that can adapt to changes in user preferences and behavior over time.
  • Address privacy concerns by developing techniques for anonymizing or obfuscating sensitive user data while preserving model effectiveness.
  • Collaborate with interdisciplinary teams including computer scientists, psychologists, and designers to ensure the usability and effectiveness of developed techniques.
  • Publish research findings in top-tier conferences and journals in the field of Human-Machine Interaction and Machine Learning

Requirements:

The ideal candidate must have a PhD degree and a strong background in machine learning, human-machine interaction or robotics.

The successful candidate should have:

  • Experience in human-machine interaction
  • Good knowledge of Machine Learning Techniques
  • Good knowledge of experimental design and statistics
  • Excellent publication record
  • Strong skills in Python
  • Willing to work in multi-disciplinary and international teams
  • Good communication skills

General information:

– Contract start date: from September 2024

– Contract duration: 24 months

– Level of study required: doctorate

– Salary: standard salary scale

– Host laboratory: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.

Person to contact:

– Mohamed Chetouani

– Tel: +33 1 44 27 63 08

– Email : mohamed.chetouani@sorbonne-universite.fr

– Application: Interested candidates should submit the following by email in a single PDF file to: mohamed.chetouani[@]sorbonne-universite.fr with the subject: “Application Post-Doc Multimodal Representation” with:

  1. Curriculum vitae with 2 references (recommendation letters are also welcome)
  2. One-page summary of research background and interests
  3. At least three papers (either published, accepted for publication, or pre-prints) demonstrating expertise in one or more of the areas mentioned above
  4. Doctoral dissertation abstract and the expected date of graduation (for those who are currently pursuing a Ph.D)

Deadline for applications: 17 May 2024

Download this offer

Job title: Post-doc position in neuro-computational AI ethics

Context:

The CAVAA European project (https://cavaa.eu/) proposes to realize a theory of awareness instantiated as an integrated computational architecture and its components to explain awareness in biological systems and engineer it in technological ones. In a world governed by hidden states, awareness enables to deal with the “invisible”, from unexplored environments (counterfactual pasts and futures) to social interaction that depends on the internal states of agents and moral norms. In particular, we will study such cognitive architecture agents’ ability and propensity of reasoning, decision-making, or revisiting past experiences, but also reflecting upon what was right or wrong given some moral norms, and which possible future states could be right or wrong. CAVAA’s awareness engineering is accompanied by an ethics framework towards human users and aware artefacts in the broader spectrum of trustworthy AI, considering shared goals, counterfactuals and projections towards new future scenarios, and prediction of the impact of choices. CAVAA aims to deliver a better user experience because of its explainability, adaptability, and legibility.

Location and environment:

The post-doc position will be located in the Institute of Intelligent Systems and Robotics (ISIR, http://www.isir.upmc.fr), Paris, France. ISIR belongs to Sorbonne Université, CNRS and INSERM, and is located in the center of Paris. Speaking or understanding French is not required. This work will be done in close collaborations with philosophers, engineers and computational neuroscientists of the CAVAA consortium.

Missions:

The post-doc work will focus on ethical reasoning through virtualization, deliberation and alignment with human values. The theoretical framework will be anchored on probabilistic model-based reinforcement learning, extended to include homeostatic, epistemic and social values, including social conventions and moral norms as starting point. Research will investigate learning through interaction with the environment and with other agents, social decision-making, mental simulation and counterfactual reasoning to inform humans about potential long-term consequences of actions. The model will be confronted to experimental data about human decision-making when confronted to various social and moral dilemmas. The model will be integrated into the CAVAA cognitive architecture and applied in artificial agents and robots in virtual and real-world scenarios involving spatial navigation and social interaction.

Required profile:

We are looking for highly motivated candidates with a strong academic record. An excellent background is expected at the interface between computational neuroscience and machine learning. Significant experience in cognitive architectures and computational modeling for neuroscience, psychology, AI or cognitive robotics will be appreciated. A strong interest in philosophy of mind and moral philosophy is expected. Eligibility: PhD degree in a quantitative discipline. There is no nationality or age criteria.

Required skills:

Mastery of reinforcement learning and game theory, very good level in applied maths, and advanced programming skills in modern C++ and python are required.

Very good level of English (written, spoken).

General information :

– Type of position : Post-Doc

– Contract start date: 01/08/2024

– Contract duration: 26 months (until 30/09/2026)

– Quota of work : 100%

– Desired experience: Beginning to 4 years

– Level of studies required: PhD

– Host laboratory: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.

Contact person :

– Mehdi Khamassi

– Tel:+33650764492

– Email : mehdi.khamassi@sorbonne-universite.fr

– Send your application by email, with [CAVAA post-doc application] in the subject line, a CV, a covering letter (max 2 pages) and a list of two references.

– Application deadline: 07/05/2024

Download this offer

Job title: Post-doctoral researcher. Simulating social touch at a distance through sounds

Context: Audio-touch: Simulating social touch at a distance through sounds

The position is funded by a grant from the French National Research Agency (Multisensory and Affective Modulation of Touch: Interaction and Intentionality). This project brings together a consortium of three research laboratories, CNRS-LISN (O. Grynszpan, F. Bimbard, E. Prigent), UTC-Heudiasyc (I. Thouvenin) and SU-ISIR (M. Auvray & C. Pelachaud). The aim of the consortium is to simulate the feeling of being touched during a social interaction in virtual environments. This involves the investigation of the multiple dimensions involved in social touch (i.e., physical sensations, emotional content, and sense of agency), the sonification of social touch, the design of virtual agents endowed with the ability to produce social touch.

Within this consortium, the aim of the research to be conducted at ISIR is to build on cutting- edge research on social touch and movement sonification to empirically investigate the conversion of social tactile interactions and their corresponding emotions into sounds. In particular, recent studies conducted at ISIR showed that participants listening to sounds recorded with our sonification technique are able to correctly categorize tactile gestures (e.g., stroking, tapping) and their valence ratings are consistent with the intended emotions (e.g., anger, joy, love). Many scientific questions arise from these results, including the multiple dimensions involved in social touch that can be sonified, the multisensory and contextual factors increasing the feeling of being touched, the perceptual and physiological responses in participants listening to these audio-touch stimuli. All these questions will constitute advances toward the aim to give remote access to meaningful social touch interactions.

Missions:

The work will consist in designing and conducting experiments in laboratory settings, analyzing the results and contribute to writing the corresponding publications.

Required profile:

PhD in cognitive sciences, cognitive psychology, or behavioral sciences.

Required skills:

Good knowledge and practice of experimental psychology methods are essential. The applicant should demonstrate a capacity to work within a team and also independently. Given the scope of the project, the applicant should be able to assume important responsibility for carrying out all aspects of the research project including data collection, statistics, basic programming, writing publications.

General information :

– Contract start date: September 01, 2024

– Contract duration: 2 years

– Number of hours worked: 100%.

– Desired experience: 1 to 10 years

– Level of studies required: PhD

– Host laboratory: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.

Contact persons :

– Malika Auvray ; auvray(at)isir.upmc.fr

– Send your application by email, with [job title] in the subject line, a CV and covering letter + contact details for 2 referees.

– Deadline for applications: 15 May 2024

Download this job offer

Post-doctoral research topic: Development of a patient-specific numerical model for surgical simulation of the mitral valve

Context:

The post-doctoral work is part of an RHU-ICELAND project that brings together several academic, hospital, and industrial partners. The project’s objective is to develop a new transcatheter mitral valve annuloplasty solution incorporating intracardiac echocardiography.

Such a solution will enable interventions without cardiopulmonary bypass, allowing, in the initial phase, mitral valve repair for high-risk patients who are not eligible for open surgery, and later for most patients requiring mitral valve repair.

Direct annuloplasty involves attaching a ring or band directly to the mitral annulus using anchors under direct echocardiographic and fluoroscopic guidance. The advantage of this technique is that it constrains the shape of the mitral annulus, closely reproducing surgical mitral annuloplasty. The RHU-ICELAND project focuses on two essential phases: the development of a numerical model of the anatomy and the robotic system for placing staples on the mitral valve, and then the design and validation of the robotic system, previously validated numerically.

Technical and Scientific Objectives:

In the first phase, the recruited post-doc will focus on the numerical modeling of the anatomical part. The goal is to start with an open-source numerical model and add functionalities (deformation model, physiological movements, etc.) to approach the desired model as closely as possible. Once the anatomical model is deemed satisfactory, the post-doc will work on the numerical modeling of the flexible endoscopic robot (conveyor), starting from an existing numerical model of a robot developed for obstetric surgery.

Experimental Validation and Project Management:

The post-doc will work closely with the academic and clinical teams involved in the project, especially when integrating the work into the final demonstrator. The post-doc will benefit from a stimulating research environment and access to clinical data from the clinical and industrial partners of the project.

Profile Sought:

– Robotics, mechatronics, simulation, and/or numerical modeling,

– Advanced programming skills (C++, Matlab, Python),

– Proficiency in a library for numerical simulation of flexible robots (e.g., SOFA) is a plus,

– Enthusiasm for interdisciplinary research and a collaborative spirit.

Hosting Structure:

The recruited candidate will join the Institute of Intelligent Systems and Robotics (ISIR) at Sorbonne Université and CNRS (Paris). ISIR is organized into several interdisciplinary teams, including AGATHE. Among the research activities addressed by researchers are microrobotics, drones, surgical robotics, bionic prostheses, social robots, and various intelligent and interactive systems, physical, virtual, or mixed reality, artificial intelligence, etc. Their applications address major societal challenges: health, industry of the future, transportation, and personal services.

The Assistance to Gesture and Therapeutic Applications (AGATHE) team, where the recruited post-doc will be integrated, conducts research in robotics for healthcare, focusing on interactive systems for expert (surgery) or pathological (disability) gesture assistance. AGATHE has extensive experience in developing advanced robotic solutions for interventional medicine (neurosurgery, ENT surgery, endovascular interventions, etc.), which is the focus of the proposed post-doc.

General Information:

– Contract start date: as soon as possible

– Contract duration: 12 months, renewable for a further 12 months

– Working time: 100%

– Desired experience: beginner at 4 years old

– Level of studies required: PhD

– Host laboratory: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.

Contact persons:

Jérôme Szewczyk (PU-Sorbonne University) and Brahim Tamadazte (DR-CNRS)

Send a single pdf file, a CV, a covering letter and any scientific articles you consider relevant to the application to sz(at)isir.upmc.fr and brahim.tamadazte(at)cnrs.fr

Download this job description

Job title: Post-doc on haptic and multisensory human-computer interactions

Context:

This post-doc is part of the NeuroHCI ANR project. The overall goal of NeuroHCI is to improve human decision making in the physical and digital worlds in interac,ve contexts. There are various scenarios in which a human makes a decision with an interac,ve system. The decision might be about a complex real-world choice assisted by a computer (e.g. medical treatment), the choice of a method to achieve a digital task (e.g. editng a photo with the preferred tool), or the way we decide the best way to perform a haptic interaction.

Missions:

The envisioned scientific approach will rely on optimizing the haptic feedback delivered to the user with relation to vision and hearing by leveraging computational models of multisensory integration. Thus, the scientific activities of the project will revolve around questions including but not limited to:

– How to ensure that the inconsistencies between what the user sees and what the user feels does not break the illusion and how to mitigate their effects on user experience?

– How visuo-haptic inconsistencies influence users’ strategies (e.g. which objects they will decide to interact with) and high-level decision making?

Required profile:

The ideal candidate must have a PhD degree and a strong background in human-computer interaction and/or cognitive science.

Required skills:

– Experience in haptics is a strong plus ;

– Strong skills in Python, Matlab or equivalent ;

– Good knowledge of experimental design, psychophysics and statistics ;

– Excellent publication record ;

– Willingness to work in a multi-disciplinary team ;

– Good communication skills.

General Information:

  • Contract start date: at the latest 06/01/2024
  • Contract duration: 24 months
  • Working time: 100%
  • Desired experience: beginner at 4 years old
  • Level of studies required: PhD
  • Host laboratory: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.

How to apply:

– David Gueorguiev ; david.gueorguiev(at)sorbonne-universite.fr

– Send your application by email, with [name of offer] in the subject line, a CV and a cover letter.

Download this offer

Post-doc : Learning in robotics, with application to grasping

Context:

During the FET Proactive DREAM project (http://dream.isir.upmc.fr/) has been defined an approach for adaptive robotics based on open-ended learning. The main goal is to allow a robot to learn without requiring a careful preparation by an expert. This approach raises many challenges, notably learning with sparse reward, representation learning (for states and actions), model learning and exploitation, transfer learning, meta-learning and generalization. These topics are considered in simulation, but also on real robotics setup, notably in the context of grasping.

Missions:

This position aims at contributing to these topics in the context of several European projects, in particular SoftManBot, Corsmal, INDEX and Learn2Grasp. Calling upon previous works in the research team, the proposed approaches need to be easy to adapt to different robotic platforms and will thus be applied to different robots (Panda arm from Franka-Emika, Baxter, PR2 or TIAGO, for instance).

Required profile:

Candidates for the position must have a PhD degree in machine learning or related field in which robotics applications (either simulated or real) have been considered.

Required skills:

An excellent background is expected in machine learning as well as an experience in robotics. Excellent programming skills in Python are expected.

General Information: 

  • Position Type: Post-doctoral researcher
  • Contract duration: 24 months
  • Level of education required: PhD
  • Remuneration : Remuneration according to experience
  • Location: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.

Contact person: 

  • Stephane Doncieux
  • stephane.doncieux(at)sorbonne-universite.fr
  • Send your application by email, with a CV and a cover letter.

Download this offer

Doctoral, Postdoctoral or research engineer positions for the HCI Sorbonne group (Human Computer Interaction)

Context:

We have multiple postdoctoral and engineering positions in the HCI group (https://hci.isir.upmc.fr) at Sorbonne Université, Paris, France.

Missions:

We are searching for curious minds who want to conduct cutting-edge research at the intersection of HCI with VR, Haptics, Robotics or AI. Possible topics/areas of research are:

  • Novel Interaction techniques in VR,
  • VR and haptics for gaming or training,
  • Computational models for learning, decision making and human performance,
  • AI-based recommendation systems.

Some of our previous work in these areas:

Required profile:

For the postdoctoral position, a Phd degree in Computer science, HCI or other field related to our research areas is required.

Required skills:

  • strong programming and analytical skills,
  • strong background in at least one of the following areas (HCI, VR, Haptics, Robotics, AI).

More information : 

  • Type of position: Postdoctoral or Research Engineer position
  • Start date: as soon as possible
  • Duration: 1 to 2 years
  • Level of study required: Master 2 (for engineer), PhD (for post-doc)
  • Location: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.

Contacts: 

  • Gilles Bailly et Sinan Haliyo
  • Email : gilles.bailly(at)sorbonne-universite.fr ; sinan.haliyo(at)sorbonne-universite.fr
  • Application: Send your application by email, with a CV and a cover letter.
  • Application deadline: None

Download the offer

PhD offers

Subject: Physics-aware deep learning for modeling spatio-temporal dynamics

Context:

Physics-aware deep learning is an emerging research field aiming at investigating the potential of AI methods to advance scientific research for the modeling of complex natural phenomena. This research topic investigates how to leverage prior knowledge of first principles (physics) together with the ability of machine learning at extracting information from data. This is a fast-growing field with the potential to boost scientific progress and to change the way we develop research in a whole range of scientific domains. An area where this idea raises high hopes is the modeling of complex dynamics characterizing natural phenomena occurring in domains as diverse as climate science, earth science, biology, fluid dynamics, etc. This will be the focus of the PhD project.

Research Directions:

The direct application of state-of-the-art deep learning (DL) methods for modeling and solving physical dynamics occurring in nature is limited by the complexity of the underlying phenomena, the need for large amounts of data and their inability to learn physically consistent laws. This has motivated the recent exploration of physics-aware methods incorporating prior knowledge, by researchers from different communities (Willard et al. 2020, Thuerey et al. 2021). Although promising and rapidly developing, this research field faces several challenges. For this PhD project we will address two main challenges, namely the construction of hybrid models for integrating physics with DL and generalization issues which condition the usability of DL for physics.

– Integrating DL and physics for spatio-temporal dynamics forecasting and solving PDEs

In physics and many related fields, partial differential equations (PDEs) are the main tool for modeling and characterizing the dynamics underlying complex phenomena. Combining PDE models with ML is then a natural idea when building physics-aware DL models and it is one of the key challenges in the field. For now, this has been explored for two main directions: (i) augmenting low resolution solvers with ML in order to reach the accuracy of high-fidelity models at a reduced computational cost (Belbute-Perez et al. 2020, Kochkov et al. 2021, Um et al. 2020), and (ii) complementing incomplete physical models with ML by integrating observation data through machine learning (Yin et al. 2021a, Dona et al. 2022). The former topic is crucial for the entire field of numerical simulation while the latter allows for explorations beyond the current limits of numerical models. Simultaneously, the recent advances in neural operators (Li et al. 2021, Lu et al. 2021, Li et al. 2022, Yin et al. 2023) offer new methods for learning and modeling dynamics at different resolutions in space and time, providing the possibility of combining and learning multiple spatio-temporal scales within a unified formalism, a challenge in ML. A first direction of the PhD will then be to investigate physics-aware ML models by exploring the potential developments of hybrid models together with neural operators.

– Domain generalization for deep learning based dynamical models

Explicit physical models come with guarantees and can be used in any context (also called domain or environment) where the model is valid. These models reflect explicit causality relations between the different variables involved in the model. This is not the case for DL: statistical models learn correlations from sample observations, their validity is usually limited to the context of the training domain, and we have no guarantee that they extrapolate to new physical environments. This is a critical issue for the adoption of ML for modeling the physical world. Models of real-world dynamics should account for a wide range of contexts resulting from different forces, different initial and boundary conditions or different prior parameters conditioning the phenomenon. Ensuring generalization to these different contexts and environments is critical for real world applications. Surprisingly, only a few works have explored this challenging direction. In relation with the construction of hybrid models as described above, one will investigate this issue along two main directions. The first one exploits ideas from learning from multiple environments through task decomposition as in (Yin et al. 2021b, Kirchmeyer et al. 2022). This is a purely data-based approach. The second one, takes a dual perspective, relying on prior physical knowledge of the system equations and directly targets the problem of solving parametric PDEs (Huang 2022), exploiting ideas from meta-learning (Finn 2016).

Required Profile:

Master degree in computer science or applied mathematics, Engineering school. Background and experience in machine learning. Good technical skills in programming.

More information:

  • Supervisor: Patrick Gallinari
  • Collaboration within the framework of the thesis: INRIA team Ange, Paris; Inria team Epione, Sophia; Institut d’Alembert, Sorbonne University.
  • Host laboratory: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.
  • Start date: October/November 2023
  • Note: The research topic is open and depending on the candidate profile could be oriented more on the theory or on the application side
  • Keywords: deep learning, physics-aware deep learning, climate data, fluid dynamics, earth science

Contact:

  • Patrick Gallinari
  • Email : patrick.gallinari(at)sorbonne-universite.fr
  • Please send a cv, motivation letter, grades obtained in master, recommendation letters when possible to patrick.gallinari(at)sorbonne-universite.fr
  • Application deadline: 15/12/2023

Download this offer

Internship offers

Subject: Evaluation and integration of causal and contrastive feedback in inverse reinforcement learning algorithms

Abstract:

Interactive Machine Learning (IML) has gained significant attention in recent years as a means for intelligent agents to learn from human feedback, demonstration, or instruction. However, many existing IML solutions primarily rely on sparse feedback, placing an unreasonable burden on the expert involved. Our project aims to address this limitation by enabling the learner to leverage richer feedback from the expert, thereby accelerating the learning process. Additionally, we seek to incorporate a model of the expert to select more informative queries, further reducing the burden placed on the expert.

This internship provides you with the opportunity to participate in our research project, where you will specifically focus on evaluating algorithms for Inverse Reinforcement Learning (IRL) and related approaches. IRL is an imitation learning technique that centres on inferring the reward function underlying expert demonstrations. To be more precise, your role will involve testing innovative approaches to incorporate causal and contrastive feedback while maintaining hypotheses about potential expert objectives.

Internship Objectives:

– Investigate methods to integrate causal and contrastive feedback into the inverse reinforcement learning process ;

– Design and implement a belief-based system that allows the learner to explicitly maintain hypotheses about the expert’s objectives ;

– Utilise received feedback to generate a posterior that informs subsequent queries and improves the learning process within the framework of IRL.

Required Profile:

– Master 2 student in computer science, intelligent systems, or related fields ;

– Interest in imitation learning, reinforcement learning, interactive robot learning, and human- machine collaboration ;

– Ability to work independently and in a team ;

– Strong written and oral communication skills in English.

Required skills :

– Proficiency in Python programming and some familiarity with GitHub and Jupyter Notebook, as well as knowledge of machine learning concepts and OpenAI Gym environments.

General information :

– Supervisor: Silvia Tulli

– Duration of internship: 6 months

– Desired level of study: Master 2

– Host laboratory: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris, France

Contact person:

– Silvia Tulli; tulli(at)isir.upmc.fr

– Send your application by e-mail, with [internship subject] in the subject line, a CV and a covering letter.

Download this offer

Subject: Social Robot Navigation with Pepper Robot

Abstract:

Human Robot Interaction is one of the main pillars of robotics that has a long way to go to adapt robots in our daily life environments. Robots working in human populated environments should be able to perceive and understand human behavior and adapt their motions to be more socially compliant. This means, they should not only guarantee the safety of the people around them, but also need to show legible motions to be more understandable by humans. In fact, by generating trajectories that are both legible and efficient, we can optimize service efficiency and customer experience in dynamic, shared spaces.

As part of euROBIN project, we are developing a social navigation stack to deliver small objects in a restaurant-like scenario, to a specific person, while acting gently and legibly so that the target person and other people in the environment can understand the robot’s intentions. This requires equipping the robot with the necessary sensors to perceive the environment, design and develop a perception system that can capture the necessary information from the environment and the people in it, and finally develop motion planning algorithms that can generate legible motions for the robot while adapting to the environment changes.

Internship Objectives:

The main objective of this internship is to develop a social navigation stack for the Pepper robot. This stack should perceive the human behavior plus any other relevant information from the scene, predict the motions and finally generate legible motion plans for the robot. But since Pepper, is not equipped with the necessary sensors to perceive the environment, we need to equip it with extra sensors and processing units. We want to leverage Intel RealSense stereo cameras and Nvidia Jetson GPUs to enhance the perception and processing capabilities of the robot. The algorithms should be implemented in ROS (preferably ROS2) and be tested on Pepper robot. But this might also require us to simulate the robot before testing on the real robot, for which we can use Gazebo or Unity. This internship is a great opportunity to dive into the ROS ecosystem, learn about social navigation and also learn about the latest technologies in computer vision for robotics.

The internship at ISIR, Sorbonne University, encompasses three primary steps. Initially, they will focus on developing the ROS stack to control the Pepper robot, connected to the Intel RealSense stereo cameras and Nvidia Jetson GPUs. The second phase involves designing and implementing the vision system to detect and track people and their gaze. The third step is implementing the planning algorithms to generate legible motions for the robot, and making some limited tests in a controlled environment. And finally, the intern will be expected to document their work and possibly present it at a conference or workshop.

Required Profile:

Motivated Master 2 students with a robust academic foundation in Computer Vision and Robot Navigation, eager to contribute to a dynamic and collaborative robotics team.

Required skills :

– programming (Python / C++)

– implement nodes and algorithms in ROS/ROS2

– debugging multi-threaded applications

– motion planning, and obstacle avoidance algorithms

– stereo vision and depth cameras

– simulation (Gazebo, Unity, Isaac Sim,…)

– basic experience with a 3D CAD design software (Solidworks, FreeCAD, …)

– and demonstrate strong problem-solving skills.

General information :

– Supervisors: Javad Amirian and Mouad Abrini

– Starting date: March 1, 2024

– Duration of internship: 5-6 months

– Level of studies required: Master 2

– Host laboratory: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.

Contact person:

– Javad Amirian

– Email : amirian[at]isir.upmc.fr

– Send your application by e-mail, with [internship subject] in the subject line, a CV and a covering letter.

– Application deadline: January 31, 2024

Download this internship offer

Subject: Learning to grasp in robotics – european network of excellence

Abstract:

Learning to grasp in robotics has received increasing attention for several years, justified by the high scientific and practical stakes associated with it [1]. This problem is quite simple in a deterministic environment: it consists in controlling a manipulator arm to reach precise positions and open or close the gripper. But these approaches are limited to heavily constrained scenarios. Despite the efforts of major academic and industrial players, performing object grasping in an uncontrolled environment is still an unsolved task [2].

First, it is a hard exploration problem: it is very difficult to make the robot produce successful grasps until an effective controller is available – in other words, until the problem is solved. Second, experiments with real robots are expensive, slow, and subject to many integration and maintenance problems. Training policies in simulation is therefore preferable, but the reality gap often makes the generated solutions inefficient on a real robot.

The most common way to tackle this problem is to simplify it by considering it as a grasp pose estimation task. This was first done in the form of in-plane position predictions, limiting the policies to top-down movements [3], and more recently by doing 6-DoF pose estimation (gripper position and orientation) [4]. However, these methods impose strong assumptions on the gripper structure, limiting the related works to parallel grippers or suction grippers.

Quality-diversity algorithms [5] are evolutionary learning methods designed to generate high- performance solutions to a given problem. Recent results from the team have shown that these methods can be used to generate large datasets of diverse and robust inputs [6], which can be transferred to real robots [7] and generalised to the whole robot’s operational space [8].

Internship Objectives:

EuROBin [9] is a network of excellence in robotics in Europe, bringing together private companies and public institutions. In November, the 2nd EuROBin event will take place during the Humanoïds conference in Nancy. A cooperative competition will be held there, in which teams will be rewarded for carrying out specific tasks with robots by exploiting the work of other European teams.

The aim of this internship is to build on the team’s previous work [10] to extend these results to the robots entered in the EuROBin competition. The aim is to study the grippers of each team involved, to adapt the results to these grippers, and to ensure that the European partners can exploit them on their robots to improve their gripping capabilities.

Required Profile:

Students with a strong academic background in Artificial Intelligence, Machine Learning or Data Science for Robotics.

Skills:

Required : Python, data science, machine learning (theory : standard methods, DL, CNN ; in practice : AI framework (PyTorch, …)), measurement and visualisation (matplotlib, seaborn).

Optional : Robotics simulators (PyBullet, Isaac Gym …), Computer vision, evolutionary algorithms, high performance computing (CPU et GPU).

General information :

– Supervisor : Stéphane Doncieux

– Start date: January or February 2024

– Duration of internship: 6 months

– Level of studies required: Currently in Master 2, or final year of engineering school.

– Host laboratory: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.

Contact persons:

– Stéphane Doncieux, Johann Huber, François Hélénon

– Email : stephane.doncieux (at) isir.upmc.fr ; johann.huber (at) isir.upmc.fr ; helenon (at) isir.upmc.fr

– Send your application by e-mail, with [apprentissage_s saisie_objets] in the subject line, a CV and a covering letter. It is strongly recommended that you also attach one or more personal projects (github, etc…).

Download this offer

Subject: Control of balance when walking on the hands

Context:

Maintaining balance during dynamic movement is a challenging task which, in humans, requires the coordinated contraction of more than 600 muscles to adjust the forces acting on the body. During walking and running, individual muscles do not contract independently, but work together as synergies resulting in meaningful motor outputs: muscles which counteract each other thus do not contract simultaneously [1]. Moreover, healthy adults maintain balance by minimising rotation around their Center of Mass (CoM, green in Figure 1.A). They achieve this by orienting the force exerted by the legs onto the ground (Ground Reaction Force – GRF, purple in Figure 1) such that it points to the CoM (Figure 1.B) [2]. Preliminary results indicate that, during successful hand-walking, subjects also maintain balance by orienting the GRF to the CoM (Figure 1.C). This requires novel muscle synergies.

Internship:

The task is to analyse experimental data of human gymnasts walking on the hands. The goal of the internship is :

1. To find a biomechanical predictor of when subjects will lose their balance, by comparing successful (Figure 1.C) and unsuccessful hand-walking trials ;

2. To identify the muscle synergies allowing subjects to maintain balance during successful hand- walking trials.

Reference :

[1] E. Bizzi and V. C. K. Cheung, ‘The neural origin of muscle synergies’, Front Comput Neurosci, vol. 7, Apr. 2013.

[2] H. Herr and M. Popovic, ‘Angular momentum in human walking’, J Exp Biol, vol. 211, no. Pt 4, pp. 467–481, Feb. 2008, doi: 10.1242/jeb.008573.

General information :

– Supervisors: Charlotte Le Mouel, CNRS Research Fellow, and Hélène Pillet, University Professor

– Starting date: February – April 2024

– Internship duration: 3 to 6 months

– Level of studies required: Master 2

– Host laboratory: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.

Contact persons:

– Charlotte Le Mouel et Hélène Pillet

– Email: charlotte.lemouel(at)normale.fr ; helene.pillet(at)ensam.eu

– Send your application by email, with [internship subject] in the subject line, a CV and a covering letter.

– Application deadline: 7 January 2024

Download this offer

Subject: Segmentation If MRI Images for bilio-pancreatic endoscopy

Abstract:

The MAAGIE project aims to develop a set of software tools to aid endoscopic navigation in the biliopancreatic tract (Fig. 1). Most of these tools are based on a 3D model of the biliopancreatic anatomy of the operated patient (Fig. 2) [1]. In particular, a current thesis focuses on automatic segmentation based on deep learning (DL) of MRI images for the reconstruction of these 3D models [2]. One of the major difficulties in this work lies in the creation of a base of 3D reference models to serve as ground truth during the training phase of the DL-based algorithms. Currently, these reference 3D models are segmented manually, which is very time-consuming or sometimes even impossible.

Internship Objectives:

The aim of this internship is to develop a computer aid for the manual segmentation of 3D models to create a reliable and sufficient reference base (we are targeting around fifty patients). To do this, two approaches will be explored in parallel:

1) We will develop a manual segmentation support environment offering mask prediction by region growth and propagation from section to section as well as tools for manual rectification of mask contours. To do this, we will use the context of the 3D slicer software namely the Volume, Segmentation and Segment Editor modules [3];

2) We will develop a CNN-based semi-automatic segmentation algorithm based on the MONAI framework [4]. The idea here is to interactively train a DL segmentation model: we first train the model with a reduced existing baseline and then the model infers (approximately) new patient cases which are corrected by an operator before to be added to the learning base etc.

Required profile: Master degree or ingineering degree – computer sc., Image processing, AI

Required skills : Autonomy, good team spirit

General information :

– Supervisors: J. Szewczyk (ISIR), M. Camus (Hôpital Saint-Antoine)

– Starting date: February 2024

– Internship duration: 6 months

– Level of studies required: Master 2 or engineering final year project

– Host laboratory: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.

Contact persons:

– Jérôme Szewczyk

– Email : jerome.szewczyk(at)sorbonne-universite.fr

Send your application by e-mail, with [internship subject] in the subject line, a CV and your M1 and M2 report cards if available.

Application deadline: December 15, 2023

Downlaod the internship offer

Subject: Electrical actuation of the SYNSYS prosthesis

Context:

The Institute of Intelligent Systems and Robotics (ISIR) is offering an exciting and innovative M2 or final year engineering school internship, in collaboration with PROTEOR, a leading company in the field of medical devices, particularly lower limb prostheses. This internship will focus on improving PROTEOR’s SYNSYS prosthesis. The SYNSYS prosthesis is a lightweight trans-femoral prosthesis with two joints, the first for the knee and the second for the ankle. Among existing prostheses, it has the best operating autonomy thanks to the use of hydraulic energy.

Internship subject:

The aim of the internship is to contribute to the evolution and optimization of PROTEOR’s SYNSYS prosthesis by integrating an electric motorization stage. The candidate will be involved in the design, development and evaluation of this new functionality, which aims to broaden the prosthesis’ range of uses. Experimental work will be central to this project.

Profile required:

– Student in the final year of a Master’s 2 or graduating from an engineering school in robotics, mechatronics or a related field.

– Solid experience in robotics and mechatronics, including the design and development of mechanical and electronic systems.

– Strong proposal skills.

– Strong interest in experimental research.

– Ability to work independently and as part of a team. – Excellent communication and writing skills.

General information :

– Supervisors: Waël Bachta, Lecturer in Robotics, and Nathanaël Jarrassé, Researcher in Robotics

– Starting date: from February 2024

– Internship duration: 5 to 6 months

– Level of studies required: Master 2

– Host laboratory: ISIR (Institut des Systèmes Intelligents et de Robotique), Campus Pierre et Marie Curie, 4 place Jussieu, 75005 Paris.

Contact persons:

– Waël Bachta and Nathanaël Jarrassé

– Email : bachta(at)isir.umpc.fr et jarrasse(at)isir.upmc.fr

Interested candidates are invited to send their CVs, covering letters and transcripts to: bachta(at)isir.umpc.fr and jarrasse(at)isir.upmc.fr. Please include “Application Stage SYNSYS – ISIR” in the subject line of the e-mail. Shortlisted candidates will be contacted for an interview.

Download the job description