PhD Position F/M Anticipation of Hand-Object Contact Configuration for Object Manipulation
Inria
Montbonnot-Saint-Martin, 38330
PhD Position F/M Anticipation of Hand-Object Contact Configuration for Object Manipulation
Le descriptif de l’offre ci-dessous est en Anglais
Type de contrat : CDD
Niveau de diplôme exigé : Thèse ou équivalent
Fonction : Doctorant
A propos du centre ou de la direction fonctionnelle
The Inria Grenoble research center groups together almost 600 people in 23 research teams and 7 research support departments.
Staff is present on three campuses in Grenoble, in close collaboration with other research and higher education institutions (University Grenoble Alpes, CNRS, CEA, INRAE, …), but also with key economic players in the area.
Inria Grenoble is active in the fields of high-performance computing, verification and embedded systems, modeling of the environment at multiple levels, and data science and artificial intelligence. The center is a top-level scientific institute with an extensive network of international collaborations in Europe and the rest of the world.
Contexte et atouts du poste
Collaboration between Inria team THOTH and the multidisciplinary institute in artificial intelligence (MIAI) in Grenoble
Research activities in MIAI aim to cover all aspects of AI and applications of AI with a current focus on embedded and hardware architectures for AI, learning and reasoning, perception and interaction, AI & society, AI for health, AI for environment & energy, and AI for industry 4.0.
This project in particular focuses on perception and robot interaction and thus will take place in close collaboration with LIG and GIPSA-Lab at the University of Grenoble.
Mission confiée
Key words
deep learning, trajectory prediction, manipulation, contact
Theme/Domain
Anticipation of Hand-Object Contact Configuration for Object Manipulation
Context and Motivation
Learning hand-object contact configurations typically use large data sets consisting of 3D object shapes together with hand (robot gripper) poses, that together determine stable object-hand configurations. Still most approaches require complicated refinement strategies that disregard infeasible hand-object configurations from physically plausible ones [1]. In this work package we propose to jointly model both, the manipulation strategy together with the final object-hand configuration that should be reached. The intuition behind this approach is that knowledge about the appropriate action (e.g. manipulation strategy – sliding, picking) for a specific scenario determines the prediction of final object-hand configurations. Thus unrealistic configurations that are either not reachable or interfere with the environment in a physically not plausible won’t be even raised as a possibly feasible option.
Contact
[email protected]
Please apply with the typical application documents: CV, certificates (MSc degree), research statement and two references. Application deadline is November 30th. We will screen applications on a rolling basis. The position will be filled as soon as a suitable candidate is found.
Main location
INRIA Montbonnot, TOTH team
Principales activités
Summary
The knowledge about possible manipulation strategies and the object lead to feasible hand-object configurations. This projects aims at determining feasible hand-object contact configurations for object manipulation.
Approach
Many learning based approaches that consider only the object and the manipulator, generate hand-object configurations that are either not reachable or interfere with the environment in an harmful manner. For realisation, we will capitalise on the recent trend of action prediction using conditional variational autoencoders (CVAE) [2,3]. CVAEs are versatile deep generative that extend the standard VAE by conditioning with auxiliary properties – thus are suitable to develop visual representations conditioned on activity information. We will follow-up on existing concepts that will be further developed:
- Learning a (latent) representation conditioned on high-level manipulation strategies.
- Semi-supervised learning for learning from just few (but correct) examples. This stands in contrast to learning from massive amounts of noisy data.
Goal
Developing representations that jointly model visual information and dynamic action understanding – motion and contact. These representations will allow fundamentally new ways of interpreting grasps as dynamic motion trajectories that are not only defined by pre-computed static hand-object configurations, further this new way of modelling hand-object configurations will allow for incorporating information about a suitable manipulation strategy.
References
[1] Mousavian, A., Eppner, C., & Fox, D. (2019). 6-dof graspnet: Variational grasp generation for object manipulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 2901-2910).
[2] Halawa, M., Hellwich, O., & Bideau, P. (2022, October). Action-based contrastive learning for trajectory prediction. In European Conference on Computer Vision (pp. 143-159). Cham: Springer Nature Switzerland.
[3] Noseworthy, M., Paul, R., Roy, S., Park, D., & Roy, N. (2020, May). Task-conditioned variational autoencoders for learning movement primitives. In Conference on robot learning (pp. 933-944). PMLR.
Compétences
- MSc degree in computer science or similar field
- Experience in robotics, machine learning, computer vision, and/or control, industrial experience is a plus.
- Experience in applying (deep) learning to robotic control problems
- Excellent software engineering and programming skills in C++ and/or Python
- Interest in interdisciplinary research in the context of the MIAI Grenoble Alpes Institute
- Excellent English writing and communication skills
Avantages
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking (90 days / year) and flexible organization of working hours
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage under conditions
Rémunération
1st and 2nd year: 2 082 euros gross salary /month
3rd year: 2 190 euros gross salary / month