Offre en lien avec l’Action/le Réseau : – — –/– — –
Laboratoire/Entreprise : IRIMAS, MSD team
Durée : 6 months
Contact : maxime.devanne@uha.fr
Date limite de publication : 2022-12-31
Contexte :
Human motion analysis is crucial for studying people and understanding how they behave, communicate and interact with real world environments. Due to the complex nature of body movements as well as the high cost of motion capture systems, acquisition of human motion is not straightforward and thus constraints data production. Hopefully, recent approaches estimating human poses from videos offer new opportunities to analyze skeleton-based human motion. While skeleton-based human motion analysis has been extensively studied for behavior understanding like action recognition, some efforts are yet to be done for the task of human motion generation. Particularly, the automatic generation of motion sequences is beneficial for rapidly increasing the amount of data and improving Deep Learning-based analysis algorithms.
Since several years, new image generation paradigms have been possible thanks to the appearance of Generative Adversarial Networks (GAN) which have proved to be extremely efficient for many image generation tasks and human posture estimation. Although these networks are very efficient, their explainability and control still remain challenging tasks. Differently, other generative models have also emerged by considering the data distribution during training like Variational AutoEncoder (VAE) and Diffusion models.
First work addressing deep generative models for human motion have considered motion capture (mocap) data allowing to accurately extract body parts positions along the time. Hence, aforementioned generative architectures have been successively employed for generating mocap-based human motion sequences.
Differently, we consider noisy skeleton data estimated from videos as it is easily applicable in real-world scenarios for the general public
Sujet :
The goal of this internship is to provide guidelines in building deep generative models for skeleton-based human motion sequences. Inspiring from recent effective Deep Learning-based approaches, the aim is to generate full skeleton-based motion sequences without access to successive poses as prior information as it can be done in prediction tasks. It is therefore crucial to investigate how deep generative models can handle such noisy and possibly incomplete data in order to generate novel motion sequences as natural and variable as possible.
In particular, the candidate will work on the following tasks:
– Deep Learning architectures for skeleton-based human motion: investigation and assessment of the influence of different deep network architectures for capturing complex human motion features. Particularly, the goal of this task is to theoretically and empirically analyze the performance of existing architectures like CNN, RNN GCN and Transformers for modeling skeleton-based human motion.
– Deep generative models adapted to skeleton data: based on studies from the previous task, the goal is to build generative models upon the previously identified meaningful spaces where skeleton sequences are represented. Therefore, the candidate will investigate different generative models, like GAN, VAE and Diffusion models, in order to propose and develop a complete Deep Learning model for generating skeleton-based human motions.
– Evaluation of deep generative models: in order to validate the proposed model, experimental evaluation is crucial. In comparison to motion recognition where classification accuracy is a natural way to assess an approach, evaluating the task of motion generation is not as straightforward. Dedicated metrics evaluating both naturalness and diversity of generated sequences as well as the impact of new generated sequences in a classification task will be considered.
Profil du candidat :
The candidate must fit the following requirements:
– Registered in Master 2 or last year of Engineering School (or equivalent) in Computer Science
– Advanced skills in Python programming are mandatory
– Good skills in Machine Learning & Deep Learning using related libraries (scikit-learn, Tensorflow, Pytorch, etc.) are required
– Knowledge and/or a first experience in human motion analysis will be appreciated
Formation et compétences requises :
The candidate must fit the following requirements:
– Registered in Master 2 or last year of Engineering School (or equivalent) in Computer Science
– Advanced skills in Python programming are mandatory
– Good skills in Machine Learning & Deep Learning using related libraries (scikit-learn, Tensorflow, Pytorch, etc.) are required
– Knowledge and/or a first experience in human motion analysis will be appreciated
Adresse d’emploi :
The proposed internship will be carried out within the MSD (Modeling and Data Science) team from the IRIMAS Institute. It will be part of the ANR DELEGATION (https://maxime-devanne.com/delegation/)
IRIMAS, Université de Haute Alsace, Rue des Frères Lumière, 68093 Mulhouse
