Offre en lien avec l’Action/le Réseau : DOING/– — –

Laboratoire/Entreprise : ICube
Durée : 6 mois
Contact : seo@unistra.fr
Date limite de publication : 2021-02-15

Contexte :
Recently, there is a huge interest in applying deep learning techniques for synthesizing novel data from the learned model. It is true also for the human shape and motion data, for which several deep learning approaches have been proposed. Examples include a feedforward neural network that maps high level control parameters to the low level human motion over a manifold space found by a convolutional autoencoder, CNN-based architecture combined with deep correlated 2D features for full shape recovery from image silhouettes, auto-conditioned recurrent neural networks to synthesize arbitrary motions with highly complex styles, RNNs (recurrent neural networks) trained for time-series prediction on shape and pose change during animation, Phase-Functioned neural network which takes the geometry of the scene into account to produce character motion along a user-defined path, networks that can produce a distribution of next-state predictions in the context of character motion generation, among others.
In this internship, we will focus on generative models of new types of data, 4D human, i.e. 3D human shape data under motions. The challenging problem of high spatiotemporal dimension of data, physical/environmental constraints, and user-defined controls will be addressed, along with the architectures of deep neural networks that can handle long sequences without an accumulation of errors.

Sujet :
The objective of this internship is to develop deep-learning frameworks for the generation of realistic and controllable 4D human models. Given the user-controllable goal (task, style, constraints, etc), the trained network should be able to generate the desired model in real-time. There are several ways to approach the problem, depending on the representation of dataset, the choice of the network architecture, and the types of goals and the way they are specified/controlled by the user. As for the network architecture, we will focus on the combinations of RNN and variational autoencoder, allowing a stochastic prediction of shape- and pose-sequences in a latent space. Several preprocessing of datasets from different sources may be required, in order to homogenize them into a uniform representation for the training. Different data representations and network hyperparameters will be experimented, to obtain the best results. Evaluation and comparison of the performance to the state-of-the-art methods is strongly recommended, whenever applicable.

Profil du candidat :
— Master student in Computer Science or in (Applied) Mathematics
— Solid programming skills in deep learning platforms: Tensorflow/Pytorch
— Background in geometric modeling and statistics
— Good communication skills

Formation et compétences requises :
Image processing, Introduction to deep learning, Computer vision, Linear algebra, Statistics

Adresse d’emploi :
ICube UMR 7357 – Laboratoire des sciences de l’ingénieur, de l’informatique et de l’imagerie
300 bd Sébastien Brant – CS 10413 – F-67412 Illkirch

Document attaché : 202012142143_sujetM2_GenerativeModels.pdf