Offre en lien avec l’Action/le Réseau : RoCED/– — –

Laboratoire/Entreprise : INRAE Toulouse
Durée : 6 mois
Contact : mathieu.fauvel@inrae.fr
Date limite de publication : 2024-02-28

Contexte :
Over the last ten years, Earth Observation (EO) has made enormous advances in terms of spatial and temporal resolutions, data
availability and open policies for end-users. The increasing availability of complementary imaging sensors allows land ecosystems
state variables and processes to be observed at different spatio-temporal scales. Big EO data can thus enable the design of new
land monitoring systems providing critical information in order to guide climate change monitoring, mitigation and adaptation.
Conventional machine learning methods are not well adapted to the complexity of multi-modal, multi-resolution satellite image
time series (SITS) with irregular sampling. Therefore they are not suitable for extracting and processing all the relevant infor-
mation. On the other hand, methods based on deep neural networks have shown to be very effective to learn low-dimensional
representations of complex data for several tasks and come with high potential for EO data. However, they often emerge from the
computer vision (CV) and natural language processing (NLP) communities and need to be extended and properly instantiated to
handle the very specificities of Earth Observation data.
Previous works at the CESBIO-lab have shown that generative encoder-decoder architectures such as the Variational Auto-
Encoder (VAE) or the U-NET models perform very well for a variety of EO tasks : estimation of biophysical parameters or
Sentinel-1 to Sentinel-2 translations, to cite a few.
However, such approaches appear to be inadequate to handle data coming from more than 2 sources and acquired at different
time and spatial resolutions, as prioritized in the RELEO chair within ANITI. In particular, the generative capability of these
models may generalize poorly to unseen regions or temporal periods. Processing such streams of data requires to jointly encode
all sources into a structured latent space where each complementary information carried by each source can be embedded while
ensuring long-term encoding of newly acquired data (from possibly new sensors).

Sujet :
The objective of this internship is to investigate Gaussian process (GP) prior for Variational Auto-Encoders (VAEs). Usually,
VAEs assume independence between samples. This assumption is generally made for sake of simplicity and computational ef-
ficiency of the training and inference steps. However, assuming independence of samples amount to ignoring the correlation
between adjacent pixels in the temporal domains. Furthermore, because of the very deterministic nature of such neural networks
architectures, they do not properly encode uncertainty related to missing/noisy data.
Adopting this GP prior is expected to model correlations between times. However, due to the irregular and unaligned nature of
SITS and their massive volume, approximation are required to maintain fast training and inference.
The work-plan of this Master internship is as follows :
1. Define GP prior VAEs for pixel satellite time series with different approximations,
2. Implement the models in PyTorch,
3. Perform experiments on massive SITS and compare with others VAEs on downstream tasks.

Profil du candidat :
Master or Engineering school students with major in applied mathematics, computer science or electrical engineering.

Formation et compétences requises :
The knowledge needed for this work includes a strong background in machine learning or data science, signal & image
processing or remote sensing data processing. Good scientific programming skills (e.g., Python) and good communication skills
in English, both written and oral are also expected. Interests in Earth observation will be appreciated.

Adresse d’emploi :
Centre d’Etudes Spatiales de la Biosph`ere (CESBIO) & INRAE

Document attaché : 202312190628_proposal_MSc_2024_ANITI.pdf