Offre en lien avec l’Action/le Réseau : MACLEAN/– — –

Laboratoire/Entreprise : laboratoire d’informatique et systèmes (LIS) UMR
Durée : 5 to 6 months
Contact : zniyed@univ-tln.fr
Date limite de publication : 2022-04-30

Contexte :
Deep Neural Networks (DNNs) demonstrate good prediction performances in numerous applications. However, the architectures of neural networks are very large, reaching several million parameters, and running them on systems with limited computing capacity (embedded systems) becomes a difficult task. For this reason, we will focus in this internship project on the compression of DNNs by tensor methods.

Sujet :
This internship project deals with the study of new compression techniques for deep neural networks, by resorting to tensor decompositions to model and factorize the DNN weights. Recent studies show that DNN weight matrices are often redundant, and by restricting their ranks, it is possible to significantly reduce the number of parameters without a significant drop in performance. In this project, we propose to convert these matrices to a tensorial format and to use multidimensional data processing methods to compress them. The goal of this internship is to study different tensor representations, such as the canonical polyadic decomposition (CPD) or Tucker decomposition (TD), for the compression of the converted multidimensional weights. Specifically, we will study the compactness of these representations and their impact on the predictive accuracy of DNNs. In a first stage, the intern student will review the existing state-of-the-art tensor-based compression techniques and will get familiar with the tensor decompositions. Then, we will compare different representations with the goal to improve them and propose new tensor-based scheme for DNN compression.

This internship can be followed by a Ph.D research project, starting October, 2022, at LIS, Toulon

Profil du candidat :
M2R or engineering school students with major in signal processing, machine learning or applied mathematics.

Formation et compétences requises :
Good python programming skills are required. The knowledge of deep learning frameworks is a desirable plus. The candidate should have good writing and oral communication skills.

Adresse d’emploi :
The intern student will join the Signal and Image (SIIM) research team at the LIS laboratory, Toulon.
The internship will be supervised by Yassine Zniyed (Associate Professor at Université de Toulon) and Thanh Phuong Nguyen (Associate Professor/HDR at Université de Toulon).

Document attaché : 202202210815_Stage_M2R_2022.pdf