Offre en lien avec l’Action/le Réseau : – — –/– — –

Laboratoire/Entreprise : Laboratoire Hubert Curien
Durée : 36 month
Contact : ievgen.redko@univ-st-etienne.fr
Date limite de publication : 2021-04-15

Contexte :
In recent years, deep learning has imposed itself as the state of the art ML method in many real-world tasks, such as computer vision or natural language processing to name a few [1]. While achieving impressive performance in practice, training DNNs requires optimizing a non-convex non-concave objective function even in the case of linear activation functions and can potentially lead to local minima that are arbitrary far from global minimum. This, however, is not the typical behaviour observed in practice, as several works [2, 3] showed empirically that even in the case of training the state-of-the-art convolutional or fully-connected feedforward neural networks one does not converge to suboptimal local minima. Such a mysterious behaviour made studying the loss surface of DNNs and characterizing their local minima one of the topics of high scientific importance for the ML community.

Sujet :
In order to provide novel insights into the behaviour of DNNs, our goal will be to study them as instances of congestion games [4], a class of games often used to model network traffic and communications. This particular choice is due, on one hand, to the fact that both DNNs and congestion games can be modeled as direct acyclic graphs (DAGs), while, on the other, congestion games are arguably among the most studied classes of games in GT that are known to exhibit many attractive properties. The approximate objectives of the Ph.D. thesis in this context will consist in:
1. Proposing a general approach of finding a congestion game equivalent to a given DNN.
2. Translating the different quantities of interest often analyzed in the context of congestion games to DNNs in order to provide a novel theoretical analysis for them.
3. Using extension theorems [5] to study the speed of convergence of online optimization strategies when applied to DNNs.
Some preliminary encouraging results obtained for such an approach have been obtained recently by supervisors and the Ph.D. candidate is expected to address the open problems mentioned in the latter paper.

Profil du candidat :
Ideal candidates will have a strong background in both machine learning and game theory, but anyone with a Master’s degree in applied/pure mathematics is encouraged to apply. Proficiency in at least one programming language commonly used in machine learning community would be a plus.

Formation et compétences requises :
Master’s degree in Applied/Pure math or Machine Learning.

Adresse d’emploi :
Saint-Etienne, France

Document attaché : 202103221643_thesis_proposal.pdf