Nicolas Castanet | ISIR, Sorbonne University

About Me

I recently obtained a Ph.D. in Reinforcement Learning at The “Institut des Systèmes Intelligents et de Robotique” (ISIR) in Sorbonne Université (Paris), within the MLIA team. I was supervised by Olivier Sigaud (ISIR, Sorbonne Université) and Sylvain Lamprier (LERIA, Université d’Angers, ex MLIA Sorbonne université). I work on goal-conditionned Deep Reinforcement Learning in continuous environment. Specifically, my objective is to design intrinsically motivated agents that sets their own goals in order to explore and expand their sets of skills. I focus on approaches that need as little prior knowledge on the environment as possible in order to scale to unsupervised settings, mainly for manipulation and navigation robotic tasks.

Research Interests

Deep Reinforcement Learning: Goal-conditionned RL, Curiculum Learning, Exploration
Representation Learning: Data compression, Contrastive Learning
Machine Learning: Generative models, Variational inference, Optimisation

News

[September. 2025] Thrilled to announce that our paper Imagine Beyond! has been accepted to Neurips 2025, I will be in San Diego for the poster session.
[May. 2025] Check out our new preprint: Imagine Beyond! We propose a new method to overcome exploration bottlenecks in Online RL coupled with representation learning leveraging Distributionally Robust Optimization.
[January. 2025] I defended my PhD Thesis on the 15 January 2025. Manuscript.
[July. 2023] I am going to present our paper SVGG at ICML 2023 during a poster session in Honolulu.
[April. 2023] Our paper Stein Variational Goal Genration (SVGG) is accepted at ICML 2023. We propose a new method to automatically generate a curriculum of goals leveraging the Bayesian Inference method Stein Variational Gradient Descent.

Publications

Neurips

Imagine Beyond! Distributionally Robust Auto-Encoding for State Space Coverage in Online Reinforcement Learning

Nicolas Castanet, Olivier Sigaud, Sylvain Lamprier (2025).

Neural Information Processing Systems(Neurips), 2025.

ICML

Stein Variational Goal Generation for adaptive Exploration in Multi-Goal Reinforcement Learning

Nicolas Castanet, Olivier Sigaud, Sylvain Lamprier

International Conference on Machine Learning (ICML), 2023.

Teaching

[ 2024-2025] Reinforcement Learning and advanced Deep Learning, 2nd year Master DAC, Sorbonne Université.
[ 2023-2024] [ 2024-2025] Python introduction in 1st year Bachelor, Sorbonne Université.
[ 2022-2023] Python game development project, development of simple IA agents, 2nd year Bachelor, Sorbonne Université.
[ 2022-2023] Business Intelligence and User Modeling, 1st year Master DAC, Sorbonne Université.