Julien Siems

About Me

I am a Ph.D. student at the University of Freiburg advised by Prof. Frank Hutter.

After graduating from the University of Freiburg in 2020 with distinction, I was an Applied Science Intern at AWS Berlin working on Neural Network Pruning, I worked on Bayesian Optimization for combinatorial auctions at the University of Zurich, did research into numerical optimization for inverse problems at TU Munich and worked as a Machine Learning Researcher at Merantix Momentum on Interpretable Machine Learning.

Research Interests

• In-Context Learning: State-Space Models, Linear Attention, Linear RNNs
• Interpretability: Generalized Additive Models

News

• [Oct. 2024] 3 papers accepted at NeurIPS workshops: “Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues” (Oral), “GAMformer” (Oral), and “Mamba4Cast” (Spotlight).
• [Jul. 2024] I co-organized the 1st Workshop on In-Context Learning (ICL) at ICML 2024 alongside Frank Hutter, Erin Grant, Beyza Ermiş, Jelena Bratulić and Noah Hollmann.
• [Mar. 2024] Our paper “Is Mamba Capable of In-Context Learning?” is accepted at the ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models (ME-FoMo) and at the AutoML Conference.
• [Sep. 2023] Our paper on concurvity (non-linear equivalent of multicollinearity) regularization was accepted at NeurIPS 2023.
• [Mar. 2023] Our paper on interpretable reinforcement learning via neural additive models was accepted at the AI4ABM Workshop at ICLR 2023.
• [Feb. 2023] We won the Energy Forecasting Challenge on Kaggle organized by Hessian AI.
• [Nov. 2022] Our paper on Bayesian Optimization Based Combinatorial Assignment was accepted at AAAI 2023.

Seminars / Lectures

• [April 2024] Is Mamba Capable of In-Context Learning? At the AutoML Online Seminar with Riccardo Grazzi (Youtube) (Extended Slides)

Publications

Powered by Jekyll and Minimal Light theme.