Julien Siems | University of Freiburg

About Me

I am a Ph.D. student at the University of Freiburg advised by Prof. Frank Hutter.

After graduating from the University of Freiburg in 2020 with distinction, I was an Applied Science Intern at AWS Berlin working on Neural Network Pruning, I worked on Bayesian Optimization for combinatorial auctions at the University of Zurich, did research into numerical optimization for inverse problems at TU Munich and worked as a Machine Learning Researcher at Merantix Momentum on Interpretable Machine Learning.

Research Interests

Linear RNNs / State-Space Models, State-Tracking, In-Context Learning

News

[Oct. 2025] “TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting” accepted at Tabular-AI Workshop at EurIPS 2025.
[Sept. 2025] I am interning at Microsoft Research Cambridge from Sept. to Nov. 2025.
[Sept. 2025] “DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Products” is accepted at NeurIPS 2025.
[Mar. 2025] “DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Products” is accepted as Oral at FM-WILD Workshop at ICLR 2025.
[Feb. 2025] “Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues” is accepted as an Oral at ICLR 2025.
[Oct. 2024] 3 papers accepted at NeurIPS workshops: “Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues” (Oral), “GAMformer” (Oral), and “Mamba4Cast” (Spotlight).
[Jul. 2024] I co-organized the 1st Workshop on In-Context Learning (ICL) at ICML 2024 alongside Frank Hutter, Erin Grant, Beyza Ermiş, Jelena Bratulić and Noah Hollmann.

Seminars / Lectures

[Mar. 2025] State-Tracking in Linear RNNs at the Advances in Sequence modeling from Algorithmic Perspectives Seminar (Youtube) (Slides)
[Jan. 2025] Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues at the AutoML Seminar. (Youtube) (Slides)
[Apr. 2024] Is Mamba Capable of In-Context Learning? At the AutoML Online Seminar (Youtube) (Extended Slides)

Publications

Tabular WS@EurIPS

TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting

Vladyslav Moroshan*, Julien Siems*, Arber Zela, Timur Carstensen, Frank Hutter (*equal contribution)

AI for Tabular Data Workshop @ EurIPS 2025

PDF Code

NeurIPS

DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Products

Julien Siems*, Timur Carstensen*, Arber Zela, Frank Hutter, Massimiliano Pontil, Riccardo Grazzi* (*equal contribution)

Neural Information Processing Systems (NeurIPS) 2025

PDF

ICLR (Oral)

Unlocking State-Tracking in Linear RNNs through Negative Eigenvalues

Riccardo Grazzi*, Julien Siems*, Jörg Franke, Arber Zela, Frank Hutter, Massimiliano Pontil (*equal contribution)

International Conference on Learning Representation 2025 (Oral)

PDF

Interpretability@NeurIPS (Oral)

GAMformer: In-Context Learning for Generalized Additive Models

Andreas Müller*, Julien Siems*, Harsha Nori, David Salinas, Arber Zela, Rich Caruana, Frank Hutter (*equal contribution)

NeurIPS Workshop on Interpretable AI Past, Present and Future (Oral)

PDF

TS@NeurIPS (Spotlight)

Mamba4Cast: Efficient Zero-Shot Time Series Forecasting with State Space Models

Sathya Kamesh Bhethanabhotla*, Omar Swelam*, Julien Siems, David Salinas, Frank Hutter (*equal contribution)

NeurIPS Workshop on Time Series in the Age of Large Models (Spotlight)

PDF

ME-FoMo@ICLR

Is Mamba Capable of In-Context Learning?

Riccardo Grazzi*, Julien Siems*, Simon Schrodi, Thomas Brox, Frank Hutter (*equal contribution)

ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models (ME-FoMo) and AutoML Conference 2024

PDF

NeurIPS

Curve Your Enthusiasm: Concurvity Regularization in Differentiable Generalized Additive Models

Julien Siems*, Konstantin Ditschuneit*, Winfried Ripken*, Alma Lindborg*, Maximilian Schambach, Johannes S. Otterbach, Martin Genzel (*equal contribution)

Neural Information Processing Systems (NeurIPS) 2023

PDF Code

AI4ABM@ICLR

Interpretable Reinforcement Learning via Neural Additive Models for Inventory Management

Julien Siems, Maximilian Schambach, Sebastian Schulze, Johannes S Otterbach

Workshop AI4ABM at International Conference on Learning Representations (ICLR) 2023

PDF

AAAI

Bayesian Optimization-Based Combinatorial Assignment

Jakob Weissteiner*, Jakob Heiss*, Julien Siems*, Sven Seuken (*equal contribution)

AAAI 2023

PDF Code

IJCAI-ECAI

Monotone-Value Neural Networks: Exploiting Preference Monotonicity in Combinatorial Assignment

Jakob Weissteiner*, Jakob Heiss*, Julien Siems*, Sven Seuken (*equal contribution)

International Joint Conference on Artificial Intelligence & European Conference on Artificial Intelligence 2022

PDF Code

ICLR

Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarks

Arber Zela*, Julien Siems*, Lucas Zimmer*, Jovita Lukasik, Margret Keuper, Frank Hutter (*equal contribution)

International Conference on Learning Representation 2022

PDF Code

AutoML @ ICML

Dynamic Pruning of a Neural Network via Gradient Signal-to-Noise Ratios

Julien Siems, Aaron Klein, Cedric Archambeau, Maren Mahsereci

8th ICML Workshop on Automated Machine Learning (AutoML) at ICML 2021

PDF

ICLR

NAS-Bench-1Shot1: Benchmarking and dissecting one-shot neural architecture search

Arber Zela*, Julien Siems*, Frank Hutter (*equal contribution)

International Conference on Learning Representation 2020

PDF Code

CVIU

Faster training of Mask R-CNN by focusing on instance boundaries

Roland Zimmermann*, Julien Siems* (*equal contribution)

Computer Vision and Image Understanding Journal

PDF Code