I study reinforcement learning with deep neural networks, focusing on transformer-based agents, representation learning, and interpretability. My work is part of the 3AI Project between hessian.AI and the Artificial Intelligence and Machine Learning Lab at TU Darmstadt.
Quick navigation:
A snapshot of the themes that guide my current work.
Learning compact and semantically meaningful state representations that make decision-making more data-efficient and robust.
Exploring attention-based architectures for long-horizon reasoning and generalization in RL.
Understanding why agents act the way they do by emphasizing object-centric views and causal cues.
Designing agents that remain reliable under distribution shifts and noisy, imperfect environments.
Grouped by venue type for quick scanning.
Selected ongoing work and recently completed projects.
The Arcade Learning Environments platform is widely used to train deep RL agents. Object-centric environments help build interpretable agents and compare against object-aware baselines. The results can be found here. We also develop new methods to adapt environments to search for misalignments and shortcut learning, enabling more robust agents while focusing on object-centric input representations.
While transformers have gained the reputation as the "Swiss army knife of AI", no one has challenged them to master the game of chess, one of the classical AI benchmarks. Simply using vision transformers (ViTs) within AlphaZero does not master the game of chess, mainly because ViTs are too slow. Even making them more efficient using a combination of MobileNet and NextViT does not beat what actually matters: a simple change of the input representation and value loss, resulting in a greater boost of up to 180 Elo points over AlphaZero. The results can be found here. Further we researched how information in games like Stratego (Information Sets) or Hex (Graph-based Inputs) can be represented to enable methods like AlphaZero to work.