Work & Research
I study reinforcement learning with deep neural networks and focus on transformer-based agents, representation learning, and interpretability. Read more about my research directions and current projects on the dedicated page.
Jannis Blüml
[ˈjanɪs ˈblyːml̩] | (er/he)
PhD student in computer science at TU Darmstadt and hessian.AI.
Reinforcement learning, representations, and interpretable agents.
Profile
I am a PhD student in computer science focused on reinforcement learning and representation learning. I collaborate with Kristian Kersting, Johannes Czech, and Quentin Delfosse in the Artificial Intelligence and Machine Learning Lab.
CV updated: 2025-12-28
Current Focus
Finishing my PhD
Currently I plan to finish my PhD thesis and submit it for defense end of 2026.
Searching for Research Visitations
For 2026, I am looking for research visitations to further my work in reinforcement learning and representations.
Education
PhD at TU Darmstadt
2021-present
M.Sc. Computer Science and M.Sc. Internet and Web-based Systems
2018-2023
Two Master of Science degrees at TU Darmstadt. My thesis explored multi-agent reinforcement learning in uncertain environments using Stratego as a case study.
B.Sc. Computer Science
2014-2018
Undergraduate work on visualizing error in dimensionality reduction, developed with the Visual Analytics group at Fraunhofer IGD.
Extracurricular Activities
Scientific Staff Representative
Department council member for Computer Science and part of the senate committee for teaching.
Student Council
Member of the Computer Science student council between 2015 and 2021.
German Children's Literature Award
Served on the young adult jury (2014-2018) to foster public engagement with children's literature.
Interests
Outside research, I recharge through food, stories, strategy games, and photography.
Cooking & Baking
Experimenting in the kitchen, from slow Sunday bakes to improvised weeknight recipes, often vegan.
Creative Writing
Not sure where this will go, currently interested in Young Adult stories usually sparked by sci-fi and fantasy worlds.
Photography
Portraits, travel, and quiet details — mostly with a Sony A7III.
Chess & Boardgames
Chess, Shogi, Xiangqi, and Go. In general, I am a fan of all kind of board games. Always up for a friendly game.
Selected Projects
Representation Matters
While transformers have gained the reputation as the "Swiss army knife of AI", no one has challenged them to master the game of chess, one of the classical AI benchmarks. Simply using vision transformers (ViTs) within AlphaZero does not master the game of chess, mainly because ViTs are too slow. Even making them more efficient using a combination of MobileNet and NextViT does not beat what actually matters: a simple change of the input representation and value loss, resulting in a greater boost of up to 180 Elo points over AlphaZero. The results can be found here.
Object-centric Environments for Atari 2600 (OCAtari)
The Arcade Learning Environments platform is the most used set of environments to train deep RL agents. It provides an easy-to-use set of diverse Atari 2600 games, that require different skills to be mastered. Having object-centric environments would help scientists both develop object extraction methods, and interpretable RL trained agents. As ALE is the most used platform to evaluate deep RL algorithm, an object centric ALE would allow comparing existing methods to object-centric explainable ones. The results can be found here. The code is on GitHub. This work was also accepted at the first Reinforcement Learning Conference (RLC).