BLISS AI Speaker Series #31: Scaling LLM Training to Thousands of GPUs

BLISS e.V. features Nouamane Tazi, who is currently a Research Engineer at HuggingFace and will discuss "Scaling LLM Training to Thousands of GPUs", lasting approximately 45 minutes. After the talk, seize the opportunity to connect with fellow AI enthusiasts to share ideas and questions while enjoying free drinks and pizza. Door close by 7.15pm, so please come early! Also, "attend"ing (RSVP) here on Meetup is strictly necessary to be guaranteed entry.
Please note that Meetup has recently been quite keen on promoting its Plus program. However, you are not obligated to purchase it, as both our events and the platform remain free.

Who is this event for?
This event is open to everyone interested in state-of-the-art AI research. We especially design it for students, PhD candidates, academic researchers, and industry professionals with a research focus in machine learning.

BLISS e.V. is, the AI organization in Berlin that connects like-minded individuals who share great interest and passion for the field of machine learning. This summer 2026, we will, again, host an exciting speaker series on site in Berlin, featuring excellent researchers from cohere, ETH Zürich, University of Oxford, HuggingFace, and Stanford University.

Abstract:

Training large language models at scale introduces a cascade of systems bottlenecks absent at smaller scales: from communication overhead and memory fragmentation to subtle numerical instabilities that surface only across thousands of devices. This talk covers the practical design choices behind scaling LLM training to thousands of GPUs: what parallelism strategies work (and when they break), how to keep training runs efficient and stable, and the engineering trade-offs that shape modern pretraining infrastructure. The presentation aims to be accessible to a broad ML audience, drawing on real-world experience from large-scale open-source training runs at Hugging Face.

Bio:

Nouamane Tazi is a Machine Learning Research Engineer at Hugging Face, specializing in training and scaling large language models. He is a co-author of SmolLM3 and The Ultra Scale Playbook, and his research spans NLP, deep learning, and scalable AI infrastructure.