Breakfast Talk: When Histology is Painted

Dennis Grinwald & Tom Neuhäuser "When Histology is Painted: Evaluating PRISM on Synthetic, Non-Biological Slide Data"

Our Lunch Talk series is starting into the winter term with a special “Breakfast Talk” on October 29 from 10-11 a.m. at Schering Stiftung Berlin (Unter den Linden 32-34, 10117 Berlin). This time Dennis Grinwald and Tom Neuhäuser from the Machine Learning Group explore how PRISM, a state-of-the-art computational pathology foundation model, reveals it blind spots by showing how PRISM responds to watercolor paintings designed to mimic histological slides. In doing so they reveal both its adaptability and fragility while offering insights into preprocessing, architecture, fine-tuning, and the broader reliability of medical AI. The work was pivotal for the exhibition of BIFOLD’s Artists in Residence currently showing at Schering Stiftung Berlin (see abstract below).

Abstract: Foundation models in computational pathology, such as PRISM, demonstrate state-of-the-art capabilities in cancer detection, subtyping, and biomarker prediction by aggregating tile-level embeddings into slide-level representations. Yet, their application to data outside the medical domain provides a unique lens on their blind spots. In collaboration with the artistic project The Neverending Cure, we subjected PRISM—originally trained on hematoxylin and eosin (H&E)-stained whole slide images—to watercolor renderings intentionally painted in the aesthetics of histopathological specimens.

This talk will detail the preprocessing steps required to transform the painted “slides” into model-compatible inputs, followed by a technical overview of PRISM’s slide encoder architecture, including its use of Virchow tile embeddings, Perceiver-based aggregation, and BioGPT-driven report generation. We then examine the application of these preprocessed drawings to the model, presenting qualitative results that demonstrate how a high-performing diagnostic model can produce confident but spurious classifications when confronted with synthetic or “species-foreign” inputs. Additional discussion will address fine-tuning and model adjustments undertaken to improve performance and reduce overfitting in this unconventional setting.

By critically analyzing PRISM’s behavior on fabricated histology-like images, we highlight both the adaptability and fragility of modern pathology foundation models. Beyond its artistic provocation, this experiment offers methodological insights into preprocessing, architectural design, and model adaptation, while also raising broader questions about reliability and trust in medical AI.

Speakers

Dennis Grinwald: I am a third year PhD student in the Machine Learning Group at the Technical University of Berlin led by Prof. Klaus-Robert Müller and at the Berlin Institute for the Foundations of Learning and Data (BIFOLD). My research focuses on multi-task and distributed machine learning. LinkedIn

Tom Neuhäuser: is a doctoral researcher in the Machine Learning Group at Technische Universität Berlin. He received his master’s degree in computer science with a specialization in cognitive systems. Before joining the Machine Learning Group, he worked as AI Lead at the green-tech start-up Hortiya. Tom’s research focuses on the application of machine learning to other sciences, with a particular interest in modeling dynamical systems. Interests: Digital Pathology, Computational Neuroscience, Dynamical Systems. LinkedIn

__

The BIFOLD Lunch Talk series gives BIFOLD members and external partners the opportunity to engage in dialogue about their research in Machine Learning and Big Data. Each Lunch Talk offers BIFOLD members, fellows and colleagues from other research institutes the chance to present their research and to network with each other.

The Lunch Talk takes place at the TU Berlin. For further information on the Lunch Talks and registration, contact Dr. Laura Wollenweber via email.