Reflecting on the blind spots of medical AI systems
The artist duo kennedy+swan is BIFOLD artist-in-residence 2024/2025
The exhibition "THE NEVERENDING CURE" was the result of the residency of Berlin-based artists Bianca Kennedy and Felix Kraus, collaborating with BIFOLD scientists and supported by the Schering Foundation.
The use of AI in medicine comes with big promises: AI is supposed to alleviate illness, reduce suffering, and extend our lifespan. Besides the immense technological potential, it is, however, important to reflect on social, ethical, and philosophical issues. What happens when sensitive health data are used to train medical AI models? None of the current systems are error-free. They hallucinate and tend to provide plausible diagnoses instead of admitting insecurities.
One such hallucination is at the center of kennedy+swan’s work The Neverending Cure. An AI model specialized in recognizing lung cancer in tissue scans is submitted to a both playful and revealing experiment. With the support of researchers from BIFOLD (Naima Elosegui Borras, Tom Neuhäuser, Dennis Grinwald), kennedy+swan exposed the algorithm to watercolors on glass – painted in the aesthetics of microscopic specimens. What looks like fine structures of real lung tissue to the human eye is recognized by the AI application as potentially diseased tissue and diagnosed as such with astonishing confidence.
In their artistic research, kennedy+swan reflect on the blind spots of medical AI systems. These systems are built on datasets that are not neutral; instead, they reflect societal dynamics complete with their contradictions and tensions. Among other things, the duo explores the distinction between “healthy” and “sick” and raises the question: Should machines also be trained with “species-foreign” content to reliably differentiate between these two conditions? How far are we from a diagnostic AI that we can trust wholeheartedly?
The watercolor lightboxes that resemble pneumotomies are supplemented by animated films that dive deeper into the microcosm of the painted tissue landscapes. From the AI analysis of the images, to a hand-crafted, extra-corporal lung, to the complete immersion in the watercolors’ “cell structures,” we get closer to the feeling of opening up the body to artificial intelligence. Already tomorrow, today’s challenges could provide crucial impulses to deliver eternal healing – fed by ever-new data that will liberate us, step by step, from our physical suffering forever. THE NEVERENDING CURE.
The opening featured BIFOLD Co-Director Klaus-Robert Müller, who welcomed the guests, Frederick Klauschen, Director of Pathology, LMU Munich, Group Leader BIFOLD/Charité – Universitätsmedizin Berlin, who gave a brief insight into the rapidly advancing use of machine learning and big data management in medicine, as well as the BIFOLD scientists, Dennis Grinwald, Naima Elosegui Borras and Tom Neuhäuser, who collaborated closely with the artists.
Photo Recap