Gazing into the Black Box AI

.

Explainable Artificial Intelligence

Artificial intelligence is already making decisions in many places, often unnoticed and without us really understanding the base of these decisions. AI is a black box - there is still a lot of research, which needs to be done. The research field that delves deeper into this question is called Explainable Artificial Intelligence, or XAI. At the Open Day of the Federal Ministry of Education and Research (BMBF) researchers from Fraunhofer HHI and BIFOLD explained how a neural network calculates its answers or makes decisions using the example of an AI-supported image evaluation process. This demonstration was called: „Ein Blick in die Blackbox Künstliche Intelligenz“.

There are many different AIs. One example are AI-supported image evaluation procedures used in cancer diagnostics. The AI can support the physician in the assessment of cell photographs. It can calculate whether the cells shown are diseased or healthy based on pictures. A doctor can use this to decide how to treat a patient further.

The presentation shown at Open Day was more straightforward. Here, the focus was not on cell images but on photos of everyday life. The presentation showed that an AI calculates its answers in probabilities and which elements of the image are used for this calculation. The evaluation started with the selection of a photo. Then, using a keyboard, one entered a simple question about the selected image. For example, on a picture showing two people, one might have asked, "Is the person on the left female?" Then, the AI answered its response on a screen. But it didn't respond with a simple "yes" or "no" answer, but it indicated a probability in percentage: Yes (70%), No (28%), Can be (1%). In addition, the AI showed which parts of the image were relevant to its decision by fading all irrelevant details to white.

"Ein Blick in die Blackbox Künstliche Intelligenz" was just one of many presentations at the Open Day of the Federal Ministry of Education and Research. BIFOLD thanks Leila Arras, Lorenz Hufe, Bruno Puri, and Dr. Ekkehard Schnoor who presented the research during the Open Day.
The presentation is based on the "Dual Recurrent Attention Units for VQA" research work, which was funded by the BMBF via Fraunhofer HHI and BIFOLD. Link: https://arxiv.org/abs/1802.00209. A selection of XAI demonstrations can be found here: https://lrpserver.hhi.fraunhofer.de/.