Towards scalable and transparent ML algorithms
Recent advances in machine learning (ML) have enabled the development of transformative applications, including protein structure prediction and natural language processing. These achievements largely stem from scaling model capacity, dataset size, and computational resources. However, this rapid progress has come at the cost of interpretability, making these systems increasingly opaque to human understanding. As transparency becomes more crucial—driven by both legislative mandates like the EU AI Act and the scientific needs to understand learned representations—this thesis seeks to bridge the gap between scalability and transparency from two opposing perspectives. First, we promote transparency for complex ML models through Explainable AI (XAI), while ensuring scalability of the overall approach. We consider occlusion-based methods to provide versatile insights independent of the ML model. These methods rely on partially occluding samples and observing the resulting changes in the model prediction. Here, pixel flipping (PF) benchmarks allow to rank XAI methods by occluding features depending on their attributed relevance. However, different occlusion strategies lead to inconsistent rankings for conventional PF measures. To circumvent this, we develop an improved PF measure that breaks this inherent dependence and ensures a consistent ranking of XAI methods regardless of the underlying occlusion strategy. By removing reliability requirements on the occlusion strategy, this ensures consistent results at minimal computational cost. We also consider Prediction Difference Analysis (PredDiff) to obtain occlusion-based explanations. PredDiff is minimally invasive as only target features of interest are occluded, ensuring reliability and computational efficiency of the overall approach. While retaining these essential properties, we develop a framework for higher-order attributions. This allows to explain the interactive structure of ML models, thereby increasing the level of transparency.
