Regularization of neural networks for quantum systems

Neural networks (NNs) applied to quantum systems offer exciting opportunities. Even for small molecules, NNs are orders of magnitude faster at predicting molecular properties than quantum chemical approximation methods. Moreover, the computational cost of such first-principles approximation methods scales superlinearly with the size of the system, which prevents their application to larger molecules. Since the computational cost of NNs scales only linearly with the system size, the simulation of large molecules with unprecedented accuracy is now becoming routine. From disciplines such as biophysics or materials science, to broad application areas like drug development, many fields will benefit from the incorporation of NN-based simulations. The enormous expressive capabilities of NNs are at the same time their strength and their weakness. There seems to be no functional relationship that NNs cannot approximate well, but their learned representations are sometimes not in line with either human intuition or the laws of physics. This leads them to occasionally extrapolate poorly outside of their training data regime. The goal of this thesis is to develop regularization techniques that incorporate known quantum chemical principles into NN models. In order to perform effective regularization, we find that a deep understanding of the model is beneficial. To this end, we show how to adapt Explainable Artifical Intelligence (XAI) techniques to models for quantum systems and uncover in detail the learned prediction strategies. Two examples for the measures we propose are a) the size of the molecular context that an NN bases its prediction on and b) the many-bodyness, which is the degree to which the molecular context modifies the interaction strength between two given atoms. Our results indicate that many known chemical principles have been learned by the models remarkably well. At the same time, we identify several key shortcomings. One of these is the finding that NN predictions are in severe violation of Newton's third law. A regularization technique we develop fixes this issue and improves the generalization capability both quantitatively and qualitatively. We also show that NNs are not confined to virtual simulation of quantum systems. In a real-world molecular manipulation task, we use Reinforcement Learning (RL) to let an NN-agent steer the tip of a Scanning Probe Microscope. The manipulation task consists of removing a molecule from a surface, which requires finding a suitable removal trajectory. We find that existing RL methods are not efficient enough for the agent to solve the task in the required timeframe. To address this issue, we devise two regularization methods that significantly improve the data efficiency of the agent, which enables it to solve the task autonomously. In summary, we show that domain-specific regularization methods for NNs applied to quantum systems are a key ingredient towards enabling wide adoption of NN-based atomistic modeling.