Research for Cybersecurity in the Era of Generative AI
With the innovative research project "AIgenCY - Opportunities and Risks of Generative AI in Cybersecurity," leading experts from academia and industry take on the challenge of exploring the implications of generative artificial intelligence (AI) for cybersecurity.
The project AIgenCY, supported by the Federal Ministry of Education and Research (BMBF), brings together specialists from the CISPA – Helmholtz Center for Information Security, the Berlin Institute for the Foundations of Learning and Data (BIFOLD) at TU Berlin, and the FU Berlin under the leadership of the Fraunhofer Institute for Applied and Integrated Security AISEC, and collaborates with the pioneering AI company, Aleph Alpha (Heidelberg). Their collaborative goal is to analyze the complex effects that generative AI has on cybersecurity, particularly those concerning critical national infrastructures.
Rapidly advancing developments in generative AI, particularly in neural networks, are revolutionizing the creation of digital content. Generative AI systems are already capable of generating and refining authentic texts, visual content, and complex program codes. These capabilities open up new opportunities but also pose significant risks – especially to the cybersecurity of critical national infrastructures and the foundations of our democracy. AIgenCY is consequently dedicated to identifying potential threats posed by AI-generated content and developing robust defense strategies to strengthen Germany's digital sovereignty while protecting critical infrastructures.
BIFOLD researcher Prof. Dr. Konrad Rieck, chair of Machine Learning and Security, explains: “As part of AIgenCY, we will scrutinize the latest AI technologies and examine their possibilities and risks for IT security. I’m confident that we will encounter some surprises along the way."
As part of the project, an experimental lab will be established to explore the applicability of generative AI technologies in practical scenarios. The research team will focus on the following core topics:
• Malware defense: Analysis of the generation of advanced malware by AI and the development of corresponding countermeasures.
• Prevention of Social Engineering, Disinformation, and Fraudulent Campaigns: Early detection and tracking of automated attacks.
• Information Collection and Processing (Reconnaissance): Understanding preventative attack strategies and their defense.
• Explainability and Inference: Interpretation of the decision-making of generative AI models and identification of systemic vulnerabilities.
• Watermarking and Anomaly Detection: Tracing data origin and detecting unusual patterns that may indicate potential attacks.
AIgenCY aims to: significantly contribute to the protection of critical infrastructure, strengthen Germany as a location for innovation, deepen expertise in the field of cybersecurity, and support the local economy. Thus, the project will play a crucial role in addressing societal challenges that arise from new AI technologies.
"With the AIgenCY project, we have the opportunity to set the course for a secure digital future and advance AI innovations for a resilient society," emphasizes Prof. Dr. Claudia Eckert, director of the Fraunhofer AISEC. "With funding of five million euros from the BMBF over the next three years, AIgenCY is well on its way to significantly shaping this relevant field of research and innovation."