Controlling Autonomous Vehicles using Continuous Adversarial Patches
Autonomous vehicles, including self-driving cars and drones, heav ily rely on machine learning models for perception and control. This reliance introduces a significant attack surface through adver sarial patches—visual patterns that mislead learning models. While prior work has demonstrated that such patches can induce misclas sifications or tracking failures, they fall short of enabling sustained control over autonomous systems in rapidly changing conditions. In this paper, we introduce continuous adversarial patches, a proof-of-concept attack that demonstrates the potential for sus 3 Felix Weißberg BIFOLD & TUBerlin Berlin, Germany Konrad Rieck 2 BIFOLD & TUBerlin Berlin, Germany 1 1 2 3 tained control over autonomous vehicles that rely on single object tracking. When displayed on a video screen, these dynamic patches Figure 1: A person-following drone is accelerated along guide tracking drones along adversary-defined trajectories. We implement a diffusion model to produce continuous adversarial patches in real time, targeting PULP-Frontnet and YOLOv5, two object detection models used in drone research. In simulation, our method demonstrates successful trajectory manipulation under con trolled conditions, including slingshot and freestyle maneuvers. We release our simulation environment and code to facilitate further investigation of this attack class and the development of robust countermeasures. Our code is available for further research under github.com/mlsec-group/continuous-patches.
