ERASE -- A Real-World Aligned Benchmark for Unlearning in Recommender Systems
Machine unlearning (MU) enables the removal of selected training data from trained models, to address privacy compliance, security, and liability issues in recommender systems. Existing MU benchmarks poorly reflect real-world recommender settings: they focus primarily on collaborative filtering, assume unrealistically large deletion requests, and overlook practical constraints such as sequential unlearning and efficiency. ERASE spans three core tasks -- collaborative filtering, session-based recommendation, and next-basket recommendation -- and includes unlearning scenarios inspired by real-world applications, such as sequentially removing sensitive interactions or spam. The benchmark covers seven unlearning algorithms, including general-purpose and recommender-specific methods, across nine public datasets and nine state-of-the-art models. Over 600 GB of reusable artifacts were produced, including extensive experimental logs and more than a thousand model checkpoints. ERASE showcases that approximate unlearning can match retraining in some settings, but robustness varies widely across datasets and architectures.
