Ensuring that the placement of continuously changing queries in a massively distributed and volatile environment remains consistently valid.
Ankit Chaudhary and his colleagues from the DIMA group at BIFOLD, led by Prof. Dr. Volker Markl, have been honored with the prestigious Best Paper Award at the 41st IEEE International Conference on Data Engineering (ICDE) in Hong Kong. Their award-winning work proposes a method to ensure that the placement of continuously changing queries in a massively distributed and volatile environment remains consistently valid.
Commonly known as the operator placement problem in the database literature, this challenge involves determining optimal locations for query operators to maintain correctness and efficiency under dynamic conditions. In particular, the frequent arrival or removal of queries and potential volatility of the infrastructure might invalidate or reduce the efficiency of previous operator placement decisions and thus might lead to constant, expensive query re-optimizations. Therefore, these changes require new solutions for operator placement, which adjust existing placement decisions upon changes to the queries and infrastructure.
In their paper, researchers proposed ISQP, a framework that keeps operator placements valid under query and infrastructure changes. ISQP performs a fine-grained identification of invalid operator placements and takes concurrent, incremental placement decisions to reduce the optimization time. ISQP works for arbitrary placement strategies, making it a general-purpose framework. Our evaluations show that ISQP reduces the optimization overhead by one order of magnitude compared to the baseline. This work was conducted in the context of NebulaStream, an IoT data management platform developed by researchers at DIMA. A key feature of NebulaStream is its ability to operate seamlessly across unified sensor–edge–cloud infrastructures, which are inherently dynamic and volatile. ISQP enables NebulaStream to maintain valid workload placement under these conditions while incurring minimal re-optimization overhead. As a result, NebulaStream becomes a practical and efficient data management solution for highly volatile infrastructures.
Paper: Incremental Stream Query Placement in Massively Distributed and Volatile Infrastructures
Authors: Ankit Chaudhary, Kaustubh Beedkar, Jeyhun Karimov, Felix Lang, Steffen Zeuch, Volker Markl
LINK
At ICDE 2025, various BIFOLD research groups showcased diverse work. The following pictures provide a glimpse of the conference.
