KRAKEN Optimizes Distributed Stream Processing
Ariane Ziehn and her colleagues from the DIMA group at BIFOLD, led by Prof. Dr. Volker Markl, have been honored with the prestigious Best Paper Award at the 20th ACM International Conference on Distributed and Event-Based Systems (DEBS 2026) in Lisbon, Portugal. Presented at the conference by Ariane Ziehn and Finn Glück, their award-winning work introduces a method that jointly decides where the operators of a continuous query are executed and how their data is exchanged between nodes, resulting in execution plans that move far less data while remaining fast.
In distributed stream processing, applications continuously analyze event streams produced across many geo-distributed devices, often by combining events from different sources using stateful binary operators such as window joins and sequences. Executing such workloads efficiently depends on two tightly coupled decisions: operator placement (on which node each computation runs) and communication (how events are exchanged between nodes, for example by actively pushing them or by pulling them on demand). Most existing planners make these decisions sequentially, first fixing the placement and then optimizing the communication under that constraint. As a result, they systematically overlook better execution plans globally.
In their paper, the researchers propose KRAKEN, an execution planner that jointly optimizes both decisions. Instead of separating them, KRAKEN evaluates placement candidates under alternative communication schemes (all-push and push-pull) before committing to atomic deployment decisions. The resulting candidate decisions are compared using a bi-objective cost model that balances transferred tuple volume with latency. Across diverse synthetic topologies and workloads, KRAKEN reduces the number of tuples transferred up to a factor of 11, while also lowering latency and planning time compared to the sequential baseline. By optimizing placement and communication together, KRAKEN makes the decentralized execution of stream queries in dynamic fog-cloud environments markedly more efficient, which is essential for Internet of Things applications running continuous analytics across the sensor-edge-cloud continuum.
