TU Berlin’s student Hendrik Makait reached the 2nd place at the ACM SIGMOD 2020 Student Research Competition at the 2020 ACM SIGMOD/PODS International Conference on the Management of Data [1] with his paper “Rethinking Message Brokers on RDMA and NVM” [2]. This is a joint work with the Data Engineering Systems Group at the Hasso Plattner Institute in Potsdam led by Prof. Rabl [3].
Short abstract:
Over the last years, message brokers have become an important part of enterprise systems. As microservice architectures become more popular and the need to analyze data produced by the individual services grows, companies increasingly rely on message brokers to orchestrate the flow of events between different applications as well as between data-producing services and stream processing engines that analyze the data in real-time. Current state-of-the-art message brokers such as Apache Kafka or Apache Pulsar were designed for slow networks and disk-based storage. In this work, we propose a new architecture that leverages remote direct memory access (RDMA) and non-volatile memory (NVM) to improve the weaknesses of existing message brokers and further scale these systems.
[1] ACM SIGMOD 2020 Student Research Competition
[2] ACM Paper “Rethinking Message Brokers on RDMA and NVM”
[3] Data Engineering Systems Group HPI Prof. Dr. Tilmann Rabl