Building an Ultra-Low Latency Message Broker Supporting Tree Data Structures

Scalac provided IT consulting and software development services on a challenging R&D project in the sports betting space. The project involved designing and building an in-memory message broker service, creating solutions to technical issues, and conducting testing. Scalac’s iterative approach enabled them to comprehend and clarify requirements quickly and engineer solutions to problems that arose during the project. They minimized latency, which is critical for the client’s business operation, and gained a very solid codebase that solved core problems. 

1. The message broker is capable of processing 50k leaf updates per second.
2. The system needed to deliver updates to subscribers in less than 1 ms after receiving an update to its internal structure.
3. Implementing a raw TCP protocol led to better results in load testing.
4. The optimization of memory usage significantly reduced GC time.
5. The solution delivered a simple and elegant code base with a refined DSL allowing for ease of use.
6. The server is stable and can handle thousands of leaf updates per second.

Ematiq, a cutting-edge technology company, was at the forefront of innovation in their industry. They recognized the need for a high-performance message broker that could facilitate seamless communication between their backend services. To maintain their position as industry leaders, they needed a message broker to handle updates to an in-memory data structure with lightning-fast speed as well as to deliver messages to relevant subscribers in less than a millisecond after receiving an update. This challenge presented an exciting opportunity for the Ematiq team to develop a customized solution to meet their specific needs and exceed industry standards. As a result, they approached the task enthusiastically, eager to create a solution that would push the boundaries of what was possible.

Ematiq turned to Scalac, a software house with expertise in building customized solutions to complex problems. Scalac’s team of developers listened carefully to Ematiq’s requirements and understood the complexity of their needs. The project’s most critical aspect was the message broker’s performance. Subscriber updates had to be delivered in less than 1 ms after receiving the publisher’s update. Moreover, the team had to respect the order of inputs, which meant that updates to the data structure and notifications to subscribers had to be made sequentially. Another challenge was to ensure that services could subscribe to this structure with paths supporting wildcards.

Scalac’s team underwent several stages of development, at each stage learning something new that allowed them to further refine their solution in the name of ever-increasing performance. The message broker application was written as a series of typed Akka Actors. Network communication was implemented using Akka gRPC, allowing them to generate a substantial amount of scaffolding code and to implement the most critical actors more quickly. The actual architecture of the actors also underwent several phases  After the system was implemented using gRPC and the load testing was undertaken, they quickly discovered they were not getting their desired performance.

Many hours of debugging of Akka’s gRPC implementation led them to conclude that the gRPC performance was buffering logic, sometimes buffering incoming messages for over 5 ms before releasing the entire buffer. This meant that messages needed to be released to their implementation faster when they had a lot of incoming requests. Instead, they decided to go to a lower level of execution to control the runtime environment. This led them to develop a raw TCP implementation.

After the TCP protocol development and implementation were completed, the team undertook another round of load testing. The results were much more promising this time, but they still could detect slow performance at times. Profiling the JVM revealed a correlation between the garbage collector activities and periods of subpar performance for their message broker. Therefore,  the next development step was to optimize the application’s memory usage. They could significantly reduce GC time by coalescing their models into a single unified type and implementing their protobuf decoders and encoders, validating the anemic model straight from raw bytes.

Finally, the team delivered a simple and elegant code base with a refined DSL allowing for ease of use. They also provided a stable server capable of handling thousands of leaf updates per second, a pair of optimized clients to communicate with the server, and a complete CI/CD pipeline with nearly complete unit and integration test coverage of both the clients and server. The client was pleased with the solution they delivered, meeting all of their requirements.

Through this experience, Ematiq realized the value of partnering with a software house that could build a customized solution tailored to their specific needs. As a result, Scalac was able to help Ematiq overcome the technical challenges they were facing and deliver a solution that was stable, performant, and met their requirements. As a result, Ematiq was able to improve its backend services and better serve its customers.