E-learning Platform Scalability Solution

Architecture scalability for a fast-growing company challenges

E-learning Platform Scalability Solution

Challenge

Finding a new approach for better scalability of the solution

Read more

Solution

Changing the architecture of a crucial part of the system, and building a new HTTP interface

Read more

Result

Better scalability, reduced number of queues, user session location transparency

Read more

Challenge

As an e-Learning company, Sumdog has developed an e-Learning system with a personalized learning experience for math and spelling practice. What’s special about the Sumdog platform is it achieves great learning results from fun, more specifically multi-player games that are proven to accelerate progress.

We dove into the existing environment built by Sumdog, and we created the solution using an Akka environment, combined with RabbitMQ working as an Enterprise Service Bus, to assure communication along with the rest of the components in the solution. Scalac engineers worked together with Sumdog to find a new approach for better scalability of the solution. Scalability has become one of the most important issues in Sumdog, as with every fast-developing company, due to the growing number of customers using the software, the evolution of existing features, as well as increasing requirements when it comes to adding new features.

Solution

At the beginning of the project, two main goals were set:
  1. To change the architecture of a crucial part of the system, to achieve scalability, by among other things:
    ▸ reducing communication queues
    ▸ providing location transparency for each user session
  2. To build a brand new HTTP interface that complements the outcomes of the previously mentioned improvements

Akka Cluster approach
To address the first project goal, we performed a detailed examination of components, taking into account the complex architecture of the existing system. After analyzing the initial situation, we decided, together with Sumdog engineers, to go in the direction of an Akka Cluster approach, which would eventually lead us to better scalability via architectural evolution rather than revolution.

The change in the architecture introduced clustering built on Akka Cluster, and resulted in:

▸ Reduction in the number of queues in RabbitMQ from thousands to just a few
▸ Smoothing the process of scaling up & down – seed nodes of a cluster working as EC2 / Docker machines in Auto-Scaling Groups in an AWS environment
▸ Using Sticky Session with RabbitMQ – to avoid unnecessary communication between nodes. The communication is driven directly to the desired node, which handles a particular user session
▸ Split Brain Resolver – to let nodes have a self-strategy how to act accordingly to changing conditions inside a cluster, to let them have control over EC2 instances (to tear it down)
▸ Support for both RabbitMQ and HTTP mode at the same time

Akka HTTP approach
To address the second project goal, and to provide a new HTTP direct interface for customers, we used a natural approach for the Akka environment and utilized the Akka-HTTP toolkit which provides HTTP-based services. After achieving goal no 1, Akka-HTTP allowed us to create convenient endpoints to reach backend logic from the frontend.

People who played a crucial role in the facilitation of the entire release process
Our Agile team:

▸ Development Engineers (Scalac):
   ▸ Scala Architect / Akka Cluster Leader
   ▸ Senior Scala Engineer / Akka Cluster
          ▸ There were two experienced Scala & Akka Cluster engineers, who were committed to Sumdog’s needs,
▸ They carefully took into consideration the existing solution and spent time before starting the project on proposing
    the best approach based on their comprehensive knowledge,
   ▸ DevOps Engineer / AWS & Cloud Engineer
          ▸ We also engaged a DevOps engineer to prepare a relevant AWS cloud environment, to demonstrate the solution.

The Client’s team:

   ▸ Team Lead / Senior Software Engineer (Sumdog)
   ▸ Senior Software Engineer (Sumdog)
   ▸ DevOps Engineer (Sumdog)

Result

After the successful implementation of the proposed Akka Cluster approach, our engineers demonstrated the principles of a developed cluster solution in a dedicated environment:

   ▸ Seamless routing of communication between shards and nodes
   ▸ Self-managing nodes of a cluster and self-healing
   ▸ Implementation of a strategy taking into account a load of each instance
   ▸ Split Brain Resolver strategy – to ensure the right behavior of particular nodes, during the partition of the network, we implemented a self-strategy for every node/EC2 instance created

Our solution achieved our agreed goals and the following key benefits:

 Scalability (main project goal):
   ▸ better scalability,
   ▸ reduced number of queues,
   ▸ user session location transparency
   ▸ auto Scale-up & scale-down implemented on both sides: AWS cloud deployment and
     self-strategy for every type of node (seed/dynamic) of the cluster
▸ New HTTP Interface (second project goal):
   ▸ brand new HTTP interface corresponding to the improved shape of the system
▸ Canary environment:
   ▸ possibility of showing a demo version of an engineered cluster solution
   ▸ possibility of verifying new versions of the application on a production cloud
     environment, accessible for a determined group of chosen users

Let’s talk about your project

Drop us a line

Learn more

Scalac worked with over 80 companies around the world.
Find out more about our consulting and development solutions.
See how our team contributed to customers’ success.

E-learning Platform Optimization Outsource Scala Development

E-learning Platform Optimization

The client expanded the development team in an established technology with a strong team of developers, alleviating the challenges and costs of recruiting at scale in a competitive market in San Francisco.
Custom Storage System Solution

Custom Storage System Solution

With a significant number of devices under constant analysis to intelligently find out if something is wrong or needs handling, Indeni needed a way to ingest a high rate of metrics into its system for storage and display.
MVP Budgeting Application

Budgeting MVP Application for NGOs

The app not only simplifies management but also allows control of the budget, which is crucial in every money to value system, identifying the needs, and distributing funds accordingly.