Migrating & building a new product for all-in-one developer platform

Scalac partnered with Conduktor to help migrate its legacy code base to Scala and help building a new product around end-to-end testing. Our developers worked seamlessly with Conduktor’s in-house team to overhaul an outdated system, deliver new features, and drive architectural improvements.

Conduktor is an all-in-one developer platform that streamlines the process of working with Apache Kafka. Founded on a love of Kafka, Conduktor is a remote company (based in USA) that employs professionals from all around the world. 

The primary challenge faced by Conduktor was that its system’s legacy Java code was causing errors, particularly in relation to routine maintenance tasks. 

One striking example involved their end-to-end testing product that enabled users to test Kafka-based applications with building blocks like “ProduceToKafka,” “ConsumeFromKafka,” “DoHTTPCall,” etc. 

Conduktor enlisted Scalac to help with the following tasks:

• Transfer parts of the old code base to Scala infrastructure, with a view to improving code quality, expanding test coverage, and streamlining the maintenance of the legacy system. 
• Increase the number of building blocks available for testing pipelines, allowing users to test more cases. 
• Develop additional features for Conduktor’s testing app, for example running testing pipelines in private environments. 
• Streamline processes like bug fixes and identification of performance issues. 

By building on work already completed on Conduktor’s migration, Scalac helped deliver the following outcomes: 

• Set up a smart migration proxy that improved on the existing simple proxy. 
• Created infrastructure in which Conduktor could selectively use both the migrated and legacy versions and track the results of migrated endpoints,
• Enabled an incremental migration of endpoints to better monitor results and troubleshoot issues produced by the new code. 
• Added an API to encode/decode Scala typed to/from jsonb PostgreSQL using quill.
• Implemented “single request to stream response”  client-server communication using a long-live gRPC bi-directional connection.

Scalac also worked more generally with Conduktor’s team to propose changes to the platform architecture, a partnership that is currently ongoing. 

Remote Scalac developers, experienced in both Scala and Java, worked with Conduktor on the legacy code migration as part of the in-house team. 

Scalac’s successful approach was made possible by the following factors:

• Conduktor provided access to infrastructure and tools like source code, development environments, software, and so on. All Scalac employees are trained to adhere to high-level security protocols.  
• Communication was through a mixture of regular and on-demand meetings, alongside Slack and email channels. Scalac developers were readily available to talk to Conduktor team members during working hours. 
• Work was undertaken in Linear and Kanban. Regular code reviews were performed before updates were merged into the main code base. 
• Scalac maintained a flexible approach, and developers could apply their skills to different projects (in conjunction with the code migration) when needed, such as developing customer portal features. 

Conduktor partnered with Scalac in December. In May, new application functionality was launched without any issues. 

Scalac delivered the following outcomes:

• Migrated 38 of 58 endpoints over a three-month period. 
• Helped launch architecture that allowed customer portals to run as separate applications installed on clients’ servers.
• Worked to build an audit log for recording system-wide activities, like user log-ins and new accounts created, for analytics purposes. 
• Developed a range of new user management features that streamlined in-house processes at Conduktor.