The case of DKE Company

DKE is the 5th largest electronic components provider in the world. For retail, they had established a MFC-based ERP Distribution Management System since 2002, written in C/C++, with over 2 million lines of codes. It has been almost 20 years since then.

The current system is facing these following problems:

  • Some technologies developed since 20 years ago, like MFC, has become obsolete, especially in presence of the new .NET and .NET core.
  • The original design is Monolithic, hence bulky with additional functions, hard to maintain, and time-consuming when building and deploying source code. System expansion, in particular, is extremely complicated.
  • Framework is hard to change, as the system is written tin C/C++ and MFC. Therefore, a full rewrite will be require to change into .NET core or C#.

DKE decided to migrate the design from monolithic to microservice, in order to utilize the many benefits of the later design like easy to deploy, able to use many technical stacks, scaling, and so on.

In theory, this is easy, but in practice, it is quite challenging. Even with multiple benefits compared to the old system, DKE engineers still face the following problems in the microservice:

  • DKE database includes over 500 tables, with usage of Oracle DB relationships to represent the very complex relations between the tables. On average, there are hundreds of thousands of records. Thus, a question arose on how to break out this database into multi-database in the microservice, as well as migrate the data?

  • Migration from mono to microservice cannot be done immediately, but rather take years. Therefore, we cannot take down the system for migration, rather have to roll-out. So, how to run these 2 system simultaneously?
  • During migration, the old system may be updated constantly, so how to reflect these changes to the new system in time?

There are also various other challenges, too many to be named.

Solution

For almost 2 years, DKE engineers had researched and developed many other methods of migrations. This article will provide an overview of several.

DKE’s field is Distributed management system, where the domain drive design method is applied for task analysis and break the system down to separate models like Order, Payment, and Sale.

From the business analysis results of DDD, engineers can now estimate the number of necessary modules and microservices to be created.

Each microservice is separated into internal service and external service. Internal service is written in NodeJS, and interacts directly with the database, then sends unmatched data to the external service (expose API) to client via a message queue. The, the internal microservice will be transformed into .NET core

The API part is written in .Netcore and uses Swagger library to generate API specs.

According to microservice best practice, in order to ensure loose coupling, the large database will be separated into smaller ones owned by microservices. Simple in theory, right? However, in practice, there are complex relationships and high dependency with one another, along with millions of record data, making break out a difficult task.

The team had therefore used partial break out and data migration. For example: one database is owned by sale services, another by payment services.

Oracle Golden Gate is tested for data migration between the old and new database.

While results for separate modules are promising, whole database breakout remains challenging, especially in maintaining ACID while moving it to the microservice’s BASE.

Therefore, migration to microservice is still halted at code logic, where all microservices point towards one singular database.

In the next article, we will cover how DKE engineers tackle this problem.

Do Trong Nguyen – FPT Software

Related posts: