In this day and age, you can easily see a camera installed anywhere you go suchlike office, restaurant, grocery store, and even on the street. We can see a lot of benefits from the camera system which has become a popular solution for security, traffic control, anomaly detection,… However, has the camera video been exploited to their full potential yet? In other words, besides the original purpose of the camera system, are there any other aspects of the video that can bring more added values?

To meet the rising demands in this digital era, uniSense, a Video Analytics Platform has been developed by FPT Software, aiming to provide various common analysis which is possible to apply for many other industries. The solution adapts to different business tasks like analyze customers’ traits and behaviors, shelves assessment, restaurant operation, etc.

The platform is a multi-purpose solution that satisfies different needs to extend business of enterprises. Hence, mobility, installation and operation have been carefully taken into account in development phase.

Specific architecture for video analytics system

In general, a Video Analytics system includes typical components such as AI model, analytics job, business job, persistence job, etc. In this approach, the overall architecture of uniSense is presented in the picture as below:

Each component has a specific role in the processing flow, generating a powerful data analytics system to serve several demands of enterprises.

1. Message Broker

The role of message broker is to remove the dependency between jobs. That is, each job only needs to listen to a certain topic, get contracted JSON data, processing data, then output back the result to the message broker (in a contracted format, too). Each job does not need to know the order of the flow. For example, Frame Extractor can extract frame images from the video and put data to message broker, without having to know which processor will consume the frames later.

2. TensorFlow Serving

TensorFlow Serving is a serving system for integrating with the machine learning model. TensorFlow Serving provides the capability to flexibly update model version, and an easy method to use model – through API. For more information about TensorFlow Serving, please follow the link

3. Processors

Processors are independent components that have their own tasks to perform. Compare to single application design, this architecture gives system:

  • The ability of extension: Freely add new features to the system without impacting old features.
  • The ability to maintain and improve: Update is apparently a required step in AI project. Applying this architecture can help us enhance each component separately that will enable architecture to be updatable with modern technology in the future.
  • The ability to reuse: Well-designed components can be reused on other workflows without modification.

With those benefits, a Video Analytics Platform can be built easily, which allows user to define their own workflow, configure external services, and get their own Video Analytics System ready.

Rapid deployment, flexible deployment model

Components is made into Docker images to enable the ability to deploy on various platform, both on local and on the cloud.

Docker is a containerization technology that enables us to easily create, deploy, and run applications. Docker uses container to package up application and dependencies (libraries) into one package. Thanks to this image-based deployment model, Docker-enabled applications can be shared and run across multiple environments. If you are interested in Docker, please follow Docker Quickstart document

By using Docker, user can update components by changing the target image. For TensorFlow Serving, it can be updated the model version over time. These capabilities help the platform quickly adapt to the real environment, where AI model will be updated based on actual input. Platform can also keep up with new technologies, which is essential to enable new AI capabilities.

Furthermore, with the help of Docker, we can choose where to run each component depends on our needs.

For example, it allows us to deploy frame extractor on each local camera system, and host other processors on the cloud to centralize monitoring. This approach can utilize powerful CPU VMs for model processing.

The deployment process in each project depends on many factors. Using Docker, we can customize deployment model based on what we want.

Adapting to various business needs

Currently, uniSense provides a wide range of common analysis such as counting people, detect age and gender, read license plate code, etc., adapting to different business needs. This Video Analytics Platform has been applied as a potential solution to analyze customers’ traits and behaviors, shelves assessment, restaurant operation…

Here are 4 recent use cases that we have worked for:

1. Analyze Customer Traits and Behaviors

In retail stores, cameras are usually installed at common places for security purposes such as front door, counters, goods shelves…. In addition to this purpose, the camera system can do more to create many valuable information that help business run more effectively by providing an extra understanding of customer behaviors as well as better services. After using the data extracting from the camera system, we come up with the evaluation for:

  • How many customers come in / come out each day?
  • Customers are male or female, in which age range?
  • How long customers have been waiting at the counter to check out?
  • Distinguish customers and employees so that the result is more accurate.

Finally, the output is visualized as below:

2. Support Shelf Assessment

In grocery stores or brand-activation events, the shelf arrangement is a critical key point where customer’s buying decision is made. It can be seen as a next level of customer experience for the retail industry when implementing AI to their operation, especially shelf assessment. Computer vision using deep neural network detects objects within images of shelves and classifies them based on category, brand, and item. At this point, uniSense supports retailer to check shelf arrangement and gather key information promptly to take action on time.

3. Restaurant Consulting

It’s a massive leap when many restaurants catch up video analytics as a key trend to their business. Video insights are automated with the help of uniSense, considered as an excellent way to increase profits and improve operating procedures. The data gathered from video can help the restaurant understand specific situations with exact time frames for:

  • Calculate how many customers at a table at a certain time
  • How long customer have to wait until they are served
  • How long customer have their meals
  • How long staffs need to clean the table after the customer left

The output can be viewed as below:

4. License Plate Reader

A common use case of video analytics platform is license plate reader, for both cars and bikes. License plate recognition is used widely today, combined with other video analytics to maximize the benefits for physical security. The platform provides customizable character identification model in order to adapt with different license plate fonts.

Speed up the business process

uniSense supports customers to fasten digitalize transformation process by providing the ability to deploy services in a short time, so the business owner can reduce their time to market.

The architecture of uniSense allows it to be extensible, maintainable, updatable, and reusable. Through the help of Docker and TensorFlow Serving, we can update the components, as well as AI-model, so the system can be flexible, and be deployed in a short time.

The more diverse video capabilities become, the more insights businesses can pull based on the video collected in your system.

In real world, when companies have lots of data, they can build an AI model to support their own business. However, the quality of AI-related roducts usually need to be continuously improved to ensure a stable system operation. Hence, the real challenges of AI project are how to quickly apply its model and rapidly update latest version, as well as testing the efficiency of improvement by the time. Besides, converting customer requirements into processing pipeline is also an important process.

The proposed architecture of uniSense helps enterprise solve above requirements. With functionality of TensorFlow Serving, uniSense provides capability to easily update new version of models as it evolved. Multi components based pipeline, combine with message based coordinator that help flexibly update business processing flow. Together, uniSense helps enterprises bring AI to solve their business problem effortlessly.

Tran Gia Quoc Hung – Solution & Technology Unit, FPT Software

Related posts: