Nowadays, cyber attack, phishing, or malware issues, etc., are becoming more commonplace and a major problem for organizations. The information system of organizations needs a solution to collect, manage and analyze information security events. SIEM (Security information and event management) is a system designed to collect log information of security event from terminals and archive data in a centralized way, allowing organizations to limit risks, save time and human resources.


Nowadays, cyber attack, phishing, or malware issues, etc. are becoming more commonplace and a major problem of organizations. To dismiss these concerns, organizations need to invest in security solutions, in-depth and multiple-layer protection. A common trend is that these solutions are good solutions, leading the market but many belong to different vendors.

This creates a lot of problems for the Security Operations Team, such as:

  • Inability to analyze the entire log information: Everyday, systems such as Firewall, IPS, OS, Database, etc., produce millions of logs. Organizations with their working teams have no way of completing the analysis with manual tools.
  • Number of false alerts: Of those millions of events, there are a great number of false and not so important messages.
  • Implementing individual protection, lack of ability to cover: Each system requires one or more administrators, and usually these people only work with their expertise. When there is a need to perform an analysis, investigate a problem in the system, they will take a long time to learn about other related systems or work with other departments, etc. So the implementing time will be long, and it is easy to ignore the events thought not dangerous. Because the administrator is not capable of observing all the security issues that are going on in the system.
  • Each system has a different log format: This makes it difficult to synchronize and analyze.

With such requirements, the organization’s information system needs a solution to gather, manage and analyze information security events. Security information and event management (SIEM) is a system designed to collect log information of security event from terminals and archive data in a centralized way. The SIEM system must collect logs from all systems that are interested by organizations, and provide tools diversely and flexible for finding, analyzing, and monitoring security events from time to time on the only display. The inter-event analysis feature allows the system to pinpoint the major security issues that the system is facing. This will allow organizations to limit risks, save time and human resources.

2. Components of the SIEM system

The SIEM system consists of several parts, each of which performs a separate task. Each component in this system can operate independently of other components but if all are not operating at the same time, there will be no efficient SIEM.

Depending on the system being used, each SIEM will have the basic components. By understanding each part of SIEM and how it works, administrators can effectively manage and solve problems as they arise.

Components of SIEM.

Input device provides data

The first component of SIEM is the input devices that provide data for SIEM. The source device can be a real one in a network such as Router, Switch or some kinds of server, and can also be logs from an application or just any other data. Determining what is in the system is very important in implementing SIEM. Identifying sources of data provider in the first place will save significant efforts and money, reduce complexity in implementation.

  • Operating Systems: Microsoft Windows with variants of Linux and UNIX, AIX, Mac OS are operating systems used commonly. Most operating systems are fundamentally different technologies and perform a specific task but one of the things that all have in common is that they generate logs. Logs show what the system has done: Who logged in? what do they do on the system? etc. Logs generated by an operating system and active users will find it very useful when dealing with security problems, diagnosing problems or just misconfigurations.
  • Devices: Actually, system administrators do not have remote access to devices in the system such as routers, switches, firewalls, and servers to perform some basic management. But they can manage the device through a special port interface. This interface can be based on webs, command lines or run through an application downloaded to the administrator’s workstation. The operating system of running network devices can be either a common one such as Microsoft Windows or an open source-based customizer such as Linux, but can be configured in the way that the operating system normally does. Devices such as routers, switches are a typical case. Being independent of vendors, so they can never access directly to their basic operating system, which can only be accessed through the command line or web interface used to manage. Devices store their logs on the system or can usually be configured to send out logs via SysLog or FTP.

  • Applications: What runs on operating systems are applications that are used for a variety of functions. In a system, there may be Domain Name Servers (DNS), Dynamic Host Configuration Protocol (DHCP), web servers, email systems and a myriad of other applications. Logs of application contain detailed information about the status of the application, such as statistics, errors, or information on messages. Some applications that generate logs will be useful for administrators, used to require to maintain and store logs in compliance with the law.
  • Defining needed log records: After identifying the source devices providing data in the system, the administrator should consider collecting logs from which devices that are necessary and important for SIEM. Some points to note in collecting logs are as follows:
  • Which source device is preferred? What should important data be collected? What is the size of logs generated during a given time period? This information is used to determine how much resources SIEM needs, especially storage space.
  • How long does this source device generate logs? This information is combined with the size of log to select the use of network access when collecting records.
  • How to connect source devices with SIEM?
  • Is it necessary to have logs in real time or set it up at a specific time of the day?

The above information is very useful in identifying the source of necessary devices for SIEM. They have too many but it is necessary to determine more precisely what is needed for SIEM. The number of users, the schedule of system maintenance, and many other factors can have a significant impact on the number of logs generated each day.

2. Method of collecting information

Collecting log records

The next component in the diagram is the log collector component. The mechanism for collecting logs depends on each device, but there are basically two methods, as follows: Push Log and Pull Log.

  • Push Log: Log records are pushed into SIEM by source devices

This method has benefits: Easy to install and configure. Normally, just set up a receiver and then connect the source device to this receiver. For example, SysLog, when configuring a source device using SysLog, the administrator can set the IP address or DNS name of a SysLog server on the network and the device will automatically send its logs through SysLog. However, this method still has some disadvantages.

Using SysLog in UDP environment (User Datagram Protocol – one of the core protocols of the TCP/IP). The nature of using SysLog in the UDP environment may be that it does not guarantee that packets arrive at the destination, since UDP is a non-directional connecting protocol. If a situation occurs on the network such as a strong virus, the administrator may not receive the SysLog information packet. One problem may be occur if the appropriate access control is not set on the log receivers, because of misconfigurations or malwares overwhelming the wrong information. That makes it difficult to detect security events. If there is a deliberate attack against SIEM, a bad guy can distort information and add trash data to SIEM. Therefore, it is very important to understand the devices that send logs to SIEM.

  • Pull Log: Log records will be retrieved by SIEM

Unlike the Push Log method in which the source device sends logs to SIEM without any interaction from SIEM, Pull Log requires SIEM to start connecting to the source device and actively retrieves the logs from those source devices by the software installed on the security device. For example, if logs are stored in a shared text file on the network, SIEM will establish the connection to retrieve the stored information and read the log files from the source device.

With the Push Log method, logs of the source device usually send the logs to SIEM as soon as it is generated. But with the Pull Log method, the connection will be created for SIEM to access the source device and retrieve logs from the source device. The duration of the connection to retrieve logs of Pull Log can be a few seconds or every hour. This time period can be configured either by option or by default for SIEM. This also requires to calculate timing cycles to pull logs into SIEM; otherwise, it will lead to the overflow and blockage of the SIEM system when too many source devices retrieve the logs at the same time.

Information collection policy: It is possible to set up a priority policy and collect to filter and consolidate the information of security events before sending it to the system. This technique allows the administrator to regulate security events and manage the information; otherwise, there will be so many security events in the network that the administrator is becoming awkward where to start.

Analyzing and Standardizing log records

Numerous logs are sent from devices and applications in the environment to the SIEM. At this point, all of the logs are in the original format, so the administrator can not do anything except for saving it somewhere. The SIEM system collects logs from a variety of devices, and transmission of the logs from source devices to SIEM need to be kept confidential, authenticated, and trusted by using Syslog or protocols such as SNMP, OPSEC, SFTP.

However, in order to make logs useful in SIEM, it is necessary to reformat them into the only standard format. Changing all types of different logs to ones with the only format is called as normalization. If the devices do not support these protocols, Agents need to be used. That’s one must-do step to receive the logs in the format that SIEM can understand. Installing agents can extend the SIEM implementing process, but the administrator will have the logs in the desired format.

Standardizing logs

The purpose of the information gathering is to capture and standardize information from various security devices and provide that information to the system for further analysis. This function is really important because the data is in different formats from different devices and different vendors.

An example in the image: Standardizing logs of these two systems, a Windows Event Log system, and a ASA Cisco. Both shows that when the same person logged into the device, the login of each vendor is different. It is important to understand the format and details of the event. Therefore, Standardizing logs is a necessity.

Security information and event management – Part 2: Techniques to determine the rule of SIEM management

Song Phuong – FPT IS

Related posts: