Log management and intelligence

From Wikipedia, the free encyclopedia
Jump to: navigation, search

Log management (LM) comprises an approach to dealing with large volumes of computer-generated log messages (also known as audit records, audit trails, event-logs, etc.). LM covers log collection, centralized aggregation, long-term retention, log analysis (in real-time and in bulk after storage) as well as log search and reporting.[1]

Log management is driven by reasons of security,[2] system and network operations (such as system or network administration) and regulatory compliance.

Effectively analyzing large volumes of diverse logs can pose many challenges — such as huge log-volumes (reaching hundreds of gigabytes of data per day for a large organization), log-format diversity, undocumented proprietary log-formats (that resist analysis) as well as the presence of false log records in some types of logs (such as intrusion-detection logs)[examples needed].

Users and potential users of LM can build their own log management and intelligence tools, assemble the functionality from various open-source components, or acquire (sub-)systems from commercial vendors. Log management is a complicated process and organizations often make mistakes while approaching it.[3]

Lately, more and more the suggestion is made to change the definition of logging. This change would keep matters both more pure and more easily maintainable.

  • Logging would then be defined as all instantly discardable data on the technical process of an application or website, as it represents and processes data and user input.
  • Auditing, then, would be used to denote data that is not immediately discardable. In other words: data that is assembled in the auditing process, is stored persistently, is protected by authorization schemes and is, always, connected to some end-user functional requirement.

Logging is technical information used for the maintenance process of applications or websites - either to define whether a reported bug is actually a bug, or to help analyze, reproduce and solve a bug, and even to help test new features in a development stage.

Deployment life-cycle[edit]

One view[citation needed] of assessing the maturity of an organization in terms of the deployment of log-management tools might use[original research?] successive categories such as:

  • Level 1: in the initial stages, organizations use different log-analyzers for analyzing the logs in the devices on the security-perimeter. They aim to identify the patterns of attack on the perimeter infrastructure of the organization.
  • Level 2: with increased use of integrated computing, organizations mandate logs to identify the access and usage of confidential data within the security-perimeter.
  • Level 3: at the next level of maturity, the log analyzer can track and monitor the performance and availability of systems at the level of the enterprise — especially of those information-assets whose availability organizations regard as vital.
  • Level 4: organizations integrate the logs of various business-applications into an enterprise log manager for better value proposition.
  • Level 5: organizations merge the physical-access monitoring and the logical-access monitoring into a single view.

See also[edit]



  • MITRE: Common Event Expression (CEE) Proposed Log Standard. Online at http://cee.mitre.org, retrieved 2010-03-03

External links[edit]