Log management and intelligence

From Wikipedia, the free encyclopedia
Jump to: navigation, search

Log management (LM) comprises an approach to dealing with large volumes of computer-generated log messages (also known as audit records, audit trails, event-logs, etc.). LM covers:[1]

  • log collection
  • centralized aggregation
  • long-term retention
  • log rotation
  • log analysis (in real-time and in bulk after storage)
  • log search and reporting.

Concerns about security,[2] system and network operations (such as system or network administration) and regulatory compliance drive log management.

Effectively analyzing large volumes of diverse logs can pose many challenges — such as:

  • huge log-volumes (reaching hundreds of gigabytes of data per day for a large organization)
  • log-format diversity
  • undocumented proprietary log-formats (that resist analysis)
  • the presence of false log records in some types of logs (such as intrusion-detection logs)[examples needed]

Users and potential users of LM can build their own log-management and intelligence tools, assemble the functionality from various open-source components, or acquire (sub-)systems from commercial vendors. Log management is a complicated process and organizations often make mistakes while approaching it.[3]

Lately[when?], more and more the suggestion is made[by whom?] to change the definition of logging. This change would keep matters both more pure and more easily maintainable:

  • Logging would then be defined as all instantly discardable data on the technical process of an application or website, as it represents and processes data and user input.
  • Auditing, then, would involve data that is not immediately discardable. In other words: data that is assembled in the auditing process, is stored persistently, is protected by authorization schemes and is, always, connected to some end-user functional requirement.

Logging can produce technical information usable for the maintenance of applications or websites. It can serve:

  • to define whether a reported bug is actually a bug
  • to help analyze, reproduce and solve bugs
  • to help test new features in a development stage

Deployment life-cycle[edit]

One view[citation needed] of assessing the maturity of an organization in terms of the deployment of log-management tools might use[original research?] successive categories such as:

  • Level 1: in the initial stages, organizations use different log-analyzers for analyzing the logs in the devices on the security-perimeter. They aim to identify the patterns of attack on the perimeter infrastructure of the organization.
  • Level 2: with increased use of integrated computing, organizations mandate logs to identify the access and usage of confidential data within the security-perimeter.
  • Level 3: at the next level of maturity, the log analyzer can track and monitor the performance and availability of systems at the level of the enterprise — especially of those information-assets whose availability organizations regard as vital.
  • Level 4: organizations integrate the logs of various business-applications into an enterprise log manager for better value proposition.
  • Level 5: organizations merge the physical-access monitoring and the logical-access monitoring into a single view.

See also[edit]

Vendors[edit]

References[edit]

  • MITRE: Common Event Expression (CEE) Proposed Log Standard. Online at http://cee.mitre.org, retrieved 2010-03-03

External links[edit]