Wire data

Wire data is the information that passes over computer and telecommunication networks defining communications between client and server devices. It is the result of decoding wire and transport protocols containing the bi-directional data payload. More precisely, wire data is the information that is communicated in each layer of the OSI model (Layer 1 not being included because those protocols are used to establish connections and do not communicate information).

Relevance

Wire data is the observed behavior and communication between networked elements which is an important source of information used by IT operations staff to troubleshoot performance issues, create activity baselines, detect anomalous activity, investigate security incidents, and discover IT assets and their dependencies. According to American IT research and advisory firm Gartner, wire data is one of five types of data that will need to be handled by emerging IT Operations Analytics platforms. Gartner defines wire data as “the data contained in the headers and payloads of packets and their associated flow data as traffic moves from one node to another across a distributed IT system.”

Real-time wire data streams are also important sources of data for business and operational intelligence teams. In these types of scenarios, wire data is used to measure order transactions for real-time reporting on transaction volume, success, and failure rates; tracking patient admission rates at hospitals; as well as reporting on the weights and measures of airplanes prior to take off.

Distinction between wire data and system self-reported data

Wire data is distinct from machine-generated data, which is system self-reported information typically in the form of logs sourced from elements like network routers, servers, and other equipment. Unlike those forms of machine-generated data, which are dependent on the logging configurations of those devices, wire data is defined by wire and transport protocols. There is a small amount of overlap between wire data and machine-generated data but also significant differences. For example, web server logs typically record HTTP status code 200 responses, indicating that a web page was served to a client. However, web servers do not log the transaction payload and so would not be able to show which HTTP status code 200 responses were for pages with a “service unavailable” message. That information is contained in the wire data or transaction payload and is not logged by the server.

According to a March 2016 Gartner research note, wire data will play a more important role than machine data for analytics in the future: "While log data will certainly have a role in future monitoring and analytics, it is wire data—radically rethought and used in new ways—that will prove to be the most critical source of data for availability and performance management over the next five years."

Examples of information derived from wire data

• Structured transactional data passed over HTTP, including information encoded using SOAP/XML
• SQL transaction details, such as errors, methods used, and stored procedures executed
• Unique customer IDs, handset type, and credit-control details defined by AVPs and Commands contained in Diameter transactions
• Cross-tier (web, database, storage, etc.) transaction metrics parsed by unique session IDs or other GUID
• Correlation of network transfer time and server processing time
• TCP mechanisms such as Nagle delays and throttling
• HTTP metadata including user-agent, session ID, status code, and IP address
• HTTP page content including page title, user ID, and transaction values

Methods of analyzing wire data

Traditional methods of capturing and analyzing wire data include offline network packet analyzers. Newer approaches receive a copy of network traffic from a port mirror (SPAN) or network tap and reassemble those packets into full per-client sessions and transaction streams, analyzing the entire transaction payload in real time and generating metadata on those transactions without storing the actual packets.

Bibliography

1. Will Cappelli, Gartner: "Use Data- and Analytics-Centric Processes With a Focus on Wire Data to Future-Proof Availability and Performance Management," March 2016
2. Will Cappelli, Gartner: “Data Growth Demands a Single, Architected IT Operations Analytics Platform,” September 2013
3. Will Cappelli, Gartner: “How ITOA Relates to Other Analytics-Driven Disciplines,” November 2013