07 Mar 2019
|Internet protocol suite|
MQTT (MQ Telemetry Transport) is an open OASIS and ISO standard (ISO/IEC PRF 20922) lightweight, publish-subscribe network protocol that transports messages between devices. The protocol usually runs over TCP/IP; however, any network protocol that provides ordered, lossless, bi-directional connections can support MQTT. It is designed for connections with remote locations where a "small code footprint" is required or the network bandwidth is limited.
Andy Stanford-Clark (IBM) and Arlen Nipper (Cirrus Link, then Eurotech) authored the first version of the protocol in 1999. It was used to monitor an oil pipeline through the desert. The goal was to have a protocol that is bandwidth-efficient, lightweight and uses little battery power, because the devices were connected via satellite link and at this time it was extremely expensive.
In 2013, IBM submitted MQTT v3.1 to the OASIS specification body with a charter that ensured only minor changes to the specification could be accepted. MQTT-SN is a variation of the main protocol aimed at embedded devices on non-TCP/IP networks, such as Zigbee.
Historically, the "MQ" in "MQTT" came from the IBM MQ (then 'MQSeries') MQ product line. However, the protocol provides publish-and-subscribe messaging (no queues, in spite of the name) and was specifically designed for resource-constrained devices and low bandwidth, high latency networks such as dial up lines and satellite links, for example.
The MQTT protocol defines two types of network entities: a message broker and a number of clients. An MQTT broker is a server that receives all messages from the clients and then routes the messages to the appropriate destination clients. An MQTT client is any device (from a micro controller up to a full-fledged server) that runs an MQTT library and connects to an MQTT broker over a network.
Information is organized in a hierarchy of topics. When a publisher has a new item of data to distribute, it sends a control message with the data to the connected broker. The broker then distributes the information to any clients that have subscribed to that topic. The publisher does not need to have any data on the number or locations of subscribers, and subscribers in turn do not have to be configured with any data about the publishers.
If a broker receives a topic for which there are no current subscribers, it will discard the topic unless the publisher indicates that the topic is to be retained. This allows new subscribers to a topic to receive the most current value rather than waiting for the next update from a publisher.
When a publishing client first connects to the broker, it can set up a default message to be sent to subscribers if the broker detects that the publishing client has unexpectedly disconnected from the broker.
Clients only interact with a broker, but a system may contain several broker servers that exchange data based on their current subscribers' topics.
A minimal MQTT control message can be as little as two bytes of data. A control message can carry nearly 256 megabytes of data if needed. There are fourteen defined message types used to connect and disconnect a client from a broker, to publish data, to acknowledge receipt of data, and to supervise the connection between client and server.
MQTT relies on the TCP protocol for data transmission. A variant, MQTT-SN, is used over other transports such as UDP or Bluetooth.
MQTT sends connection credentials in plain text format and does not include any measures for security or authentication. This can be provided by the underlying TCP transport using measures to protect the integrity of transferred information from interception or duplication.
A software running on a computer (running on-premises or in the cloud), could be self-built or hosted by a third party. Available as open source and proprietary with extra features added.
The broker acts as a post office, MQTT doesn’t use the address of the intended recipient but uses the subject line called “Topic”, and anyone who wants a copy of that message will subscribe to that topic. Multiple clients can receive the message from a single broker (one to many capability). Similarly, multiple publishers can publish topics to a single subscriber (many to one).
Each client can both produce and receive data by both publishing and subscribing, i.e. the devices can publish sensor data and still be able to receive the configuration information or control commands (MQTT is a bi-directional communication protocol). This helps in both sharing data, managing and controlling devices.
With MQTT broker architecture the devices and application becomes decoupled and more secure. MQTT uses Transport Layer Security (TLS) encryption with user name, password protected connections, and optional certifications that requires clients to provide a certificate file that matches with the server’s. The clients are unaware of each others IP address.
In case of single source of failure, broker software and clients have an automatic handover to Redundant/automatic backup broker. The backup broker can also be setup to share the load of clients across multiple servers onsite, cloud, or the combination of both.
The broker can support both standard MQTT and MQTT for compliant specifications such as Sparkplug, can be done with same server, same time and with same levels of security.
The broker can store the data in the form of retained messages (need to subscribe with database client) so that new subscribers to the topic can get the last value straight away.
The broker also keeps track of all the session’s information as the devices goes on and off called “persistent sessions”.
The main advantages of MQTT broker are:
- Eliminates vulnerable and insecure client connections
- Can easily scale from a single device to thousands
- Manages and tracks all client connection states, including security credentials and certificates
- Reduced network strain without compromising the security (cellular or satellite network)
Waits for a connection to be established with the server and creates a link between the nodes.
Waits for the MQTT client to finish any work it must do, and for the TCP/IP session to disconnect.
Returns immediately to the application thread after passing the request to the MQTT client.
Quality of service (QoS)
Each connection to the broker can specify a quality of service measure. These are classified in increasing order of overhead:
- At most once - the message is sent only once and the client and broker take no additional steps to acknowledge delivery (fire and forget).
- At least once - the message is re-tried by the sender multiple times until acknowledgement is received (acknowledged delivery).
- Exactly once - the sender and receiver engage in a two-level handshake to ensure only one copy of the message is received (assured delivery).
 This field does not affect handling of the underlying TCP data transmissions; it is only used between MQTT senders and receivers.
There are several projects that implement MQTT. Examples are:
- Facebook Messenger. Facebook has used aspects of MQTT in Facebook Messenger for online chat. However, it is unclear how much of MQTT is used or for what.
- IECC Scalable, DeltaRail's latest version of their IECC Signaling Control System uses MQTT for communications within the various parts of the system and other components of the signaling system. It provides the underlying communications framework for a system that is compliant with the CENELEC standards for safety-critical communications.
- Amazon Web Services announced Amazon IoT based on MQTT in 2015.
- The Open Geospatial Consortium SensorThings API standard specification has a MQTT extension in the standard as an additional message protocol binding. It was demonstrated in a US Department of Homeland Security IoT Pilot.
- The OpenStack Upstream Infrastructure's services are connected by an MQTT unified message bus with Mosquitto as the MQTT broker.
- Adafruit launched a free MQTT cloud service for IoT experimenters and learners called Adafruit IO in 2015.
- The IoT Guru provides free MQTT server as cloud service.
- Microsoft Azure IoT Hub uses MQTT as its main protocol for telemetry messages.
- XIM, Inc. launched an MQTT client called MQTT Buddy in 2017. It's a MQTT app for Android and iOS, but not F-Droid, users available in English, Russian and Chinese languages.
- Node-RED supports MQTT with TLS nodes as of version 0.14.
- Open-source software home automation platform Home Assistant is MQTT enabled and offers four options for MQTT brokers.
- Pimatic home automation framework for Raspberry Pi and based on Node.js offers MQTT plugin providing full support for MQTT protocol.
- McAfee OpenDXL is based on MQTT with enhancements to the messaging brokers themselves so that they can intrinsically understand the DXL message format in support of advanced features such as services, request/response (point-to-point) messaging, service fail over, and service zones.
- ejabberd supports MQTT as of version 19.02.
- EMQ X Scalable and Reliable Real-time MQTT Message Broker for IoT in 5G Era. 
- Comparison of MQTT implementations
- Apache ActiveMQ
- Advanced Message Queuing Protocol (AMQP)
- Streaming Text Oriented Messaging Protocol (STOMP)
- Constrained Application Protocol (CoAP)
- Extensible Messaging and Presence Protocol (XMPP)
- "MQTT 3.1.1 specification". OASIS. December 10, 2015. Retrieved April 25, 2017.
- "ISO/IEC 20922:2016 Information technology -- MQ Telemetry Transport (MQTT) v3.1.1". iso.org. International Organization for Standardization. June 15, 2016.
- "MQTT Version 5.0 OASIS Standard Specification" (PDF). OASIS. Retrieved 15 October 2019.
- "10th birthday party". MQTT.org. July 2009. Retrieved April 25, 2015.
- "OASIS MQ Telemetry Transport (MQTT) Technical Committee". OASIS. Retrieved May 9, 2014.
- Stanford-Clark, Andy; Hong Linh Truong (November 14, 2013). "MQTT For Sensor Networks (MQTT-SN) Protocol Specification Version 1.2" (PDF). mqtt.org. MQTT. p. 27. Retrieved May 9, 2014.
- "IBM MQ". IBM. Retrieved November 18, 2013.
- Piper, Andy (February 19, 2013). "Choosing Your Messaging Protocol: AMQP, MQTT, or STOMP". blogs.vmware.com. VMware Blogs. p. 1. Retrieved October 23, 2013.
- Yuan, Michael. "Getting to know MQTT". IBM Developer. Retrieved 13 October 2019.
- "Client, Broker / Server and Connection Establishment - MQTT Essentials: Part 3". hivemq.com. Retrieved 13 October 2019.
- "MQTT Sparkplug/Tahu". www.cirrus-link.com. Retrieved November 5, 2019.
- "IBM Knowledge Center". www.ibm.com. Retrieved 2018-01-30.
- Zhang, Lucy (August 12, 2011). "Building Facebook Messenger". facebook.com/Engineering. Facebook. p. 1. Retrieved October 15, 2015.
By maintaining an MQTT connection and routing messages through our chat pipeline, we were able to often achieve phone-to-phone delivery in the hundreds of milliseconds, rather than multiple seconds.
- Wood, Daren; Robson, Dave (August 13, 2012). "Message broker technology for flexible signalling control" (PDF). irse.org. IRSE. p. 7. Retrieved March 31, 2014.
- Barr, Jeff (October 8, 2015). "AWS IoT - Cloud Services for Connected Devices". aws.amazon.com. Amazon Web Services. p. 1. Retrieved October 21, 2015.
- "AWS IoT". aws.amazon.com/iot. Amazon Web Services. p. 1. Retrieved July 1, 2017.
- Brothers, Reginald (January 25, 2016). "S&T's Internet of Things Pilot Demonstrates 'State of the Practical'". dhs.gov. p. 1. Retrieved March 31, 2016.
- "OpenStack Firehose - The unified message bus for Infra services". docs.openstack.org. OpenStack Infrastructure Team. p. 1. Retrieved August 30, 2016.
- "Coming Soon: Adafruit IO". blog.adafruit.com. Adafruit Industries. September 16, 2014. p. 1. Retrieved March 29, 2017.
- "The Internet of Things for Everyone". io.adafruit.com. Adafruit. p. 1. Retrieved July 1, 2017.
- "IoT Guru MQTT tutorial". iotguru.cloud.
- "The IoT Guru". iotguru.cloud.
- Dotchkoff, Konstantin; Betts, Dominic; Kshirsagar, Dhanashri; mastermanu; Damaggio, Elio (March 1, 2017). "Understanding Microsoft Azure MQTT Support". docs.microsoft.com. Microsoft. p. 1. Retrieved March 29, 2017.
- "The story of MQTT Buddy begins!". mqtt.ximxim.com. XIM, Inc. February 24, 2017. p. 1. Retrieved July 1, 2017.
- "MQTT Buddy". mqtt.ximxim.com. XIM, Inc. Retrieved July 1, 2017.
- Community staff writer (June 14, 2016). "Version 0.14 released". nodered.org/blog. Node-RED. Retrieved July 6, 2016.
MQTT with TLS support
- Home Assistant Community (August 7, 2015). "MQTT". home-assistant.io. Home Assistant Community. Retrieved August 4, 2017.
- Home Assistant Community (August 7, 2015). "MQTT Brokers". home-assistant.io. Home Assistant Community. Retrieved August 4, 2017.
The MQTT component needs you to run an MQTT broker for Home Assistant to connect to. There are four options, each with various degrees of ease of setup and privacy.
- Kail, Marek (October 16, 2016). "pimatic-mqtt". pimatic.org. Oliver Schneider. Retrieved August 11, 2017.
- "What protocol does OpenDXL use? - OpenDXL". OpenDXL. Retrieved 2017-10-13.
- McDonald, Ted. "Architecture Guide McAfee Data Exchange Layer (DXL)" (PDF).
- ProcessOne, Marek Foss (2019-02-26). "ejabberd 19.02: the MQTT Edition". ProcessOne — Blog. Retrieved 2019-03-04.
- "EMQ X Broker - High Performance MQTT Message Broker — EMQ X - MQTT Messaging Broker 3.2.0 documentation". docs.emqx.io. EMQ. Retrieved 2019-12-16.