MQTT Protocol

MQTT (Message Queuing Telemetry Transport) is a lightweight, publish-subscribe messaging protocol designed for low-bandwidth, high-latency, or unreliable networks. It is widely used in IoT (Internet of Things) applications for efficient communication between devices.

It operates over TCP/IP (Transmission Control Protocol / Internet Protocol)and is optimized for low power consumption, minimal data packets, and efficient network usage.

It is a protocol that is mainly used to connect devices with each other on the internet with backend services. The underlying protocol of the Internet is TCP/IP, built on top of the TCP/IP stack, MQTT is today a standard protocol for IoT communication. It is a TCP-based protocol relying on the publish-subscribe model. This communication protocol is suitable for transmitting data between resource-constrained devices having low bandwidth and low power requirements. Hence this messaging protocol is widely used for communication in the IoT Framework.

Let’s take below a simaple example of sensors based Smart Agriculture. In the subscriber-publisher model, the one which sends data/ command is termed a publisher, and the one which receives data/ command is termed a subscriber.

The moisture sensor publishes moisture data and sends it to the broker, which when received by the subscriber can generate an alert based on the threshold value. If the moisture level is low, and the user wants to turn ON the sprinkler, then the user device publishes data/command to turn the sprinkler ON and the sprinkler receives (working as a subscriber here) command and actuates the water valve.

Image Credit: Psiborg

How MQTT Works

1. Client-Server Model
   i. Client
   ii. Server
2. Publisher-Subscriber Model
   i. Topic
   ii. Publisher / Brocker
   iii. Subscriber

1. Client-Server Model

MQTT follows a client-server (or client-broker) model:

I. Clients:

• Devices (e.g., IoT sensors, mobile apps, embedded systems) that send or receive messages. Any application or device running the MQTT client library is an MQTT client.
• For example, an instant messaging app that uses MQTT is a client, various sensors that use MQTT to report data are a client, and various MQTT testing tools are also a client.

ii. Broker:

• A central server (e.g., Eclipse Paho, Mosquitto, HiveMQ, EMQX) that routes messages between clients. The MQTT Broker handles client connection, disconnection, subscription, and unsubscription requests, and routing messages.
• A powerful MQTT broker can support massive connections and million-level message throughput, helping IoT service providers focus on business and quickly create a reliable MQTT application. The heart of any publish/subscribe protocol is the MQTT broker.
• A broker can handle up to thousands of concurrently connected MQTT customers, depending on how it is implemented.
• All communications must be received by the broker, who will then sort them, ascertain who subscribed to each one, and deliver the messages to the clients who have subscribed. All persistent customers’ sessions, including missed messages and subscriptions, are likewise kept by the Broker.

2. Publish-Subscribe Model

Instead of direct communication, MQTT uses a publish-subscribe model:

I. Publisher (client) sends messages to a Topic.

Topic: With MQ Telemetry Transport, resource-constrained IoT devices can send or publish information on a specific topic to a server that acts as an MQTT message broker. The broker then transmits the information to those customers who have previously subscribed to the customer’s topic. To a human, a topic looks like a hierarchical file path. Customers can subscribe to a specific hierarchy level of a topic or use a wildcard character to subscribe to multiple levels.
The MQTT protocol is a good choice for wireless networks that have varying latency due to occasional bandwidth limitations or unreliable connections. If the connection from a subscribing client to a broker is interrupted, the broker buffers the messages and sends them to the subscriber when the subscriber is back online. If the connection from the Publishing Client to the Broker is disconnected without notification, the Broker can disconnect and send the Subscriber a cached message with instructions from the Publisher.

• MQTT messages are sent to a topic (like a messaging channel).
• Subscribers can use wildcards (+, #) to listen to multiple topics.
• Single-level : “+” or (home/+/temperature) : Single-level wildcard represented by “+” or (home/+/temperature) Symbol can replace single level in topic. If the client wants information about all temperature present inside the house, it will subscribe to the topic :
• Multi-Level: “#” or "home/groundfloor/#" : Multi-level wildcard represented by “#” symbol can replace multiple levels in topic. If a client wants information about all objects present inside the kitchen, living room, bedroom, or any other room on ground floor, it will subscribe to topic:

ii. Broker

• Receives the message and forwards it to all subscribed clients
• The heart of any publish/subscribe protocol is the MQTT broker. A broker can handle up to thousands of concurrently connected MQTT customers, depending on how it is implemented.
• The MQTT Broker handles client connection, disconnection, subscription, and unsubscription requests, and routing messages. A powerful MQTT broker can support massive connections and million-level message throughput, helping IoT service providers focus on business and quickly create a reliable MQTT application.
• All communications must be received by the broker, who will then sort them, ascertain who subscribed to each one, and deliver the messages to the clients who have subscribed. All persistent customers’ sessions, including missed messages and subscriptions, are likewise kept by the Broker.
• The MQTT broker is the center of every Publish / Subscribe protocol. Depending on the implementation, a broker can manage up to thousands of simultaneously connected MQTT clients. The broker is responsible for receiving all messages, filtering the messages, determining who subscribed to each message and sending the message to those subscribed clients. The Broker also holds the sessions of all persistent clients, including subscriptions and missed messages.
• Another task of the Broker is the authentication and authorization of clients. Usually the broker is extensible, which facilitates custom authentification, authorization and integration with backend systems. Integration is especially important, because the Broker is often the component directly exposed on the Internet, serves many clients and has to forward messages to downstream analysis and processing systems. In short, the Broker is the central hub through which every message must be routed. It is therefore important that your broker is highly scalable, can be integrated into back-end systems, is easy to monitor and, of course, is fail-safe. MQTT brokers used in the industry are, for example, the HiveMQ MQTT Broker and mosquitto from cedalo. Cloud providers such as Microsoft and Amazon also provide their own MQTT brokers with Azure IoT Hub and AWS IoT Core.

iii. Subscriber (client)

• Listens to a specific topic and processes incoming messages.
• All devices and software, such as the OPC Router, that are connected to the broker in some way are referred to as MQTT clients. A client can send messages to the broker (publish) and receive messages from the broker (subscribe). When sending a message to the broker, an MQTT topic must be specified, which can be used to further process the message. Messages can be sent with different Quality of Service (QoS):
• Quality of Service (QoS) Levels : MQTT is mainly used to ensure reliability by adding three distinct flags. This is referred to as Quality of Service (QoS). MQTT offers three QoS (Quality of Service) levels:

a). QoS 0 (At most once) → No guarantee of delivery. At this level, there is no guarantee of delivery. Neither does the recipient acknowledge receiving the message nor does the sender store and re-transmit it. This requires the lowest amount of network traffic.

b). QoS 1 (At least once) → Guaranteed delivery, but may receive duplicates. This level makes sure that the recipient receives the message at least once. The sender stores the message until it does not receive an acknowledgment from the recipient. There is a possibility that duplicate messages are also received by the recipient.

c). QoS 2 (Exactly once) → Guaranteed delivery with no duplicates (most reliable but slowest). This is the highest level of service in MQTT. At this level, it is guaranteed that each message will only be received once. This level is the safest and the slowest and is suitable for critical scenarios that require a message to be guaranteed delivered and not duplicated.

Get Abhishek Rai’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

🔹 Example:

• Sensor A publishes { "temperature": 20 } to topic: "home/temperature".
• Mobile App subscribes to "home/temperature".
• Broker delivers the message to the app.

Below is the Diagram for The MQTT protocol operates on a publish and subscription model. This means that devices (known as clients) can publish messages to a broker (the backend system), which then distributes these messages to other devices that have subscribed to the relevant topics. MQTT’s simplicity, efficiency, and ability to provide real-time communication make it an ideal protocol for a wide range of applications, from smart devices to industrial automation systems. Below is a detailed diagram of how the MQTT protocol operates with IoT devices:

Image Credit: Opc-router

Who uses MQTT?

MQTT is adopted across different industries due to its versatility. Here are some key sectors where MQTT is prominently used:

• Industrial automation: In manufacturing and production environments, MQTT monitors and controls equipment, ensuring smooth and efficient operations.
• Automotive: MQTT is used in connected car systems for real-time data exchange, including vehicle diagnostics, navigation, and infotainment.
• Healthcare: In medical devices and health monitoring systems, MQTT facilitates the real-time transmission of patient data, ensuring timely interventions and improved patient care.
• Energy and utilities: MQTT helps manage smart grids, monitor energy consumption, and control utility services.

Image Credit : Fiixsoftware

Key Components of MQTT

1. Topics

With MQ Telemetry Transport, resource-constrained IoT devices can send or publish information on a specific topic to a server that acts as an MQTT message broker. The broker then transmits the information to those customers who have previously subscribed to the customer’s topic. To a human, a topic looks like a hierarchical file path. Customers can subscribe to a specific hierarchy level of a topic or use a wildcard character to subscribe to multiple levels.

2. Quality of Service (QoS) Levels

MQTT is mainly used to ensure reliability by adding three distinct flags. This is referred to as Quality of Service (QoS). MQTT offers three QoS (Quality of Service) levels:

• QoS 0 (At most once) → No guarantee of delivery. At this level, there is no guarantee of delivery. Neither does the recipient acknowledge receiving the message nor does the sender store and re-transmit it. This requires the lowest amount of network traffic.
• QoS 1 (At least once) → Guaranteed delivery, but may receive duplicates. This level makes sure that the recipient receives the message at least once. The sender stores the message until it does not receive an acknowledgment from the recipient. There is a possibility that duplicate messages are also received by the recipient.
• QoS 2 (Exactly once) → Guaranteed delivery with no duplicates (most reliable but slowest). This is the highest level of service in MQTT. At this level, it is guaranteed that each message will only be received once. This level is the safest and the slowest and is suitable for critical scenarios that require a message to be guaranteed delivered and not duplicated.

3. Retained Messages

• Broker can keep the last message for a topic so that new subscribers receive the latest data immediately.

4. Last Will & Testament (LWT)

• If a client disconnects unexpectedly, the broker sends a predefined “Last Will” message to notify other clients.

5. Persistent Sessions

• Clients can maintain session state, so they receive missed messages when reconnecting.

Universal protocol for cloud and IoT application

Image Credit: HW-group.com

How is MQTT used?

Using MQTT involves several steps, from setting up an MQTT broker to configuring clients for publishing and subscribing messages. Below is a detailed guide on how to use MQTT:

1. Set up an MQTT broker: An MQTT broker is the central hub through which all messages are sent and received. Some popular examples include: Eclipse Mosquitto, HiveMQ, and EMQX.
2. Configure the MQTT broker: Configuration settings may vary depending on the broker used. For Mosquitto, you can edit the configuration file (/etc/mosquitto/mosquitto.conf) to customize settings like listener ports, authentication, and access control.
3. Set up MQTT clients: Clients can be devices, applications, or any entity that publishes or subscribes to topics. Clients need to use an MQTT library for their respective programming environments. For example, the Paho MQTT library is popular for Python.
4. Publish messages: The client publishes messages to a specific topic.
5. Subscribe to messages: The client subscribes to a topic to receive messages.
6. Test the setup: Once the broker and client are set up, it’s important to test the system to make sure messages are being sent and received correctly.
7. Monitor and manage: Once the system is set up it’s important to monitor and manage MQTT topics and messages. Tools like MQTT Explorer provide an interface for monitoring and managing message flows and troubleshooting.

Why Is MQTT the Best Protocol for IoT?

MQTT has emerged as one of the best IoT protocols due to its unique features and capabilities tailored to the specific needs of IoT systems. Some of the key reasons include:

• Lightweight: IoT devices are often constrained in terms of processing power, memory, and energy consumption. MQTT’s minimal overhead and small packet size make it ideal for these devices, as it consumes fewer resources, enabling efficient communication even with limited capabilities.
• Reliable: IoT networks can experience high latency or unstable connections. MQTT’s support for different QoS levels, session awareness, and persistent connections ensures reliable message delivery even in challenging conditions, making it well-suited for IoT applications.
• Secure communications: Security is crucial in IoT networks as they often transmit sensitive data. MQTT supports Transport Layer Security (TLS) and Secure Sockets Layer (SSL) encryption, ensuring data confidentiality during transmission. Additionally, it provides authentication and authorization mechanisms through username/password credentials or client certificates, safeguarding access to the network and its resources.
• Bi-directionality: MQTT’s publish-subscribe model allows for seamless bi-directional communication between devices. Clients can both publish messages to topics and subscribe to receive messages on specific topics, enabling effective data exchange in diverse IoT ecosystems without direct coupling between devices. This model also simplifies the integration of new devices, ensuring easy scalability.
• Continuous, stateful sessions: MQTT allows clients to maintain stateful sessions with the broker, enabling the system to remember subscriptions and undelivered messages even after disconnection. Clients can also specify a keep-alive interval during connection, which prompts the broker to periodically check the connection status. If the connection is lost, the broker stores undelivered messages (depending on the QoS level) and attempts to deliver them when the client reconnects. This feature ensures reliable communication and reduces the risk of data loss due to intermittent connectivity.
• Large-scale IoT device support: IoT systems often involve a large number of devices, requiring a protocol that can handle massive-scale deployments. MQTT’s lightweight nature, low bandwidth consumption, and efficient use of resources make it well-suited for large-scale IoT applications. The publish-subscribe pattern allows MQTT to scale effectively, as it decouples sender and receiver, reducing network traffic and resource usage. Furthermore, the protocol’s support for different QoS levels allows customization of message delivery based on the application’s requirements, ensuring optimal performance in various scenarios.
• Language support: IoT systems often include devices and applications developed using various programming languages. MQTT’s broad language support enables easy integration with multiple platforms and technologies, fostering seamless communication and interoperability in diverse IoT ecosystems. You can visit our MQTT Client Programming blog series to learn how to use MQTT in PHP, Node.js, Python, Golang, Node.js, and other programming languages.

Comparison:

MQTT vs Socket.IO vs GATT vs HTTPS vs GraphQL

— — — —

— — — -

HiveMQ MQTT vs Paho MQTT

HiveMQ MQTT and Paho MQTT are both MQTT libraries but serve different purposes, and each is suited for different use cases. Here’s a comparison of where and why you might use each one:

1. HiveMQ MQTT

HiveMQ is an MQTT broker, primarily focused on providing a reliable and scalable MQTT server for handling connections between clients. It’s designed for enterprise-level use cases and is often used to manage high throughput and handle large numbers of IoT devices or clients.

Use cases for HiveMQ:

• IoT Systems: HiveMQ is typically used in IoT systems where reliable and scalable message delivery is required across large numbers of devices.
• Enterprise MQTT Broker: It is used in enterprise environments where high availability and clustering are necessary for distributing MQTT messages.
• High-Volume, Low-Latency Messaging: HiveMQ supports high-volume environments, like data streaming or telemetry collection, requiring low latency and real-time messaging.
• Cloud Integration: It supports MQTT over WebSockets, which is ideal for cloud integrations or web-based IoT applications.

Key Features:

• Broker Solution: HiveMQ is a server-side broker, not just a client library.
• Enterprise-Ready: Built for large-scale deployments with clustering and fault tolerance.
• WebSocket Support: Allows MQTT communication with web applications using WebSockets.
• Security: Provides features like authentication, encryption, and access control.
• Extensions: Supports plugins and integrations with enterprise systems.

2. Paho MQTT

Paho is an MQTT client library from Eclipse Foundation, which is typically used for implementing MQTT clients on different platforms. It supports a wide range of programming languages, including Java, Python, C, and more. It can be used for both client-side MQTT communication and testing purposes.

Use cases for Paho:

• IoT Devices and Clients: Paho is ideal for building MQTT clients that connect to any MQTT broker (including HiveMQ or any other broker).
• Embedded Systems: It’s used in embedded IoT devices or applications where lightweight libraries are required.
• Mobile Applications: For Android and other mobile platforms, Paho provides an easy-to-use MQTT client library.
• Prototyping and Development: Often used for creating simple MQTT clients for testing and prototyping systems before deploying them to production.

Key Features:

• Client Library: Paho is a client library, not a broker.
• Cross-Platform: It supports multiple platforms, including Android, Java, C, and Python.
• Lightweight: The library is optimized for low-resource environments, making it suitable for embedded devices.
• Simple API: It offers an easy-to-use API for connecting to an MQTT broker and publishing/subscribing to topics.
• Security: Paho also supports TLS/SSL and authentication for secure communication with brokers.

Example — 1:

Paho MQTT (For IoT, Sensors, Connected Cars)

📡 Best for: IoT, real-time monitoring, low-power devices.
💡 Library: Eclipse Paho

Setup (Gradle)

implementation 'org.eclipse.paho:org.eclipse.paho.client.mqttv3:1.2.5'
implementation 'org.eclipse.paho:org.eclipse.paho.android.service:1.1.1'

Code Example: MQTT Connection, Subscribe & Publish and

import android.content.Context
import org.eclipse.paho.android.service.MqttAndroidClient
import org.eclipse.paho.client.mqttv3.*
import javax.net.ssl.SSLSocketFactory


class MqttManager(context: Context) {
    // MQTT Broker URL with SSL security enabled
    private val brokerUrl = "ssl://broker.hivemq.com:8883" // Secure MQTT broker
    private val clientId = "AndroidClient" // Unique client ID for connection
    private val topic = "iot/device" // Topic to publish/subscribe to
    private val username = "yourUsername" // Optional username for authentication
    private val password = "yourPassword" // Optional password for authentication

    

// Creating MQTT client instance
    private val mqttClient = MqttAndroidClient(context, brokerUrl, clientId)




    /**
     * Connects to the MQTT broker with security and timeout settings.
     * Uses SSL for a secure connection and sets a keep-alive interval.
     */
    fun connect() {
        val options = MqttConnectOptions().apply {
            isCleanSession = true // Ensures no previous session data is stored
            userName = username // Sets MQTT username if required
            password = password.toCharArray() // Sets MQTT password if required
            socketFactory = SSLSocketFactory.getDefault() // Enables SSL security
            connectionTimeout = 10 // Custom timeout in seconds before giving up
            keepAliveInterval = 30 // Keeps connection alive by sending pings every 30 seconds
        }




        // Attempt to connect to the broker
        mqttClient.connect(options, null, object : IMqttActionListener {
            override fun onSuccess(asyncActionToken: IMqttToken?) {
                println("Connected to MQTT broker")
                subscribe() // Automatically subscribe to topic after connecting
            }

            override fun onFailure(asyncActionToken: IMqttToken?, exception: Throwable?) {
                println("Connection failed: ${exception?.message}") // Handle connection failure
            }
        })
    }





    /**
     * Subscribes to a topic to receive messages.
     * When a message is received, it is printed to the console with a description.
     */
    fun subscribe() {
        mqttClient.subscribe(topic, 1) { _, message ->
            val receivedMessage = message.payload.decodeToString() // Convert byte array to string
            val description = "Received message: $receivedMessage with QoS ${message.qos}" // QoS level indicates message reliability
            println(description) // Display received message
        }
    }




    /**
     * Publishes a message to the MQTT topic.
     * Includes an optional description for logging purposes.
     * @param message The message content to be sent.
     * @param description Additional details about the message (default: empty string).
     * qos = 1 : QoS 1 ensures the message is delivered at least once
        QoS 0 (At most once) → No guarantee of delivery.
        QoS 1 (At least once) → Guaranteed delivery, but may receive duplicates.
        QoS 2 (Exactly once) → Guaranteed delivery with no duplicates (most reliable but slowest).
     */
    fun publish(message: String, description: String = "") {
        val mqttMessage = MqttMessage(message.toByteArray()).apply { qos = 1 } // QoS 1 ensures the message is delivered at least once
        mqttClient.publish(topic, mqttMessage) // Publish the message to the broker
        println("Published: $message | Description: $description") // Log published message
    }


}

Example — 2

HiveMQ MQTT (For IoT, Sensors, Connected Cars)

📡 Although HiveMQ itself doesn’t have a specific Android library, you can use Paho MQTT for Android, as it is lightweight and works well for client-side MQTT communication.
💡 Library: Eclipse Paho

Setup (Gradle)

HiveMQ MQTT client implementation, explaining authentication, QoS, SSL/TLS security, and message handling.

Paho MQTT Client for Android

Add this dependency to your build.gradle file:

implementation 'org.eclipse.paho:client.mqttv3:1.2.5'

If you want to use the HiveMQ MQTT Client:

You can also integrate the HiveMQ MQTT Client library, but it is primarily intended for server-side applications, so make sure you test thoroughly for compatibility with Android. You can add the following dependency for HiveMQ MQTT Client:

implementation 'com.hivemq:hivemq-mqtt-client:1.1.0'

— — — -

import android.content.Context
import com.hivemq.client.mqtt.MqttClient
import com.hivemq.client.mqtt.datatypes.MqttQos
import com.hivemq.client.mqtt.mqtt3.Mqtt3AsyncClient
import java.nio.charset.StandardCharsets

class HiveMqttManager(context: Context) {
    // HiveMQ MQTT Broker URL (Secure Connection)
    private val brokerUrl = "ssl://broker.hivemq.com:8883" // Using SSL/TLS for encryption
    private val clientId = "AndroidClient" // Unique client ID to identify this device
    private val topic = "iot/device" // MQTT topic for publishing and subscribing
    private val username = "yourUsername" // Optional username for authentication
    private val password = "yourPassword" // Optional password for authentication



    // Creating an asynchronous MQTT client using HiveMQ's MQTT Client Library
    private val mqttClient: Mqtt3AsyncClient = MqttClient.builder()
        .useMqttVersion3() // Using MQTT version 3.1.1
        .identifier(clientId) // Setting client ID for connection
        .serverHost("broker.hivemq.com") // Setting the broker host
        .serverPort(8883) // Secure port for SSL/TLS connections
        .sslWithDefaultConfig() // Enables SSL/TLS security
        .buildAsync() // Builds an asynchronous client for non-blocking operations




    /**
     * Connects to the HiveMQ broker securely using authentication if provided.
     * It applies SSL/TLS encryption and keeps the session alive.
     */
    fun connect() {
        mqttClient.connectWith()
            .simpleAuth() // Enables authentication
            .username(username) // Assigns username
            .password(password.toByteArray()) // Assigns password (converted to byte array)
            .applySimpleAuth()
            .send()
            .whenComplete { _, throwable ->
                if (throwable == null) {
                    println("Connected to HiveMQ broker successfully")
                    subscribe() // Auto-subscribe to topic after successful connection
                } else {
                    println("Connection failed: ${throwable.message}")
                }
            }
    }




    /**
     * Subscribes to an MQTT topic to receive messages from the broker.
     * The QoS level ensures at least one delivery of messages.
     */
    fun subscribe() {
        mqttClient.subscribeWith()
            .topicFilter(topic) // Specify the topic to listen to
            .qos(MqttQos.AT_LEAST_ONCE) // Ensures messages are delivered at least once
            .callback { publish ->
                val receivedMessage = String(publish.payloadAsBytes, StandardCharsets.UTF_8)
                val description = "Received message: $receivedMessage with QoS ${publish.qos}"
                println(description) // Logs received message with QoS level
            }
            .send()
    }




    /**
     * Publishes a message to the specified MQTT topic.
     * Uses QoS level 1 (At least once) for guaranteed delivery.
     *
     * @param message The actual message content to send.
     * @param description Additional logging information (optional).
     */
    fun publish(message: String, description: String = "") {
        mqttClient.publishWith()
            .topic(topic) // Defines the topic to send data to
            .payload(message.toByteArray()) // Converts message to byte array for transmission
            .qos(MqttQos.AT_LEAST_ONCE) // Guarantees at least one delivery
            .send()
            .whenComplete { _, throwable ->
                if (throwable == null) {
                    println("Published successfully: $message | Description: $description")
                } else {
                    println("Publish failed: ${throwable.message}")
                }
            }
    }
}