<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by EMQ Technologies on Medium]]></title>
        <description><![CDATA[Stories by EMQ Technologies on Medium]]></description>
        <link>https://medium.com/@emqx?source=rss-2287ea9742a6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Cp7OwW2ZTRJQ4xaj8Dg7zg@2x.png</url>
            <title>Stories by EMQ Technologies on Medium</title>
            <link>https://medium.com/@emqx?source=rss-2287ea9742a6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 13:36:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@emqx/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Getting Started with MQTT on Zephyr RTOS: From TCP Quickstart to mTLS with EMQX]]></title>
            <link>https://emqx.medium.com/getting-started-with-mqtt-on-zephyr-rtos-from-tcp-quickstart-to-mtls-with-emqx-e448d1fadaa0?source=rss-2287ea9742a6------2</link>
            <guid isPermaLink="false">https://medium.com/p/e448d1fadaa0</guid>
            <dc:creator><![CDATA[EMQ Technologies]]></dc:creator>
            <pubDate>Thu, 23 Jul 2026 09:00:09 GMT</pubDate>
            <atom:updated>2026-07-23T09:00:09.131Z</atom:updated>
            <content:encoded><![CDATA[<h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#i-introduction">I. Introduction</a></h3><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_11-project-overview">1.1 Project Overview</a></h3><p>This post demonstrates how to connect an embedded device running Zephyr RTOS to an EMQX message broker via the <a href="https://www.emqx.com/en/blog/the-easiest-guide-to-getting-started-with-mqtt">MQTT protocol</a>, and gradually upgrade the connection from plaintext TCP to secure communication using mutual TLS (mTLS) authentication. All code has been verified on the native_sim/native/64 simulation platform, allowing you to replicate the entire workflow within a Linux container without requiring actual hardware.</p><p>This sample project offers two optional operational modes to suit your needs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gddYZM_dw3MFck8L1TI3YA.png" /></figure><p><strong>Mode Selection Recommendation:</strong></p><ul><li>If you only want to quickly experience <a href="https://www.emqx.com/en/blog/mqtt-5-introduction-to-publish-subscribe-model">MQTT publish/subscribe</a>, choose <strong>NSOS/TCP-only</strong> (no sudo or certificates required).</li><li>If you need to evaluate the feasibility of TLS/mTLS on Zephyr, choose <strong>TAP+TLS</strong> (requires a Linux host/container with NET_ADMIN privileges and /dev/net/tun).</li></ul><p>The system architecture is illustrated below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*90fvtTgpMzJThlM3" /></figure><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_12-emqx-a-cloud-native-mqtt-broker-for-modern-iot">1.2 EMQX: A Cloud-Native MQTT Broker for Modern IoT</a></h3><p>EMQX is a massive-scale, distributed IoT connectivity platform. A single node can stably support 1.5 million concurrent MQTT connections, while a cluster can scale horizontally to 100 million concurrent connections, delivering million-level message throughput per second with millisecond-level latency.</p><p><strong>Key Features:</strong></p><ul><li><a href="https://www.emqx.com/en/blog/mqtt-over-quic">MQTT over QUIC</a><strong>:</strong> EMQX pioneered in introducing <a href="https://www.emqx.com/en/blog/quic-protocol-the-features-use-cases-and-impact-for-iot-iov">QUIC</a> into the MQTT protocol, significantly improving connection stability and message throughput under weak networks or frequent topology changes (e.g., Connected Vehicles).</li><li><strong>Multi-Protocol Gateway:</strong> Beyond MQTT, it natively supports HTTP, WebSocket, LwM2M/CoAP, and other protocols, providing a unified device access entry.</li><li><strong>Message Queues (EMQX 6.x):</strong> Supports named persistent queues to decouple publishing and subscribing. Messages are automatically buffered when consumers are offline and replayed upon reconnection, ideal for intermittent connection scenarios typical of Zephyr IoT edge devices.</li><li><strong>MQTT Streams (EMQX 6.x):</strong> A persistent, replayable message stream model that supports replaying historical data by timestamp or offset, removing the need for external streaming systems like Kafka.</li><li><strong>Data Integration and Processing:</strong> The built-in rule engine extracts, filters, and transforms MQTT messages in real-time using SQL syntax. It provides out-of-the-box integration with mainstream systems such as Kafka, PostgreSQL, TimescaleDB, InfluxDB, MongoDB, AWS S3, and Google BigQuery.</li></ul><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_13-zephyr-rtos-a-high-performance-real-time-operating-system-for-iot">1.3 Zephyr RTOS: A High-Performance Real-Time Operating System for IoT</a></h3><p><a href="https://www.zephyrproject.org/">Zephyr RTOS</a> is an open-source, Apache 2.0-licensed real-time operating system hosted by the Linux Foundation, specifically engineered for resource-constrained embedded systems. It scales seamlessly from simple environmental sensors and wearable LEDs to complex embedded controllers, smartwatches, and wireless IoT applications.</p><p>Zephyr supports over a dozen CPU architectures, including ARM Cortex-M/A/R, RISC-V (32/64-bit), x86 (32/64-bit), ARC, MIPS, Xtensa, and SPARC, and is compatible with <a href="https://docs.zephyrproject.org/latest/boards/index.html">hundreds of development boards</a>. This article utilizes the native_sim platform to simulate execution on x86, enabling complete development and debugging without physical hardware.</p><p><strong>Core Technical Highlights:</strong></p><ul><li><strong>Native Network Stack:</strong> Zephyr features a robust, fully-featured network stack supporting LwM2M, BSD sockets, TCP/UDP/IPv4/IPv6, DNS, TLS/mbedTLS, HTTP, MQTT, CoAP, and optional OpenThread mesh networking. The stack is highly tailorable, compiling only the specific protocol modules your application requires.</li><li><strong>Modular Configuration &amp; Static Allocation:</strong> Applications use <strong>Kconfig</strong> to include only necessary features and define resource limits. Hardware is described via <strong>Devicetree</strong>, enabling static allocation of system resources at compile time, which significantly reduces code size and optimizes performance.</li><li><strong>Built-in Security and Reliability:</strong> Features highly configurable stack overflow protection, kernel object and driver permission tracking, thread-level memory isolation, and a comprehensive security vulnerability reporting and CVE tracking mechanism.</li><li><strong>Rich OS Services:</strong> Includes a full Bluetooth 5.0 Low Energy stack (with Mesh support), virtual file systems (ext2, FatFS, LittleFS), persistent storage (NVS/Settings), a multi-backend logging system, and an interactive full-featured Shell.</li><li>native_sim <strong>Simulation Platform:</strong> Compiles Zephyr directly into a native Linux executable, simulating subsystems like networking, file systems, and Bluetooth. This capability drastically accelerates development and testing iterations, serving as the core engine for the native_sim/native/64 platform used in this guide.</li></ul><p><strong>Key Zephyr Components in This Project:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VnJg9WSMW0rjrP1ZQClWuw.png" /></figure><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#ii-environment-setup">II. Environment Setup</a></h3><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_21-emqx-installation-and-rapid-deployment">2.1 EMQX Installation and Rapid Deployment</a></h3><p>EMQX supports various installation methods. This section introduces four methods in order of recommendation; you can choose the one that best fits your workflow.</p><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#method-1-docker-containerized-deployment-recommended">Method 1: Docker Containerized Deployment (Recommended)</a></h4><p>Containerization is the fastest way to experience EMQX and is the preferred choice in the official Quick Start Guide. Ensure that <a href="https://www.docker.com/">Docker</a> is installed and running before executing the command.</p><pre># Download and start the latest EMQX Enterprise edition<br>docker run -d --name emqx \<br>  -p 1883:1883 \<br>  -p 8083:8083 \<br>  -p 8084:8084 \<br>  -p 8883:8883 \<br>  -p 18083:18083 \<br>  emqx/emqx-enterprise:latest</pre><p>Once started, access the Dashboard via your browser at http://localhost:18083 (replace localhost with your actual server IP). Log in using the default credentials: Username admin, Password public.</p><p><strong>Quick Verification with MQTTX (Optional):</strong> <a href="https://mqttx.app/zh">MQTTX</a> is an official open-source, cross-platform <a href="https://www.emqx.com/en/blog/introduction-to-mqtt-5">MQTT 5.0</a> client tool developed by EMQ, available as a Web version (no installation required) and a Desktop application. You can visit <a href="https://mqttx.app/web-client">MQTTX Web</a>, create a new connection pointing to ws://localhost:8083, and quickly subscribe to or publish messages to verify broker connectivity.</p><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#method-2-package-deployment-macos-linux">Method 2: Package Deployment (macOS / Linux)</a></h4><p>If you need to deploy EMQX directly on a physical or virtual machine for subsequent performance tuning, you can use the official operating system installation packages. Supported platforms include RedHat, CentOS, RockyLinux, Ubuntu, Debian, macOS, and more.</p><p>Taking <strong>macOS</strong> as an example:</p><pre># 1. Obtain the latest installation package from the official download page:<br>#    https://www.emqx.com/en/downloads-and-install/enterprise?os=macOS<br><br># 2. Extract and start EMQX in the foreground to monitor logs easily<br>./emqx/bin/emqx foreground<br><br># 3. Access Dashboard：http://localhost:18083（admin / public）<br><br># 4. Stop EMQX when finished<br>./emqx/bin/emqx stop</pre><blockquote>Note: For installation commands corresponding to other Linux distributions, please refer to the <a href="https://docs.emqx.com/en/emqx/latest/deploy/install.html">EMQX Official Installation Guide</a>.</blockquote><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#method-3-emqx-cloud-fully-managed-service">Method 3: EMQX Cloud Fully Managed Service</a></h4><p>In addition to self-hosted deployment, EMQX offers a fully managed cloud service called EMQX Cloud. With a few simple registration steps, you can spin up a fully managed, maintenance-free MQTT message broker without the hassle of setting up and managing your own infrastructure. This is ideal for rapid staging or production-scale deployments.</p><blockquote>You can visit the <a href="https://www.emqx.com/en/cloud">EMQX Cloud Registration Page</a> for a free trial. For a detailed primer, see the <a href="https://docs.emqx.com/en/cloud/latest/quick_start/introduction.html">EMQX Cloud Quick Start Guide</a>.</blockquote><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#method-4-public-test-broker-alternative-for-quick-testing">Method 4: Public Test Broker (Alternative for Quick Testing)</a></h4><p>If you cannot install Docker or set up a local deployment immediately, EMQX provides a free public broker for quick connectivity testing:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PDOLViH1HfILhCK62t6JGw.png" /></figure><pre># Quickly verify connectivity within your container<br>mqtt_cli conn -h broker.emqx.io -p 1883</pre><blockquote><strong>Warning:</strong> The public broker is a shared public environment. It does not guarantee stability, data persistence, or message privacy. It is strictly for quick evaluation and should <strong>not</strong> be used for formal development or production tasks.</blockquote><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_22-docker-based-zephyr-environment-setup-and-west-toolchain-preparation">2.2 Docker-Based Zephyr Environment Setup and West Toolchain Preparation</a></h3><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#requirements">Requirements</a></h4><ul><li>Linux host (Kernel $\ge$ 5.x)</li><li>Docker installed and running</li><li>TUN module loaded</li></ul><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#step-1-verify-the-tun-module">Step 1: Verify the TUN Module</a></h4><pre>lsmod | grep tun<br># If not loaded, run:<br>sudo modprobe tun</pre><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#step-2-run-the-docker-container">Step 2: Run the Docker Container</a></h4><p>The official Zephyr build image (ghcr.io/zephyrproject-rtos/zephyr-build) comes pre-configured with west, CMake, Python venv, Zephyr SDK, and GCC toolchains, eliminating manual setup.</p><pre># Pull the official image<br>docker pull ghcr.io/zephyrproject-rtos/zephyr-build:main<br><br># Start the container（--cap-add=NET_ADMIN and --device=/dev/net/tun are required for TAP mode）<br>docker run -d \<br>    --name zephyr-tap \<br>    --cap-add=NET_ADMIN \<br>    --device=/dev/net/tun \<br>    -p 2222:22 \<br>    -v ~/Projects/EMQ/ZephyrProject:/workdir \<br>    ghcr.io/zephyrproject-rtos/zephyr-build:main \<br>    sleep infinity<br><br># Enter the container<br>docker exec -it zephyr-tap bash</pre><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#step-3-initialize-the-zephyr-workspace">Step 3: Initialize the Zephyr Workspace</a></h4><pre># Create workspace<br>mkdir -p /workdir &amp;&amp; cd /workdir<br><br># Initialize Zephyr repository<br>west init -m https://github.com/zephyrproject-rtos/zephyr --mr v4.1-branch<br>west update<br><br># Export Zephyr CMake package<br>west zephyr-export<br><br># Activate Python virtual environment<br>source /var/lib/zephyr/venv/bin/activate<br>echo &#39;source /var/lib/zephyr/venv/bin/activate&#39; &gt;&gt; ~/.bashrc</pre><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#step-4-install-iptables-required-for-tap-mode-network-forwarding">Step 4: Install iptables (required for TAP mode network forwarding)</a></h4><pre>apt-get update &amp;&amp; apt-get install -y --no-install-recommends \<br>    iptables \<br>    &amp;&amp; rm -rf /var/lib/apt/lists/*</pre><blockquote><strong>Note for macOS / Docker Desktop Users:</strong> TAP mode is restrictive on macOS Docker Desktop. Use <strong>NSOS/TCP-only</strong> mode instead, or switch to a Linux host/WSL2 environment.</blockquote><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#iii-zephyr-mqtt-demo-walkthrough">III. Zephyr MQTT Demo Walkthrough</a></h3><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_31-project-structure-and-core-logic">3.1 Project Structure and Core Logic</a></h3><p>The repository follows a standard Zephyr application layout. The core client logic resides in src/main.c (~650 lines), while the remaining files handle mode configuration and automated startup scripts:</p><pre>mqtt-client-C-Zephyr/<br>├── src/main.c                  # MQTT client core logic<br>├── prj.conf                    # Shared base Kconfig (Networking/DNS/Shell/MQTT)<br>├── prj-nsos.conf               # NSOS/TCP-only mode Kconfig overlay<br>├── prj-tap-tls.conf            # TAP+TLS mode Kconfig overlay<br>├── run-zephyr-nsos.sh          # One-click startup script for NSOS mode<br>└── run-zephyr-tap.sh           # One-click startup script for TAP mode (includes network setup)</pre><p>The execution flow within src/main.c operates as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Z-vTXm0T8L5C6n46" /></figure><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_32-command-line-arguments-and-core-kconfig-options">3.2 Command-Line Arguments and Core Kconfig Options</a></h3><p>The mqtt_cli command provides three sub-commands sharing a unified set of connection and TLS arguments:</p><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#general-connection-arguments-all-sub-commands">General Connection Arguments (All Sub-commands)</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_2LWoMI_-_Irqeg-tD0QuQ.png" /></figure><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#tls-arguments-taptls-mode-only-requires-config_mqtt_lib_tlsy">TLS Arguments (TAP+TLS Mode Only; requires </a><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#tls-arguments-taptls-mode-only-requires-config_mqtt_lib_tlsy">CONFIG_MQTT_LIB_TLS=y)</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zrKaWv3DlTgx_LbP2DBk2w.png" /></figure><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#sub-commands-and-specific-arguments">Sub-commands and Specific Arguments</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kFOogFuGWErlC-QbSdEPQg.png" /></figure><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#two-tier-kconfig-structure">Two-Tier Kconfig Structure</a></h4><p>Base configurations (Network stack, IPv4, TCP, DNS, MQTT, Shell, etc.) are defined in prj.conf. Mode-specific optimizations are injected via <strong>overlay files</strong>:</p><ul><li><strong>NSOS/TCP-only (</strong>prj-nsos.conf<strong>)</strong>: Enables CONFIG_NET_SOCKETS_OFFLOAD=y and CONFIG_NET_NATIVE_OFFLOADED_SOCKETS=y. Zephyr socket calls bypass the internal stack and forward directly to the host POSIX sockets. TLS is <strong>not supported</strong> here, as SO_TLS setsockopt is unavailable on offloaded sockets.</li><li><strong>TAP+TLS (</strong>prj-tap-tls.conf<strong>)</strong>: Spins up the complete mbedTLS stack (allocated with a 30KB heap, ECDHE-RSA-AES-128-GCM cipher suites, PEM parsing, and PSA key types). Zephyr assigns itself a dedicated IP (192.0.2.1) and accesses external networks using NAT through the host via the zeth virtual interface.</li></ul><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_33-core-code-breakdown">3.3 Core Code Breakdown</a></h3><p>This section dissects the three most critical components within src/main.c: the Event Callback, the Connection Engine, and TLS Credential Lifecycle Management.</p><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#code-block-1-mqtt-event-callback">Code Block 1: MQTT Event Callback</a></h4><pre>void mqtt_evt_handler(struct mqtt_client *client, const struct mqtt_evt *evt)<br>{<br>    switch (evt-&gt;type) {<br>    case MQTT_EVT_CONNACK:<br>        if (evt-&gt;result == 0) {<br>            is_connected = true;                   // ① Connection success flag<br>        } else {<br>            LOG_ERR(&quot;MQTT connection refused: %d&quot;, evt-&gt;result);<br>        }<br>        break;<br>    case MQTT_EVT_DISCONNECT:<br>        is_connected = false;<br>        break;<br>    case MQTT_EVT_PUBLISH: {<br>        const struct mqtt_publish_param *pub = &amp;evt-&gt;param.publish;<br>        // ② Auto-acknowledge QoS 1/2 messages to prevent broker retransmission<br>        if (pub-&gt;message.topic.qos == MQTT_QOS_1_AT_LEAST_ONCE) {<br>            const struct mqtt_puback_param ack = { .message_id = pub-&gt;message_id };<br>            mqtt_publish_qos1_ack(client, &amp;ack);<br>        } else if (pub-&gt;message.topic.qos == MQTT_QOS_2_EXACTLY_ONCE) {<br>            const struct mqtt_pubrec_param rec = { .message_id = pub-&gt;message_id };<br>            mqtt_publish_qos2_receive(client, &amp;rec);<br>        }<br>        uint8_t payload_buf[128];<br>        int len = MIN(pub-&gt;message.payload.len, sizeof(payload_buf) - 1);<br>        int rc = mqtt_read_publish_payload(client, payload_buf, len);<br>        if (rc &gt;= 0) {<br>            payload_buf[rc] = &#39;\0&#39;;<br>            // ③ Use shell_print via global shell pointer to output data.<br>            //   Using LOG_INF on native_sim causes duplicate prints due to dual-channel routing.<br>            shell_print(mqtt_evt_shell,<br>                &quot;[Received Msg] Topic: %.*s | Payload: %s&quot;,<br>                pub-&gt;message.topic.topic.size,<br>                pub-&gt;message.topic.topic.utf8, payload_buf);<br>        }<br>        break;<br>    }<br>    }<br>}</pre><p><strong>Analysis:</strong></p><ul><li>When the Zephyr <a href="https://www.emqx.com/en/blog/mqtt-client-tools">MQTT library</a> receives a CONNACK packet with a result == 0 (indicating the broker accepted the connection), the global flag is_connected is set to true.</li><li>For QoS 1 and QoS 2 messages, you must explicitly invoke mqtt_publish_qos1_ack() or mqtt_publish_qos2_receive() to acknowledge receipt. Failing to do so causes the broker to repeatedly retransmit messages (modeled after Zephyr&#39;s official secure_mqtt_sensor_actuator sample).</li><li>Upon receiving a message, output is handled via shell_print using the global shell pointer mqtt_evt_shell. On the native_sim platform, standard logging functions like LOG_INF and printk output through both the native UART PTY channel and the log backend, causing duplicate stdout entries. shell_print targets a single UART channel, eliminating duplication.</li></ul><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#code-block-2-unified-connection-engine-common_mqtt_connect">Code Block 2: Unified Connection Engine </a><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#code-block-2-unified-connection-engine-common_mqtt_connect">common_mqtt_connect()</a></h4><pre>static int common_mqtt_connect(const struct shell *sh, struct mqtt_conn_params *p)<br>{<br>    is_connected = false;<br>    mqtt_evt_shell = sh;  // Store the shell pointer for event callback use<br>    // ① DNS Resolution: Hostname -&gt; IP Address<br>    struct zsock_addrinfo hints = { .ai_family = AF_INET, .ai_socktype = SOCK_STREAM };<br>    struct zsock_addrinfo *res = NULL;<br>    char port_str[6];<br>    snprintf(port_str, sizeof(port_str), &quot;%d&quot;, p-&gt;port);<br>    int rc = zsock_getaddrinfo(p-&gt;host, port_str, &amp;hints, &amp;res);<br>    if (rc != 0) { /* error */ return rc; }<br>    memcpy(&amp;broker_addr, res-&gt;ai_addr, res-&gt;ai_addrlen);<br>    zsock_freeaddrinfo(res);<br>    // ② Initialize the MQTT client struct<br>    mqtt_client_init(&amp;client_ctx);<br>    client_ctx.broker = &amp;broker_addr;<br>    client_ctx.evt_cb = mqtt_evt_handler;<br>    client_ctx.client_id.utf8 = (uint8_t *)client_id_global;<br>    client_ctx.client_id.size = strlen(client_id_global);<br>    client_ctx.protocol_version = MQTT_VERSION_3_1_1;<br>    client_ctx.rx_buf = rx_buffer;<br>    client_ctx.rx_buf_size = sizeof(rx_buffer);<br>    client_ctx.tx_buf = tx_buffer;<br>    client_ctx.tx_buf_size = sizeof(tx_buffer);<br>    client_ctx.keepalive = p-&gt;keepalive;<br>    // ③ TLS Credential Loading (TLS mode only, see Code Block 3)<br>    if (p-&gt;use_tls) {<br>#ifdef CONFIG_MQTT_LIB_TLS<br>        /* ... TLS credential loading and transport config ... */<br>        client_ctx.transport.type = MQTT_TRANSPORT_SECURE;<br>#else<br>        shell_error(sh, &quot;Error: TLS options provided but TLS support is disabled&quot;);<br>        return -ENOTSUP;<br>#endif<br>    } else {<br>        client_ctx.transport.type = MQTT_TRANSPORT_NON_SECURE;<br>    }<br>    // ④ Initiate MQTT CONNECT<br>    rc = mqtt_connect(&amp;client_ctx);<br>    if (rc != 0) { /* error */ return rc; }<br>    // ⑤ Poll and wait for CONNACK (5-second timeout)<br>    uint32_t timeout = 0;<br>    while (!is_connected &amp;&amp; timeout &lt; 5000) {<br>        struct zsock_pollfd fds[1] = {<br>            { .fd = get_client_fd(&amp;client_ctx), .events = ZSOCK_POLLIN }<br>        };<br>        if (zsock_poll(fds, 1, 100) &gt; 0) {<br>            mqtt_input(&amp;client_ctx);  // Drives the event callback<br>        }<br>        k_msleep(100);<br>        timeout += 100;<br>    }<br>    if (!is_connected) { /* timeout error */ return -ETIMEDOUT; }<br>    shell_print(sh, &quot;Connection successful!&quot;);<br>    return 0;<br>}</pre><p><strong>Analysis:</strong></p><p>common_mqtt_connect() serves as the shared entry point for the conn, sub, and pub sub-commands.</p><ul><li>Invokes zsock_getaddrinfo() for DNS resolution. Under NSOS mode, this relies directly on the host&#39;s glibc. Under TAP mode, it utilizes Zephyr&#39;s built-in DNS stack querying 8.8.8.8.</li><li>Populates the mqtt_client structure, mapping memory buffers, event callbacks, Client ID, and the MQTT protocol version.</li><li>Loads TLS credentials conditionally based on configuration.</li><li>Calls mqtt_connect() to dispatch the MQTT CONNECT packet.</li><li>Polls the socket at 100ms intervals. Upon capturing an MQTT_EVT_CONNACK, the callback sets is_connected to true, breaking the loop and printing a success confirmation.</li></ul><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#code-block-3-tls-credential-loading-and-lifecycle-management">Code Block 3: TLS Credential Loading and Lifecycle Management</a></h4><pre>// Clear stale credentials before reconnecting (avoids cross-connection sec_tag remnants)<br>static void clear_registered_credential(enum tls_credential_type type)<br>{<br>    tls_credential_delete(101, type);       // ① Delete from the Zephyr credential cache<br>    struct credential_slot *slot = credential_slot_for_type(type);<br>    if (slot &amp;&amp; slot-&gt;buf) {<br>        free(slot-&gt;buf);                    // ② Free the stale buffer<br>        slot-&gt;buf = NULL;<br>    }<br>}<br>static int load_and_register_credential(const struct shell *sh,<br>                                         const char *path,<br>                                         enum tls_credential_type type)<br>{<br>    // ③ Malloc separate buffers for each credential type (do not share a static buffer)<br>    uint8_t *file_buf = malloc(3073);  // 3072 + 1 null terminator<br>    size_t br = fread(file_buf, 1, 3072, f);<br>    fclose(f);<br>    file_buf[br] = &#39;\0&#39;;               // ④ NUL terminate (required by crt_is_pem())<br>    clear_registered_credential(type); // ⑤ Clear before loading<br>    int rc = tls_credential_add(101, type, file_buf, br + 1);<br>    // ...<br>    slot-&gt;buf = file_buf;              // ⑥ Keep track of the pointer for future free calls<br>    return 0;<br>}<br>// Execution sequence within common_mqtt_connect():<br>clear_registered_credential(TLS_CREDENTIAL_CA_CERTIFICATE);<br>clear_registered_credential(TLS_CREDENTIAL_PUBLIC_CERTIFICATE);<br>clear_registered_credential(TLS_CREDENTIAL_PRIVATE_KEY);<br>load_and_register_credential(sh, p-&gt;ca_path,   TLS_CREDENTIAL_CA_CERTIFICATE);<br>load_and_register_credential(sh, p-&gt;cert_path, TLS_CREDENTIAL_PUBLIC_CERTIFICATE);<br>load_and_register_credential(sh, p-&gt;key_path,  TLS_CREDENTIAL_PRIVATE_KEY);<br>client_ctx.transport.tls.config.sec_tag_list = sec_tag_list;  // {101}<br>client_ctx.transport.tls.config.peer_verify = p-&gt;insecure<br>    ? TLS_PEER_VERIFY_NONE : TLS_PEER_VERIFY_REQUIRED;<br>client_ctx.transport.tls.config.hostname = p-&gt;host;</pre><p><strong>Analysis:</strong></p><p>Zephyr’s tls_credential_add() requires the passed buffer to remain valid throughout the credential&#39;s active lifecycle. Consequently, you cannot use stack-allocated or shared static buffers, as subsequent calls will corrupt previously loaded assets.</p><p>This implementation handles this constraint by allocating (malloc) individual, independent buffers for the CA, Certificate, and Key components. Stale allocations are purged via tls_credential_delete() and free() prior to any new connection attempt. This pattern is a verified best practice: if old credential data is not purged, certificate leftovers between sessions will cause the mTLS handshake to fail with error code -0x4e / -103.</p><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#iv-hands-on-connecting-zephyr-to-emqx-via-plaintext-tcp">IV. Hands-on: Connecting Zephyr to EMQX via Plaintext TCP</a></h3><p>This chapter utilizes the <strong>NSOS/TCP-only</strong> mode. It requires no sudo privileges or certificates, making it the simplest configuration to build and launch.</p><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_41-minimal-project-configuration">4.1 Minimal Project Configuration</a></h3><p>Execute the following commands inside the container to build the application:</p><pre>cd /workdir/mqtt-client-C-Zephyr<br># NSOS Build for NSOS mode<br>west build -d build-nsos -p always -b native_sim/native/64 . \<br>    -- -DOVERLAY_CONFIG=prj-nsos.conf</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gcPU8kpvnf6ZdY_H" /></figure><p><em>Upon a successful build, the terminal will output:</em> [211/211] Running utility command for native_runner_executable.</p><p>The Zephyr build system automatically merges prj.conf (base configurations) and prj-nsos.conf (NSOS overlay). The critical Kconfig options are as follows:</p><pre># prj-nsos.conf — Enable NSOS networking in three lines<br>CONFIG_NET_DRIVERS=y<br>CONFIG_NET_SOCKETS_OFFLOAD=y<br>CONFIG_NET_NATIVE_OFFLOADED_SOCKETS=y<br># CONFIG_ETH_NATIVE_TAP is not set<br># Shared network/DNS/MQTT configurations in prj.conf remain unchanged<br>CONFIG_NETWORKING=y<br>CONFIG_NET_IPV4=y<br>CONFIG_NET_TCP=y<br>CONFIG_NET_SOCKETS=y<br>CONFIG_DNS_SERVER1=&quot;8.8.8.8&quot;<br>CONFIG_MQTT_LIB=y</pre><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_42-startup-and-connectivity-verification">4.2 Startup and Connectivity Verification</a></h3><pre># Start Zephyr with an interactive shell<br>./run-zephyr-nsos.sh<br># Once the uart:~$ prompt appears, test the connection<br>uart:~$ mqtt_cli conn -h 100.108.113.19 -p 1883</pre><p>Expected Output:</p><pre>Connecting to 100.108.113.19:1883 (ID: zephyr-emqx-123456, Keepalive: 60) ...<br>[TLS] Calling mqtt_connect (transport.type=0)...<br>Connection successful!<br>Entered conn blocking maintenance mode. Press Ctrl+C to terminate simulation process.</pre><p>Verification Results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ibYDCoDqmQUU7y09" /></figure><ul><li>The Zephyr shell prints Connection successful! upon completing the handshake.</li><li>Navigating to the <strong>Connections</strong> page on the EMQX Dashboard will display the new connection (e.g., zephyr-emqx-679688 with a randomized Client ID) showing a status of Connected, protocol MQTT, and version 3.1.1.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oZaLdUGQN95yRj4P" /></figure><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_43-implementation-highlights">4.3 Implementation Highlights</a></h3><p>The core workflow within common_mqtt_connect() (src/main.c) executes as follows:</p><ol><li><strong>DNS Resolution:</strong> Invokes zsock_getaddrinfo() to resolve hostnames into IP addresses. In NSOS mode, this call maps directly to the host&#39;s glibc implementation of getaddrinfo(), automatically utilizing the host&#39;s DNS configuration.</li><li><strong>MQTT Client Initialization:</strong> Calls mqtt_client_init(&amp;client_ctx) and binds the event callback handler mqtt_evt_handler.</li><li><strong>Establishing Connection:</strong> Invokes mqtt_connect(&amp;client_ctx) to dispatch the TCP handshake and subsequent MQTT CONNECT packet.</li><li><strong>Awaiting </strong><a href="https://www.emqx.com/en/blog/mqtt5-new-features-reason-code-and-ack">CONNACK</a><strong>:</strong> Polls the socket file descriptor. Upon receiving MQTT_EVT_CONNACK, the global is_connected flag is set to true, and Connection successful! is printed.</li><li><strong>Event Dispatching:</strong> The conn sub-command enters a heartbeat maintenance loop; sub calls mqtt_subscribe() and transitions into a long-polling listening loop; pub triggers mqtt_publish() and gracefully disconnects.</li></ol><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_44-message-publishing-and-subscription-demo">4.4 Message Publishing and Subscription Demo</a></h3><p>Once the connection is established, you can experience MQTT’s core publish/subscribe model. We recommend opening two terminals: one running the Zephyr shell as the subscriber, and another using mqtt_cli pub or the EMQX Dashboard&#39;s built-in WebSocket client as the publisher.</p><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#scenario-1-subscribing-and-receiving-messages">Scenario 1: Subscribing and Receiving Messages</a></h4><p>Subscribe to the topic test/zephyr/demo with QoS 1 in the Zephyr shell:</p><pre>uart:~$ mqtt_cli sub -h 100.108.113.19 -t test/zephyr/demo -q 1</pre><p>Expected output：</p><pre>Connecting to 100.108.113.19:1883 (ID: zephyr-emqx-654321, Keepalive: 60) ...<br>Connection successful!<br>Subscribing to topic: &#39;test/zephyr/demo&#39; (QoS 1) ...<br>Entered sub listening state. Waiting for messages... (Press Ctrl+C to exit)</pre><p>Zephyr is now in a continuous listening loop. Open a separate terminal inside the container and use mqtt_cli pub to send a message to the same topic:</p><pre># Execute in a separate terminal on the host or within the container<br>mqtt_cli pub -h 100.108.113.19 -t test/zephyr/demo -m &quot;Hello Zephyr&quot; -q 1</pre><p>The Zephyr shell will instantly display the received message:</p><pre>[Received Msg] Topic: test/zephyr/demo | Payload: Hello Zephyr</pre><p>Verification Results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8Lqn2qqDKzE4n-Pn" /></figure><p>The terminal logs will show a side-by-side or sequential confirmation: the mqtt_cli pub command successfully dispatches the message on one side, while the Zephyr subscriber shell immediately prints the received payload log on the other.</p><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#scenario-2-publishing-messages">Scenario 2: Publishing Messages</a></h4><p>Zephyr can also act as a message producer, publishing a payload and automatically disconnecting upon completion:</p><pre>uart:~$ mqtt_cli pub -h 100.108.113.19 -t test/zephyr/demo -m &quot;Hello from Zephyr!&quot; -q 1</pre><p>Expected output:</p><pre>Connecting to 100.108.113.19:1883 (ID: zephyr-emqx-789012, Keepalive: 60) ...<br>Connection successful!<br>[Published 1/1] Topic=&#39;test/zephyr/demo&#39; | Payload=&#39;Hello from Zephyr!&#39;<br>Publish finished, gracefully disconnecting and exiting...</pre><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#scenario-3-batch-and-interval-publishing">Scenario 3: Batch and Interval Publishing</a></h4><p>Simulate periodic telemetry reporting by using the -L (count) and -I (interval in milliseconds) arguments:</p><pre>uart:~$ mqtt_cli pub -h 100.108.113.19 -t test/zephyr/demo -m &quot;Batch msg&quot; -q 0 -L 3 -I 500</pre><p>Expected output:</p><pre>Connection successful!<br>[Published 1/3] Topic=&#39;test/zephyr/demo&#39; | Payload=&#39;Batch msg&#39;<br>[Published 2/3] Topic=&#39;test/zephyr/demo&#39; | Payload=&#39;Batch msg&#39;<br>[Published 3/3] Topic=&#39;test/zephyr/demo&#39; | Payload=&#39;Batch msg&#39;<br>Publish finished, gracefully disconnecting and exiting...</pre><p>The terminal will print three consecutive [Published N/3] entries spaced exactly 500ms apart.</p><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#verification">Verification</a></h4><p>After executing any of the scenarios above, open the EMQX Dashboard (http://localhost:18083). You can track the zephyr-emqx-xxxxxx client state under the <strong>Connections</strong> tab, and monitor message inbound/outbound statistics for test/zephyr/demo via <strong>Topics</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*f0_Yu2d6yKwlEgb6" /></figure><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#v-advanced-connecting-zephyr-to-emqx-via-secure-tls">V. Advanced: Connecting Zephyr to EMQX via Secure TLS</a></h3><p>This chapter utilizes the <strong>TAP+TLS</strong> mode. It requires a Linux host, a container with NET_ADMIN privileges and /dev/net/tun, and the preparation of TLS certificate materials.</p><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_51-security-prerequisites-obtaining-ssltls-certificates">5.1 Security Prerequisites: Obtaining SSL/TLS Certificates</a></h3><p>According to the official <a href="https://docs.emqx.com/en/emqx/latest/network/tls-certificate.html">EMQX TLS certificate documentation</a>, there are three primary ways to acquire SSL/TLS certificates:</p><ol><li><strong>Self-Signed Certificates:</strong> Generated independently by creating your own Root CA. This method poses security risks and is strictly recommended for local testing or controlled environments.</li><li><strong>Trusted CA Certificates:</strong> Issued by recognized Certificate Authorities (e.g., Let’s Encrypt, DigiCert), which is mandatory for production environments.</li><li><strong>Cloud Provider Certificates:</strong> Managed certificates provisioned via cloud platforms (e.g., AWS, Google Cloud).</li></ol><blockquote>⚠️ <strong>Important:</strong> This guide utilizes self-signed certificates for development and testing purposes only. Deployments in production environments must use certificates issued by a trusted public CA.</blockquote><p><strong>Certificate Generation via OpenSSL (aligned with the official EMQX documentation):</strong></p><pre># 1. Generate a self-signed Root CA<br>openssl genrsa -out rootCA.key 2048<br>openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 3650 -out rootCA.crt<br># 2. Issue the Server Certificate (serverAuth) — used by EMQX<br>openssl genrsa -out mqtt-server.key 2048<br>openssl req -new -key mqtt-server.key -out mqtt-server.csr<br>openssl x509 -req -in mqtt-server.csr -CA rootCA.crt -CAkey rootCA.key \<br>    -CAcreateserial -out mqtt-server.crt -days 365 \<br>    -extfile server-ext.cnf<br># 3. Issue the Client Certificate (clientAuth) — used by Zephyr<br>openssl genrsa -out mqtt-client.key 2048<br>openssl req -new -key mqtt-client.key -out mqtt-client.csr<br>openssl x509 -req -in mqtt-client.csr -CA rootCA.crt -CAkey rootCA.key \<br>    -CAcreateserial -out mqtt-client.crt -days 365 \<br>    -extfile client-ext.cnf</pre><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_52-embedded-setup-zephyr-credential-loading">5.2 Embedded Setup: Zephyr Credential Loading</a></h3><p>The credential loading mechanism within src/main.c manages the active lifecycle of TLS certificates as follows:</p><pre>// Clear residual credentials from the previous session and free allocations<br>clear_registered_credential(TLS_CREDENTIAL_CA_CERTIFICATE);<br>clear_registered_credential(TLS_CREDENTIAL_PUBLIC_CERTIFICATE);<br>clear_registered_credential(TLS_CREDENTIAL_PRIVATE_KEY);<br>// Read certificates from storage, malloc separate buffers, and map to sec_tag 101<br>load_and_register_credential(sh, p-&gt;ca_path,   TLS_CREDENTIAL_CA_CERTIFICATE);<br>load_and_register_credential(sh, p-&gt;cert_path, TLS_CREDENTIAL_PUBLIC_CERTIFICATE);<br>load_and_register_credential(sh, p-&gt;key_path,  TLS_CREDENTIAL_PRIVATE_KEY);<br>// Configure TLS transport parameters<br>client_ctx.transport.type = MQTT_TRANSPORT_SECURE;<br>client_ctx.transport.tls.config.sec_tag_list = sec_tag_list;    // {101}<br>client_ctx.transport.tls.config.peer_verify = p-&gt;insecure<br>    ? TLS_PEER_VERIFY_NONE : TLS_PEER_VERIFY_REQUIRED;<br>client_ctx.transport.tls.config.hostname = p-&gt;host;</pre><blockquote><strong>Note:</strong> Every invocation of mqtt_cli (conn/sub/pub) reloads credentials into the same security tag (sec_tag=101). If stale certificates from previous sessions are not purged, cross-connection pollution will cause the mTLS handshake to fail with error code -0x4e / -103. This cleanup requirement is why load_and_register_credential executes tls_credential_delete(101, type) before committing any new buffers.</blockquote><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_53-tls-project-configuration-and-compilation">5.3 TLS Project Configuration and Compilation</a></h3><p>Execute the following commands inside the container to build the application:</p><pre>cd /workdir/mqtt-client-C-Zephyr<br># Build for TAP + TLS mode (automatically merges prj.conf + prj-tap-tls.conf)<br>west build -d build-tap-tls -p always -b native_sim/native/64 . \<br>    -- -DOVERLAY_CONFIG=prj-tap-tls.conf</pre><p>Upon a successful build, the terminal will output: [333/333] Running utility command for native_runner_executable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nANXvhekaSVOcx3X" /></figure><p>The critical TLS-related Kconfig options within prj-tap-tls.conf are as follows:</p><pre># MQTT TLS Transport Layer<br>CONFIG_MQTT_LIB_TLS=y<br>CONFIG_NET_SOCKETS_SOCKOPT_TLS=y<br># mbedTLS Stack Configuration<br>CONFIG_MBEDTLS=y<br>CONFIG_MBEDTLS_ENABLE_HEAP=y<br>CONFIG_MBEDTLS_HEAP_SIZE=30000<br>CONFIG_MBEDTLS_SSL_IN_CONTENT_LEN=2048<br>CONFIG_MBEDTLS_SSL_OUT_CONTENT_LEN=2048<br># Cipher Suite: ECDHE-RSA Key Exchange + AES-128-GCM (verified compatible with EMQX)<br>CONFIG_MBEDTLS_CIPHERSUITE_TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256=y<br># PEM Certificate Parsing Support<br>CONFIG_MBEDTLS_PEM_PARSE_C=y<br>CONFIG_MBEDTLS_PEM_WRITE_C=y<br># PSA Key Types (required for TLS handshake)<br>CONFIG_PSA_WANT_KEY_TYPE_RSA_PUBLIC_KEY=y<br>CONFIG_PSA_WANT_KEY_TYPE_ECC_PUBLIC_KEY=y<br># TAP Networking Configuration<br>CONFIG_NET_CONFIG_SETTINGS=y<br>CONFIG_NET_CONFIG_MY_IPV4_ADDR=&quot;192.0.2.1&quot;<br>CONFIG_NET_CONFIG_MY_IPV4_GW=&quot;192.0.2.2&quot;<br>CONFIG_NET_CONFIG_PEER_IPV4_ADDR=&quot;192.0.2.2&quot;</pre><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_54-startup-and-tls-connection-verification">5.4 Startup and TLS Connection Verification</a></h3><pre># Execute with sudo to configure iptables and the zeth interface<br>sudo ./run-zephyr-tap.sh</pre><p>The script automatically handles the following sequence:</p><p>Launches Zephyr -&gt; waits for the zeth virtual network interface -&gt; assigns IP 192.0.2.2/24 -&gt; enables IP forwarding -&gt; configures iptables NAT/FORWARD rules -&gt; drops into the foreground interactive shell.</p><p><strong>Network Ready Hint:</strong> ✅ Network Ready | Zephyr: 192.0.2.1 | Host: 192.0.2.2</p><p>Run the TLS connection test (targeting your local EMQX TLS listener, e.g., port 8084):</p><pre>uart:~$ mqtt_cli conn -h 100.108.113.19 -p 8084 \<br>    --ca certs/self_certs/rootCA.crt \<br>    --cert certs/self_certs/mqtt-client.crt \<br>    --key certs/self_certs/mqtt-client.key \<br>    --insecure</pre><p>Expected output:</p><pre>[TLS] CA certificate loaded OK (tag 101)<br>[TLS] Config: peer_verify=0, sec_tag_count=1, hostname=100.108.113.19<br>Connecting to 100.108.113.19:8084 (ID: zephyr-emqx-345678, Keepalive: 60) ...<br>[TLS] Calling mqtt_connect (transport.type=1)...<br>Connection successful!<br>Entered conn blocking maintenance mode. Press Ctrl+C to terminate simulation process.</pre><p>Verification results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7j8PyDAt6tGqoBAi" /></figure><p>The Zephyr shell outputs the complete TLS connection sequence log, including explicit confirmations for CA certificate loaded OK, peer_verify, hostname, and ultimately Connection successful!.</p><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_55-tls-encrypted-message-publishing-and-subscription-demo">5.5 TLS Encrypted Message Publishing and Subscription Demo</a></h3><p>Similar to the TCP mode, the Zephyr TLS client fully supports standard MQTT publishing and subscription. This demo targets the local EMQX TLS listener (100.108.113.19:8084).</p><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#scenario-1-tls-subscription-and-encrypted-message-receipt">Scenario 1: TLS Subscription and Encrypted Message Receipt</a></h4><pre>uart:~$ mqtt_cli sub -h 100.108.113.19 -p 8084 \<br>    --ca certs/self_certs/rootCA.crt \<br>    --cert certs/self_certs/mqtt-client.crt \<br>    --key certs/self_certs/mqtt-client.key \<br>    --insecure \<br>    -t test/tls/demo -q 1</pre><p>Expected output:</p><pre>[TLS] CA certificate loaded OK (tag 101)<br>Connecting to 100.108.113.19:8084 (ID: zephyr-emqx-... ) ...<br>Connection successful!<br>Subscribing to topic: &#39;test/tls/demo&#39; (QoS 1) ...<br>Entered sub listening state. Waiting for messages... (Press Ctrl+C to exit)</pre><p>Publish an encrypted message to the same TLS port from the host or another terminal (client certificates are required for mTLS verification):</p><pre>mqtt_cli pub -h 100.108.113.19 -p 8084 \<br>    --ca certs/self_certs/rootCA.crt \<br>    --cert certs/self_certs/mqtt-client.crt \<br>    --key certs/self_certs/mqtt-client.key \<br>    -t test/tls/demo -m &quot;Hello via TLS!&quot; -q 1</pre><p>The Zephyr subscriber shell will instantly output:</p><pre>[Received Msg] Topic: test/tls/demo | Payload: Hello via TLS!</pre><h4><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#scenario-2-tls-encrypted-publishing">Scenario 2: TLS Encrypted Publishing</a></h4><pre>uart:~$ mqtt_cli pub -h 100.108.113.19 -p 8084 \<br>    --ca certs/self_certs/rootCA.crt \<br>    --cert certs/self_certs/mqtt-client.crt \<br>    --key certs/self_certs/mqtt-client.key \<br>    -t test/tls/demo -m &quot;Hello from Zephyr via TLS!&quot; -q 1</pre><p>Expected output:</p><pre>[TLS] CA certificate loaded OK (tag 101)<br>Connection successful!<br>[Published 1/1] Topic=&#39;test/tls/demo&#39; | Payload=&#39;Hello from Zephyr via TLS!&#39;<br>Publish finished, gracefully disconnecting and exiting...</pre><p>Verification results:</p><p>The terminal displays the complete confirmation flow, finalized by the [Published 1/1] log on the Zephyr TLS publisher side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*AqOUilHp2YuoqlT3" /></figure><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#vi-project-configuration-and-execution-environment-comparison">VI. Project Configuration and Execution Environment Comparison</a></h3><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_61-dual-mode-comprehensive-comparison">6.1 Dual-Mode Comprehensive Comparison</a></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ot9nzPmSnOtI8Di7F1JXWQ.png" /></figure><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#_62-build-command-comparison">6.2 Build Command Comparison</a></h3><pre># NSOS / TCP-only<br>west build -d build-nsos -p always -b native_sim/native/64 . \<br>    -- -DOVERLAY_CONFIG=prj-nsos.conf<br># TAP + TLS<br>west build -d build-tap-tls -p always -b native_sim/native/64 . \<br>    -- -DOVERLAY_CONFIG=prj-tap-tls.conf</pre><p>Both targets utilize prj.conf as their shared foundation. By modifying only the overlay file designated via -DOVERLAY_CONFIG, the Zephyr build system automatically merges the underlying Kconfig definitions.</p><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#vii-troubleshooting-and-faq">VII. Troubleshooting and FAQ</a></h3><p><strong>Q1: Why do I get an error when passing TLS arguments (like</strong> --ca<strong>) in NSOS mode?</strong></p><ul><li><strong>Root Cause:</strong> NSOS mode does not enable CONFIG_MQTT_LIB_TLS. Consequently, the TLS code block is compiled out via #ifdef directives.</li><li><strong>Solution:</strong> Switch to the <strong>TAP+TLS</strong> mode, or stick to plaintext MQTT on port 1883.</li></ul><p><strong>Q2: Why does</strong> run-zephyr-tap.sh <strong>throw a &quot;zeth interface timeout&quot; error in TAP mode?</strong></p><ul><li><strong>Root Cause:</strong> The Docker container lacks NET_ADMIN privileges or access to the /dev/net/tun device.</li><li><strong>Solution:</strong> Recreate the container using the --cap-add=NET_ADMIN --device=/dev/net/tun flags, and verify that lsmod | grep tun returns an active module on the host.</li></ul><p><strong>Q3: What causes TLS connection failures with error code</strong> -0x4e <strong>or</strong> Underlying connection failed: -103<strong>?</strong></p><ul><li><strong>Root Cause:</strong> Error -0x4e maps to MBEDTLS_ERR_NET_SEND_FAILED. This typically stems from mixed-up certificate roles (e.g., using a server certificate as a client certificate), residual stale credentials in the sec_tag cache, or mismatched broker-side configurations.</li><li><strong>Solution:</strong> 1. Ensure the client uses a certificate generated with the clientAuth Extended Key Usage (EKU). 2. Verify that client certificate authentication is correctly enabled on the broker. 3. Ensure old TLS credentials are explicitly purged before every connection attempt (this logic is built into this demo project).</li></ul><p><strong>Q4: Why is TAP mode unavailable on macOS Docker Desktop?</strong></p><ul><li><strong>Root Cause:</strong> The underlying LinuxKit subsystem used by Docker Desktop for Mac imposes network virtualization restrictions that break full TCP forwarding over TAP/TUN devices.</li><li><strong>Solution:</strong> Use <strong>NSOS/TCP-only</strong> mode for local Mac development, or switch to a native Linux host or WSL2 environment.</li></ul><p><strong>Q5: Can I use self-signed certificates in a production environment?</strong></p><ul><li><strong>No.</strong> Self-signed certificates are strictly for local evaluation or controlled intranet environments. For production deployments, always use certificates issued by a publicly trusted CA (e.g., Let’s Encrypt, DigiCert), remove the --insecure flag, and enable strict full-chain certificate verification.</li></ul><h3><a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos#viii-summary-and-outlook">VIII. Summary and Outlook</a></h3><p>This guide demonstrated how to build an MQTT client from scratch on Zephyr RTOS and connect it to an EMQX Broker using two distinct modes:</p><ul><li><strong>NSOS/TCP-only Mode:</strong> Get up and running in just three steps: west build $\rightarrow$ ./run-zephyr-nsos.sh $\rightarrow$ mqtt_cli conn. This mode is perfect for quickly validating the entire MQTT publish/subscribe workflow without requiring elevated host privileges or certificate overhead.</li><li><strong>TAP+TLS Mode:</strong> Enables robust TLS 1.2 encryption and mutual authentication (mTLS) on Zephyr’s independent native network stack. It routes traffic through the zeth virtual interface and utilizes host-side iptables NAT for internet access, fulfilling security evaluations for embedded device communications.</li></ul><p>Using the mqtt_cli sub and mqtt_cli pub commands, you can easily verify message mechanics directly from the Zephyr shell, while leveraging the EMQX Dashboard to observe real-time device connection states and data traffic metrics.</p><p>Next Steps and Future Extensions:</p><ul><li><strong>Data Integration:</strong> Leverage the EMQX Rule Engine to bridge Zephyr edge data to external storage platforms like Kafka, PostgreSQL, or TimescaleDB in real-time.</li><li><strong>Command Downlink:</strong> Implement reliable device control mechanisms utilizing the Request-Response pattern native to MQTT 5.0.</li><li><strong>Over-the-Air (OTA) Updates:</strong> Use EMQX’s File Transfer capability to orchestrate secure firmware OTA updates on Zephyr endpoints.</li><li><strong>Hardware Deployment:</strong> Port this implementation to physical hardware (e.g., nRF52840, ESP32) to evaluate real-world power consumption and connection stability.</li></ul><p><em>Originally published at</em> <a href="https://www.emqx.com/en/blog/getting-started-with-mqtt-on-zephyr-rtos"><em>www.emqx.com</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e448d1fadaa0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Zigbee2MQTT in 2026: Local Smart-Home Control over MQTT]]></title>
            <link>https://emqx.medium.com/zigbee2mqtt-in-2026-local-smart-home-control-over-mqtt-1907bf058c94?source=rss-2287ea9742a6------2</link>
            <guid isPermaLink="false">https://medium.com/p/1907bf058c94</guid>
            <dc:creator><![CDATA[EMQ Technologies]]></dc:creator>
            <pubDate>Thu, 23 Jul 2026 08:52:29 GMT</pubDate>
            <atom:updated>2026-07-23T08:52:29.222Z</atom:updated>
            <content:encoded><![CDATA[<p>If you want local control of your Zigbee bulbs, sensors, and switches without a vendor hub or cloud account, Zigbee2MQTT is still one of the most practical ways to get it in 2026. This guide runs it with Docker and EMQX, covers what changed in the 2.x releases, shows where Matter fits, and explains what a full <a href="https://www.emqx.com/en/blog/the-ultimate-guide-to-mqtt-broker-comparison">MQTT broker</a> adds as your setup grows.</p><p><strong>TL;DR</strong></p><ul><li>Zigbee2MQTT gives you local control of Zigbee devices over MQTT, with support for 5,473 devices and no vendor cloud.</li><li>This guide uses EMQX as the broker; if you only need bare local pub/sub, Mosquitto works too.</li><li>EMQX adds a dashboard, authentication, rules, data export, and bridging once you grow past one local network.</li><li>Matter is complementary: good for new cross-ecosystem devices, not a wholesale replacement for an existing Zigbee network.</li></ul><p>This is part of a series of articles about <a href="https://www.emqx.com/en/blog/the-easiest-guide-to-getting-started-with-mqtt">MQTT</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XtIzPyJiSNvL0fqy" /></figure><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#what-is-zigbee">What Is Zigbee?</a></h3><p>Zigbee is a low-power wireless protocol for short-range device communication, approved as an IEEE standard in 2004. It is widely used in home automation, smart energy, and building control. Zigbee uses a mesh topology: mains-powered devices act as routers and relay traffic for each other, so the network extends as you add devices and battery-powered end devices can stay asleep most of the time. That mesh design is also why Zigbee end devices run for a year or more on a coin cell.</p><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#what-is-zigbee2mqtt">What Is Zigbee2MQTT?</a></h3><p>Zigbee2MQTT is open-source software that bridges Zigbee devices to MQTT. It talks to a Zigbee coordinator (a USB or network radio), then publishes device state to an MQTT broker and accepts commands back. Your automation platform speaks plain MQTT and never needs to know a device is Zigbee underneath.</p><p>The point is independence. You do not need the manufacturer’s hub or cloud. A generic coordinator radio plus Zigbee2MQTT controls devices from many vendors through one interface, and every message stays on your own network.</p><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#zigbee-vs-matter-in-2026">Zigbee vs Matter in 2026</a></h3><p>If you are starting a smart home in 2026, the obvious question is whether to skip Zigbee and go straight to Matter. Before comparing them, it’s worth clarifying how Zigbee, Thread, and Matter relate to one another, since these terms are often confused.</p><p><strong>Thread</strong> is a low-power wireless mesh, the same family of radio Zigbee uses. On its own, it only moves data between devices; it cannot control them. <strong>Matter</strong> is the application standard that sits on top and does the controlling, and it lets devices work across the Apple, Google, Amazon, and Samsung ecosystems. Matter runs over Thread or over ordinary Wi-Fi and Ethernet, so not every Matter device uses Thread.</p><p>In other words, when people compare “Zigbee vs Matter,” they are often mixing different layers of the technology stack. A more direct protocol comparison is Zigbee versus Thread, while Matter provides the interoperability layer that Zigbee achieves through Zigbee2MQTT and its extensive device database.</p><p>Here is how the two stack up for a 2026 build:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JFX7YBELxyRSxdFp-WfM9Q.png" /></figure><p>Matter’s catalog is growing, but Zigbee still has the larger supported-device list and a longer track record with low-power sensors. For most builders, the two are complementary: use Matter for new cross-ecosystem gear, and keep Zigbee2MQTT for device breadth and full MQTT-level control.</p><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#why-zigbee2mqtt">Why Zigbee2MQTT</a></h3><p>Zigbee2MQTT has a few practical advantages over a proprietary hub:</p><ul><li><strong>Your data stays local.</strong> Many Zigbee products route control through a vendor cloud, which limits you and raises privacy questions. With Zigbee2MQTT, every message stays on your own network.</li><li><strong>You are not locked to one brand.</strong> It supports 5,473 devices from 577 vendors (see the <a href="https://www.zigbee2mqtt.io/supported-devices/">official list</a>), so you pick hardware on price and features.</li><li><strong>Everything is MQTT.</strong> You can wire devices into any automation engine, write your own rules, and integrate with other systems without proprietary glue.</li></ul><blockquote>Related reading: <a href="https://www.emqx.com/en/blog/essential-things-to-know-about-mqtt-security">MQTT security</a>.</blockquote><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#what-changed-in-zigbee2mqtt-20">What Changed in Zigbee2MQTT 2.0</a></h3><p>If you last used Zigbee2MQTT a couple of years ago, note that <strong>2.0</strong> was a breaking release, and the current version is <strong>2.12.0</strong> (June 2026). For a new install, the changes worth knowing are:</p><ul><li><strong>Legacy behavior changed.</strong> The legacy API and legacy availability payloads were removed, and legacy action sensors are off by default, so older integrations built on them need updating.</li><li><strong>Config migration is automatic.</strong> On first start, 2.0 migrates your configuration.yaml and records what it changed in migration-1-to-2.log in your data directory.</li><li><strong>Pairing moved out of the config file.</strong> You no longer set permit_join in configuration.yaml. Enable joining from the web frontend (the join button opens the network for 254 seconds) or by publishing to zigbee2mqtt/bridge/request/permit_join, and leave joining off in normal operation so stray devices cannot pair.</li><li><strong>The web frontend is the main control surface.</strong> An alternative frontend, windfront, can manage several Zigbee2MQTT instances from one UI if you run more than one network.</li></ul><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#set-up-zigbee2mqtt-with-docker-and-emqx">Set Up Zigbee2MQTT with Docker and EMQX</a></h3><p>This walkthrough uses Docker and Docker Compose.</p><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#find-your-zigbee-adapter">Find your Zigbee adapter</a></h3><p><strong>USB adapter:</strong></p><p>Plug it in, then list the stable by-id path. Use this rather than /dev/ttyUSB0, which can change across reboots:</p><pre>$ ls -l /dev/serial/by-id/</pre><p>You will see a symlink such as /dev/serial/by-id/usb-Texas_Instruments_TI_CC2531_USB_CDC___0X00124B-if00 pointing at /dev/ttyACM0 or /dev/ttyUSB0. Note the by-id path; you map it into the container. Most current coordinators are Silicon Labs (EmberZNet) or Texas Instruments (zStack) based.</p><p><strong>Network adapter:</strong></p><p>Zigbee2MQTT auto-discovers network coordinators that support mDNS, so you usually do not need to find their IP by hand. For adapters without mDNS, set serial.port to the adapter&#39;s socket URL (for example tcp://192.168.1.50:6638) in configuration.yaml.</p><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#run-zigbee2mqtt-with-emqx">Run Zigbee2MQTT with EMQX</a></h3><p>Create a project folder and a docker-compose.yml. This runs EMQX as the broker alongside Zigbee2MQTT:</p><pre>services:<br>  emqx:<br>    image: emqx/emqx-enterprise:6.2.1<br>    restart: unless-stopped<br>    ports:<br>      - &quot;1883:1883&quot;      # MQTT<br>      - &quot;18083:18083&quot;    # dashboard<br>    volumes:<br>      - ./emqx-data:/opt/emqx/data<br><br>  zigbee2mqtt:<br>    container_name: zigbee2mqtt<br>    restart: unless-stopped<br>    image: ghcr.io/koenkk/zigbee2mqtt<br>    volumes:<br>      - ./zigbee2mqtt-data:/app/data<br>      - /run/udev:/run/udev:ro<br>    ports:<br>      - &quot;8080:8080&quot;<br>    environment:<br>      - TZ=Europe/Berlin<br>    devices:<br>      # replace with your own /dev/serial/by-id/... path from the step above<br>      - /dev/serial/by-id/usb-Texas_Instruments_TI_CC2531_USB_CDC___0X00124B-if00:/dev/ttyACM0</pre><p>Adjust the device path and timezone for your machine. The Zigbee2MQTT image is published at ghcr.io/koenkk/zigbee2mqtt, and modern Compose files no longer use the version: key.</p><p>Bring the stack up with docker compose up -d, then open &lt;http://&lt;host&gt;&gt;:8080. On first start, Zigbee2MQTT 2.x runs a web onboarding wizard: it auto-detects the adapter you mapped in and asks for your MQTT server. Enter mqtt://emqx:1883 (the EMQX service, addressed by its Compose name). A fresh EMQX has no authentication configured, so Zigbee2MQTT connects without a username or password; add authentication from the dashboard once everything works.</p><p>If you would rather configure it up front and skip the wizard, create zigbee2mqtt-data/configuration.yaml before the first start:</p><pre>version: 5<br>mqtt:<br>  server: mqtt://emqx:1883<br>serial:<br>  port: /dev/ttyACM0<br>frontend:<br>  enabled: true<br>  port: 8080</pre><p>serial.port is the in-container path from the devices: mapping, not the host by-id path. Zigbee2MQTT generates its network keys on first start.</p><p>Prefer a minimal broker with nothing else attached? eclipse-mosquitto:2.0 is a drop-in replacement for the emqx service. You lose the dashboard, rule engine, and bridging covered below, which is fine if all you want is local message passing.</p><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#pair-a-device">Pair a device</a></h3><p>Find your device in the <a href="https://www.zigbee2mqtt.io/supported-devices/">supported devices list</a> and follow its pairing steps; if none are listed, a factory reset usually triggers pairing. Enable joining first, either with the frontend join button (it opens the network for 254 seconds) or by publishing a request to the zigbee2mqtt/bridge/request/permit_join topic, then trigger pairing on the device. A successful join shows up in the log:</p><pre>Zigbee2MQTT:info  Successfully interviewed &#39;0x00158d0001dc126a&#39;, device has successfully been paired</pre><p>Turn joining off again once your devices are added.</p><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#what-emqx-adds-beyond-message-passing">What EMQX Adds Beyond Message Passing</a></h3><p>With EMQX as the broker, you get more than a place for Zigbee2MQTT to publish.</p><p>The <strong>dashboard</strong> at &lt;http://&lt;host&gt;&gt;:18083 (default login admin / public, which it prompts you to change) shows connected clients, subscriptions, topics, and message rates, so you can watch your Zigbee traffic instead of guessing at it.</p><p>The <strong>rule engine</strong> can filter and transform device payloads and forward them to a time-series database, a data warehouse, or Kafka without bolting on extra services. See <a href="https://www.emqx.com/en/blog/building-an-iot-time-series-data-application-with-mqtt-and-influxdb">building an IoT time-series application with MQTT and InfluxDB</a> for a Zigbee-friendly pattern.</p><p>When one home grows into several sites, EMQX <strong>clustering</strong> links nodes into a single logical broker and MQTT bridging connects them to each other or up to the cloud. That is the path from a Raspberry Pi in one house to a fleet on the broker you already started with. If you would rather not run that upstream broker yourself, <a href="https://www.emqx.com/en/cloud">EMQX Cloud</a> is the managed option. Learn more on the <a href="https://www.emqx.com/en">EMQX website</a>.</p><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#a-lighter-emqx-for-small-gateways-coming-in-630">A Lighter EMQX for Small Gateways (Coming in 6.3.0)</a></h3><p>A full broker asks more of a Raspberry Pi than a minimal one does, which is the usual reason people run Mosquitto on small gateways. EMQX 6.2.1 runs on Pi-class hardware, and the upcoming <strong>6.3.0</strong> release adds an essential mode that trims the footprint further for resource-constrained devices.</p><p>Setting EMQX_FEATURES=ESSENTIAL selects a slim feature set aimed at edge and gateway deployments. Early testing puts the broker around 100 MB of RAM in this mode, light enough to sit next to Zigbee2MQTT on the same small device while keeping the dashboard, rules, and bridging. The exact figure is still settling ahead of release, so treat 100 MB as an early number rather than a guarantee.</p><p>Once 6.3.0 ships, switch the emqx image to the new release and add one environment variable; the rest of the service stays the same:</p><pre>emqx:<br>    image: emqx/emqx-enterprise:6.3.0<br>    environment:<br>      - EMQX_FEATURES=ESSENTIAL<br>    # ports and volumes unchanged</pre><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#zigbee2mqtt-faq">Zigbee2MQTT FAQ</a></h3><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#does-zigbee2mqtt-need-an-mqtt-broker">Does Zigbee2MQTT need an MQTT broker?</a></h3><p>Yes. Zigbee2MQTT publishes Zigbee device state to an MQTT broker and receives commands through <a href="https://www.emqx.com/en/blog/advanced-features-of-mqtt-topics">MQTT topics</a>. For a single local setup, a minimal broker like Mosquitto can be enough. For visibility, authentication, rules, data export, or bridging, a fuller broker such as EMQX helps.</p><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#is-zigbee2mqtt-local">Is Zigbee2MQTT local?</a></h3><p>Yes. Zigbee2MQTT runs entirely on your local network with a Zigbee coordinator, Zigbee2MQTT, and an MQTT broker. Any cloud connection is optional and depends on what you wire up downstream.</p><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#zigbee2mqtt-vs-matter-which-should-i-use">Zigbee2MQTT vs Matter: which should I use?</a></h3><p>Use Zigbee2MQTT for broad support of existing Zigbee devices and MQTT-level automation. Use Matter for new cross-ecosystem devices. Many smart-home setups run both.</p><h3><a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026#can-i-use-emqx-with-zigbee2mqtt">Can I use EMQX with Zigbee2MQTT?</a></h3><p>Yes. EMQX runs as the MQTT broker for Zigbee2MQTT, and adds a dashboard, authentication, rules, data export, and bridging when a setup grows beyond local pub/sub.</p><p><strong>Related resources</strong></p><ul><li><a href="https://www.emqx.com/en/blog/home-assistant-and-mqtt-4-things-you-could-build">Home Assistant and MQTT: 4 Things You Could Build</a></li><li><a href="https://www.emqx.com/en/blog/set-up-emqx-cloud-mqtt-broker-with-openhab">MQTT with openHAB: A Step-by-Step Tutorial</a></li><li><a href="https://www.emqx.com/en/blog/a-developer-s-journey-with-esp32-and-mqtt-broker">A Developer’s Journey with ESP32 and MQTT Broker</a></li></ul><p><em>Originally published at</em> <a href="https://www.emqx.com/en/blog/zigbee2mqtt-in-2026"><em>www.emqx.com</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1907bf058c94" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Comparing EMQX MQTT Streams and NATS JetStream for Durable Device Telemetry]]></title>
            <link>https://emqx.medium.com/comparing-emqx-mqtt-streams-and-nats-jetstream-for-durable-device-telemetry-62af6cec07d5?source=rss-2287ea9742a6------2</link>
            <guid isPermaLink="false">https://medium.com/p/62af6cec07d5</guid>
            <dc:creator><![CDATA[EMQ Technologies]]></dc:creator>
            <pubDate>Tue, 21 Jul 2026 09:46:40 GMT</pubDate>
            <atom:updated>2026-07-21T09:46:40.478Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction: Streamlining IoT Telemetry with Built-in Durable Streams</h3><p>You are collecting data from a fleet of devices, and live pub/sub is no longer enough. You need to keep that data and read it back later: a service you deploy next month should be able to replay what the sensors reported today, and after a bad deploy you should be able to reprocess from a point in time. <strong>That is a durable stream.</strong></p><p>The classic way to get one in an MQTT shop has been to run Kafka behind the broker and bridge messages into it. NATS folds that capability into the server as JetStream; EMQX folds it into the broker as MQTT Streams, so when Kafka was in the path only to retain and replay the data, you can now keep that in the broker.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*s_jSit-bK_YkCQ_S" /></figure><p>The streaming layers look alike. What differs is the system each one lives in.</p><ul><li>EMQX is a full <a href="https://www.emqx.com/en/blog/the-ultimate-guide-to-mqtt-broker-comparison">MQTT broker</a>, and a device fleet leans on MQTT guarantees that sit around the stream: a last-will message when a device drops off, a retained message so a reconnecting client gets current state at once, a durable per-client session, and QoS 2 for the messages that must not be lost or doubled.</li><li>NATS is a cloud-native messaging system, strong at streaming and request-reply, but it was built for services talking to services, and most of those device guarantees are not part of it.</li></ul><p>This post builds the same durable-telemetry scenario on both, each fed by its native clients, and maps where they match and where they diverge, on the stream itself and on the device features around it.</p><p>The code is public and available for you to explore at: <a href="https://github.com/emqx/emqx-streams-demo">github.com/emqx/emqx-streams-demo</a>.</p><h3>What JetStream is</h3><p><a href="https://docs.nats.io/nats-concepts/jetstream">JetStream</a> is the persistence layer built into the NATS server. Enable it and the same server that does pub/sub also gives you durable streams bound to subjects, consumers that track their own position and redeliver on missed acknowledgments, a key-value store, an object store, and work-queue retention where a message is removed once a consumer acknowledges it. Publisher deduplication is built in: stamp a Nats-Msg-Id header and the server drops duplicates inside a configurable window.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pWl_CLlaFZVbZl_V" /></figure><p>That is a lot of capability with no external dependencies, and it is why teams building cloud-native backbones reach for NATS. JetStream consumes and produces through NATS clients speaking the NATS protocol.</p><h3>What MQTT Streams is</h3><p><a href="https://docs.emqx.com/en/emqx/latest/mqtt-stream/mqtt-stream-concept.html">MQTT Streams</a> arrived in <a href="https://www.emqx.com/en/products/emqx">EMQX Enterprise</a> 6.1. A stream is a named, durable store that captures every MQTT message matching a topic filter. Two types exist: append-only streams for history and replay, and last-value streams that keep the newest message per key, which is how a dashboard gets current state.</p><p>Producers are untouched. Any <a href="https://www.emqx.com/en/blog/mqtt-client-tools">MQTT client</a> publishing to a matching topic feeds the stream, and the device fleet does not know streams exist. Consumers subscribe with <a href="https://www.emqx.com/en/blog/introduction-to-mqtt-5">MQTT 5.0</a> to $stream/&lt;name&gt; and pick the start position with a stream-offset subscription property: earliest, latest, or a timestamp. History arrives first, then the subscription keeps tailing. Replay is an MQTT subscription, nothing else.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VBNyDy3DUuC8lB_I" /></figure><p>Work queues are a separate EMQX feature, <a href="https://www.emqx.com/en/blog/real-time-mqtt-pub-sub-and-durable-queues-natively-in-emqx">Message Queues</a>, consumed via $queue/&lt;name&gt;: the broker delivers each message to exactly one of the subscribed workers.</p><h3>The Same Scenario, Built Twice</h3><p>The demo runs both builds from one compose file. Each fleet speaks its broker’s native protocol: MQTT sensors publish to EMQX, and a separate fleet publishes to NATS over the NATS protocol. The stacks are symmetric, one broker plus a sensor fleet plus four consumers on each side, with no gateway in either. That keeps the comparison about features and fit rather than plumbing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PSEgFfcQ-UfPs2xk" /></figure><p>Four consumers exercise the durable-streaming need on both sides: a live tail, a late joiner that replays the full history in order, a dashboard that reads current state on a cold start, and a pool of two workers that split dispatched tasks one-to-one. Both systems do all four. The differences are elsewhere.</p><h3>What Actually Differs</h3><p>On the core job, the two are close. Each replays history in order, load-balances a work queue, and serves the latest value per sensor:</p><ul><li><a href="https://www.emqx.com/en/blog/the-easiest-guide-to-getting-started-with-mqtt"><strong>MQTT</strong></a> achieves this through retained messages and last-value streams.</li><li><strong>NATS</strong> achieves this through a KV bucket (written by the producer) or a last-per-subject stream.</li></ul><p>Where they diverge, however, comes down to five key architectural areas:</p><h3>1. Durable Consumers</h3><ul><li><strong>NATS JetStream:</strong> Keeps a named consumer’s position on the server. If a consumer crashes, it simply reattaches and resumes exactly where it stopped. For a long-lived pipeline that must never miss or reprocess data, JetStream’s model requires less operational effort to get right.</li><li><strong>EMQX Streams:</strong> The stream consumer picks a start point and tracks its own position to resume from.</li></ul><h3>2. Device-Centric Features around the Stream</h3><p>An IoT fleet needs more than just a log of readings. MQTT natively includes three core features designed for device management, whereas native NATS does not support them at the protocol level:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jwcuFj4aT3BWc2OFIiC3Gg.png" /></figure><p>You can build each of these features in the application layer with NATS, but that is your code to write and run.</p><p>NATS does match MQTT on the basics (wildcards, load-balanced subscriptions, headers, and per-message TTL since v2.11), and its request-reply pattern is more first-class; the gap is specifically these device-centric features.</p><h3>3. Endpoint Protocols</h3><p>This is usually the ultimate deciding factor in technical selection:</p><ul><li><strong>MQTT Dominance:</strong> The device and OT installed base is overwhelmingly MQTT, much of it consisting of third-party or fixed-firmware hardware whose protocol you cannot change. If your endpoints <em>must</em> use MQTT, EMQX puts <strong>durable streams natively right where the devices already are</strong>.</li><li><strong>NATS Ecosystem:</strong> When you fully control the endpoints and they can run a NATS client, the MQTT ecosystem advantage falls away. The choice then comes down to pure features, where NATS’s breadth is a real draw.</li></ul><h3>4. Open Standard vs. Single Implementation</h3><ul><li><strong>MQTT (EMQX):</strong> An OASIS and ISO/IEC 20922 standard with many interoperable brokers (EMQX, HiveMQ, Mosquitto, cloud IoT services) and clients. The practical effect is at the device layer: because devices speak a standard protocol rather than a vendor-locked one, <strong>you can change brokers without re-touching the fleet</strong>.</li><li><strong>NATS:</strong> Has a <strong>single implementation</strong>, the CNCF nats-server. It is Apache-2.0 licensed, so you have access to the source code, but there is no alternative server to migrate to, and the protocol tracks one project&#39;s roadmap.</li></ul><p>In fairness, the streaming feature is proprietary on both sides, MQTT Streams to EMQX and JetStream to NATS; the portability MQTT buys you is at the protocol and device layer, not the stream.</p><h3>5. Licensing</h3><ul><li><strong>NATS:</strong> Fully open-source under <strong>Apache-2.0</strong>.</li><li><strong>EMQX:</strong> Features a single edition since v5.9 that is <strong>free for single-node and non-commercial use</strong>, but requires a license for clustering or commercial production.</li></ul><p>If an OSS-only stack is a hard constraint for your architecture, this settles the choice early.</p><h3>Beyond Streaming: Platform Capabilities</h3><p>Beyond data streaming, each technology functions as a much broader platform. Your choice here depends heavily on your extended architecture:</p><ul><li><strong>NATS adds:</strong> A key-value store, an object store, and first-class request-reply.</li><li><strong>EMQX adds:</strong> A SQL rule engine and data bridges directly to Kafka and various databases.</li></ul><p>Data ordering is <strong>per key on both sides</strong> as configured here (per sensor), and the replay consumers in both repositories verify this behavior.</p><h3>How to Choose</h3><ul><li><strong>Choose NATS JetStream</strong> when your producers and consumers speak NATS, or you build them to, or when you want an Apache-2.0 system that is also a key-value, object, and request-reply platform. That is what JetStream is built for, and a fleet of NATS-native services is a fine architecture.</li><li><strong>Choose EMQX MQTT Streams</strong> when your fleet is MQTT, as most IoT and OT fleets are or must be, and the job is replay, current state, or work distribution on that data. The stream then lives in the same system as the device connections and the rule processing, with nothing to translate.</li></ul><h3>The Wider Pattern</h3><p>Messaging systems have been growing each other’s primitives for years: Kafka added queue-style consumption with share groups, RabbitMQ added replayable streams, and MQTT brokers are now adding durable logs and queues of their own. The protocol label on the box says less and less about what the system can do; what still differs is which clients connect natively and where the data enters. For device fleets, that entry point is usually MQTT, and that is the strongest reason to want the stream where the devices already are.</p><p>Run both builds yourself: <a href="https://github.com/emqx/emqx-streams-demo">github.com/emqx/emqx-streams-demo</a>. Fifteen minutes, Docker only, and the README walks through replay, current state, work queues, and a broker restart.</p><p><em>Originally published at</em> <a href="https://www.emqx.com/en/blog/comparing-emqx-mqtt-streams-and-nats-jetstream"><em>www.emqx.com</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=62af6cec07d5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What a PV Cooling Paper Reveals About Distributed IoT Control]]></title>
            <link>https://emqx.medium.com/what-a-pv-cooling-paper-reveals-about-distributed-iot-control-5b3055bf78f3?source=rss-2287ea9742a6------2</link>
            <guid isPermaLink="false">https://medium.com/p/5b3055bf78f3</guid>
            <dc:creator><![CDATA[EMQ Technologies]]></dc:creator>
            <pubDate>Tue, 21 Jul 2026 09:41:53 GMT</pubDate>
            <atom:updated>2026-07-21T09:41:53.362Z</atom:updated>
            <content:encoded><![CDATA[<h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#introduction">Introduction</a></h3><p>A 600 Wp experimental PV installation in the Czech Republic ran for 52 days in the field with active water cooling. Over that period, the cooling produced more extra electricity than the pumping system consumed: an Energy ROI of 1.07 on representative high-irradiance days. That’s unusual. Most active PV-cooling studies describe pumps that eat more electricity than the cooling saves, even when the peak-gain numbers look attractive.</p><p>Novak and Landkamer’s <a href="https://doi.org/10.1016/j.ecmx.2026.101743">new paper</a> in <em>Energy Conversion and Management: X</em> explains where the positive balance comes from. The cooling hardware isn’t new: Kuo and Lo published on active water cooling back in 2013, and <a href="https://www.pv-magazine.com/2026/04/10/internet-of-things-could-bring-solar-module-water-cooling-closer-to-commercial-viability/"><em>pv magazine</em></a> recently framed this paper as moving the technology “closer to commercial viability.” What’s new is a control algorithm that learned to skip cooling when rain was about to do the job for free.</p><p>This post is about why that algorithm is inseparable from the distributed architecture that runs it. The control algorithm and the edge/fog layout that carries it are two views of the same structural property. The architecture <em>is</em> the algorithm.</p><h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#the-experiment-setup">The Experiment Setup</a></h3><p>Novak and Landkamer tested cooling with a matched-pair design: two parallel branches of PV panels running side by side, one water-cooled and one as an uncooled reference. Both branches see the same sunlight and power the same constant electrical load, so any difference in their daily energy output is attributable to the cooling effect alone.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*90ik4gOz4MDmuJGY" /></figure><p>The field test ran for 52 valid daylight days across spring through early fall 2024. Each branch held two PV panels in series, with a 600 Wp rating per branch.</p><h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#what-the-paper-measured">What the Paper Measured</a></h3><p>On a representative high-irradiance day, the cooled branch produced <strong>about 56 Wh more electricity</strong> than the reference. The pump drew 6.0 W while active, consuming about <strong>53 Wh</strong> over the day. Divide the extra energy by the pump’s consumption and the Energy ROI is <strong>1.07</strong>: the cooling generated slightly more electricity than the pumping consumed.</p><p>Across all 52 days, the mean daily gain was 5.84% (95% CI 5.04%, 6.65%), with peak daily gains of 7.38% on high-irradiance days. Total instantaneous control-side load (ESP32 edge nodes + Raspberry Pi fog + pump) never exceeded 10 W, under 2% of the branch’s 600 Wp nominal rating.</p><p>These numbers are solid but not remarkable on their own; many cooling studies report similar peak gains. In §4.4, the authors compare the implemented weather-aware adaptive strategy against a reconstructed temperature-threshold baseline using the same 52-day dataset.</p><h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#the-weather-aware-strategy-is-doing-the-work">The Weather-Aware Strategy is Doing the Work</a></h3><p>The comparison:</p><ul><li><strong>Temperature-threshold baseline</strong> (the first-draft design): run the pump whenever panel temperature exceeds a cutoff.</li><li><strong>Weather-aware adaptive strategy</strong> (what they actually shipped): run the pump only when temperature exceeds the cutoff <em>and</em> a short-term weather outlook (fused from barometric pressure trends and AS3935 lightning activity) doesn’t indicate imminent precipitation.</li></ul><p>With the adaptive strategy, the pump ran <strong>18–27% less</strong> than it would have under the threshold-only baseline. On storm-transition days, <strong>30% less</strong>. And despite the pump running much less, the daily energy gain dropped by <strong>under 10%</strong>. The strategy was selectively skipping pump activations that wouldn’t have paid off, because the panel would cool passively within hours anyway, once the rain arrived.</p><p>The paper doesn’t compute this next step explicitly, but the arithmetic follows from the daily numbers above. A threshold-only baseline would consume roughly 20% more pump energy (~63 Wh/day instead of 53) for about 10% more gain (~62 Wh/day instead of 56). Plug those into the ROI formula and the baseline lands somewhere between 0.92 and 1.00.</p><p>Without the weather-aware suppression, this would have been yet another cooling system that consumes more than it recovers. <em>pv magazine</em>’s “closer to commercial viability” framing is, at the technical core, an algorithmic story rather than a hardware one.</p><h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#how-the-algorithm-works">How the Algorithm Works</a></h3><p>The algorithm has three pieces, each running on a different class of hardware.</p><h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#edge-weather-sensing-on-its-own-esp32">Edge: Weather Sensing on Its Own ESP32</a></h3><p>A dedicated ESP32 runs a barometric pressure sensor and ams OSRAM’s AS3935 Franklin lightning detection IC.</p><ul><li><strong>Pressure:</strong> sampled every 5 seconds, averaged over 30-minute windows. The node keeps a rolling 3-hour buffer of seven averaged samples and computes the gradient as a discrete temporal derivative.</li><li><strong>Lightning:</strong> treated as a multi-state indicator (near / distant / idle), not a binary event.</li></ul><p>The two signals fuse locally into a compact categorical output: <em>stable</em>, <em>deteriorating</em>, or <em>probable precipitation</em>. That category is the only thing this edge publishes over <a href="https://www.emqx.com/en/blog/the-easiest-guide-to-getting-started-with-mqtt">MQTT</a>. The raw pressure samples never leave the node.</p><h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#fog-decision-making-on-a-raspberry-pi">Fog: Decision-Making on a Raspberry Pi</a></h3><p>A Raspberry Pi running a <a href="https://www.emqx.com/en/blog/how-to-use-mqtt-in-python">Python MQTT</a> client subscribes to four inputs:</p><ul><li>The weather state, from the weather edge node.</li><li>Panel surface temperature, from a separate ESP32 with thermistors calibrated via the Steinhart-Hart equation.</li><li>Coolant reservoir level.</li><li>Coolant temperature.</li></ul><p>A rule set gates pump activation: every condition must indicate cooling is <em>needed now</em> and <em>not about to be made redundant</em> by incoming precipitation. If yes, the fog publishes activatePump. Otherwise, deactivatePump.</p><h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#actuator-command-execution-only">Actuator: Command Execution Only</a></h3><p>The pump’s ESP32 subscribes to activation commands and executes them. No decision logic lives at this layer.</p><h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#three-properties-that-shape-what-the-architecture-must-do">Three Properties That Shape What the Architecture Must Do</a></h3><p>The algorithm is structured:</p><ol><li><strong>Multi-signal.</strong> Panel temperature, ambient air, humidity, barometric pressure, lightning events, coolant level, coolant temperature. Each sensor has its own placement, sampling rate, and failure mode.</li><li><strong>Multi-location.</strong> Panel thermistors on the panels. Ambient sensors in ambient air. The lightning detector placed to see RF sferics without switching noise. Coolant sensors in the reservoir. No single mounting location holds them all.</li><li><strong>Multi-timescale.</strong> 5-second edge sampling. 30-minute pressure averaging. 3-hour gradient windows. Per-event lightning processing. Pump decisions within a minute of threshold crossing. That’s several orders of magnitude in one algorithm.</li></ol><p>These three properties are what the architecture has to carry.</p><h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#why-the-architecture-fits-the-algorithm">Why the Architecture Fits the Algorithm</a></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RPNRozEYOqb488Df" /></figure><p>A single-microcontroller implementation is possible in principle: long wires from every sensor to a central MCU, ADC multiplexing, per-signal timers, one fused control loop, and a pump relay at the end. The same authors built exactly that in their <a href="https://doi.org/10.1016/j.rser.2023.113347">2023 paper in <em>Renewable and Sustainable Energy Reviews</em></a>. The new paper is explicitly an argument against that design.</p><p>What goes wrong in the single-MCU version:</p><ul><li><strong>Wiring and noise.</strong> Long analog cable runs pick up EMI, degrade accuracy, and constrain where sensors can sit. The paper observes that centralized control “increases wiring complexity, and restricts the number of deployable sensors.”</li><li><strong>Extensibility.</strong> Adding a signal means a new cable, a re-flash, and a regression test of the whole system. With the MQTT layout: “New nodes simply register as <a href="https://www.emqx.com/en/blog/mqtt-client-tools">MQTT clients</a> within the predefined topic structure.”</li><li><strong>Failure isolation.</strong> A corroded connector or shorted cable in any signal path can cascade into the central controller and corrupt the fused state. No compartmentalization.</li><li><strong>Timescale coupling.</strong> One clock, forcing the system either to over-sample signals that don’t need it or to miss signals that do.</li></ul><p>The three-layer split addresses each of these:</p><ul><li><strong>Edge.</strong> One sensor (or a tight group), one node, one power budget per ESP32. Local calibration and preprocessing happen on the node. A failed edge node drops its own signal and nothing else. Adding a new signal = new ESP32 + new topic.</li><li><strong>Fog.</strong> The Raspberry Pi subscribes to signals it needs and fuses them at the cadence it needs them, independent of how fast any edge is sampling. Python on a general-purpose OS is good ground for evolving logic.</li><li><strong>Actuator.</strong> The pump’s ESP32 executes received commands. Control logic lives elsewhere, so changes don’t touch field hardware.</li><li><strong>MQTT as the only coupling.</strong> No bus wiring, no shared clock, no shared memory. Topics are the interface between layers.</li></ul><p>The three properties of the algorithm map directly onto architectural primitives:</p><ul><li><strong>Multi-signal</strong> → pub/sub fan-in.</li><li><strong>Multi-location</strong> → physical distribution of independent nodes.</li><li><strong>Multi-timescale</strong> → independent liveness with per-node timers.</li></ul><p>The algorithm and the architecture are the same structural properties in two different vocabularies: one about information flow, one about hardware layout.</p><h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#the-argument-is-about-the-pattern-not-the-protocol">The Argument is About the Pattern, Not the Protocol</a></h3><p>None of this is MQTT-specific. The paper describes its communication layer as “Wi-Fi or, alternatively, LoRaWAN for long-range low-power operation,” and the same topology works over <a href="https://datatracker.ietf.org/doc/html/rfc7641">CoAP with its Observe extension</a> or <a href="https://docs.oasis-open.org/amqp/core/v1.0/amqp-core-overview-v1.0.html">AMQP 1.0</a>. What’s load-bearing is the <em>lightweight pub/sub with a broker</em> pattern. MQTT happens to be the canonical IoT instance, which is why the paper uses it. The structural argument is about the pattern.</p><h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#at-production-scale">At Production Scale</a></h3><p>The paper’s four-panel installation fits in a lab. A real deployment spans dozens to hundreds of branches, typically organized into zones with their own fog controllers. The paper’s discussion section addresses this directly: “MQTT communication overhead scales approximately linearly with the number of clients and the publish/subscribe model avoids point-to-point connections,” and the likely bottleneck sits at the fog layer rather than the broker.</p><p>At that scale, the broker runs as a cluster rather than a single process, and the paper’s fog-layer decision logic has a natural home inside it. <a href="https://www.emqx.com/en">EMQX</a> handles the broker concerns (topic hierarchy, ACLs, shared subscriptions) and evaluates the pump decisions in its rule engine, which collapses the separate fog-layer tier the paper needed in its lab setup.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*K6EmpDrH9tBJufjF" /></figure><h3><a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control#the-broader-point">The Broader Point</a></h3><p>A 600 Wp PV branch produced 1.07 units of extra electricity for every unit the cooling system consumed, field-validated across 52 days of real Central European weather. That positive balance came from cooling <em>less often</em>, not <em>harder</em>. Weather-aware suppression of pump activation is what made the cooling worth doing.</p><p>In distributed IoT control, what’s worth building and what’s buildable are the same question. The architecture is not scaffolding for the algorithm. It’s part of the algorithm.</p><p><em>Originally published at</em> <a href="https://www.emqx.com/en/blog/what-a-pv-cooling-paper-reveals-about-distributed-iot-control"><em>www.emqx.com</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5b3055bf78f3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[MQTT Streams in EMQX: Bringing Replayable Data and Durable Pipelines to MQTT]]></title>
            <link>https://emqx.medium.com/mqtt-streams-in-emqx-bringing-replayable-data-and-durable-pipelines-to-mqtt-e9348fd27f80?source=rss-2287ea9742a6------2</link>
            <guid isPermaLink="false">https://medium.com/p/e9348fd27f80</guid>
            <category><![CDATA[mqtt]]></category>
            <dc:creator><![CDATA[EMQ Technologies]]></dc:creator>
            <pubDate>Wed, 01 Jul 2026 08:59:22 GMT</pubDate>
            <atom:updated>2026-07-01T08:59:22.621Z</atom:updated>
            <content:encoded><![CDATA[<p>With <a href="https://www.emqx.com/en/blog/emqx-6-2-0-release-notes">EMQX 6.2</a> now released, we are continuing to expand what MQTT can support across real-time systems, AI workloads, and industrial architectures.</p><p>Before looking ahead, it is worth taking a closer look at one of the most important capabilities introduced in <a href="https://www.emqx.com/en/blog/emqx-6-1-0-release-notes">EMQX 6.1</a>: MQTT Streams.</p><p>For years, MQTT has been known for real-time data movement, but it has lacked durability and replay, two capabilities that modern systems increasingly expect. MQTT Streams change that.</p><h3><a href="https://www.emqx.com/en/blog/mqtt-streams-in-emqx#the-limitation-of-real-time-only-messaging">The Limitation of Real-Time Only Messaging</a></h3><p>In a standard MQTT setup, messages are delivered in real time to subscribers and, once consumed, are no longer available.</p><p>This works well when systems are always online and continuously processing data. In practice, however, real-world systems are rarely that simple.</p><p>Consumers go offline, pipelines fail, and new applications often need access to historical data. Teams also need to replay events to debug issues or rebuild downstream systems. Without replay, gaps inevitably appear.</p><p>To address this, many teams introduce a separate streaming system such as Kafka. MQTT handles ingestion, while another system handles durability and replay. While effective, this approach adds complexity, cost, and operational overhead.</p><h3><a href="https://www.emqx.com/en/blog/mqtt-streams-in-emqx#what-modern-iot-systems-need">What Modern IoT Systems Need</a></h3><p>As deployments grow, expectations evolve.</p><p>Teams need durable data retention, replayable event streams, and the ability to consume data from specific points in time. They also require reliable pipelines to support analytics and AI workloads.</p><p>These are no longer advanced capabilities. They are foundational requirements for modern data architectures. Until recently, MQTT and streaming systems have largely existed in separate domains.</p><h3><a href="https://www.emqx.com/en/blog/mqtt-streams-in-emqx#mqtt-streams-in-emqx-61">MQTT Streams in EMQX 6.1</a></h3><p>MQTT Streams were introduced in EMQX 6.1 to bring durable, replayable data streams directly into MQTT.</p><p>Instead of treating MQTT messages as transient events, streams organize them as an ordered, append-only log. This allows data to be retained and consumed independently of when it was originally published.</p><p>Subscribers can read from a stream using offsets, similar to modern streaming systems. This makes it possible to replay past messages, recover from failures, and build reliable data pipelines without introducing another system.</p><p>At a high level, MQTT Streams introduce three key capabilities. Messages are stored durably, allowing them to persist beyond real-time delivery. Consumers can replay data using offsets to read from any point in the stream. And stream-based consumption operates alongside traditional pub/sub, giving teams flexibility in how data is accessed and processed.</p><h3><a href="https://www.emqx.com/en/blog/mqtt-streams-in-emqx#how-it-works">How It Works</a></h3><p>MQTT Streams introduce a new way to organize and consume data.</p><p>Messages are written into streams and stored as an ordered sequence. Consumers subscribe to that stream and control where they begin reading, either from the latest messages or from an earlier point in time.</p><p>Each stream maintains an ordered sequence of messages within its shards, and consumers track their position using offsets. In EMQX, an offset represents a specific point in time within the stream, allowing consumers to replay data or resume processing from where they left off.</p><p>This model enables multiple consumers to read the same data independently, each at their own pace, without interfering with one another.</p><p>In practice, this supports several important patterns, including reprocessing historical data, backfilling analytics pipelines, recovering from downstream outages, and debugging issues using real event data.</p><p>Importantly, this does not replace traditional MQTT pub/sub. Instead, it extends it. Real-time subscribers can continue operating as before, while stream consumers can take advantage of durability and replay when needed.</p><p>While the underlying model is similar to modern streaming systems, EMQX makes it straightforward to configure and manage streams directly within the platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pJgcte9qocNwA6i0" /></figure><blockquote>Example EMQX Streams configuration, including retention policies, stream key settings, and shard controls for managing durable message storage.</blockquote><h3><a href="https://www.emqx.com/en/blog/mqtt-streams-in-emqx#why-this-matters">Why This Matters</a></h3><p>MQTT Streams significantly expand what MQTT can be used for.</p><p>Instead of acting only as a real-time transport layer, MQTT can now serve as a durable data backbone for IoT and AI systems.</p><p>This shift simplifies architecture by removing the need for a separate streaming system. It reduces operational overhead by minimizing the number of components to manage and accelerates development by allowing teams to build replayable pipelines directly on MQTT. It also creates a more consistent data model, with a single system supporting both real-time and historical data access.</p><p>For many teams, this eliminates an entire layer of infrastructure.</p><h3><a href="https://www.emqx.com/en/blog/mqtt-streams-in-emqx#where-mqtt-streams-fit">Where MQTT Streams Fit</a></h3><p>MQTT Streams are especially valuable in scenarios where data needs to be both real-time and durable.</p><ul><li>In industrial environments, where systems frequently disconnect or operate intermittently, streams ensure that data is not lost and can be replayed once systems come back online.</li><li>In analytics and AI pipelines, streams make it possible to backfill historical data or reprocess events as models evolve.</li><li>In event-driven architectures, they provide a reliable way to recover from downstream failures without losing critical data.</li></ul><h3><a href="https://www.emqx.com/en/blog/mqtt-streams-in-emqx#from-messaging-to-data-backbone">From Messaging to Data Backbone</a></h3><p>MQTT has long been known for efficient real-time communication.</p><p>With the introduction of streams in EMQX 6.1, it becomes something more. It becomes a system that not only moves data, but also retains, replays, and supports downstream processing at scale.</p><p>MQTT Streams transform MQTT from a transient messaging protocol into a durable data backbone for modern IoT and AI systems.</p><p>As EMQX continues to evolve with releases like 6.2, this foundation becomes increasingly important, enabling not just real-time data movement but reliable, replayable, and coordinated systems at scale.</p><p><em>Originally published at</em> <a href="https://www.emqx.com/en/blog/mqtt-streams-in-emqx"><em>www.emqx.com</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9348fd27f80" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Your UNS Architecture Requires Native MQTT Governance and How EMQX Enforces It]]></title>
            <link>https://emqx.medium.com/why-your-uns-architecture-requires-native-mqtt-governance-and-how-emqx-enforces-it-45021c875e22?source=rss-2287ea9742a6------2</link>
            <guid isPermaLink="false">https://medium.com/p/45021c875e22</guid>
            <category><![CDATA[mqtt]]></category>
            <category><![CDATA[unified-name-space]]></category>
            <dc:creator><![CDATA[EMQ Technologies]]></dc:creator>
            <pubDate>Wed, 03 Jun 2026 08:00:39 GMT</pubDate>
            <atom:updated>2026-06-03T08:00:39.753Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>EMQX 6.2 has recently been released, introducing a broker-level Unified Namespace governance to enforce topic structure and data contracts at the source.</blockquote><blockquote>We’d been debating topic and schema governance for UNS inside EMQX for a long time. The urgency crystallized after a hallway conversation with a user at an industry conference who described how a single misconfigured gateway had quietly polluted their data lake. We went back, built and shipped the first version as a plugin in EMQX 6.2 so teams could act immediately. Looking ahead, we plan to promote UNS Governance to a native feature in EMQX 6.3 to make it even easier to adopt and operate at scale.</blockquote><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#the-hidden-cost-of-an-ungoverned-unified-namespace">The Hidden Cost of an Ungoverned Unified Namespace</a></h3><p>The Unified Namespace (UNS) has become the de facto architecture for modern industrial IoT. By organizing all operational data into a single, hierarchical MQTT topic tree — typically following ISA-95 — manufacturers gain a shared data layer that connects OT, IT, and cloud systems without point-to-point integrations.</p><p>But there’s a problem nobody talks about at the conference keynotes: <strong>topic sprawl</strong>.</p><p>Six months after go-live, your clean ISA-95 hierarchy looks like this:</p><pre>v1/WTP1/Intake/PUMP01/Sensors/TRB001/Reading     &lt;- correct<br>v1/wtp1/Intake/PUMP01/Sensors/TRB001/Reading     &lt;- wrong case<br>v1/WTP1/Boiling/TANK01/Sensors/TMP001/Reading     &lt;- &quot;Boiling&quot; is not a valid area<br>v1/WTP1/Intake/filter-3/Sensors/PRS001/Reading    &lt;- wrong ID format<br>v1/WTP1/Intake/PUMP01/Debug/Log                   &lt;- unauthorized branch<br>v1/WTP1/Intake/PUMP01/Sensors/TRB001/Reading      &lt;- payload: {&quot;value&quot;:&quot;not-a-number&quot;}</pre><p>Every malformed topic or invalid payload is a silent failure. Downstream analytics ingest garbage. Historian queries return incomplete datasets. Dashboards show gaps. And the root cause? Someone on a different shift deployed an edge gateway with a typo in the topic configuration.</p><p>Bad data quality in manufacturing costs an average of 15–25% of operating revenue, according to industry estimates. In a Unified Namespace, a single misconfigured device can corrupt the data layer for every consumer downstream.</p><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#what-is-mqtt-topic-governance">What Is MQTT Topic Governance?</a></h3><p>Topic governance is the enforcement of a defined schema on your MQTT topic tree. Just as a relational database enforces table schemas and column types, topic governance validates that every MQTT message published to your broker:</p><ol><li><strong>Follows an approved topic hierarchy</strong>: only recognized paths are allowed</li><li><strong>Uses valid segment values</strong>: site IDs, area names, and equipment IDs match defined patterns</li><li><strong>Carries a well-formed payload</strong>: JSON structure, required fields, and value types are validated</li></ol><p>You want your UNS to be a contract but not just a suggestion.</p><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#introducing-uns-governance-for-emqx">Introducing UNS Governance for EMQX</a></h3><p>EMQX 6.2 introduces the <strong>UNS Governance plugin</strong>: a native, broker-level enforcement layer that validates every MQTT publish against your Unified Namespace model.</p><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#how-it-works">How It Works</a></h3><p>UNS Governance operates as part of EMQX’s message processing pipeline. When a client publishes a message:</p><ol><li><strong>Topic-filter pre-check</strong>: The plugin matches the publish topic against compiled topic filters derived from your active models. This is a fast, O(1) lookup, not a tree traversal for every message.</li><li><strong>Topic structure validation</strong>: The matched model validates each topic segment. Variables (like {site_id} or {area_id}) are checked against regex patterns or enumerated value lists.</li><li><strong>Payload schema validation</strong>: If enabled, the JSON payload is validated against the endpoint’s schema definition — required fields, data types, value enumerations, and whether additional properties are allowed.</li><li><strong>Enforcement action</strong>: Invalid messages are rejected (for topic violations) or silently dropped (for payload violations), with detailed counters and recent-drop event logs for observability.</li></ol><pre>MQTT Publish<br>    |<br>    v<br>+---------------------+<br>|  Topic Filter Match  |---- No match ---&gt; Reject (topic_nomatch)<br>+---------+-----------+<br>          | match<br>          v<br>+---------------------+<br>|  Segment Validation  |---- Invalid ----&gt; Reject (topic_invalid)<br>+---------+-----------+<br>          | valid<br>          v<br>+---------------------+<br>|  Endpoint Check      |---- Not leaf ---&gt; Reject (not_endpoint)<br>+---------+-----------+<br>          | endpoint<br>          v<br>+---------------------+<br>|  Payload Validation  |---- Invalid ----&gt; Drop (payload_invalid)<br>+---------+-----------+<br>          | valid<br>          v<br>      Delivered</pre><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#key-capabilities">Key Capabilities</a></h3><p><strong>Declarative topic tree models.</strong> Define your UNS schema as a JSON model with ISA-95-aligned hierarchy, variable constraints, and payload schemas. No code required.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WdcU47KdPpdkq63O" /></figure><p>The screenshot above is the topic tree editor UI, and below is the raw model spec.</p><pre>{<br>  &quot;id&quot;: &quot;water-treatment&quot;,<br>  &quot;variable_types&quot;: {<br>    &quot;site_id&quot;:   { &quot;type&quot;: &quot;string&quot;, &quot;pattern&quot;: &quot;^WTP[0-9]{1,3}$&quot; },<br>    &quot;area_id&quot;:   { &quot;type&quot;: &quot;enum&quot;, &quot;values&quot;: [&quot;Intake&quot;, &quot;Coagulation&quot;, &quot;Filtration&quot;, &quot;Disinfection&quot;] },<br>    &quot;unit_id&quot;:   { &quot;type&quot;: &quot;string&quot;, &quot;pattern&quot;: &quot;^[A-Z]{2,6}[0-9]{1,3}$&quot; }<br>  },<br>  &quot;tree&quot;: {<br>    &quot;v1&quot;: {<br>      &quot;children&quot;: {<br>        &quot;{site_id}&quot;: {<br>          &quot;children&quot;: {<br>            &quot;{area_id}&quot;: {<br>              &quot;children&quot;: {<br>                &quot;{unit_id}&quot;: {<br>                  &quot;children&quot;: {<br>                    &quot;Status&quot;:  { &quot;_payload&quot;: &quot;equipment_status&quot; },<br>                    &quot;Alarm&quot;:   { &quot;_payload&quot;: &quot;alarm&quot; },<br>                    &quot;Sensors&quot;: { &quot;children&quot;: { &quot;{sensor_id}&quot;: { &quot;children&quot;: {<br>                      &quot;Reading&quot;: { &quot;_payload&quot;: &quot;sensor_reading&quot; }<br>                    }}}}<br>                  }<br>                }<br>              }<br>            }<br>          }<br>        }<br>      }<br>    }<br>  }<br>}</pre><p><strong>JSON Schema payload validation.</strong> Each endpoint in your topic tree can reference a payload schema that enforces required fields, data types, and value enumerations.</p><pre>{<br>  &quot;sensor_reading&quot;: {<br>    &quot;type&quot;: &quot;object&quot;,<br>    &quot;required&quot;: [&quot;value&quot;, &quot;unit&quot;, &quot;ts&quot;],<br>    &quot;properties&quot;: {<br>      &quot;value&quot;:   { &quot;type&quot;: &quot;number&quot; },<br>      &quot;unit&quot;:    { &quot;type&quot;: &quot;string&quot;, &quot;enum&quot;: [&quot;mg/L&quot;, &quot;NTU&quot;, &quot;pH&quot;, &quot;degC&quot;, &quot;mbar&quot;, &quot;m3/h&quot;] },<br>      &quot;ts&quot;:      { &quot;type&quot;: &quot;integer&quot; },<br>      &quot;quality&quot;: { &quot;type&quot;: &quot;string&quot;, &quot;enum&quot;: [&quot;good&quot;, &quot;uncertain&quot;, &quot;bad&quot;] }<br>    },<br>    &quot;additionalProperties&quot;: false<br>  }</pre><p><strong>Real-time observability.</strong> A built-in dashboard shows cluster-wide metrics — total messages, allowed, dropped — broken down by rejection reason and per-model. A recent-drops event log shows exactly which topics failed and why.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*y8szTvn7GpazbgeT" /></figure><p>Realtime Statistics</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3xz_5uaHb0QjifrC" /></figure><p>Recent Events</p><p><strong>Multiple model support.</strong> Run different UNS models for different namespaces (e.g., v1/ for production, v2/ for a pilot line) with deterministic model selection and per-model metrics.</p><p><strong>Exempt topics.</strong> System topics, diagnostics, or legacy integrations can bypass governance via configurable topic filter exemptions.</p><p><strong>Bootstrap models.</strong> Ship a default UNS model with the plugin so governance is active from first boot — no manual configuration required.</p><p><strong>Visual schema editor.</strong> A built-in web UI lets engineers browse the topic tree, edit payload schemas, and validate topics interactively through the EMQX Dashboard.</p><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#real-world-example-1-municipal-water-treatment">Real-World Example 1: Municipal Water Treatment</a></h3><p>Consider a municipal water treatment plant running EMQX as the UNS backbone. The ISA-95 topic hierarchy covers six process areas, from raw water intake through distribution:</p><pre>v1/{site_id}/{area_id}/{unit_id}/Sensors/{sensor_id}/Reading<br>v1/{site_id}/{area_id}/{unit_id}/Status<br>v1/{site_id}/{area_id}/{unit_id}/Alarm<br>v1/{site_id}/{area_id}/{unit_id}/Dosing</pre><p><strong>Without governance</strong>, a misconfigured PLC publishes turbidity readings to v1/wtp1/Intake/PUMP01/Sensors/TRB001/Reading (lowercase site ID). The historian ingests it under a different key. The compliance dashboard shows a gap in turbidity monitoring. An operator manually investigates — hours later.</p><p><strong>With UNS Governance</strong>, the publish is rejected at the broker with a topic_invalid reason. The device receives a PUBACK error code. The operations team sees the rejection in the Stats dashboard immediately. The PLC vendor fixes the configuration. Total time to resolution: minutes, not hours.</p><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#real-world-example-2-multi-team-plant-management">Real-World Example 2: Multi-Team Plant Management</a></h3><p>Consider a large manufacturing plant where different operational teams are responsible for distinct sectors, each deploying and managing their own edge devices. For instance, one team manages the assembly line, another handles quality control, and a third manages raw material intake. Each team has its own developers configuring MQTT publishers for their respective devices.</p><p>Without governance, a developer on the quality control team might inadvertently configure a device to publish data to a topic like v1/FACTORY_A/Quality/InspectionLine01/Sensors/Temperature/reading using a different casing or an unrecognized segment value, e.g., v1/factory_a/Quality/InspectionLine01/Sensors/Temperature/reading (lowercase factory ID). Or, a new team member, unaware of the established schema, might introduce a new topic branch like v1/FACTORY_A/Quality/TemporaryTesting/Data. These seemingly small deviations lead to fragmented data, where analytics dashboards fail to correlate data across the entire plant, and critical production insights are missed. Data scientists attempting to build predictive models struggle with inconsistent topic paths and unreliable data sets, leading to a lack of trust in the overall data.</p><p>With UNS Governance, any message published with a non-conforming topic path or an invalid payload schema is immediately rejected at the broker. The team responsible gets instant feedback to correct misconfigurations before bad data pollutes the UNS. Centralized enforcement keeps every team aligned to a single contract, ensuring trustworthy data across the entire plant and enabling accurate, real-time decisions.</p><p>Payload validation catches subtler issues too. A sensor gateway sends {&quot;value&quot;:&quot;not-a-number&quot;,&quot;unit&quot;:&quot;NTU&quot;,&quot;ts&quot;:1712678400000} — the value field is a string instead of a number. Without governance, this silently corrupts downstream analytics. With governance, the message is dropped, the counter increments, and the event appears in the recent-drops log with the exact validation error.</p><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#why-enforce-governance-at-the-broker">Why Enforce Governance at the Broker?</a></h3><p>There are three places you can enforce a UNS schema: at the edge device, in application middleware, or at the broker. Here’s why the broker is the right layer:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*teMYbDQiw3JAzWcYZ4yl9g.png" /></figure><p>Broker-level enforcement is the only approach that guarantees <strong>no invalid message reaches any subscriber</strong>. It’s also the only approach that scales with your MQTT infrastructure rather than requiring a parallel validation fleet.</p><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#getting-started-in-10-minutes">Getting Started in 10 Minutes</a></h3><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#step-1-install-the-plugin">Step 1: Install the Plugin</a></h3><p>Upload the UNS Governance plugin package through the EMQX Dashboard (Management -&gt; Plugins) or via the REST API. The plugin is compatible with EMQX 6.1 and later.</p><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#step-2-define-your-model">Step 2: Define Your Model</a></h3><p>Create a JSON model that describes your UNS topic hierarchy. Start with the bundled ISA-95 template and customize the variable constraints and payload schemas for your environment.</p><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#step-3-activate-and-monitor">Step 3: Activate and Monitor</a></h3><p>Upload the model via the plugin API or Dashboard UI. Set on_mismatch: deny and validate_payload: true. Every non-conforming publish is now rejected or dropped, with full observability through the Stats dashboard.</p><pre># Upload and activate a model<br>curl -H &quot;Authorization: Bearer $TOKEN&quot; \<br>     -H &quot;Content-Type: application/json&quot; \<br>     -X POST http://localhost:18083/api/v5/plugin_api/emqx_unsgov/models \<br>     -d &#39;{&quot;activate&quot;: true, &quot;model&quot;: &lt;your-model-json&gt;}&#39;<br># Check enforcement stats<br>curl -H &quot;Authorization: Bearer $TOKEN&quot; \<br>     http://localhost:18083/api/v5/plugin_api/emqx_unsgov/stats</pre><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#step-4-iterate">Step 4: Iterate</a></h3><p>Use the visual schema editor to refine your model. Add payload schemas for new endpoint types. Review recent-drops events to identify misconfigured devices. Your UNS schema evolves with your operations — governed from day one.</p><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#built-for-enterprise-scale">Built for Enterprise Scale</a></h3><p>UNS Governance is built as a native EMQX plugin, which means:</p><ul><li><strong>Cluster-wide enforcement</strong>: Models and configuration replicate across all nodes automatically. Stats are aggregated cluster-wide.</li><li><strong>Hot reconfiguration</strong>: Activate, deactivate, or update models without broker restart.</li><li><strong>Prometheus integration</strong>: All counters are exposed in Prometheus text format at /metrics for integration with Grafana, Datadog, or your existing monitoring stack.</li><li><strong>Zero-downtime deployment</strong>: Install or upgrade the plugin without disrupting connected clients.</li></ul><p>Combined with EMQX’s proven ability to handle 100M+ concurrent connections, UNS Governance scales from a single-site pilot to a global multi-plant deployment.</p><h3><a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns#from-data-chaos-to-data-contracts">From Data Chaos to Data Contracts</a></h3><p>The Unified Namespace is only as valuable as the data flowing through it. Without governance, it’s a convention that erodes over time. With EMQX UNS Governance, it becomes an enforceable contract — validated at wire speed, observable in real time, and managed from a single pane of glass.</p><p><strong>Your 10 minutes start now.</strong> Install the plugin, upload a model, and watch every non-conforming message get caught before it reaches a single subscriber.</p><p>Ready to enforce your Unified Namespace? <a href="https://www.emqx.com/en/downloads-and-install/enterprise">Download EMQX 6.2</a> and <a href="https://docs.emqx.com/en/emqx/v6.2/extensions/plugin-catalog/emqx-unsgov.html">UNS Governance Plugin</a>. Or <a href="https://www.emqx.com/en/contact">contact our solutions team</a> to discuss your UNS architecture.</p><p><em>Originally published at</em> <a href="https://www.emqx.com/en/blog/native-mqtt-governance-for-uns"><em>www.emqx.com</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=45021c875e22" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A2A over MQTT: How AI Agents Find Each Other at Scale]]></title>
            <link>https://emqx.medium.com/a2a-over-mqtt-how-ai-agents-find-each-other-at-scale-d051af693678?source=rss-2287ea9742a6------2</link>
            <guid isPermaLink="false">https://medium.com/p/d051af693678</guid>
            <category><![CDATA[mqtt]]></category>
            <dc:creator><![CDATA[EMQ Technologies]]></dc:creator>
            <pubDate>Wed, 27 May 2026 09:33:52 GMT</pubDate>
            <atom:updated>2026-05-27T09:33:52.466Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Fragmentation of AI: Cross-Runtime Agent Coordination</h3><p>AI agents can already use tools. Whether through <a href="https://www.emqx.com/en/blog/mcp-over-mqtt">MCP</a> servers or through skills (agent-native capabilities defined as prompts with tool access, which many practitioners now prefer for their simplicity), agents read databases, call APIs, and control devices. This layer is maturing fast.</p><p>Composing multiple agents within a single team is also a solved problem. Frameworks like CrewAI, LangGraph, and AutoGen let you define agents with different roles and wire them into pipelines. If all your agents share a codebase and a runtime, these tools work well.</p><p>But consider a smart home with AI agents: A lighting agent, a thermostat agent, a security agent, and an energy optimization agent. Each was built by a different vendor. Different codebases, different programming languages, different deployment environments, different release cycles. You want a workflow that spans all four: “when I leave home, lock the doors, turn off the lights, lower the heating.” The kind of cross-agent coordination that makes the whole system more than the sum of its parts.</p><p>No framework can compose agents that don’t share a runtime. No skill or MCP server handles agent-to-agent communication. The missing piece is a standard protocol that lets independently deployed agents discover each other’s capabilities and coordinate work across organizational boundaries.</p><p>This is what Google’s <a href="https://a2a-protocol.org/latest">A2A protocol</a> addresses.</p><h3>A2A Protocol: A Standardized Era of Agent Interoperability</h3><p>A2A defines four core concepts:</p><ul><li><strong>Agent Cards</strong> — structured metadata describing an agent’s capabilities, interfaces, and security requirements</li><li><strong>Messages</strong> — single communications between agents: a request, a reply, a clarification</li><li><strong>Tasks</strong> — stateful units of work that span one or more messages and progress through lifecycle states: submitted, working, completed</li><li><strong>Artifacts</strong> — results produced incrementally as a task progresses</li></ul><p>The protocol standardizes how agents find each other and delegate work, regardless of who built them or where they run.</p><p>With A2A in the picture, the landscape breaks down into three categories:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*izhPezHmvE0E9VBrQl7FAQ.png" /></figure><p>A2A is transport-agnostic by design. The reference implementation runs over HTTP and server-sent events. But the coordination patterns that multi-agent systems demand (discovery, presence tracking, fan-out, load-balanced dispatch) are a poor fit for point-to-point request-response. A <a href="https://www.emqx.com/en/blog/why-mqtt-is-the-missing-infrastructure-layer-for-agentic-ai">previous post</a> in this series made the detailed case for why MQTT is a more natural transport for agent coordination. This post goes deeper: how agent discovery actually works over MQTT at the protocol level, what coordination patterns it enables, and how <a href="https://www.emqx.com/en/blog/emqx-6-2-0-release-notes">EMQX 6.2</a> makes it production-ready.</p><h3>What an MQTT Broker Gives You for Free</h3><p>Before the protocol mechanics, a quick orientation. The coordination primitives that multi-agent systems need aren’t separate systems in an MQTT stack: retained messages deliver always-available state to any new subscriber (discovery), Last Will and Testament fires crash notifications without health-check polling (presence), shared subscriptions distribute work across competing consumers (load balancing), and topic-level ACLs constrain each client to paths matching its own {org_id}/{unit_id}/{agent_id} identity (authorization scoping). The same protocol runs on an ESP32 or in the cloud, with no gateway translation. One operational surface instead of five. <a href="https://www.emqx.com/en/blog/why-mqtt-is-the-missing-infrastructure-layer-for-agentic-ai">Our previous post</a> makes the full case; the rest of this one assumes it and goes a layer deeper.</p><h3>The Protocol: How A2A Discovery Works over MQTT</h3><p>The <a href="https://github.com/emqx/mqtt-for-ai/tree/main/a2a-over-mqtt">A2A-over-MQTT specification</a> defines how agents discover, communicate, and coordinate through an MQTT broker. It’s an open spec, hosted on GitHub, designed to work with any MQTT v5 broker.</p><h3>Topic Namespace</h3><p>All A2A communication lives under the $a2a/v1/ prefix, organized into four namespaces:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*BnGc9N22CU7rzlDD" /></figure><p>The hierarchy is multi-tenant by design: organization, unit, agent. Wildcard discovery follows naturally:</p><ul><li>$a2a/v1/discovery/{org_id}/+/+ finds every agent in an organization</li><li>$a2a/v1/discovery/+/+/+ finds every agent on the broker</li></ul><p>The {reply_suffix} segment on the reply topic is a requester-generated token with high collision resistance, so concurrent in-flight requests don&#39;t overlap reply streams.</p><h3>Agent Cards</h3><p>An Agent Card is a JSON document that describes what an agent can do and how to talk to it. Published as a retained message on the discovery topic, it’s the agent’s “business card” on the network:</p><pre>{<br>  &quot;name&quot;: &quot;energy-optimizer&quot;,<br>  &quot;description&quot;: &quot;Optimizes energy usage across connected devices&quot;,<br>  &quot;version&quot;: &quot;1.0&quot;,<br>  &quot;defaultInputModes&quot;: [&quot;application/json&quot;],<br>  &quot;defaultOutputModes&quot;: [&quot;application/json&quot;],<br>  &quot;capabilities&quot;: {<br>    &quot;streaming&quot;: true,<br>    &quot;pushNotifications&quot;: true<br>  },<br>  &quot;skills&quot;: [<br>    {<br>      &quot;id&quot;: &quot;optimize-schedule&quot;,<br>      &quot;name&quot;: &quot;Optimize Energy Schedule&quot;,<br>      &quot;description&quot;: &quot;Generates an optimal schedule based on tariffs and occupancy&quot;,<br>      &quot;tags&quot;: [&quot;energy&quot;, &quot;optimization&quot;]<br>    }<br>  ],<br>  &quot;supportedInterfaces&quot;: [<br>    {<br>      &quot;url&quot;: &quot;mqtt://broker.emqx.io:8883&quot;,<br>      &quot;protocolBinding&quot;: &quot;a2a-over-mqtt/0.1&quot;,<br>      &quot;protocolVersion&quot;: &quot;0.1&quot;<br>    }<br>  ],<br>  &quot;securitySchemes&quot;: {<br>    &quot;oauth2&quot;: {<br>      &quot;oauth2SecurityScheme&quot;: {<br>        &quot;flows&quot;: {<br>          &quot;clientCredentials&quot;: {<br>            &quot;tokenUrl&quot;: &quot;https://auth.example.com/token&quot;,<br>            &quot;scopes&quot;: { &quot;energy:read&quot;: &quot;Read data&quot;, &quot;energy:control&quot;: &quot;Control devices&quot; }<br>          }<br>        }<br>      }<br>    }<br>  }<br>}</pre><p>The example omits optional fields for brevity; the <a href="https://a2a-protocol.org/latest/definitions/#json">A2A JSON Schema</a> defines the full AgentCard structure.</p><p>Because cards are retained messages, a new subscriber doesn’t need to wait for agents to re-announce themselves. Subscribe to the discovery wildcard and the broker delivers every existing card immediately. When an agent updates its capabilities, it publishes a new card; subscribers receive the update as a normal message.</p><p>Agents can also be discovered via HTTP /.well-known/agent-card.json for interoperability with standard A2A clients. When both transports are available, clients pick MQTT from supportedInterfaces.</p><h3>Request/Reply Flow</h3><p>Here’s what a complete interaction looks like on the wire:</p><ol><li><strong>Discover.</strong> The requester subscribes to $a2a/v1/discovery/{org_id}/+/+ and receives retained Agent Cards for all available agents.</li><li><strong>Subscribe to replies.</strong> The requester subscribes to its own reply topic: $a2a/v1/reply/{org_id}/{unit_id}/{agent_id}/{reply_suffix}.</li><li><strong>Publish request.</strong> The requester publishes to the target agent’s request topic, including:</li></ol><ul><li>MQTT v5 <strong>Response Topic</strong>, pointing to the requester’s reply topic</li><li>MQTT v5 <strong>Correlation Data</strong>, a unique identifier for this specific request</li><li>Payload containing the A2A JSON-RPC request with Task.id</li></ul><p><strong>4. Broker routes.</strong> The broker delivers the message to the agent subscribed on the request topic.</p><p><strong>5. Agent replies.</strong> The agent processes the task and publishes the result to the Response Topic, copying back the Correlation Data and including the task status in the payload.</p><p><strong>6. Broker delivers.</strong> The requester receives the reply on its subscription, matched by Correlation Data.</p><p>The entire flow uses standard MQTT v5 features. Response Topic and Correlation Data were added to the spec precisely for request-response patterns. QoS 1 (at-least-once delivery) is recommended. No custom headers, no protocol hacks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/760/0*M2hgPRKGQC18VU2r" /></figure><h3>What Discovery Enables</h3><p>Discovery isn’t the end goal. It’s the foundation that makes real coordination possible. Once agents can find each other and establish communication channels, several powerful patterns emerge, all built on the same MQTT primitives.</p><h3>Streaming Task Updates</h3><p>Not every task completes in a single response. A research agent compiling a report, an analytics agent processing a large dataset, a planning agent evaluating options. These take time.</p><p>A2A over MQTT handles this naturally. The agent streams back multiple replies on the requester’s Response Topic, all sharing the same Correlation Data. Each reply carries a task status: submitted→ working (with partial artifacts) → completed. The requester stays subscribed until it receives a terminal state: completed, failed, canceled, or rejected. Artifacts arrive incrementally as the agent produces them.</p><p>This is discrete event streaming, not a held-open connection. Each status update is a separate MQTT PUBLISH. If the requester disconnects briefly and reconnects, QoS 1 ensures pending updates are delivered.</p><h3>Agent Pools and Load Balancing</h3><p>When you need multiple instances of the same agent (say, ten instances of a translation agent handling concurrent requests), pool topics and shared subscriptions provide load balancing without external infrastructure.</p><p>Agents subscribe to a pool topic using MQTT shared subscriptions: $share/{group}/$a2a/v1/request/{org_id}/{unit_id}/pool/translators. The broker picks one subscriber per message and delivers the request to that instance only.</p><p>The selected agent includes its specific identity in the reply via the a2a-responder-agent-idMQTT v5 User Property. The requester now knows exactly which instance handled the request and routes follow-up messages directly to that agent&#39;s individual request topic, maintaining session affinity without sticky sessions or cookies.</p><p>Scaling is adding subscribers. An agent instance spins up, subscribes to the shared group, and starts receiving requests. No load balancer reconfiguration, no routing table updates.</p><h3>Presence and Liveness</h3><p>LWT gives you crash detection for free (as covered above), but the A2A-over-MQTT spec builds a richer presence model on top. When an agent publishes its card, it attaches a2a-status=onlineas an MQTT v5 User Property. Its LWT carries the same card with a2a-status=offline. The spec also defines a2a-status-source to distinguish three cases:</p><ul><li>agent: graceful shutdown, the agent published the status itself</li><li>lwt: unexpected disconnect, the broker fired the will</li><li>broker: status managed by the broker (used by the EMQX A2A Registry)</li></ul><p>This distinction matters. A supervisor that sees lwt knows to respawn the agent. A dashboard that sees agent knows the shutdown was intentional and doesn&#39;t page anyone.</p><h3>Task Handover</h3><p>Sometimes the right agent to start a task isn’t the right agent to finish it. A generalist front-desk agent receives a request, evaluates it, and decides a specialist is better suited. It replies with a2a-responder-agent-id pointing to the specialist agent. The requester re-routes follow-up messages directly to the specialist&#39;s request topic.</p><p>The Task ID persists across the handover. From the requester’s perspective, it’s a seamless continuation. The front-desk agent never needs to proxy traffic or remain in the data path. This pattern is natural for hierarchical agent architectures where a triage agent routes work to domain specialists.</p><h3>Multi-Turn Conversations</h3><p>A single exchange isn’t always enough. A2A defines two multi-turn shapes, and both work cleanly over MQTT.</p><p><strong>Interrupted task: same Task ID.</strong> A booking agent may need clarification: “What dates? Economy or business?” It replies with input-required status instead of completed, pausing the task pending more input. The requester sends the answer on the same request topic, reusing the <strong>same Task ID</strong>; the agent resumes where it left off. Correlation Data is fresh on each publish (it&#39;s per-exchange), but Task ID is stable for the life of the task.</p><p><strong>Multi-step session: different Task IDs, shared Context ID.</strong> A workflow of distinct tasks (“book the flight, then reserve the hotel”) can be grouped into one conversation via <strong>Context ID</strong>. Each task has its own UUIDv4 Task ID; all of them share one UUIDv4 Context ID so the agent can maintain state (conversation history, LLM context) across the whole session.</p><p>Context ID is optional by design. Simple request-reply interactions don’t need it. Interrupted tasks and multi-step sessions, that’s where it earns its place.</p><h3>From Spec to Production: The A2A Registry in EMQX 6.2</h3><p>Everything above works on any MQTT v5 broker, the spec is deliberately broker-neutral. But running discovery at scale exposes two protocol-adjacent questions the spec doesn’t answer. The <a href="https://www.emqx.com/en/blog/emqx-6-2-0-release-notes">A2A Agent Registry</a> in EMQX 6.2 addresses both; a <a href="https://www.emqx.com/en/blog/how-mqtt-turns-iot-fleets-into-selfcoordinating-systems">previous post</a> covers its full capabilities (schema validation, broker-managed status tracking, Dashboard and CLI management).</p><p><strong>Multiple registration paths, one namespace.</strong> Agents can self-register via MQTT retained publish, operators can register through the Dashboard, and automation can use the REST API. All three produce the same artifact: a retained message on $a2a/v1/discovery/. Subscribers see no difference, so you can bootstrap manually in development and switch to self-registration in production without touching the discovery logic.</p><p><strong>Rate and size limits on registration.</strong> When hundreds of agents reconnect after a broker restart or network partition, the discovery namespace can flood with registration publishes. Configurable rate limits and card-size caps absorb these bursts without degrading the broker.</p><h3>What Comes Next</h3><p>The stack for AI agent coordination is crystallizing. Skills and MCP give agents hands: the ability to use tools and access data. A2A gives agents identity (discoverable presence on a network) and voice (a structured protocol for delegating and coordinating work). The MQTT broker is the coordination fabric that connects them, absorbing the infrastructure for discovery, presence, load balancing, and authorization that would otherwise be separate systems to build and operate.</p><p>If you want to build on this:</p><ul><li>The <a href="https://github.com/emqx/mqtt-for-ai/tree/main/a2a-over-mqtt">A2A-over-MQTT specification</a> is open source. <a href="https://github.com/id/skitter">Skitter</a> is a reference implementation you can clone and run today.</li><li><a href="https://www.emqx.com/en/cloud/serverless-mqtt">EMQX Cloud Serverless</a> offers a forever-free tier with a production-grade MQTT broker in seconds, enough to run a multi-agent system without managing infrastructure.</li><li>For the full architectural argument behind MQTT as the agent coordination layer, see the <a href="https://www.emqx.com/en/blog/why-mqtt-is-the-missing-infrastructure-layer-for-agentic-ai">first post</a> in this series. For how this applies to per-device IoT agents, see <a href="https://www.emqx.com/en/blog/how-mqtt-turns-iot-fleets-into-selfcoordinating-systems">the second</a>.</li></ul><p><em>Originally published at</em> <a href="https://www.emqx.com/en/blog/a2a-over-mqtt"><em>www.emqx.com</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d051af693678" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From MQTT to Parquet: Analyzing IoT Data on Amazon S3 with EMQX and DuckDB]]></title>
            <link>https://emqx.medium.com/from-mqtt-to-parquet-analyzing-iot-data-on-amazon-s3-with-emqx-and-duckdb-bc364963da83?source=rss-2287ea9742a6------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc364963da83</guid>
            <category><![CDATA[mqtt]]></category>
            <dc:creator><![CDATA[EMQ Technologies]]></dc:creator>
            <pubDate>Fri, 20 Mar 2026 01:30:02 GMT</pubDate>
            <atom:updated>2026-03-20T01:30:02.653Z</atom:updated>
            <content:encoded><![CDATA[<p>Modern IoT systems generate massive streams of MQTT messages. Getting value from this data means more than just moving it around; you need to store it efficiently and make it easy to query.</p><p>With EMQX’s Amazon S3 integration and <strong>Parquet</strong> support for aggregated uploads, you can turn your MQTT broker into a powerful data ingestion layer for your data lake. In this post, we’ll walk through a practical use case: <strong>Persist MQTT messages to S3 in Parquet format, then analyze them with DuckDB.</strong></p><p>We already cover the configuration details in our documentation for exporting aggregated data in Parquet to <strong>Amazon S3</strong> and other storages. Here, we’ll focus on the <em>end-to-end workflow</em> and what it enables for your IoT analytics.</p><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#why-parquet-for-mqtt-data">Why Parquet for MQTT Data?</a></h3><p>Before jumping into the example, let’s clarify why Parquet is a great fit for MQTT workloads:</p><ul><li><strong>Columnar storage</strong>: Ideal for analytical queries (e.g., “avg temperature by device per hour”), since only the needed columns are scanned.</li><li><strong>High compression</strong>: Typically <strong>much smaller</strong> than CSV, reducing S3 storage costs.</li><li><strong>Query-optimized</strong>: Works natively with modern analytics tools (Athena, Spark, DuckDB, etc.).</li><li><strong>Schema-aware</strong>: Strong typing for timestamps, integers, floats, booleans, etc., which fits structured telemetry data.</li></ul><p>With EMQX, Parquet is now an option for <strong>aggregated uploads</strong> to S3 and Azure Blob Storage, alongside CSV and JSON Lines.</p><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#use-case-overview">Use Case Overview</a></h3><p>Imagine you operate thousands of IoT devices publishing telemetry to EMQX:</p><ul><li>Topic: sensors/&lt;device_id&gt;/telemetry</li><li>Payload (JSON):</li></ul><pre>{<br>  &quot;device_id&quot;: &quot;sensor-001&quot;,<br>  &quot;temperature&quot;: 24.7,<br>  &quot;humidity&quot;: 55.2,<br>  &quot;ts&quot;: 1738588245123<br>}</pre><p>Your goals:</p><ol><li><strong>Persist</strong> all telemetry data to <strong>Amazon S3</strong> in an efficient, analytics-friendly format (Parquet).</li><li><strong>Analyze</strong> this data using <strong>DuckDB</strong>, both ad hoc (local) and potentially in serverless environments.</li></ol><p>We’ll implement:</p><ul><li>EMQX <strong>Connector</strong>: Amazon S3.</li><li>EMQX <strong>Action</strong>: Aggregated upload in <strong>Parquet</strong> format.</li><li>EMQX <strong>Rule</strong>: Select MQTT fields and send them to the S3 Aggregated action.</li><li><strong>DuckDB</strong>: Query the resulting Parquet files directly from S3.</li></ul><blockquote><strong><em>Note:</em></strong><em> For detailed step-by-step S3 configuration and UI screenshots, refer to the </em><a href="https://docs.emqx.com/en/emqx/v6.1/data-integration/s3.html"><em>official documentation</em></a><em> on exporting aggregated data to S3 in Parquet format. In this post, we focus on the end-to-end flow and key configuration points.</em></blockquote><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#step-1-configure-the-s3-connector">Step 1: Configure the S3 Connector</a></h3><p>Create an <strong>Amazon S3 connector</strong> in EMQX (via Dashboard or config file). At a high level, you will specify:</p><ul><li><strong>S3 endpoint</strong> (e.g., s3.us-east-1.amazonaws.com)</li><li><strong>Access key &amp; secret</strong> with permissions to write to your chosen bucket.</li><li>Optional: access mode, encryption, and other S3 options.</li></ul><p>Once the connector is configured, you’re ready to define the S3 Aggregated Action with Parquet output.</p><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#step-2-create-an-s3-action-with-parquet-aggregation">Step 2: Create an S3 Action with Parquet Aggregation</a></h3><p>Next, create an <strong>Action</strong> that uses the S3 connector and selects <strong>Parquet</strong> as the aggregated upload format.</p><p>Key aspects to configure:</p><ol><li><strong>Upload mode:</strong> Aggregated</li><li><strong>Bucket name</strong>, e.g., mqtt-telemetry-parquet.</li><li><strong>Aggregation format:</strong> Parquet Choose <strong>Parquet</strong> instead of CSV/JSONL for aggregated uploads.</li><li><strong>Parquet schema definition:</strong> Parquet files follow a strict schema to organize and compress data efficiently. Either refer to an Avro Schema already registered in Schema Registry, or define one inline that matches your incoming data. e.g.,</li></ol><pre>{<br>  &quot;fields&quot;: [<br>    {<br>      &quot;field-id&quot;: 1,<br>      &quot;name&quot;: &quot;device_id&quot;,<br>      &quot;type&quot;: &quot;string&quot;<br>    },<br>    {<br>      &quot;field-id&quot;: 2,<br>      &quot;name&quot;: &quot;temperature&quot;,<br>      &quot;type&quot;: &quot;float&quot;<br>    },<br>    {<br>      &quot;field-id&quot;: 3,<br>      &quot;name&quot;: &quot;humidity&quot;,<br>      &quot;type&quot;: &quot;float&quot;<br>    },<br>    {<br>      &quot;default&quot;: null,<br>      &quot;field-id&quot;: 4,<br>      &quot;name&quot;: &quot;ts&quot;,<br>      &quot;type&quot;: [<br>        &quot;null&quot;,<br>        {<br>          &quot;adjust-to-utc&quot;: false,<br>          &quot;logicalType&quot;: &quot;timestamp-micros&quot;,<br>          &quot;type&quot;: &quot;long&quot;<br>        }<br>      ]<br>    }<br>  ],<br>  &quot;name&quot;: &quot;root&quot;,<br>  &quot;type&quot;: &quot;record&quot;<br>}</pre><p><strong>5. Upload triggers</strong> (same semantics as existing aggregation):</p><ul><li>max_records – Maximum number of records per file, e.g. 10_000.</li><li>time_interval – Maximum time window, e.g., 60s.</li></ul><p>EMQX triggers the upload when <strong>either</strong> condition is met.</p><p><strong>6. Compression</strong></p><p>Parquet files support multiple compression algorithms, typically:</p><ul><li>snappy (default; good balance of speed and size)</li><li>gzip</li><li>zstd</li></ul><p>For most IoT analytics workloads, snappy is a great default.</p><p><strong>7. S3 object key strategy</strong></p><p>Use time-based partitioning to keep directories manageable and to optimize downstream queries, as well as to avoid data collisions in data coming from different nodes and actions in the cluster. Example object path pattern:</p><pre>${action}/${node}/${datetime.rfc3339utc}/N${sequence}.parquet</pre><p>Once saved, this action will collect records from the rule, batch them, and upload <strong>compressed Parquet files</strong> to your S3 bucket.</p><p>Finally, let’s set up the Rule SQL to tie the Action with incoming device data to be ingested.</p><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#step-3-define-the-data-shape-with-emqx-rule-sql">Step 3: Define the Data Shape with EMQX Rule SQL</a></h3><p>In EMQX, the <strong>Rule SQL</strong> defines both the data routing logic and its transformations. Fields selected in the SELECT clause become available for our Action to use in columns in the Parquet file.</p><p>For our telemetry example:</p><pre>SELECT<br>  payload.device_id          AS device_id,<br>  payload.temperature        AS temperature,<br>  payload.humidity           AS humidity,<br>  publish_received_at * 1000 AS publish_received_at<br>FROM<br>  &quot;sensors/+/telemetry&quot;</pre><p>What this does:</p><ol><li>Subscribes to sensors/+/telemetry (all devices).</li></ol><p>2. Extracts relevant metrics from the JSON payload.</p><p>3. Casts EMQX message metadata into a well-defined schema that matches the defined Parquet schema:</p><ul><li>device_id: string</li><li>temperature: float</li><li>humidity: float</li><li>publish_received_at: integer (timestamp in microseconds)</li></ul><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#step-4-inspect-parquet-files-in-s3">Step 4: Inspect Parquet Files in S3</a></h3><p>After some data flows through, you should see objects like:</p><pre>s3://mqtt-telemetry-parquet/my_action/emqx@node1.my.cluster/2026-02-03T14_19_32Z/N0.parquet<br>s3://mqtt-telemetry-parquet/my_action/emqx@node1.my.cluster/2026-02-03T14_19_32Z/N1.parquet<br>...</pre><p>Each file holds up to max_records rows (or less, if time_interval triggers earlier). Thanks to Parquet + compression, file sizes should be significantly smaller than equivalent CSV.</p><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#step-5-analyze-the-data-with-duckdb">Step 5: Analyze the Data with DuckDB</a></h3><p><a href="https://duckdb.org/">DuckDB</a> is an in-process analytical database that excels at querying Parquet data, both locally and from object storage.</p><p>Here’s how to query your S3-based Parquet data from DuckDB.</p><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#option-a-download-files-locally">Option A: Download Files Locally</a></h3><p>For quick experiments, you can pull files locally (e.g., via aws s3 cp s3://... ./data/) and run DuckDB on them:</p><pre>-- In DuckDB shell<br>CREATE TABLE telemetry AS<br>SELECT *<br>FROM read_parquet(&#39;data/my_action/emqx@node1.my.cluster/2026-02-03T14_19_32Z/N0.parquet&#39;);<br>​<br>-- Example queries<br>SELECT<br>  device_id,<br>  date_trunc(&#39;hour&#39;, publish_received_at) AS hour,<br>  avg(temperature) AS avg_temp,<br>  avg(humidity)    AS avg_humidity<br>FROM telemetry<br>GROUP BY device_id, hour<br>ORDER BY hour, device_id;</pre><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#option-b-query-directly-from-s3">Option B: Query Directly from S3</a></h3><p>DuckDB can read from S3 without downloading all files locally, by configuring S3 access:</p><pre>INSTALL httpfs;<br>LOAD httpfs;<br>​<br>CREATE SECRET (<br>    TYPE s3,<br>    KEY_ID &#39;AKIAIOSFODNN7EXAMPLE&#39;,<br>    SECRET &#39;wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY&#39;,<br>    REGION &#39;us-east-1&#39;<br>);</pre><p>Then query straight from S3:</p><pre>SELECT<br>  device_id,<br>  date_trunc(&#39;hour&#39;, publish_received_at) AS hour,<br>  avg(temperature) AS avg_temp<br>FROM read_parquet(<br>  &#39;s3://mqtt-telemetry-parquet/my_action/*/*/N*.parquet&#39;<br>)<br>GROUP BY device_id, hour<br>ORDER BY hour, device_id<br>LIMIT 100;</pre><p>You can run DuckDB:</p><ul><li>Locally on your laptop.</li><li>Inside a container or a lightweight API service.</li><li>In serverless environments for ad hoc analytics or dashboards.</li></ul><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#best-practices-tips">Best Practices &amp; Tips</a></h3><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#_1-choose-sensible-aggregation-windows">1. Choose Sensible Aggregation Windows</a></h3><ul><li>High-frequency telemetry → smaller intervals (10–30s) or smaller max_records.</li><li>Low-frequency or sparse data → larger intervals (e.g., 1–5 minutes) to avoid too many tiny files.</li></ul><p><strong>Goal:</strong> Balance file size (for efficient queries) and data latency (how fresh your analytics need to be).</p><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#_2-design-a-query-friendly-partition-layout">2. Design a Query-Friendly Partition Layout</a></h3><p>In this example, we partitioned by the simple aggregation timestamp. Depending on your data rate, further partitioning by date/time could be an option:</p><ul><li>year=YYYY/month=MM/day=DD/hour=HH</li><li>Optionally device_id if you often query per device, but be careful of too many small partitions.</li></ul><p>This allows tools like DuckDB, Spark, and Athena to <strong>prune</strong> partitions efficiently and reduce I/O.</p><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#_3-keep-schemas-stable">3. Keep Schemas Stable</a></h3><p>Adding/removing columns is supported by Parquet, but frequent schema changes can complicate downstream queries. When evolving the schema:</p><ul><li>Add new columns rather than frequently renaming/removing.</li><li>Use views in DuckDB to hide schema complexity from consumers.</li></ul><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#_4-use-snappy-compression-by-default">4. Use Snappy Compression by Default</a></h3><p>snappy usually gives the best trade-off for IoT workloads:</p><ul><li>Fast compression/decompression.</li><li>Good enough size reduction for most telemetry data.</li></ul><p>Use gzip or zstd only if storage cost is critical and you can tolerate slower CPU performance during reads.</p><h3><a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet#what-you-gain-with-emqx-s3-parquet-duckdb">What You Gain with EMQX + S3 Parquet + DuckDB</a></h3><p>By combining EMQX’s Parquet support for S3 with DuckDB, you get:</p><ol><li><strong>A simple IoT data lake ingestion layer</strong>: MQTT → EMQX → S3 Parquet, no custom ingestion service required.</li></ol><p><strong>2. Cost-efficient, analytics-ready storage</strong>: Columnar, compressed Parquet files directly queryable by modern tools.</p><p><strong>3. Flexible analytics workflows</strong>: Use DuckDB for:</p><ul><li>Ad hoc exploratory analysis.</li><li>Lightweight dashboards.</li><li>Embedded analytics in internal tools or backends.</li></ul><p><strong>4. Future-proof architecture</strong>: The same Parquet files can be used by:</p><ul><li>AWS Athena / Glue</li><li>Spark / Flink</li><li>Pandas/Polars (via Parquet readers)</li><li>Data warehouses that support external tables on S3.</li></ul><p>By streaming MQTT data into S3 as compact, analytics‑ready Parquet files, EMQX turns your broker into a powerful bridge between real‑time IoT workloads and modern data platforms. Paired with DuckDB, you get instant, SQL‑driven insight from raw device data without heavy infrastructure or complex pipelines. Whether you’re validating a new product feature, monitoring fleet health, or building a full IoT data lake, this lightweight architecture lets your team move faster, cut costs, and unlock more value from every message your devices send.</p><p><em>Originally published at</em> <a href="https://www.emqx.com/en/blog/from-mqtt-to-parquet"><em>www.emqx.com</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc364963da83" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering MQTT in C: A Step-by-Step Tutorial Using the Paho C Client]]></title>
            <link>https://emqx.medium.com/mastering-mqtt-in-c-a-step-by-step-tutorial-using-the-paho-c-client-461bf0a94db7?source=rss-2287ea9742a6------2</link>
            <guid isPermaLink="false">https://medium.com/p/461bf0a94db7</guid>
            <category><![CDATA[mqtt]]></category>
            <dc:creator><![CDATA[EMQ Technologies]]></dc:creator>
            <pubDate>Tue, 10 Mar 2026 10:01:31 GMT</pubDate>
            <atom:updated>2026-03-10T10:01:31.969Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction: Why C for MQTT and Why Paho?</h3><p>When building robust, high-performance Internet of Things (IoT) solutions, the choice of programming language often comes down to resource efficiency and execution speed. This is where <strong>C language</strong> and the <strong>MQTT C</strong> client shine.</p><p>C provides unparalleled control over system resources, making it the ideal choice for:</p><ul><li><strong>Embedded Systems:</strong> Microcontrollers and resource-constrained devices where memory and CPU cycles are limited.</li><li><strong>High Performance:</strong> Applications requiring the lowest possible latency and direct hardware interaction.</li></ul><p>The most widely adopted and authoritative library for C-based MQTT development is the <strong>Eclipse Paho MQTT C Client</strong> (eclipse-paho/paho.mqtt.c). Paho is the industry standard, providing stable, feature-rich implementations for various platforms.</p><p>This tutorial will guide you through setting up the Paho C Client and building a complete Pub/Sub application, connecting to a powerful <a href="https://www.emqx.com/en/blog/the-ultimate-guide-to-mqtt-broker-comparison">MQTT Broker</a>.</p><h3>Setting Up the Development Environment</h3><h3>Prerequisite: The MQTT Broker</h3><p>To successfully run any MQTT application, you need a highly available and reliable broker.</p><p>For this tutorial, we will use the free public <strong>EMQX Broker</strong>, known for its scalability, full <a href="https://www.emqx.com/en/blog/introduction-to-mqtt-5">MQTT 5.0</a> support, and performance under massive loads — the perfect backend for your high-performance C clients.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X-0KvzY1x1xXBqxaxzt-AQ.png" /></figure><blockquote><strong><em>Pro Tip:</em></strong><em> For production environments requiring security, high concurrency (up to 100 million connections), and enterprise features, consider using </em><strong><em>EMQX Cloud</em></strong><em> or </em><strong><em>EMQX Enterprise</em></strong><em>.</em></blockquote><h3>Installing the Paho C Library</h3><p>The Paho C Client is generally compiled from source. Here are the steps for a Unix-like system (Linux/macOS).</p><ol><li>Clone the Repository:</li></ol><pre>git clone https://github.com/eclipse-paho/paho.mqtt.c.git<br>cd paho.mqtt.c</pre><p>2. Compile and Install:</p><pre># Build the library<br>make<br># Install to system libraries<br>sudo make install</pre><p>This process installs the necessary header files (MQTTClient.h, etc.) and the library files for linking.</p><h3>Step-by-Step Guide: Building Your First MQTT C Client</h3><p>We will use the synchronous Paho API (MQTTClient.h) for simplicity. Create a file named paho_c_example.c.</p><h3>Full Example Code</h3><p>Here is the complete code. We will break down each function call below.</p><pre>#include &quot;MQTTClient.h&quot;<br>#include &lt;stdio.h&gt;<br>#include &lt;stdlib.h&gt;<br>#include &lt;string.h&gt;<br>​<br>// EMQX Connection Details<br>#define ADDRESS     &quot;tcp://broker.emqx.io:1883&quot;<br>#define CLIENTID    &quot;Paho_C_Tutorial_Client&quot;<br>#define TOPIC       &quot;paho/c/tutorial/test&quot;<br>#define PAYLOAD     &quot;Hello from the Paho C Client!&quot;<br>#define QOS         1<br>#define TIMEOUT     10000L // 10 seconds<br>​<br>// The callback function executed when a message is received<br>int messageArrived(void *context, char *topicName, int topicLen, MQTTClient_message *message) {<br>    printf(&quot;Message received on topic: %s\n&quot;, topicName);<br>    printf(&quot;Payload: %.*s\n&quot;, message-&gt;payloadlen, (char*)message-&gt;payload);<br>    MQTTClient_freeMessage(&amp;message);<br>    MQTTClient_free(topicName);<br>    return 1;<br>}<br>​<br>int main(int argc, char* argv[]) {<br>    MQTTClient client;<br>    MQTTClient_connectOptions conn_opts = MQTTClient_connectOptions_initializer;<br>    int rc;<br>​<br>    // --- Step 1: Create the Client ---<br>    MQTTClient_create(&amp;client, ADDRESS, CLIENTID,<br>        MQTTCLIENT_PERSISTENCE_NONE, NULL);<br>    <br>    // Set the message callback function<br>    MQTTClient_setCallbacks(client, NULL, NULL, messageArrived, NULL);<br>​<br>    // --- Step 2: Configure Connection Options ---<br>    conn_opts.keepAliveInterval = 20; // Send PINGREQ every 20 seconds<br>    conn_opts.cleansession = 1;       // Start a new session every time<br>​<br>    // --- Step 3: Connect to the Broker ---<br>    printf(&quot;Attempting to connect to EMQX Broker...\n&quot;);<br>    if ((rc = MQTTClient_connect(client, &amp;conn_opts)) != MQTTCLIENT_SUCCESS) {<br>        printf(&quot;Failed to connect to EMQX, return code %d\n&quot;, rc);<br>        return rc;<br>    }<br>    printf(&quot;Successfully connected to EMQX!\n&quot;);<br>​<br>    // --- Step 4: Subscribe to the Topic ---<br>    printf(&quot;Subscribing to topic \&quot;%s\&quot; with QoS %d\n&quot;, TOPIC, QOS);<br>    MQTTClient_subscribe(client, TOPIC, QOS);<br>    <br>    // --- Step 5: Publish a Message ---<br>    MQTTClient_message pubmsg = MQTTClient_message_initializer;<br>    pubmsg.payload = PAYLOAD;<br>    pubmsg.payloadlen = (int)strlen(PAYLOAD);<br>    pubmsg.qos = QOS;<br>    pubmsg.retained = 0;<br>    <br>    MQTTClient_deliveryToken token;<br>    printf(&quot;Publishing message: \&quot;%s\&quot;\n&quot;, PAYLOAD);<br>    <br>    // Publish and wait for delivery confirmation (Synchronous)<br>    MQTTClient_publishMessage(client, TOPIC, &amp;pubmsg, &amp;token);<br>    rc = MQTTClient_waitForCompletion(client, token, TIMEOUT);<br>    printf(&quot;Message with token value %d delivered.\n&quot;, token);<br>​<br>    // Keep the main thread alive to receive messages (Subscriber loop)<br>    printf(&quot;Client is listening for messages. Press Enter to disconnect...\n&quot;);<br>    getchar();<br>​<br>    // --- Step 6: Disconnect and Cleanup ---<br>    MQTTClient_disconnect(client, 10000);<br>    MQTTClient_destroy(&amp;client);<br>    return rc;<br>}</pre><h3>Breakdown and Explanation</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*omWn9ac32w_NpIRP_lgKDg.png" /></figure><h3>Compiling and Running the Code</h3><ol><li>Compile:</li></ol><pre>gcc paho_c_example.c -o mqtt_c_client -lpaho-mqtt3c</pre><p>2. Run:</p><pre>./mqtt_c_client</pre><p>You will see the client connect, publish, and then listen for any messages published to the same topic.</p><h3>Advanced MQTT C &amp; EMQX Integration</h3><p>The real power of the <strong>MQTT C</strong> client is realized when paired with a high-performance broker like EMQX.</p><h3>The Security Imperative: Implementing TLS/SSL</h3><p>In production, unencrypted connections are unacceptable. Paho provides secure versions of its libraries (paho-mqtt3cs and paho-mqtt3as).</p><ul><li><strong>Paho C Implementation:</strong> Use the MQTTClient_SSLOptions structure and update the connection address (e.g., ssl://broker.emqx.io:8883).</li><li><strong>EMQX Value:</strong> EMQX natively supports TLS/SSL on multiple ports and offers advanced security features like <strong>Client Certificate Authentication</strong> and integration with external security backends, providing the trust layer required for C clients deployed in the field.</li></ul><h3>High Availability with EMQX Cluster</h3><p>A single C client must maintain a stable connection, but network instability is inevitable in IoT environments.</p><ul><li><strong>The MQTT C Pain Point:</strong> The client connects to one broker instance at a time.</li><li><strong>EMQX Solution (Resilience):</strong> The <strong>EMQX Cluster</strong> architecture ensures high availability. If the C client’s connected node fails, its built-in reconnection logic will automatically attempt to connect to another available node in the EMQX cluster, ensuring minimal downtime and preserving session state if cleansession=0 was used.</li></ul><h3>FAQ</h3><h3>What is the difference between MQTTClient.h and MQTTAsync.h?</h3><ul><li>MQTTClient.h <strong>(Synchronous/Classic):</strong> Uses blocking calls (e.g., MQTTClient_connect() blocks until connected). It is simpler to use but less suitable for high-concurrency applications or single-threaded embedded systems where blocking is undesirable.</li><li>MQTTAsync.h <strong>(Asynchronous):</strong> Uses non-blocking calls and relies on callbacks for event notification (connection established, message received, etc.). This is generally preferred for high-performance applications where the client needs to manage multiple tasks concurrently.</li></ul><h3>How do I handle large numbers of concurrent MQTT C clients in production?</h3><p>To scale your MQTT C application, you need a robust broker. <strong>EMQX</strong> is specifically designed for this purpose, supporting millions of concurrent connections through its distributed cluster architecture. Key practices include:</p><ol><li><strong>Use unique Client IDs.</strong></li><li><strong>Enable TLS/SSL</strong> for all C clients.</li><li><strong>Optimize Keep-Alive:</strong> Set a reasonable keepAliveInterval (e.g., 60 seconds) to maintain connectivity without excessive overhead.</li><li><strong>Use QoS 1 or 2</strong> only when message delivery guarantee is critical; otherwise, stick to QoS 0 for maximum throughput.</li></ol><h3>Why is my Paho C client suddenly disconnecting from the broker?</h3><p>Common reasons for unexpected disconnections include:</p><ul><li><strong>Keep-Alive Timeout:</strong> If the client or broker fails to send a message or PINGREQ/PINGRESP within $1.5 \times$ the keepAliveInterval.</li><li><strong>Network Instability:</strong> Sudden loss of TCP connection (common in cellular or Wi-Fi environments).</li><li><strong>Broker Overload:</strong> While rare with EMQX, an overloaded broker may drop connections. <strong>EMQX</strong> provides detailed logs and monitoring metrics to identify resource bottlenecks quickly.</li><li><strong>Authorization Failure:</strong> The client ID, Username, or Password may have become invalid, causing the broker (like EMQX) to reject the connection immediately.</li></ul><h3>Can I use MQTT 5.0 features with the Paho C Client?</h3><p>Yes. The Paho C Client fully supports the MQTT 5.0 specification. To use it, you must configure the connection options with the appropriate MQTT version when creating the client (e.g., setting the version field in the appropriate initialization structure). Advanced EMQX features like <strong>Session Expiry</strong> and <strong>User Properties</strong> are fully accessible via the Paho C MQTT 5.0 API.</p><h3>Conclusion</h3><p>You have successfully navigated the complexities of C-based networking to build a fully functional <strong>MQTT C</strong> client using the robust <strong>Paho C Client</strong> library. This foundation is crucial for developing efficient, reliable, and resource-friendly applications for embedded and edge computing.</p><p>Ready to scale your solution beyond a single test connection? <strong>EMQX</strong> is the world’s most scalable MQTT Broker, designed to handle the millions of high-performance C clients your IoT project demands.</p><p><em>Originally published at</em> <a href="https://www.emqx.com/en/blog/mastering-mqtt-in-c"><em>www.emqx.com</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=461bf0a94db7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using curl for MQTT: Connect, Publish, and Subscribe with Secure IoT Communication]]></title>
            <link>https://emqx.medium.com/using-curl-for-mqtt-connect-publish-and-subscribe-with-secure-iot-communication-acb14f7fd037?source=rss-2287ea9742a6------2</link>
            <guid isPermaLink="false">https://medium.com/p/acb14f7fd037</guid>
            <dc:creator><![CDATA[EMQ Technologies]]></dc:creator>
            <pubDate>Tue, 10 Mar 2026 09:52:43 GMT</pubDate>
            <atom:updated>2026-03-10T09:52:43.861Z</atom:updated>
            <content:encoded><![CDATA[<h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#introduction">Introduction</a></h3><p>curl is the ubiquitous command-line tool for transferring data, and since 2020, it speaks MQTT. With curl ≥8.19.0 (shipping early March 2026), you can now also use <strong>MQTTS</strong> (MQTT over TLS) for secure IoT communication directly from your terminal. This means developers can test, script, and automate MQTT workflows without installing dedicated client libraries.</p><p>This guide shows you how to connect, publish, and subscribe to an <a href="https://www.emqx.com/en/products/emqx">MQTT Broker</a> using nothing but curl.</p><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#why-use-curl-for-mqtt">Why Use curl for MQTT?</a></h3><ul><li><strong>Zero Dependencies</strong>: No Python, Node.js, or dedicated MQTT libraries required, just curl.</li><li><strong>Universal Availability:</strong> Pre-installed on most Linux/macOS systems; easily added to Windows and containers.</li><li><strong>Scriptable:</strong> Easily integrate into shell scripts for prototyping and testing.</li><li><strong>TLS Support (8.19.0+):</strong> Secure connections via mqtts:// for production-grade testing.</li><li><strong>Familiar Syntax:</strong> If you know HTTP with curl, MQTT feels natural.</li></ul><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#prerequisites">Prerequisites</a></h3><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#curl-version-requirements">curl Version Requirements</a></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RYCyeSzJiOJY5-fe_xY6kw.png" /></figure><p>Check your installed version:</p><pre>curl --version</pre><p>Look for mqtt (and mqtts in 8.19.0+) in the <strong>Protocols</strong> line:</p><pre>curl 8.19.0-DEV (aarch64-apple-darwin25.2.0) libcurl/8.19.0-DEV OpenSSL/3.6.0 zlib/1.2.12 brotli/1.2.0 zstd/1.5.7 AppleIDN libssh2/1.11.1 nghttp2/1.68.0 ngtcp2/1.20.0 nghttp3/1.15.0 librtmp/2.3 mit-krb5/1.7-prerelease OpenLDAP/2.4.28/Apple<br>Release-Date: [unreleased]<br>Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt mqtts pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss<br>Features: alt-svc AppleSecTrust AsynchDNS brotli GSS-API HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd</pre><blockquote><strong><em>Note:</em></strong><em> If your version is older, upgrade via your package manager or download from </em><a href="https://curl.se/download.html"><em>curl.se/download</em></a><em>. On macOS, use </em><em>brew install curl for the latest version.</em></blockquote><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#mqtt-broker-setup">MQTT Broker Setup</a></h3><p>You need an MQTT broker to connect to. This guide uses EMQX, the world’s most scalable MQTT platform.</p><p><strong>Option 1: Free Public Broker (Testing)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VObwYuZgjwA5tgH508OkDQ.png" /></figure><p><strong>Option 2: EMQX Serverless (Production)</strong></p><p>For production workloads with authentication, dedicated resources, and enterprise features, try <a href="https://accounts.emqx.com/signup?continue=https://cloud-intl.emqx.com/console/">EMQX Serverless</a>.</p><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#understanding-curls-mqtt-url-scheme">Understanding curl’s MQTT URL Scheme</a></h3><p>curl uses a URL-based syntax for MQTT operations:</p><pre>mqtt[s]://[user:password@]broker[:port]/topic</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h4e_TYlodCaScyOxXkriFg.png" /></figure><p>The port is optional; curl uses the standard default ports (1883 for mqtt://, 8883 for mqtts://). You can override them if your broker uses non-standard ports.</p><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#curls-mqtt-output-format">curl’s MQTT Output Format</a></h3><p>When subscribing, curl outputs received MQTT messages in a binary format:</p><pre>[2 bytes: topic length (big-endian)] [topic string] [payload]</pre><p>For example, a message “hello” on topic “curl/test” appears as:</p><pre>00 09 c u r l / t e s t h e l l o<br>└─┬─┘ └───────┬───────┘ └───┬───┘<br>  │           │             │<br>  │           │             └── payload (no length prefix)<br>  │           └── topic: &quot;curl/test&quot; (9 bytes)<br>  └── topic length: 9 (0x0009)</pre><p>Raw output looks like this (topic and payload concatenated):</p><pre>curl/testhello</pre><p>See <a href="https://emqx.atlassian.net/wiki/spaces/~630c7c543778a7aadf18038b/blog/2026/01/23/2369093659/Using+curl+for+MQTT+Connect+Publish+and+Subscribe+with+Secure+IoT+Communication#parsing-mqtt-messages-for-human-readable-output">Parsing MQTT Messages</a> below for how to make this human-readable.</p><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#step-1-subscribe-to-an-mqtt-topic">Step 1: Subscribe to an MQTT Topic</a></h3><p>Subscribing with curl keeps the connection open and outputs incoming messages to stdout.</p><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#basic-subscription-unencrypted">Basic Subscription (Unencrypted)</a></h3><pre># Subscribe to the &quot;curl/test&quot; topic on the public EMQX broker<br>curl -N mqtt://broker.emqx.io/curl/test</pre><p>The -N flag disables output buffering, ensuring messages appear immediately as they arrive. Without it, curl buffers the output and you won&#39;t see anything until the buffer fills or the connection closes.</p><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#secure-subscription-with-mqtts-curl-8190">Secure Subscription with MQTTS (curl ≥8.19.0)</a></h3><pre># Subscribe over TLS<br>curl -N mqtts://broker.emqx.io/curl/test</pre><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#subscription-with-authentication">Subscription with Authentication</a></h3><p>When connecting to a broker that requires credentials (like EMQX Serverless):</p><pre># Replace with your actual credentials<br>curl -N -u &quot;your-username:your-password&quot; mqtts://your-broker.emqxsl.com/curl/test</pre><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#parsing-mqtt-messages-for-human-readable-output">Parsing MQTT Messages for Human-Readable Output</a></h3><p>To display messages in a readable format, pipe curl’s output through a parser.</p><p><strong>Bash one-liner:</strong></p><pre>curl -sN mqtt://broker.emqx.io/curl/test | \<br>  while IFS= read -r -d $&#39;\0&#39; d; \<br>    do \<br>      [ -n &quot;$d&quot; ] &amp;&amp; \<br>        l=$(printf &quot;%d&quot; &quot;&#39;${d:0:1}&quot;) &amp;&amp; \<br>        echo &quot;[${d:1:$l}] ${d:1+$l}&quot;; \<br>    done</pre><p>This splits on null bytes (the first byte of the 2-byte topic length for topics under 256 characters), extracts the topic length from the second byte, and prints [topic] payload for each message.</p><p><strong>Save to file and inspect:</strong></p><pre># Save raw output to file (Ctrl+C to stop)<br>curl -sN mqtt://broker.emqx.io/curl/test &gt; messages.bin</pre><pre># View with hexdump to see the structure<br>hexdump -C messages.bin</pre><p><strong>As a reusable shell function:</strong></p><pre>mqtt_subscribe() {<br>  curl -sN &quot;$1&quot; | while IFS= read -r -d $&#39;\0&#39; d; do<br>    [ -n &quot;$d&quot; ] &amp;&amp; l=$(printf &quot;%d&quot; &quot;&#39;${d:0:1}&quot;) &amp;&amp; echo &quot;[${d:1:$l}] ${d:1+$l}&quot;<br>  done<br>}</pre><pre># Usage<br>mqtt_subscribe &quot;mqtt://broker.emqx.io/curl/test&quot;</pre><blockquote><strong><em>Note:</em></strong><em> For production use or complex parsing needs, consider </em><a href="https://mqttx.app/cli"><em>MQTTX CLI</em></a><em> which provides properly formatted output and full MQTT 5.0 support.</em></blockquote><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#step-2-publish-a-message-to-an-mqtt-topic">Step 2: Publish a Message to an MQTT Topic</a></h3><p>To publish, use curl’s -d (data) flag with the message payload.</p><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#basic-publish-unencrypted">Basic Publish (Unencrypted)</a></h3><pre># Publish &quot;Hello from curl&quot; to the &quot;curl/test&quot; topic<br>curl -v -d &quot;Hello from curl&quot; mqtt://broker.emqx.io/curl/test</pre><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#secure-publish-with-mqtts-curl-8190">Secure Publish with MQTTS (curl ≥8.19.0)</a></h3><pre># Publish over TLS<br>curl -v -d &quot;Secure message from curl&quot; mqtts://broker.emqx.io/curl/test</pre><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#publishing-json-payloads">Publishing JSON Payloads</a></h3><p>For IoT sensor data, JSON is a common format:</p><pre># Publish a JSON temperature reading<br>curl -v -d &#39;{&quot;sensor_id&quot;: &quot;temp-001&quot;, &quot;value&quot;: 23.5, &quot;unit&quot;: &quot;celsius&quot;}&#39; \<br>  mqtt://broker.emqx.io/sensors/temperature</pre><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#publishing-with-authentication">Publishing with Authentication</a></h3><pre>curl -v -u &quot;your-username:your-password&quot; \<br>  -d &#39;{&quot;status&quot;: &quot;online&quot;, &quot;timestamp&quot;: 1706000000}&#39; \<br>  mqtts://your-broker.emqxsl.com/devices/status</pre><blockquote><strong><em>Tip:</em></strong><em> To test the full pub/sub flow, open two terminal windows side by side. Run a subscribe command (Step 1) in one terminal, then publish messages (Step 2) from the other terminal.</em></blockquote><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#relevant-curl-options">Relevant curl Options</a></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RQIG_y0Pn7I7sKs1FILEVg.png" /></figure><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#tls-certificate-verification-curl-8190">TLS Certificate Verification (curl ≥8.19.0)</a></h3><p>For production MQTTS connections with custom certificates:</p><pre># With CA certificate verification<br>curl --cacert /etc/ssl/certs/emqx-ca.crt \<br>  -d &quot;Verified secure message&quot; \<br>  mqtts://your-broker.emqxsl.com/secure/topic</pre><pre># With mutual TLS (mTLS) client authentication<br>curl --cacert /etc/ssl/certs/ca.crt \<br>  --cert /etc/ssl/certs/client.crt \<br>  --key /etc/ssl/private/client.key \<br>  -d &quot;mTLS authenticated message&quot; \<br>  mqtts://your-broker.emqxsl.com/secure/topic</pre><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#use-cases-when-to-use-curl-for-mqtt">Use Cases: When to Use curl for MQTT</a></h3><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#_1-broker-connectivity-testing">1. Broker Connectivity Testing</a></h3><p>Verify your MQTT broker is reachable (DNS resolution, TCP connection) using verbose mode:</p><pre>curl -v mqtt://broker.emqx.io/test</pre><p>Example output showing successful connection:</p><pre>curl -v mqtt://broker.emqx.io:1883/curl/test<br>* Host broker.emqx.io:1883 was resolved.<br>* IPv6: (none)<br>* IPv4: 34.243.217.54, 35.172.255.228, 44.232.241.40<br>*   Trying 34.243.217.54:1883...<br>* Connected to broker.emqx.io (34.243.217.54) port 1883<br>* Using client id &#39;curlblZBtS6c&#39;<br>&gt; MQTT&lt;<br>       curlblZBtS6cmqtt_doing: state [0]</pre><p>This verifies network connectivity and DNS resolution. Note that curl only supports QoS 0 (fire-and-forget), so a successful publish does not guarantee the message was received by subscribers.</p><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#_2-shell-scripting-for-iot-prototyping">2. Shell Scripting for IoT Prototyping</a></h3><p>Simulate a temperature sensor publishing readings every 5 seconds:</p><pre>#!/bin/bash<br># simulate-sensor.sh<br>BROKER=&quot;mqtt://broker.emqx.io&quot;<br>TOPIC=&quot;sensors/room1/temperature&quot;<br><br>while true; do<br>  # Generate random temperature between 20-30°C<br>  TEMP=$(awk -v min=20 -v max=30 &#39;BEGIN{srand(); print min+rand()*(max-min)}&#39;)<br>  PAYLOAD=&quot;{\&quot;temperature\&quot;: $TEMP, \&quot;timestamp\&quot;: $(date +%s)}&quot;<br>  <br>  curl -s -d &quot;$PAYLOAD&quot; &quot;$BROKER/$TOPIC&quot;<br>  echo &quot;Published: $PAYLOAD&quot;<br>  <br>  sleep 5<br>done</pre><p>This is useful for prototyping and testing subscriber applications. For production workloads requiring delivery guarantees, use a client that supports QoS 1 or 2.</p><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#limitations-of-curl-for-mqtt">Limitations of curl for MQTT</a></h3><p>While curl is excellent for quick tests and scripting, be aware of these limitations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ATfvqoY06ykZBLAUy5HiCw.png" /></figure><p>For advanced MQTT features, consider <a href="https://mqttx.app/cli">MQTTX CLI</a>, a purpose-built MQTT command-line tool from EMQ that supports MQTT 5.0, QoS levels, wildcards, and more.</p><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#verifying-curl-mqtt-support">Verifying curl MQTT Support</a></h3><p>If MQTT commands fail, verify your curl build includes MQTT protocol support:</p><pre># Check if mqtt is listed in protocols<br>curl --version | grep -i protocols<br>​<br># Test MQTT support explicitly<br>curl -V 2&gt;&amp;1 | grep -q mqtt &amp;&amp; echo &quot;MQTT supported&quot; || echo &quot;MQTT NOT supported&quot;</pre><p>If MQTT is missing, you may need to:</p><ol><li>Upgrade to a newer curl version</li><li>Install curl built with MQTT support (some minimal distributions exclude it)</li><li>Compile curl from source with --enable-mqtt</li></ol><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#conclusion">Conclusion</a></h3><p>curl’s MQTT support transforms the familiar command-line tool into a powerful IoT testing and scripting companion. With MQTTS arriving in curl 8.19.0, you can now securely interact with MQTT brokers using the same tool you already use for HTTP APIs.</p><p><strong>Key takeaways:</strong></p><ul><li>Use mqtt:// for testing and mqtts:// (curl ≥8.19.0) for secure connections</li><li>curl excels at connectivity testing, prototyping, and quick ad-hoc messaging</li><li>Remember: curl only supports QoS 0; for delivery guarantees, use <a href="https://mqttx.app/cli">MQTTX CLI</a> or client libraries</li></ul><p>Ready to scale beyond testing? <a href="https://www.emqx.com/en/products/emqx">EMQX</a> handles millions of concurrent MQTT connections with enterprise-grade security, the perfect backend for your curl-powered IoT workflows.</p><h3><a href="https://www.emqx.com/en/blog/using-curl-for-mqtt#related-resources">Related Resources</a></h3><ul><li><a href="https://www.emqx.com/en/blog/the-ultimate-guide-to-mqtt-broker-comparison">MQTT Broker: How It Works, Popular Options, and Quickstart</a></li><li><a href="https://mqttx.app/cli">MQTTX CLI: A Powerful Command Line MQTT Client</a></li><li><a href="https://www.emqx.com/en/blog/the-easiest-guide-to-getting-started-with-mqtt">Mastering MQTT: The Ultimate Beginner’s Guide for 2025</a></li><li><a href="https://www.emqx.com/en/blog/fortifying-mqtt-communication-security-with-ssl-tls">Securing MQTT Communication with TLS/SSL</a></li><li><a href="https://daniel.haxx.se/blog/2020/04/14/curl-mqtt-true/">curl MQTT Announcement (2020)</a></li><li><a href="https://daniel.haxx.se/blog/2026/01/19/now-with-mqtts/">curl MQTTS Announcement (2026)</a></li></ul><p><em>Originally published at</em> <a href="https://www.emqx.com/en/blog/using-curl-for-mqtt"><em>www.emqx.com</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=acb14f7fd037" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>