<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Christos Vasilakis on Medium]]></title>
        <description><![CDATA[Stories by Christos Vasilakis on Medium]]></description>
        <link>https://medium.com/@cvasilak?source=rss-d86a0cc0e70b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*RgOoY89poPWDUKnlX0w_cA@2x.jpeg</url>
            <title>Stories by Christos Vasilakis on Medium</title>
            <link>https://medium.com/@cvasilak?source=rss-d86a0cc0e70b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 19:00:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@cvasilak/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Device Management with Lightweight M2M (LwM2M) and Pelion Client]]></title>
            <link>https://cvasilak.medium.com/device-management-with-lightweight-m2m-lwm2m-and-pelion-client-8d2b2c151662?source=rss-d86a0cc0e70b------2</link>
            <guid isPermaLink="false">https://medium.com/p/8d2b2c151662</guid>
            <category><![CDATA[iot-platform]]></category>
            <category><![CDATA[m2m]]></category>
            <category><![CDATA[lwm2m]]></category>
            <category><![CDATA[iot]]></category>
            <category><![CDATA[device-management]]></category>
            <dc:creator><![CDATA[Christos Vasilakis]]></dc:creator>
            <pubDate>Thu, 09 Jul 2020 15:21:20 GMT</pubDate>
            <atom:updated>2020-07-09T15:21:20.625Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/369/1*8zeBADyTtru6XS6leBS70Q.png" /></figure><p>When trying to implement IoT solutions, developers and solution architects face the dilemma of which IoT protocol to choose. Many protocols are available (such as MQTT, CoAP, DDS, AMQP and so on), making the decision difficult, time-consuming and prone to misjudgement. Although each technology has its merits and a place in the IoT ecosystem, few can claim the label of a device management protocol that allows the efficient connection and management of millions of constrained IoT devices in the field. This blog post explores the Lightweight M2M protocol (LwM2M) that is used as the foundation protocol of the <a href="https://www.pelion.com/iot-device-management/">Pelion Device Management Service</a> of the <a href="https://www.pelion.com/">Pelion IoT Platform</a> and then walks through an example running the Pelion Client to demonstrate the LwM2M concepts “in action”. Then, this post describes the value-added services provided by the platform on top of the LwM2M foundation.</p><p>Let’s start first with the basics of device management.</p><h3>Five pillars of Device Management</h3><p>A device management protocol should provide the following:</p><ol><li><strong>Secure Bootstrapping</strong><br>The mechanisms to securely provision a device with the necessary PKI cryptographic elements (keys/certificates) to successfully connect to a device management server. Further, Access Controls Lists (ACLs) should limit the “allowed” interactions to predefined and strict areas.</li><li><strong>Remote Management</strong><br>The mechanisms to configure the device with newly updated settings to reflect a new desired state required by the business application or as a response to an external stimulus that prompts a change to the device. Further, the ability to trigger actuators on the device to perform actions.</li><li><strong>Firmware Update</strong><br>The mechanism to implement a secure and efficient firmware over the air update (FOTA) to update the application running on the device with new functionality or to fix vulnerabilities discovered after the device was shipped on the field.</li><li><strong>Fault Management</strong><br>The mechanism for the device to report errors.</li><li><strong>Reporting</strong><br>The mechanisms that allow the device to notify changes in its current status and allow it to be queried for specific parts of its configuration settings.</li></ol><p>The scope of a device management protocol is wide and encompasses all the real-world” issues IoT developers face when trying to efficiently and scalably manage their production devices in the field. <a href="https://www.omaspecworks.org/">OMA SpecWorks</a>, a standardization body formed by a number of companies (<a href="https://omaspecworks.org/membership/current-members/">including Arm</a>) started a combined effort in 2013 to create a next-generation IoT device management protocol specifically targeted for M2M communication, which will serve the needs for the explosion of connected devices.</p><blockquote><em>You may not know but </em><a href="https://www.omaspecworks.org/"><em>OMA SpecWorks</em></a><em> are the original creators of the </em><a href="https://en.wikipedia.org/wiki/OMA_Device_Management"><em>OMA-DM management protocol</em></a><em> that allowed mobile operators to efficiently manage mobile phones, PDAs and tablets. If you use a mobile phone, it’s likely that you are already using the work the standardization body has done.</em></blockquote><p>In February 2017, the <a href="http://www.openmobilealliance.org/release/LightweightM2M/V1_0-20170208-A/OMA-TS-LightweightM2M-V1_0-20170208-A.pdf">first version</a> of the specification was released, followed by three more revisions. A fourth is currently in the ‘candidate’ phase. Throughout the years, the specification has incorporated many technological advances and improved a number of areas as part of the feedback gathered by the increasing number of companies deploying LwM2M in production. At the moment, LwM2M stands as a mature device management protocol, ready to be deployed in the field.</p><p>Let’s have a high-level look at how LwM2M works starting with the “interfaces” concept.</p><blockquote><em>If the word specification scares you, fear not! The LightweightM2M specifications are easy-to-follow and well-written documents and if you have a basic technical savity you can easily follow along and understand the internals of how LwM2M protocol works if that interests you!</em></blockquote><h3>LwM2M walkthrough</h3><p>Architecturally, LwM2M is a client-server protocol. The IOT device plays the role of the client while the server is the Device Management server where devices register, making them available to be managed. The communication between the two is bidirectional; a device reports back information either as a response to a request the server initiates or on a predefined time interval.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/0*bt7-XEqsNA_IFbXI.png" /></figure><p>Underneath, LwM2M relies on the IETF standardized <a href="https://tools.ietf.org/html/rfc7252">CoAP protocol</a> to provide the semantics and structure of how the different parties should communicate. CoAP itself is a specialized protocol, where the efficiency of the communication (to allow it to run on lossy networks) and of the runtime (to allow it to run on low-power constrained devices) played a vital role in its development. You may think of CoAP as analogous to the HTTP protocol (in fact it is modelled very closely), which defines the structure of the communication, and LwM2M as the Web browser that builds on top of it to provide value-added services (or whatever the application is that is based on HTTP).</p><p>The diagrams below depict the protocol stack of LwM2M:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/0*fPZDTX2CPL5HHYii.png" /></figure><p>First are “Objects”, described in detail later, which are the managed properties the device exposes to the outside world. Because LwM2M is based on CoAP, it inherits its properties, such as UDP as the transport protocol and DTLS for securing the path. SMS may feel strange to you, but LwM2M was designed to operate on that transport, too. (The OMA-DM protocol, a precursor to LwM2M, might played a role here!). In more recent versions of LwM2M (v1.1), the protocol introduced support for TCP as the communication protocol (following the <a href="https://tools.ietf.org/html/rfc8323">advancements of CoAP</a>), as well as catering to the Low-Power Wide-Area Networks (LPWAN) that are starting to appear around us (such as NB-IOT, LoRA and so on).</p><blockquote><em>If you are in the market looking for solutions on how to best connect your devices using LPWAN technologies, </em><a href="https://www.pelion.com/iot-connectivity-management/"><em>Pelion Connectivity Management</em></a><em> can be a great starting point. We’ll love to hear from you and your specific use cases and problems you are facing!</em></blockquote><p>The specification defines “four interfaces” that match the device management concepts described previously:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/699/0*ruODmDFpvEV-kMTq.png" /></figure><h3>Object Model</h3><p>One important characteristic of LwM2M is that it opted to use a standardized format to describe the “Objects” a device supports based on the work done by the <a href="https://omaspecworks.org/about/the-oma-specworks-work-program/ipso-smart-objects-working-group/">IPSO Smart Objects</a> working group (now part of OMA SpecWorks). This was done deliberately to allow maximum interoperability between devices from different vendors. As long as vendors supported this common format, their devices can be easily plugged in and communicate with other devices that supported it.</p><blockquote><em>You can find a list of current registered IPSO Smart Objects </em><a href="http://www.openmobilealliance.org/wp/omna/lwm2m/lwm2mregistry.html"><em>here.</em></a></blockquote><p>Let’s see what an “Object” looks like when extracted from the <a href="http://www.openmobilealliance.org/wp/omna/lwm2m/lwm2mregistry.html">OMA SpecWorks Registry</a>. Below is a description of the “Temperature“ Object, used to report measurements of a temperature sensor:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/0*epf7GsohUduEu7rT.png" /></figure><p>Each “Object” is globally assigned an ID, and it can contain one or more “Resources” describing its properties (which themselves are assigned an ID).</p><p>Each “Resource” can:</p><ul><li>Be either readable/writable or executable (‘Operations’).</li><li>Be single or have multiple instances (‘Instances’).</li><li>Be mandatory or optional (‘Mandatory’).</li><li>Have a specific data type associated with it (‘Type’).</li></ul><p>During the ‘Registration’ phase with the LwM2M server, the device indicates the list of Objects it supports:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/440/0*vMkLme0shFiR3Clx.png" /></figure><p>You can use the IDs for Objects and Resources to construct a URI targeting a specific resource or Object you would like to access with the management interface.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/361/0*ewxk6SIE7pyk_VM3.png" /></figure><blockquote><em>Notice the similarities with web addressing. As we described earlier, LwM2M relies on COAP protocol, so it adheres the REST paradigm to access the properties of a device.</em></blockquote><p>The specification predefined eight standard Objects that every LwM2M implementation must adhere to, so the <a href="https://omaspecworks.org/about/the-oma-specworks-work-program/ipso-smart-objects-working-group/">IPSO Smart Objects</a> working group and external vendors can step in and provide additional Objects to suit their specific needs. Below is a list of those standard objects defined by the specification:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/0*N1kyKL3wBBRkCsCv.png" /></figure><h3>Pelion Client Example</h3><p>Now that this post has discussed the basic concepts behind LwM2M protocol, it will show through an example how those concepts translate in action using the <a href="https://www.pelion.com/">Pelion IoT Platform</a>. The official <a href="https://github.com/ARMmbed/mbed-cloud-client-example">reference example</a>, provided by the platform, connects to <a href="https://www.pelion.com/iot-device-management/">Pelion Device Management Service</a> and interacts with the device’s Objects and Resources. This tutorial uses the <a href="https://portal.mbedcloud.com/">Pelion Device Management Portal</a> to interact with the service, but you can also choose to use the <a href="https://www.pelion.com/docs/device-management/current/service-api-references/index.html">exposed API’s</a> provided, in case you need to develop custom integration points or web applications on top of the service.</p><blockquote><em>For an example integration with the </em><a href="https://www.pelion.com/iot-device-management/"><em>Device Management Service</em></a><em>, check the Arduino use case published in our </em><a href="https://blog.pelion.com/post/arduino-pelion"><em>blog.</em></a></blockquote><blockquote><em>For an example web application build on top of the </em><a href="https://www.pelion.com/iot-device-management/"><em>Device Management Service</em></a><em>, check our </em><a href="https://www.pelion.com/docs/device-management/current/integrate-web-app/building-and-deploying-a-web-application-using-the-connect-apis.html"><em>web application tutorial.</em></a></blockquote><p>This demonstration uses an <a href="https://os.mbed.com/platforms/ST-Discovery-L475E-IOT01A/">ST Discovery-L475E-IOT01A</a> board, but you can use the provided example to connect and manage any device, ranging from low-constrained devices (such as <a href="https://os.mbed.com/platforms/?q=&amp;Mbed+Enabled=Pelion+Device+Ready"><em>Mbed Enabled</em></a> or <a href="https://www.pelion.com/docs/device-management/current/connecting/nxp-mcuxpresso.html">Amazon FreeRTOS</a> compatible boards) to mid-range devices (such as <a href="https://www.pelion.com/docs/device-management/current/connecting/linux-on-raspberry-pi-3.html">Raspberry Pis</a> and generic <a href="https://www.pelion.com/docs/device-management/current/connecting/linux-on-pc.html">Linux boxes</a>). See our <a href="https://www.pelion.com/docs/device-management/current/connecting/device-management-client-tutorials.html">documentation</a> for instructions on how to get started on each platform.</p><p>Flash the example on the board. The device bootstraps and connects to Device Management service. Click on the Device Directory section in the Portal to see its status as <em>Connected</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7cxMP1hrHSZFf3fB.png" /></figure><p>Clicking on the device reveals a right pane with various device details and metadata, including the <em>Object and Resources</em> this device exposes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ioQ9KqCMNnuP8s1a.png" /></figure><p>Exercise the <em>‘Information Reporting’</em> interface of LwM2M to subscribe to a specific <em>Resource</em>. This notifies you when the value of that <em>Resource</em> changes on the device. The example code is set to periodically change the value of <a href="http://www.openmobilealliance.org/tech/profiles/lwm2m/3200.xml"><em>/3200/0/5501 — Digital Input Counter</em></a> every 5s.</p><blockquote><em>In LwM2M terms, this operation is called creating an </em>Observation<em> for an Object Instance or Resource.</em></blockquote><p>Click on the Resource. The service sends an LwM2M <em>Observation</em> request for that specific Resource on the device to start receiving the new values as they change. Notice in the console that the device echoes the initial <em>observation request</em> received from the server (<em>‘subscribed’</em>), the periodic emission, as well as the delivery of the notifications with the new values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lJqXOpSs9iztKTl_.gif" /></figure><p>It’s time to exercise the <em>‘Device Management &amp; Service Enablement interface’</em> of LwM2M. Although this tutorial uses a subset of the operations of this interface, mainly <em>Read/Write and Execute</em>, it should give you an idea of how this interface works.</p><p>a. Start with the Execute action. The example exposes a blinking action on resource (<a href="http://www.openmobilealliance.org/tech/profiles/lwm2m/3201.xml"><em>/3201/0/5850 — Digital Output On/Off</em></a>). When invoked, the LED of the device blinks with a specific pattern (which we’ll adjust later) and for a predefined interval.</p><blockquote><em>NOTE: If you browse the definition of the Object </em><a href="http://www.openmobilealliance.org/tech/profiles/lwm2m/3201.xml">/3201</a><em> in the </em><a href="http://www.openmobilealliance.org/wp/omna/lwm2m/lwm2mregistry.html"><em>Object Registry</em></a><em>, you’ll notice that the </em>/5850 — On/Off<em> Resource is missing. That is because this specific resource is called a </em>‘Reusable Resource’<em> and is defined outside an Object definition. Many Objects, particularly those targetting similar domain, may need similar resources so instead of replicating those in each Object definition, a </em>‘Reusable Resource’<em> allows itself to be embedded to multiple Objects.</em></blockquote><p>Click on the <em>/3201/0/5850</em> resource to invoke the blinking operation. The operation requires no-params, so click the Send button using empty params. When the device receives the request, it starts blinking the internal LED.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7VnIM5klFeXTf-9Qa3DZgg.gif" /></figure><p>Click on the <em>/3201/0/5853</em> Resource to read and write on it. This resource indicates the number of milliseconds the LED stays active on each blink. Lowering it makes the LED blink faster. When clicked, the Portal displays the current value of the Resource. Edit the value, and click ‘Send’. This causes the service to issue a write request on the device to update it. If you try to invoke the blinking operation again, you’ll notice the device blinks faster.</p><blockquote><em>NOTE: Similar to</em> /5850<em> we discussed previously, the </em>/5853<em> Resource is itself a</em> ‘Reusable Resource’</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Upr51narNER_vsAZ5mkrfQ.gif" /></figure><p>This tutorial demonstrated the most basic operations of LwM2M, mainly to <em>Read/Write</em> and <em>Execute</em> an action on a <em>Resource</em>. Please see our documentation for more detailed information regarding <a href="https://www.pelion.com/docs/device-management/current/resources/index.html">managing resources</a>.</p><h3>Building on top of LwM2M</h3><p>Having LwM2M as its foundation protocol allows the <a href="https://www.pelion.com/">Pelion IoT platform</a> to enjoy the benefits of a mature and open standard device management protocol developed in cooperation with many companies already deploying it in the production field. On top of that, Pelion IoT Platform introduces value-added services that solve issues our customers are facing when trying to deploy real-world IoT solutions.</p><p>In particular:</p><ul><li>Trusted Device onboarding and service provisioning of the device before it leaves the factory.</li><li>Device Grouping and filtering for flexible and efficient targetting of operations to a subset of devices.</li><li>A scalable update process able to handle firmware updates for millions of devices. This includes delta updates for cutting network transmission costs to the minimum required.</li><li>Operational Monitoring of the devices (e.g. memory use, CPU behaviour) to proactively detect incorrect behaviour (e.g after a software update).</li><li>Handle long-sleeping devices with <em>‘last-reported-state’</em> and <em>‘desired-state’</em> giving the ability of business applications to continue to operate even when the device is offline.</li><li>Handle efficiently different network types with variable transmission latency characteristics.</li><li><a href="https://www.pelion.com/docs/">and much more.</a></li></ul><p>Upcoming blogs posts will cover in detail the different value-added services provided by the platform and how each one helps solve real-world IoT deployment issues. In the meantime, we provide <a href="https://os.mbed.com/pelion-free-tier/">free-tier access</a> to <a href="https://www.pelion.com/">Pelion IoT platform</a>, so you can easily <a href="https://www.pelion.com/guides/connect-device-to-pelion/">get started</a>. Why not give it a try?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8d2b2c151662" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Delta Firmware Updates over Cellular LPWAN]]></title>
            <link>https://cvasilak.medium.com/delta-firmware-updates-over-cellular-lpwan-1297222bfab5?source=rss-d86a0cc0e70b------2</link>
            <guid isPermaLink="false">https://medium.com/p/1297222bfab5</guid>
            <category><![CDATA[device-management]]></category>
            <category><![CDATA[firmware-update]]></category>
            <category><![CDATA[iot-platform]]></category>
            <category><![CDATA[arm]]></category>
            <category><![CDATA[iot]]></category>
            <dc:creator><![CDATA[Christos Vasilakis]]></dc:creator>
            <pubDate>Wed, 22 Apr 2020 22:54:54 GMT</pubDate>
            <atom:updated>2020-04-22T22:54:54.486Z</atom:updated>
            <content:encoded><![CDATA[<p>Since the <a href="https://www.pelion.com/docs/device-management/current/release-notes/previous-versions.html">October release</a> of <a href="https://www.pelion.com/">Pelion Device Management Services</a>, the platform introduced support for updating a device firmware using a feature called <a href="https://www.pelion.com/docs/device-management/current/updating-firmware/firmware-images.html">delta updates</a>. For some time now, it was on my list of things to reserve time and play with it and write down my thoughts. This blog is the result of my experiment.</p><p>Delta updates are a critical feature when trying to update IoT devices in the field cause in the majority of situations if not all, the network bandwidth is limited and costly. Any mechanism that can be employed to reduce network usage has a huge effect on savings, both in terms of power usage of the device while receiving the firmware and staying awake and the associated bandwidth cost.</p><p>In essence, a delta update is a mechanism to push a delta (the binary difference) of the firmware currently running in the device with the newly updated one. Pelion employs the well-established and popular <a href="http://www.daemonology.net/bsdiff/">bsdiff</a> algorithm (with some Arm extensions) to generate the binary delta, resulting in considerable size savings as we’ll see later on.</p><blockquote><em>For the interested reader who wants to read more about bsdiff internals can have a look at the paper </em><a href="http://www.daemonology.net/papers/bsdiff.pdf"><em>‘Naive differences of executable code’</em></a><em> from the original author of bsdiff, </em><a href="https://twitter.com/cperciva?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor"><em>Colin Percival</em></a><em>.</em></blockquote><p>To demonstrate those size savings we will take the official <a href="https://github.com/ARMmbed/mbed-os-example-pelion">mbed-cloud-client example</a> that is used as our reference code to connect to Pelion cloud, produce delta firmware updates and exercise them under different networks types, Wi-Fi, Cat-M1 and NB-IoT. We will mark down the delta size reductions as well as the time spent to deliver those firmware’s to devices using across the different network types.</p><p>But first, let’s describe the steps required to produce a delta firmware image:</p><blockquote><em>NOTE: We showcase the delta firmware generation using our command-line tools to shed a light on the “behind-the-scenes” process. That said, our integrated development environment </em><a href="https://os.mbed.com/studio/"><em>Mbed Studio</em></a><em> provides </em><a href="https://os.mbed.com/docs/mbed-studio/current/device-management/index.html#update-workflow"><em>built-in support</em></a><em> for the generation and publishing of firmware with delta generation support currently in development (scheduled for the upcoming v1.0)</em></blockquote><h3>Producing the Delta image — In a nutshell</h3><p>In our documentation, you can find a <a href="https://www.pelion.com/docs/device-management/current/updating-firmware/updating-end-to-end-tutorials.html">step-by-step tutorial</a> on how to prepare and launch firmware (and delta) updates with explanations of the different concepts, but in summary, it involves the following:</p><blockquote><em>NOTE: The same steps to produce a delta firmware are applicable on any board supported by </em><a href="https://www.pelion.com/docs/device-management/current/connecting/tutorial-pelion-mbedos.html"><em>Pelion Device Management Client library</em></a><em> as long as is </em><a href="https://os.mbed.com/platforms/?q=&amp;Mbed+Enabled=Pelion+Device+Ready"><em>‘Pelion Device Ready’</em></a><em> certified and able to connect to </em><a href="https://www.pelion.com/docs/device-management/current/welcome/index.html"><em>Pelion Device Management Service</em></a><em> with some form of network connectivity.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/0*_KBeAtAiI-hPGDQn.png" /></figure><p>Let’s start by cloning the <a href="https://github.com/ARMmbed/mbed-os-example-pelion">mbed-os-example-pelion</a> repository:</p><pre>$ mbed import <a href="https://github.com/ARMmbed/mbed-os-example-pelion">https://github.com/ARMmbed/mbed-os-example-pelion</a><br>      <br> [mbed] Working path &quot;/Users/chrvas02/Projects&quot; (directory)<br> [mbed] Importing program &quot;mbed-os-example-pelion&quot; from &quot;https://github.com/ARMmbed/mbed-os-example-pelion&quot; at latest revision in the current branch<br> [mbed] Adding library &quot;mbed-cloud-client&quot; from &quot;https://github.com/ARMmbed/mbed-cloud-client&quot; at rev #7b583acf30ca<br> [mbed] Adding library &quot;mbed-os&quot; from &quot;https://github.com/ARMmbed/mbed-os&quot; at rev #e642a7d8b360<br> [mbed] Adding library &quot;drivers/COMPONENT_WIFI_ISM43362&quot; from &quot;https://github.com/ARMmbed/wifi-ism43362&quot; at rev #1978369b2310<br> [mbed] Adding library &quot;drivers/COMPONENT_WIFI_ESP32&quot; from &quot;https://github.com/ARMmbed/esp32-driver&quot; at rev #d6bd83f8710b<br> [mbed] Adding library &quot;drivers/storage/COMPONENT_NUSD&quot; from &quot;https://github.com/OpenNuvoton/NuMaker-mbed-SD-driver&quot; at rev #56406071a446<br> [mbed] Adding library &quot;TESTS/pelion-e2e-python-test-library&quot; from &quot;https://github.com/ARMmbed/pelion-e2e-python-test-library&quot; at rev #4a3a533f8b21</pre><p>Prior to building and flashing on the device, we need to create both a <a href="https://www.pelion.com/docs/device-management/current/updating-firmware/setting-up.html">device certificate</a> and an <a href="https://www.pelion.com/docs/device-management/current/updating-firmware/update-auth-cert.html">authentication certificate</a>. The former is used by the device to be able to connect to <a href="https://portal.mbedcloud.com/">Pelion Device Management Service</a> while the latter is used to ensure the updates come from a trusted source.</p><p>For development purposes, we can utilize the ‘mbed dm init’ to create them automatically for us, passing only our desired device metadata:</p><blockquote><em>NOTE: We are using developer certificates in our example but in a production setting you should really follow the guidelines as described in our </em><a href="https://www.pelion.com/docs/device-management-provision/1.2/introduction/index.html"><em>Factory Provisioning documentation</em></a><em>.</em></blockquote><pre>$ mbed dm init -a &quot;&lt;API_KEY&gt;&quot; -d &quot;letsmakecoffe.com&quot; --model-name &quot;awesome-coffe-machine&quot; -q --force<br>   <br>[mbed] Working path &quot;/Users/chrvas02/Projects/mbed-os-example-pelion&quot; (program)<br>[INFO] 2020-03-13 15:54:32 - __main__ - Found developer certificate named Dev-Certificate<br>[INFO] 2020-03-13 15:54:32 - __main__ - Writing developer certificate Dev-Certificate into c file mbed_cloud_dev_credentials.c<br>[WARNING]: Certificates generated with this tool are self-signed and for testing only<br>[WARNING]: This certificate is valid for 90 days. For production,use certificates with at least 10 years validity.<br>[INFO] 2020-03-13 15:54:32 - manifesttool.init - Certificate written to .update-certificates/default.der<br>[INFO] 2020-03-13 15:54:32 - manifesttool.init - Private key written to .update-certificates/default.key.pem<br>[INFO] 2020-03-13 15:54:32 - manifesttool.init - Default settings written to .manifest_tool.json<br>[INFO] 2020-03-13 15:54:32 - manifesttool.init - Wrote default resource values to update_default_resources.c</pre><p>The tool generated all the required developer certificates, the private key to sign our future updates and the <a href="https://www.pelion.com/docs/device-management/current/updating-firmware/firmware-manifests.html">manifest file</a> ‘.manifest_tool.json’ file which contains the metadata of our chosen configuration. This file will be used later in the tutorial when we create our delta image:</p><blockquote><em>NOTE: </em><a href="https://www.pelion.com/docs/device-management/current/updating-firmware/firmware-manifests.html"><em>Manifests</em></a><em> play a crucial role in a device firmware update. That said, in a future blog post we’ll describe the background behind the manifest and offer some tips and advice when generating them. Stay tuned!</em></blockquote><pre>$ cat .manifest_tool.json | jq</pre><pre>{<br>    &quot;classId&quot;: &quot;b5b0afb3-06e9-5bc6-81b6-a413ddaa4645&quot;,<br>    &quot;default-certificates&quot;: [<br>    {<br>       &quot;file&quot;: &quot;.update-certificates/default.der&quot;<br>    }<br>   ],<br>   &quot;deviceURNs&quot;: [],<br>   &quot;modelName&quot;: &quot;awesome-coffe-machine&quot;,<br>   &quot;private-key&quot;: &quot;.update-certificates/default.key.pem&quot;,<br>   &quot;vendorDomain&quot;: &quot;letsmakecoffe.com&quot;,<br>   &quot;vendorId&quot;: &quot;0e56af43-f31f-574c-aa45-ad54915ca9fe&quot;<br>  }</pre><p>We now need to set up network connectivity and configure accordingly the project’s configuration “mbed_app.json” file with the appropriate parameters.</p><h3>Cellular</h3><p>For Cellular connectivity, mandatory parameters are the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/493/0*VHEGjaUOEdDBolBS.png" /></figure><h3>Wi-Fi</h3><p>For Wi-Fi connectivity, wireless credentials must be entered:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/603/0*Y0AfBCxML4NWGtkT.png" /></figure><blockquote><em>NOTE: Depending on the type of board and the wireless/cellular module installed, a number of extra parameters might also be required and you should consult the documentation of your board for more information.</em></blockquote><p>Now we are ready to compile and flash the initial firmware on the device:</p><pre>$ mbed compile -m auto --flash</pre><pre>[mbed] Working path &quot;/Users/chrvas02/Projects/mbed-os-example-pelion&quot; (program)<br>[mbed] Detected &quot;DISCO_L475VG_IOT01A&quot; connected to &quot;/Volumes/DIS_L4IOT&quot; and using com port &quot;/dev/tty.usbmodem141203&quot;<br>Building project mbed-os-example-pelion (DISCO_L475VG_IOT01A, GCC_ARM)<br>Scan: mbed-os-example-pelion<br>Using ROM regions bootloader, header, application in this build.<br>  Region bootloader: size 0x9000, offset 0x8000000<br>  Region header: size 0x70, offset 0x8011000<br>  Region application: size 0xeec00, offset 0x8011400<br> Compile [  0.1%]: CertificateEnrollmentClient.cpp<br> Compile [  0.2%]: BufferedPrint.c<br> Compile [  0.3%]: CertificateEnrollmentClientCommon.cpp...<br> ...<br> ...<br> Update Image: ./BUILD/DISCO_L475VG_IOT01A/GCC_ARM/mbed-os-example-pelion_update.bin. [1]<br> Image: ./BUILD/DISCO_L475VG_IOT01A/GCC_ARM/mbed-os-example-pelion.bin.        [2]</pre><p>Notice at the end of the compilation process, the compiler produces two images:</p><ol><li>The update image [1], which contains only the application code and is used for updating the device.</li><li>The full image [2], which combines the application with the <a href="https://www.pelion.com/docs/device-management/current/updating-firmware/firmware-images.html">bootloader and metadata</a>. This image is initially flashed on the device.</li></ol><p>Let’s open a serial terminal to the board to verify that indeed the board has connected to <a href="https://portal.mbedcloud.com/">Pelion Device Management Service</a>:</p><blockquote><em>I am using </em><a href="https://freeware.the-meiers.org/"><em>CoolTerm app</em></a><em> on a Mac to access serial, if you are on some other platform take a look at our </em><a href="https://os.mbed.com/docs/mbed-os/latest/tutorials/serial-comm.html#using-terminal-applications"><em>documentation</em></a><em> for a list of terminal applications you can use.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/0*Mt3dXVcme6m_WzY0.gif" /></figure><p>And Pelion Portal should display the device as connected:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hgVFtRjqeIdoV4oB.png" /></figure><p>Let’s save this initial image in a ‘firmwares/’ directory on the project and give it a new name.</p><pre>$ mkdir firmwares/<br>$ cp ./BUILD/DISCO_L475VG_IOT01A/GCC_ARM/mbed-os-example-pelion_update.bin firmwares/mbed-os-example- <br>pelion_original.bin</pre><p>Modify the code to simulate an update of the running firmware and produce a new image. Open ‘main.cpp’ in the project’s root directory and add a ‘printf()’ statement:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/679/0*hxZ622phH6XacNVp.png" /></figure><p>And compile again:</p><pre>$ mbed compile -m auto</pre><blockquote><em>This time we don’t flash on the device (no ‘–flash’ argument passed) since we are only interested in the compiler artifacts.</em></blockquote><p>Let’s copy the resulted firmware to our ‘firmwares/’ directory and give it a new name:</p><pre>$ cp ./BUILD/DISCO_L475VG_IOT01A/GCC_ARM/mbed-os-example-pelion_update.bin firmwares/mbed-os-example-pelion_update.bin</pre><p>Now, we have two firmwares, the original firmware that was flashed on the device ‘mbed-os-example-pelion_original.bin’ and the new one ‘mbed-os-example-pelion_update.bin’ with our simulated change:</p><pre>$ ls -l firmwares/</pre><pre>total 1504<br>-rw-r--r--  1 chrvas02  staff  381408  25 Mar 13:59 mbed-os-example-pelion_original.bin<br>-rw-r--r--  1 chrvas02  staff  381432  25 Mar 14:05 mbed-os-example-pelion_update.bin</pre><p>We are now ready to create the delta firmware image using the <a href="https://github.com/ARMmbed/mbed-cloud-client-example/tree/master/delta-tool">delta-tool</a>.</p><p>To do so we need to pass both firmwares and the ‘manifest_tool.json’ configuration file generated by the ‘manifest-tool’ in our previous step. Further, we need to specify the filename of the configuration file that which will contain the metadata of the delta update (’-o delta-tool-generated-manifest.json’)</p><pre>$ python3 delta-tool/tools/delta-tool.py \<br>-b delta-tool/bsdiff/bsdiff \<br>firmwares/mbed-os-example-pelion_original.bin \<br>firmwares/mbed-os-example-pelion_update.bin \<br>-i .manifest_tool.json \<br>-d firmwares/delta_image.bin \<br>-o delta-tool-generated-manifest.json</pre><p>After executing the command, our delta firmware ‘delta_image.bin’ file is produced:</p><pre>$ ls -l firmwares/</pre><pre>total 1552<br>-rw-r--r--  1 chrvas02  staff   23984  25 Mar 14:08 delta_image.bin<br>-rw-r--r--  1 chrvas02  staff  381408  25 Mar 13:59 mbed-os-example-pelion_original.bin<br>-rw-r--r--  1 chrvas02  staff  381432  25 Mar 14:05 mbed-os-example-pelion_update.bin</pre><blockquote><em>Notice that the delta firmware is reduced to ~24K from the ~381K of the updated firmware, a considerable improvement!</em></blockquote><p>And the generated ‘delta-tool-generated-manifest.json’ with the delta update metadata:</p><pre>$ cat delta-tool-generated-manifest.json | jq</pre><pre>{<br>  &quot;classId&quot;: &quot;b5b0afb3-06e9-5bc6-81b6-a413ddaa4645&quot;,<br>  &quot;default-certificates&quot;: [<br>    {<br>      &quot;file&quot;: &quot;.update-certificates/default.der&quot;<br>    }<br>  ],<br>  &quot;deviceURNs&quot;: [],<br>  &quot;modelName&quot;: &quot;awesome-coffe-machine&quot;,<br>  &quot;private-key&quot;: &quot;.update-certificates/default.key.pem&quot;,<br>  &quot;vendorDomain&quot;: &quot;letsmakecoffe.com&quot;,<br>  &quot;vendorId&quot;: &quot;0e56af43-f31f-574c-aa45-ad54915ca9fe&quot;,<br>  &quot;payloadHash&quot;: &quot;aa147139f2d5dc37d84883771af356745471250420243f07e098790fcc008e6f&quot;,<br>  &quot;payloadSize&quot;: 23984,<br>  &quot;payloadFile&quot;: &quot;firmwares/delta_image.bin&quot;,<br>  &quot;payloadFormat&quot;: &quot;bsdiff-stream&quot;,<br>  &quot;installedFile&quot;: &quot;firmwares/mbed-os-example-pelion_update.bin&quot;,<br>  &quot;deltaFile&quot;: &quot;firmwares/delta_image.bin&quot;,<br>  &quot;precursorFile&quot;: &quot;firmwares/mbed-os-example-pelion_original.bin&quot;<br>}</pre><p>We are now ready to upload the update manifest and the delta firmware image to Pelion Device Management Service and prepare for the update campaign. For this, we utilize the ‘manifest-tool update prepare’ command to perform it for us:</p><pre>$ manifest-tool update prepare -a &lt;API_KEY&gt; -i delta-tool-generated-manifest.json</pre><pre>[INFO] 2020-03-25 14:13:45 - manifesttool.prepare - Using delta_image.bin-2020-03-25T14:13:45 as payload name.<br>[INFO] 2020-03-25 14:13:45 - manifesttool.prepare - Using delta_image.bin-2020-03-25T14:13:45-manifest as manifest name.<br>[INFO] 2020-03-25 14:13:46 - manifesttool.prepare - Created new firmware at http://firmware-catalog-media-ca57.s3.dualstack.us-east-1.amazonaws.com/delta_image_b76d958802a64f32a484e77207a79339.bin<br>[INFO] 2020-03-25 14:13:46 - manifesttool.prepare - Created new manifest at http://firmware-catalog-media-ca57.s3.dualstack.us-east-1.amazonaws.com/manifest_dd62b06f53e641dea190795a0ef5c0db<br>[INFO] 2020-03-25 14:13:46 - manifesttool.prepare - Manifest ID: 017135e055c70000000000010010017c</pre><p>Now that both the firmware and the manifest file are uploaded to Pelion Device Management Service, we are ready to start the campaign to update our device. Although it’s possible to use the command line to perform the operation using the <a href="https://www.pelion.com/docs/device-management/current/updating-firmware/update-api-tutorial.html">Pelion service APIs</a>, instead we will utilize the Pelion portal this time. Take a look at the video below that demonstrates creating and starting the update campaign:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FvEHhoX7B2O4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DvEHhoX7B2O4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FvEHhoX7B2O4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d873a859c4410bfd7b1dfb1a3ed0f9d7/href">https://medium.com/media/d873a859c4410bfd7b1dfb1a3ed0f9d7/href</a></iframe><h3>Analysis</h3><p>Now that we’ve described how delta firmware updates work and the size reduction advantage they bring when producing the image, let’s exercise them across wireless and cellular and mark down the time spent to deliver those updates. A quick search online will reveal plenty of material that describe the technical merits of each cellular technology and areas where it excels (or fails) in IoT uses cases, so we won’t spend time repeating the same information here. For the interesting reader though who would like to learn more, we recommend watching the <a href="https://learn.arm.com/webinar-nb-iot-the-future-iot-connectivity-on-demand.html">webinar comparing NB-IOT and Cat-M1</a> created by the <a href="https://www.arm.com/products/iot/pelion-iot-platform/connectivity-management">Pelion Connectivity team</a> here at Arm, that goes into full details on the differences between these two cellular technologies. Further, it includes a lengthy QA section of many common questions asked from our users and developers and is worth watching.</p><p>For reference, here is a summary table that highlights the differences:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/0*1KRxxeqaGVOFbXiJ.png" /></figure><p>The following tables are the result of our experiment running a firmware update across the different connectivity types. For each type, we record the time taken to update the firmware using both a full and a delta image (top and bottom respectively):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/0*4hGOdEYV0eJd10kL.png" /></figure><p>If we collect the results and attempt to create a graph, we get the following picture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/0*6msJdXMJOhfZlRJB.png" /></figure><p>Clearly and regardless of the chosen connectivity type, the impact that the delta update has on limiting the transmission time is significant, over ~90% in all cases. Although configurations may vary across different use cases and environments, employing delta firmware updates can prove a significant aid in the never-ending quest to save bandwidth costs and preserve the battery of the device.</p><p>You can find the sheet with the data in our <a href="https://drive.google.com/file/d/1bid-CvXzMI8E-5TXq5SLRtxaNL1VMv6O/view?usp=sharing">shared document </a>plus a description of the hardware and mobile operators used to conduct the testing as well as the logs from the devices.</p><h3>Conclusion</h3><p>Updating the firmware of the device is a critical feature of any IoT platform that ensures the device runs the latest code with any security fixes released after its deployment in the production field. Further, optimizing the delivery of the firmware using delta updates aids in saving network bandwidth costs and preserving the device battery to expand its lifespan. Keep in mind though, regardless of how far these optimizations can go, firmware updates remains an energy-heavy operation and as noted in our <a href="https://os.mbed.com/docs/mbed-os/latest/mbed-os-pelion/stage-4-pilot.html">“Going to Production guide”</a>:</p><p><em>“You should understand the energy requirements of both firmware update and your device’s normal operations, and plan update frequencies that don’t drastically reduce the time between battery changes. For this, remember that the accessibility of the battery is important — a device placed in plain sight (for example, mounted on a wall in a normal house) can have a shorter battery life than a device buried in the ground along a pipeline.”</em></p><p><a href="https://www.pelion.com/">Pelion Device Management Service</a> provides robust support for firmware and delta updates and already is used by many of our customers to implement their production update strategy. We provide <a href="https://os.mbed.com/pelion-free-tier">free-tier access</a> to the platform for our users and developers so they can easily get started so why not creating an account and exercise the delta update mechanism we discussed here.</p><p>Are you next?</p><h3>Acknowledgements</h3><p>I would like to sincerely thank my colleague <a href="https://www.linkedin.com/in/daniel-lee-7bb30a136/">Daniel Lee</a> for his valuable help during the conducting of tests and in the writing of this post.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1297222bfab5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pelion Device Management with Arduino IoT cloud Integration]]></title>
            <link>https://cvasilak.medium.com/pelion-device-management-with-arduino-iot-cloud-integration-1b03d342d8d4?source=rss-d86a0cc0e70b------2</link>
            <guid isPermaLink="false">https://medium.com/p/1b03d342d8d4</guid>
            <category><![CDATA[arm]]></category>
            <category><![CDATA[arduino]]></category>
            <category><![CDATA[iot]]></category>
            <dc:creator><![CDATA[Christos Vasilakis]]></dc:creator>
            <pubDate>Mon, 23 Mar 2020 15:05:07 GMT</pubDate>
            <atom:updated>2020-03-30T11:48:36.206Z</atom:updated>
            <content:encoded><![CDATA[<h3>Pelion Device Management with Arduino IoT Cloud Integration</h3><p>One of the best features of <a href="https://www.pelion.com/iot-device-management/">Pelion Device Management Service</a> for IoT developers is the ability to develop deep integrations using our <a href="https://www.pelion.com/docs/device-management/current/service-api-references/service-api-documentation.html">service APIs</a>. Every company has different needs and requirements when they’re developing complex IoT solutions in the field. Having an extensive API that permits full platform access lets you customize your integration for scenarios we couldn’t have even imagined when we started developing it. You can use the APIs to create custom scripts to batch onboard devices, perform firmware updates, gather monitoring statistics, and <a href="https://www.pelion.com/docs/device-management/current/service-api-references/service-api-documentation.html">much more</a>. You can even build your own Device Management Portal on top of the service!</p><p>Developer teams from many companies already use our APIs to develop integrations with Pelion into their systems. We listen to their feedback, prioritize our roadmaps, and strive to best serve their needs.</p><p>When <a href="https://www.arduino.cc/">Arduino</a> approached us about developing an extension for their <a href="https://create.arduino.cc/">Arduino IoT Cloud</a>, integrating Pelion-managed devices with their own portal, we were delighted:</p><ul><li>Million of IoT developers trust Arduino, from hobbyists all the way up to professional users prototyping pre-production ideas. Offering a production-grade device management platform within their IoT cloud system lets you transition smoothly to a production setting.</li><li>The chance to collect feedback on our Device Management Service from a large pool of Arduino IoT developers, which guides our continuous improvements to the platform.</li><li>And last but not least, this partnership gives us a tremendous opportunity to bring awareness of our Pelion Device Management service to IoT developers who aren’t familiar with our strong offering.</li></ul><p>In this blog post, we describe our integration with <a href="https://create.arduino.cc/">Arduino IoT Cloud</a>. We’ll connect a device running <a href="https://www.mbed.com/en/">Mbed OS</a> with the <a href="https://www.pelion.com/docs/device-management/current/connecting/device-management-client-tutorials.html">Device management Client library</a> installed, to <a href="https://www.pelion.com/docs/device-management/current/introduction/index.html">Pelion Device Management Service</a>. We’ll then switch to Arduino IoT Cloud and browse available features. We assume you’re already familiar with Pelion Device Management Service. If not, please have a look at the introductory material in the Pelion <a href="https://www.pelion.com/docs/device-management/current/introduction/index.html">documentation</a>. For more information on Arduino IoT Cloud, read the <a href="https://create.arduino.cc/projecthub/133030/iot-cloud-getting-started-c93255">getting started guide</a> on Projecthub.</p><blockquote><em>We provide </em><a href="https://os.mbed.com/pelion-free-tier"><em>free access</em></a><em> to Pelion Device Management, so you can easily get started with the platform, offering device management and </em><a href="https://www.pelion.com/docs/device-management/current/updating-firmware/index.html"><em>firmware updates (plus delta updates)</em></a><em> for up to 100 devices.</em></blockquote><h3>Mapping Pelion Device Management to Arduino IoT Cloud</h3><p>When a device connects to Device Management, it registers a list of <a href="http://www.openmobilealliance.org/wp/OMNA/LwM2M/LwM2MRegistry.html">LwM2M Resources</a> that map to the device characteristics the developer wants to expose and make available to be managed. Below is a screenshot of Device Management Portal, showing a list of resources supported by the example device:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_9sntoW97yEN2Czu.png" /></figure><p>In Arduino IoT Cloud, these resources map directly to device <a href="https://www.arduino.cc/en/IoT/HomePage">Properties</a>, and the data types and permissions of each property are filled automatically by the metadata of the LwM2M resource in Pelion:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/605/0*sw2F3HA-F3ZxcNhK.png" /></figure><p>Notice that the Update mechanism of all Properties is set to On change type by default. During the initial device import, <a href="https://www.pelion.com/docs/device-management/current/device-management/resource-change-webapp.html">PreSubscriptions on observable resources</a> in Device Management are created and notify Arduino IoT Cloud whenever a resource changes.</p><p>This lets Arduino create a dashboard, which is continuously updated when a value on the device changes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*Nmm9TTdS7ffsFuTn.gif" /></figure><p>The integration also supports <a href="https://www.pelion.com/docs/device-management/current/device-management/handle-resource-webapp.html">sending requests</a> through the Device Management APIs to the device in order to change the value of a resource or to execute an action. In the following demo, we have a device that exposes two resources, the Blinking patternresource that sets how the LED on the device blinks and the Blink action to execute it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*xTRfS3vepsORI8Xw.gif" /></figure><h3>Demo</h3><p>If you want to see the integration in action, have a look at the video below:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FDKs5NlkPiY4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDKs5NlkPiY4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FDKs5NlkPiY4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/7bd84e51abf88f785e7e210ac68d27e8/href">https://medium.com/media/7bd84e51abf88f785e7e210ac68d27e8/href</a></iframe><h3>Conclusion</h3><p><a href="https://www.pelion.com/iot-device-management/">Pelion Device Management</a> offers an extensive API that allows companies to develop deep integrations with a production-grade and scalable device management platform. <a href="https://create.arduino.cc/">Arduino IoT Cloud</a> integration is a great example of what‘s possible. The extensibility of the platform has enabled many of our customers to address their specific needs and use cases. Are you next?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1b03d342d8d4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A prototype of utilizing Apache Kafka and Lightweight M2M protocol as the backbone for cloud/edge…]]></title>
            <link>https://cvasilak.medium.com/a-prototype-of-utilizing-apache-kafka-and-lightweight-m2m-protocol-as-the-backbone-for-cloud-edge-bf6d9c8f3b89?source=rss-d86a0cc0e70b------2</link>
            <guid isPermaLink="false">https://medium.com/p/bf6d9c8f3b89</guid>
            <category><![CDATA[kafka]]></category>
            <category><![CDATA[lwm2m]]></category>
            <category><![CDATA[lightweightm2m]]></category>
            <category><![CDATA[iot]]></category>
            <category><![CDATA[apache-kafka]]></category>
            <dc:creator><![CDATA[Christos Vasilakis]]></dc:creator>
            <pubDate>Fri, 19 Oct 2018 11:55:10 GMT</pubDate>
            <atom:updated>2018-10-19T11:55:10.849Z</atom:updated>
            <content:encoded><![CDATA[<h3>A prototype of utilizing Apache Kafka and Lightweight M2M protocol as the backbone for cloud/edge IoT integration</h3><p>In this post I am going to describe a sample architecture that utilizes <a href="https://kafka.apache.org/">Apache Kafka</a> and <a href="https://www.omaspecworks.org/what-is-oma-specworks/iot/lightweight-m2m-lwm2m/">Lightweight M2M</a> as the backbone for cloud/edge IoT integration. I have started working on in as part of my MSc thesis but since completing it, have spend more time to further refine, experiment, improve and learn in- between.</p><p>The following diagram depicts an overview of the architecture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ysve5cagXUiriy_5W5wQ9w.png" /></figure><p>The diagram emphasizes two important points. In particular:</p><ul><li><strong>The usage of Apache Kafka broker and it’s ecosystem of frameworks, both in the cloud as well as in the edge.</strong><br>Edge hardware is becoming increasingly powerful (and cheap). Depending on your requirements and trade offs you are willing to accept, a running Kafka broker can be made to work even on hardware with limited resources. Running a Kafka broker at the edge and on the cloud can provide us with several advantages as we will explain later in this post.</li><li><strong>The usage of the <em>standardized</em> </strong><a href="http://www.openmobilealliance.org/release/LightweightM2M/V1_0_2-20180209-A/OMA-TS-LightweightM2M-V1_0_2-20180209-A.pdf"><strong>Lightweight M2M protocol (LWM2M)</strong></a><strong> (with the </strong><a href="https://www.eclipse.org/leshan/"><strong>Eclipse Leshan</strong></a><strong> implementation) at the edge.<br></strong>LWM2M protocol provides extensive support for connecting sensors, reading and writing attributes and executing operations, all in an efficient, bandwidth friendly and secure manner. Further, it is currently well supported on two popular embedded operating systems, <a href="https://docs.zephyrproject.org/latest/samples/net/lwm2m_client/README.html">Zephyr</a> and <a href="https://github.com/contiki-ng/contiki-ng/wiki/Tutorial:-LWM2M-and-IPSO-Objects">Contiki-ng</a>, make it really easy for developers to start experimenting on real-hardware.</li></ul><h3>Mode of Operation</h3><p>The basic operation is simple. At the edge hardware we run both a <a href="https://github.com/zeelos/leshan-server-kafka">Leshan server</a> where sensors connect to, as well as a local Kafka broker. The Leshan Server has been modified to convert LWM2M protocol messages received by sensor devices to <a href="https://github.com/zeelos/leshan-avro-schema">Avro encoding schema</a> and then route them to the local Kafka broker for storage and processing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4mv41lNLr2xcYQr2wHB9Yg.png" /><figcaption><a href="https://github.com/zeelos/cockpit-leshan">Leshan plugin</a> for <a href="https://cockpit-project.org/">Cockpit</a> to monitor and administer the gateway and the LWM2M sensors attached to it</figcaption></figure><p>The messages are sent to a set of predefined topics that convey to LWM2M interfaces:</p><ul><li><strong>Client Registration<br> </strong>iot.&lt;gateway-name&gt;<strong><em>.</em></strong><em>registrations</em></li><li><strong>Device management and service enablement<br></strong>iot.&lt;gateway-name&gt;<strong><em>.</em></strong><em>management.req</em><br>iot.&lt;gateway-name&gt;<strong><em>.</em></strong><em>management.rep</em></li><li><strong>Information Reporting<br></strong>iot.&lt;gateway-name&gt;<em>.observations</em></li></ul><p>A series of deployed <a href="https://github.com/zeelos/kafka-streams-leshan">Kafka Stream analytics</a> running at the edge then consume those messages, doing further processing and outputting the results to the local Kafka broker.</p><p>At the same time, topics from the local Kafka broker are replicated to the cloud Kafka broker (using <a href="https://docs.confluent.io/current/multi-dc-replicator/mirrormaker.html">Apache MirrorMaker</a>) and be made available for other services to consume. In particular, a set of <a href="https://docs.confluent.io/current/connect/index.html">Kafka Connect</a> connectors sink the time-series data to an <a href="https://www.influxdata.com/time-series-platform/influxdb/">InfluxDB</a> database whereas for visualizing the Leshan LWM2M model, another connector sinks the data to an <a href="https://orientdb.com/">OrientDB</a> graph database.</p><blockquote>NOTE: Since data are flowing into Kafka <a href="https://www.confluent.io/hub/">any other connector</a> can be used. For example, you can use the <a href="https://docs.confluent.io/current/connect/kafka-connect-elasticsearch/index.html">Elastic Search connector</a> to store the time series or/and the LWM2M model data to <a href="https://www.elastic.co/products/elasticsearch">Elasticsearch</a>.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-Sd5yot2ZPIv01wVBxcMBg.png" /><figcaption>Grafana visualizing LWM2M sensor time-series data stored in InfluxDB</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F-2yPeAWUMSOMphOefabEQ.png" /><figcaption>OrientDB visualizing a LWM2M gateway and sensors connected to it with their supported resources</figcaption></figure><h3>Advantages of utilizing Apache Kafka</h3><p>Running Apache Kafka at the edge and on the cloud provides us with the following advantages:</p><ul><li><strong>Flexibility in the storage and processing of data.<br></strong>It’s ecosystem of frameworks provides the flexibility required to support many scenarios both in type of storage and processing of the data. For example, depending on corporate investments, we can use Kafka Connect connectors to store asset and time-series data to different types of databases (InfluxDB, Elastic, Cassandra etc) without limiting the users to one specific technology. For processing, either Kafka Streams, KSQL or Apache Spark can be used.</li><li><strong>Disconnected and offline operation of edge gateways.<br></strong>Gateways can remain operational independently of the connection to the cloud, thus saving costs. All the messages coming from sensors are stored in the Kafka broker locally and in-order. When the connection to the cloud resumes, we can selectively replicate the data to the cloud with configurable Kafka supported compression schema’s (Snappy, Gzip), thus saving bandwidth costs. Further, many of the Kafka components are designed to survive temporary network failures with an excessive list of knobs to configure depending on the requirements.</li><li><strong>Scalability and Fault-tolerance.<br></strong>Depending on our requirements, we can scale the cluster to support a large number of gateways and sensor devices.</li></ul><h3>Kafka Livin’ on the Edge</h3><p>Initially and in order to save resources, we deployed an MQTT Mosquitto server at the edge, where messages coming from Leshan where routed at. Then, a Kafka Connect MQTT connector running in the cloud, connected to the MQTT broker, picked up those messages and stored them in the cloud broker. That is, the connector only acted as a form of replication of the messages from the MQTT server to cloud Kafka and back.</p><p>Although this worked, we thought it will be advantageous if we had a Kafka broker running in the edge. Mainly because:</p><ul><li>Kafka already comes with <a href="https://www.confluent.io/white-paper/disaster-recovery-for-multi-datacenter-apache-kafka-deployments/">support of replication</a> in the form of <a href="https://docs.confluent.io/current/multi-dc-replicator/mirrormaker.html">Apache MirrorMaker</a> or Confluent’s own <a href="https://docs.confluent.io/current/multi-dc-replicator/index.html">Replicator</a>.</li><li>Eliminating another dependency to a database to store the messages since they are stored to Kafka itself (single source of truth).</li><li>Ability to deploy <a href="https://github.com/zeelos/kafka-streams-leshan">Kafka Stream analytics</a> at the edge for real-time processing of incoming messages.</li><li>A running Kafka broker at the edge allows us to develop new event-driven micro-services to support new requirements.</li></ul><blockquote>Confluent in its recent release, provides an <a href="https://docs.confluent.io/current/kafka-mqtt/kafka-mqtt/docs/index.html">MQTT proxy service</a> that routes messages directly to a Kafka broker, located in the cloud or (possible) in the edge. Depending on your requirements, this would work really well and we can clearly see use cases that this will be useful, i.e sensors using MQTT sending messages to the proxy and stored in Kafka. In our case, we use LWM2M protocol sending messages directly to Kafka instead of MQTT thus we didn’t have a requirement to support it.</blockquote><p>Since we had an <a href="https://www.asus.com/us/Single-Board-Computer/Tinker-Board/">Asus Tinkerboard</a> and a <a href="https://www.pine64.org/?page_id=7147">Rock64</a> single board computers at hand, we thought it will be interested to evaluate whether Kafka can run on this type of hardware (Arm) and if it is, what were the limitations. Since we were using docker, we first had to produce <strong>Arm based</strong> docker images of Confluent’s Kafka distribution (see <a href="https://github.com/zeelos/cp-docker-images/tree/5.0.0-post-arm32v7">arm32v7</a> and <a href="https://github.com/zeelos/cp-docker-images/tree/5.0.0-post-arm64v8">arm64v8</a> branches) since they are not officially provided. Besides Arm hardware, we have also used an x86 based <a href="https://www.aaeon.com/en/p/iot-gateway-ups-gws01">AAEON UP Squared board</a>, a fan-less industrial IoT gateway (with 8G of memory). For the x86 board specifically, we further produced <a href="https://www.eclipse.org/openj9/">OpenJ9</a> based images of Kafka (see <a href="https://github.com/zeelos/cp-docker-images/tree/5.0.0-post-openj9">openj9</a> branch), which further cut’s down memory requirements on that platform.</p><p>After producing the docker images, it was simply writing the proper scripts to bootstrap the services. We have written a detailed setup guide in the <a href="https://github.com/zeelos/zeelos">project’s Github page</a>, where we describe in more details the various services.</p><p>From our observations running Kafka at the edge, we can assess that is very low on usage of the CPU even with high read/write load, but it does use the available memory of the system (mainly for pagecache) so having hardware with enough memory will help. In our case, the hardware we used had 2G, 4G and 8G wired memory and it was capable enough to support our requirements. As an indication, the Asus Tinkerboard (2G) was able to manage one hundred sensors (100) which were continuously sending observations every second to Leshan and then routed to Kafka, while simultaneously running <a href="https://github.com/zeelos/kafka-streams-leshan">two Kafka Streams analytics jobs</a> processing those messages. Further, all incoming observations where replicated to the cloud Kafka broker by MirrorMaker. Again, it all depends on your requirements and trade offs that you are will willing to accept, but it sure does demonstrate the amount of hardware power that currently exists (and what is coming..). That said, I am wondering <a href="http://www.embedded-computing.com/news-releases/logic-supply-now-shipping-fanless-nuc-with-8th-generation-intel-kaby-lake-core-i7-cpu">what this recently released</a> <a href="http://www.embedded-computing.com/news-releases/logic-supply-now-shipping-fanless-nuc-with-8th-generation-intel-kaby-lake-core-i7-cpu">industrial IoT gateway with 32G of memory and SSD storage</a> is capable of ? ;)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5y1xG5ZyFGte4h-thkyYxQ.jpeg" /><figcaption>An Asus Tinkerboard (left) and an UP Squared board (right) both running Kafka and synchronizing sensor data received by Zephyr and Contiki-ng LWM2M sensors to the cloud Kafka broker.</figcaption></figure><h3>Advantages of utilizing Lightweight M2M (LWM2M)</h3><p>As stated earlier, Lightweight M2M (LWM2M) protocol provides support for connecting and managing sensors at the field. The advantages of LWM2M can be summarized to:</p><ul><li>At it’s core, the protocol has been designed to support embedded hardware with limited resources and limited bandwidth connectivity. The fact that is well supported and actively developed by the two of the most popular embedded operating systems, <a href="https://docs.zephyrproject.org/latest/samples/net/lwm2m_client/README.html">Zephyr</a> and <a href="https://github.com/contiki-ng/contiki-ng/wiki/Tutorial:-LWM2M-and-IPSO-Objects">Contiki-ng </a>reinforces this fact.</li><li>Designed by telecom providers to support real-world scenarios in managing million of devices at the field, so it’s usage and limitations are well understood and documented.</li><li>The specification is an <em>open standard</em><strong><em> </em></strong>and <em>actively developed</em> incorporating newest developments such as support for LORA and NB-IoT.</li><li>Availability of robust implementations for a variety of languages which makes it easy for the developer to get started.</li><li>The protocol provides enough flexibility in it’s design that allows routing of many existing protocols such as <a href="https://en.wikipedia.org/wiki/Modbus">Modbus</a> or <a href="https://en.wikipedia.org/wiki/OPC_Unified_Architecture">OPC UA.</a> As a matter of fact, we created a <a href="https://github.com/zeelos/leshan-client-modbus">Modbus adapter </a>that showcase this functionality (with an OPC-UA adapter currently in the works). Supporting and routing existing protocols over LWM2M is an important feature and the Open Mobile Alliance has standardize the process in the <a href="http://www.openmobilealliance.org/release/LightweightM2M/V1_1-20180710-A/OMA-RD-LightweightM2M-V1_1-20180710-A.pdf">latest version (v1.1) of the specification</a> (check the <em>‘LwM2M Gateway functionality’ </em>section)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rghfI0nEmsGNrPXncYtEaw.png" /><figcaption>Modbus sensor being routed over LWM2M</figcaption></figure><h3>Conclusion</h3><p>In this post we described a sample architecture where a Kafka broker together with it’s ecosystem of frameworks, is used at the edge and on the cloud. Further Lightweight M2M is used to connect sensors at the edge and route their messages to Kafka. Our main motivation was to simplify developer and user cognitive effort on the usage of the platform and provide enough flexibility for future expansion and usage. We firmly believe that Kafka and Lightweight M2M provide an excellent ground to implement various IoT scenarios.</p><p>Please have a look at the <a href="https://github.com/zeelos/zeelos">project github page</a> and I be happy to hear your thoughts and suggestions. Further, I will be attending <a href="https://www.eclipsecon.org/europe2018">EclipseCon Community and IoT day next week</a>, so if you are going to be around I will be happy to meet and chat all things IoT!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bf6d9c8f3b89" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>