<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jonathan Seow on Medium]]></title>
        <description><![CDATA[Stories by Jonathan Seow on Medium]]></description>
        <link>https://medium.com/@jonathanseow?source=rss-cb7169f0fc76------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*EJUb8qckuPOjFort</url>
            <title>Stories by Jonathan Seow on Medium</title>
            <link>https://medium.com/@jonathanseow?source=rss-cb7169f0fc76------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 23:34:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jonathanseow/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Optimize CPU Usage with Pipelining and Worker Pool]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://jonathanseow.medium.com/optimize-cpu-usage-with-pipelining-and-worker-pool-cd70620df213?source=rss-cb7169f0fc76------2"><img src="https://cdn-images-1.medium.com/max/2600/1*rfhDnxcLtZCiM7vwzwt0yw.jpeg" width="5472"></a></p><p class="medium-feed-snippet">Today, I want to tackle a straightforward interview question with my readers. I will discuss various ideas relevant to this question and&#x2026;</p><p class="medium-feed-link"><a href="https://jonathanseow.medium.com/optimize-cpu-usage-with-pipelining-and-worker-pool-cd70620df213?source=rss-cb7169f0fc76------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://jonathanseow.medium.com/optimize-cpu-usage-with-pipelining-and-worker-pool-cd70620df213?source=rss-cb7169f0fc76------2</link>
            <guid isPermaLink="false">https://medium.com/p/cd70620df213</guid>
            <category><![CDATA[system-design-concepts]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[software-interview]]></category>
            <dc:creator><![CDATA[Jonathan Seow]]></dc:creator>
            <pubDate>Sun, 18 Aug 2024 05:29:44 GMT</pubDate>
            <atom:updated>2024-08-18T05:29:44.039Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Lifecycle of an Elasticsearch Index Request]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/geekculture/lifecycle-of-an-elasticsearch-index-request-6538c5ca50d3?source=rss-cb7169f0fc76------2"><img src="https://cdn-images-1.medium.com/max/2600/1*UuRayOHJN1AeWRaZsyjN8A.jpeg" width="5184"></a></p><p class="medium-feed-snippet">What happens when you index a document in Elasticsearch?</p><p class="medium-feed-link"><a href="https://medium.com/geekculture/lifecycle-of-an-elasticsearch-index-request-6538c5ca50d3?source=rss-cb7169f0fc76------2">Continue reading on Geek Culture »</a></p></div>]]></description>
            <link>https://medium.com/geekculture/lifecycle-of-an-elasticsearch-index-request-6538c5ca50d3?source=rss-cb7169f0fc76------2</link>
            <guid isPermaLink="false">https://medium.com/p/6538c5ca50d3</guid>
            <category><![CDATA[elasticsearch]]></category>
            <category><![CDATA[software-architecture]]></category>
            <dc:creator><![CDATA[Jonathan Seow]]></dc:creator>
            <pubDate>Tue, 31 Jan 2023 07:02:31 GMT</pubDate>
            <atom:updated>2023-01-31T07:02:31.194Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[An Overview of MySQL Replication]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/codex/an-overview-of-mysql-replication-9e5635cc3a8a?source=rss-cb7169f0fc76------2"><img src="https://cdn-images-1.medium.com/max/2600/1*zXUAcpnHq6irXB5t3_fNAw.jpeg" width="3840"></a></p><p class="medium-feed-snippet">Replication is copying data from a source database to one or many replica databases. This is commonly known as master-slave replication.</p><p class="medium-feed-link"><a href="https://medium.com/codex/an-overview-of-mysql-replication-9e5635cc3a8a?source=rss-cb7169f0fc76------2">Continue reading on CodeX »</a></p></div>]]></description>
            <link>https://medium.com/codex/an-overview-of-mysql-replication-9e5635cc3a8a?source=rss-cb7169f0fc76------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e5635cc3a8a</guid>
            <category><![CDATA[software-design]]></category>
            <category><![CDATA[replication]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[mysql]]></category>
            <dc:creator><![CDATA[Jonathan Seow]]></dc:creator>
            <pubDate>Sun, 22 Jan 2023 10:51:02 GMT</pubDate>
            <atom:updated>2023-01-22T10:51:02.790Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[4 Practices to Plan a Large-scale Data Migration]]></title>
            <link>https://medium.com/better-programming/4-practices-to-plan-a-large-scale-data-migration-8edd6c921aa5?source=rss-cb7169f0fc76------2</link>
            <guid isPermaLink="false">https://medium.com/p/8edd6c921aa5</guid>
            <category><![CDATA[data-migration]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[design-systems]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[big-data]]></category>
            <dc:creator><![CDATA[Jonathan Seow]]></dc:creator>
            <pubDate>Mon, 07 Mar 2022 18:26:07 GMT</pubDate>
            <atom:updated>2022-10-03T23:46:04.586Z</atom:updated>
            <content:encoded><![CDATA[<h4>What I learned from migrating 25 billion records</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vFP7LMm9D1nYu-Dz7sPc3Q.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@cgbriggs19?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Chris Briggs</a> on <a href="https://unsplash.com/s/photos/migration?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Data migration is simply the process of moving data from one location to another.</p><p>As businesses grow, data migration is almost unavoidable. You may need larger storage to support your data growth. Or, you need to change data format due to evolving requirements.</p><p>I have recently completed a data migration with Elasticsearch. There were a total of <em>25 billion documents</em>, and the entire process took almost <em>14 days</em>!</p><p>As easy as it sounds, data migration is no easy feat. There are many caveats to a smooth migration without system downtime.</p><p>In this article, I will share with you the knowledge I have gained after completing a large-scale data migration. They are practices that you can follow regardless of your system architecture. Let’s get started! 🏃</p><h3>The Read-Write Flag</h3><p>There are two problems you need to consider before performing data migration in a continuous, live system.</p><p>One, how do you ensure data in the new location stays updated? At this point, your APIs are still writing to the old storage. Any updates to data that have been migrated will be missed in the new storage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*--G-wNehpNdTautWj866jA.png" /><figcaption>Migrate Row 1</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ySq6J6wIoCMWjUaeNxoDAw.png" /><figcaption>Row 1 in new storage not updated</figcaption></figure><p>Two, how do you smoothly migrate your APIs to use the new storage? Similarly, if anything bad happens, you need to roll back your APIs back to old storage. A smooth transition <strong>at any time</strong> between old and new storage is the key to migration without downtime.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/1*5PfyYObnvJkABkz68YERyA.png" /><figcaption>Transition without downtime</figcaption></figure><p>To solve these problems, we will introduce a <strong>read-write flag</strong>. A flag is a switch that turns certain functionalities on or off during runtime, without any code change.</p><p>Our flag has 4 values, each representing a specific way the APIs interact with data storage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/1*Brt85r9k8u7B7xUydwpljA.png" /><figcaption>The four stages of migration</figcaption></figure><p>At the start, the APIs interact only with the old storage, which is the default behavior.</p><p>When you toggle the flag to 1, the APIs read data from the old storage, but write to <strong>both storages</strong>. Writing data to both locations must begin before data migration starts. This ensures the new storage stays updated and the old storage remains functional.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*08xwD8BzwhSbNorps6luQA.png" /><figcaption>“double write”</figcaption></figure><p>Once the migration has ended, the APIs can read data from the new storage. If there are unexpected errors, you can immediately switch the flag back to 1. Thanks to “double write”, the old storage is still usable.</p><p>Finally, when all is good, you can toggle your APIs to use the new storage entirely. This marks the end of the migration.</p><p>Notice how the data flow is <strong>dynamically</strong> controlled by one single switch. Throughout the process, no code change or deployment is required. All the related logic can be pre-prepared before the migration starts.</p><p>At the code level, we can easily perform conditional checks with the read-write flag. The pseudocode looks something like below.</p><pre>// 4 values of the flag</pre><pre>oldOnly = 0<br>writeBoth = 1<br>readNew = 2<br>newOnly = 3</pre><pre>// helper functions for condition checks</pre><pre>function isReadNew(flag) {<br>    return flag == readNew || flag == newOnly<br>}</pre><pre>func isWriteOld(flag) {<br>    return flag != newOnly<br>}</pre><pre>func isWriteNew(flag) {<br>    return flag != oldOnly<br>}</pre><pre>// read logic</pre><pre>if isReadNew(flag) {<br>    read_new()<br>} else {<br>    read_old()<br>}</pre><pre>// write logic</pre><pre>if isWriteOld(flag) {<br>    write_old()<br>}<br>if isWriteNew(flag) {<br>    write_new()<br>}</pre><h3>Resource Planning</h3><p>One major problem with “double write” is that it essentially <strong>doubles</strong> the usual write traffic to your data storage system.</p><p>Depending on your system, it may not be able to hold this extra pressure. The migration might add latencies to your normal API traffic. In the worst case, it can completely disrupt the system, which is bad news for your client!</p><p>This is where you need to do your <strong>due diligence</strong> in understanding the capacity of your system. One way of doing so is to perform <a href="https://en.wikipedia.org/wiki/Stress_testing_(software)">stress testing</a> to estimate the maximum write traffic your data system can support.</p><p>Generally, stress testing involves spinning up multiple background threads and continuously “spam” concurrent requests to your system. You can imagine each thread as one unforgiving client.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P8vGsVwka3ofDVoJN498GA.png" /><figcaption>Stress testing</figcaption></figure><p>I won’t go into the details of stress testing as it deserves a separate article. I have tried stress testing with a tool called <a href="https://github.com/wg/wrk">wrk</a> and with a Go script I have written myself.</p><p>After you are confident with the stress test results, you can start asking yourself some questions.</p><p>Should your new data storage be in a separate physical machine? Can the old and new data storage be in the same cluster? Can they be in the same database instance?</p><p>These questions should be answered along with a deep understanding of the data storage you are using. Different storage systems react differently to high write traffic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-4hKXOKBIl_0CUtunNI19g.png" /><figcaption>Examples of data storage system</figcaption></figure><p>Understanding the impact a data migration can have on your system is important in preparing for the unexpected!</p><h3>Consistency Check</h3><p>Even with the read-write flag and delicate resource planning, errors can still happen! For example, data inserts may timeout due to network issues.</p><blockquote>“Anything that can go wrong will go wrong.” — Murphy’s Law</blockquote><p>Before you switch the read-write flag to 2 (read new), it is recommended to perform a <strong>consistency check</strong>.</p><p>As the name suggests, a consistency check compares data between the old and new storage. It treats the old storage as the source of truth. If there are inconsistencies, the check performs a data fix.</p><p>If your data size is huge, the consistency check should be done with an automated script. Depending on your tolerance for errors, you can perform either a full or sampled consistency check.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h4LW3V1YKcbh_BFYcQVYFQ.png" /><figcaption>Consistency check</figcaption></figure><p>The check script should also report the percentage of inconsistencies found. If they are above your expected threshold, you might want to investigate the root cause before switching the APIs to read new.</p><p>Meanwhile, the data fix can be done synchronously or asynchronously.</p><p>If synchronous, the check script will apply fixes immediately after inconsistencies are discovered. Else, the script can push the fix into a queue for a worker to apply it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1005/1*uu1EazcKA4boJJXN-oOpBA.png" /><figcaption>Synchronous fix</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p95vRmlODLgDc-lsMsG-Hw.png" /><figcaption>Asynchronous fix</figcaption></figure><p>Does the fix method matter though? Well, it depends… Pause here if you wish to think about it 😏</p><p>Let’s say your API is writing data into the new storage synchronously. But, you chose the asynchronous fix in a consistency check.</p><p>A queue can experience momentary delays. By the time the worker picks up a “fix”, the storage could already contain newer updates from the APIs. The worker will then overwrite the more updated data with the outdated “fix”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*M90gj8JNFLs1EPWLyhq1lQ.gif" /><figcaption>Delays in an asynchronous fix</figcaption></figure><p>Generally, a synchronous fix is enough as it is straightforward.</p><p>What I want to emphasize here is that data migration is complicated when data starts flowing in all places. You must understand how your data behaves, or else you will be struck by weird inconsistencies.</p><h3>The Importance of Logging</h3><p>If you have been programming for a while, you will agree that system logs are developers’ best friends. In data migration, I want to highlight two types of logs that will make your life ten times easier.</p><h4>Error Logs</h4><p>Remember, errors can happen at any time, and you need to be prepared for them. When an error occurs, your migration script should log it out for debugging purposes.</p><p>Moreover, instead of relying on a consistency check to fix the errors, I highly advise you to fix the errors <em>as they happen</em>.</p><p>If migration takes 14 days, a full data check will also take another 14 days. This may be infeasible if your project is in a rush for completion.</p><p>This is where error logs are extremely helpful. You can perform a fix manually by checking the logs for data that are not migrated correctly. If you are fancy enough, you can even use a script to parse the logs and do the fix!</p><p>With this practice, the consistency check will be just an “insurance”. A sampled consistency check at the end will be faster and more than enough.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qXwk6AB0OGA9MqBhcC3iww.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@visuals?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">visuals</a> on <a href="https://unsplash.com/s/photos/error-logs?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h4>Progress Logs</h4><p>Migrating a large amount of data is an excruciating process. It is best to know when it will end so that you can plan your celebration!</p><p>Jokes aside, knowing the progress of your migration tells you how much extra write traffic you have introduced to your data system. Plus, you need to be answerable to your stakeholders and tech lead on the progress.</p><p>What I did was I have a background thread that logs out the amount of data that has been migrated every minute. Since I know the total amount of data, I can estimate how much time the entire process needs.</p><p>In short, you must think carefully about the information you require to understand the migration process. Log them out as much as possible. Once you start the migration, it will be too late to regret the logs you missed out on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yctDEp8rYkyl5zqmiWBXAA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@mike_van_den_bos?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Mike van den Bos</a> on <a href="https://unsplash.com/s/photos/loading?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h3>Final Thoughts</h3><p>In conclusion, I have covered four practices that can help you better design and plan a migration project. You can (and should) adjust these practices according to your needs.</p><p>Some of these practices are overkill if the data size is small. When dealing with a large amount of data, however, they can be extremely helpful.</p><p>There is so much more information I wish to share, but this article is getting too long. Feel free to leave comments so that we can exchange ideas and discuss more. Until then, good luck with your data migration!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8edd6c921aa5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/better-programming/4-practices-to-plan-a-large-scale-data-migration-8edd6c921aa5">4 Practices to Plan a Large-scale Data Migration</a> was originally published in <a href="https://betterprogramming.pub/">Better Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Two Tips to Better Handle Go Concurrency]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/better-programming/two-tips-to-better-handle-go-concurrency-d369b86a32f4?source=rss-cb7169f0fc76------2"><img src="https://cdn-images-1.medium.com/max/2600/1*qdT44yZ1dGFJ13ocV4AqqQ.jpeg" width="4513"></a></p><p class="medium-feed-snippet">Here are two mistakes I made with goroutines, and how you can avoid them</p><p class="medium-feed-link"><a href="https://medium.com/better-programming/two-tips-to-better-handle-go-concurrency-d369b86a32f4?source=rss-cb7169f0fc76------2">Continue reading on Better Programming »</a></p></div>]]></description>
            <link>https://medium.com/better-programming/two-tips-to-better-handle-go-concurrency-d369b86a32f4?source=rss-cb7169f0fc76------2</link>
            <guid isPermaLink="false">https://medium.com/p/d369b86a32f4</guid>
            <category><![CDATA[golang-development]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[golang-tutorial]]></category>
            <category><![CDATA[goroutines]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Jonathan Seow]]></dc:creator>
            <pubDate>Mon, 28 Feb 2022 17:43:48 GMT</pubDate>
            <atom:updated>2023-01-22T16:06:25.787Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How I Reindex Elasticsearch Without Downtime]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/better-programming/how-i-reindex-elasticsearch-without-downtime-6e8a6a512070?source=rss-cb7169f0fc76------2"><img src="https://cdn-images-1.medium.com/max/2600/1*AD6zBUpnf2W221eut7Go4Q.jpeg" width="3999"></a></p><p class="medium-feed-link"><a href="https://medium.com/better-programming/how-i-reindex-elasticsearch-without-downtime-6e8a6a512070?source=rss-cb7169f0fc76------2">Continue reading on Better Programming »</a></p></div>]]></description>
            <link>https://medium.com/better-programming/how-i-reindex-elasticsearch-without-downtime-6e8a6a512070?source=rss-cb7169f0fc76------2</link>
            <guid isPermaLink="false">https://medium.com/p/6e8a6a512070</guid>
            <category><![CDATA[backend-development]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Jonathan Seow]]></dc:creator>
            <pubDate>Mon, 07 Feb 2022 16:34:47 GMT</pubDate>
            <atom:updated>2022-02-26T02:53:28.618Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Build a TCP Connection Pool From Scratch With Go]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/better-programming/build-a-tcp-connection-pool-from-scratch-with-go-d7747023fe14?source=rss-cb7169f0fc76------2"><img src="https://cdn-images-1.medium.com/max/2600/1*vQ4M5Rrt34VlA50qfq_NQQ.jpeg" width="4898"></a></p><p class="medium-feed-snippet">Here are some basic ideas on how connections are handled between scalable software systems</p><p class="medium-feed-link"><a href="https://medium.com/better-programming/build-a-tcp-connection-pool-from-scratch-with-go-d7747023fe14?source=rss-cb7169f0fc76------2">Continue reading on Better Programming »</a></p></div>]]></description>
            <link>https://medium.com/better-programming/build-a-tcp-connection-pool-from-scratch-with-go-d7747023fe14?source=rss-cb7169f0fc76------2</link>
            <guid isPermaLink="false">https://medium.com/p/d7747023fe14</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Jonathan Seow]]></dc:creator>
            <pubDate>Mon, 03 Jan 2022 15:06:04 GMT</pubDate>
            <atom:updated>2022-01-03T15:06:04.124Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Deliver JSON Data from Go APIs]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/geekculture/deliver-json-data-from-go-apis-f0f3e433e18e?source=rss-cb7169f0fc76------2"><img src="https://cdn-images-1.medium.com/max/2600/1*uuhsaEPY4zSfd9Uvmr2Lrg.jpeg" width="4979"></a></p><p class="medium-feed-snippet">Learn to encode data into JSON with Marshal and Encoder in Go</p><p class="medium-feed-link"><a href="https://medium.com/geekculture/deliver-json-data-from-go-apis-f0f3e433e18e?source=rss-cb7169f0fc76------2">Continue reading on Geek Culture »</a></p></div>]]></description>
            <link>https://medium.com/geekculture/deliver-json-data-from-go-apis-f0f3e433e18e?source=rss-cb7169f0fc76------2</link>
            <guid isPermaLink="false">https://medium.com/p/f0f3e433e18e</guid>
            <category><![CDATA[api]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[programming-languages]]></category>
            <category><![CDATA[go]]></category>
            <dc:creator><![CDATA[Jonathan Seow]]></dc:creator>
            <pubDate>Wed, 26 May 2021 10:42:05 GMT</pubDate>
            <atom:updated>2021-05-26T10:42:05.684Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Learn Go Middlewares by Examples]]></title>
            <link>https://medium.com/geekculture/learn-go-middlewares-by-examples-da5dc4a3b9aa?source=rss-cb7169f0fc76------2</link>
            <guid isPermaLink="false">https://medium.com/p/da5dc4a3b9aa</guid>
            <category><![CDATA[go]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[programming-languages]]></category>
            <category><![CDATA[backend-development]]></category>
            <dc:creator><![CDATA[Jonathan Seow]]></dc:creator>
            <pubDate>Mon, 10 May 2021 11:02:34 GMT</pubDate>
            <atom:updated>2022-02-26T15:35:05.570Z</atom:updated>
            <content:encoded><![CDATA[<h4>Centralize and reuse common functionalities with middlewares</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*feMTbNGkDO37R9c1kLHMTA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@ecasap?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Elaine Casap</a> on Unsplash</figcaption></figure><p>Middlewares are one of the most important concepts in backend engineering. They are standalone, reusable pieces of software that link different systems together.</p><p>In web development, it is common to place one or many middlewares between the client and server. This essentially creates <em>a bridge between data and user interfaces</em>.</p><p>Each middleware acts independently on an HTTP request or response. The output of one middleware can be the input of another middleware. This forms <em>a chain of middlewares</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/879/1*3U2QnpCYn5o7vgv_CzO4tQ.png" /><figcaption>Two middlewares between client and server</figcaption></figure><p>Why middlewares? In web applications, middlewares allow us to <strong>centralize and reuse<em> </em></strong>common functionalities on each request or response. For example, you can have a middleware that logs every HTTP request.</p><p>To design Go middlewares, there are certain rules and patterns that we have to follow. This article will teach you the concepts of a Go middleware and create two of them for a simple application!</p><p>I assume you have Go installed and are familiar with Go’s interfaces, structs, and the <a href="https://golang.org/pkg/net/http/">http</a> package.</p><p>Let’s get started! 🏃</p><h3>Getting Started</h3><p>Let us first create a directory to house our mini Go project. Open your terminal, navigate to your favorite location and run the command below. This creates our root directory named go-middleware .</p><pre>$ mkdir go-middleware</pre><p>Then, change into the directory and create a main.go file.</p><pre>$ cd go-middleware<br>$ touch main.go</pre><p>Time to start coding! Open your code editor and let’s add some boilerplate code to kickstart a basic Go HTTP server.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5f07fd898315237e8483f2e6aa73d8f7/href">https://medium.com/media/5f07fd898315237e8483f2e6aa73d8f7/href</a></iframe><p>I have declared a new <a href="https://golang.org/pkg/net/http/#ServeMux">ServeMux</a> (a router in Go’s terminology) with the http.NewServeMux() function. Then, I registered the home function as a <strong>handler </strong>for any HTTP requests to the /home route. Finally, I started an HTTP server at <a href="http://localhost:4000/">localhost:4000</a> with the custom router.</p><p>Once you are done, at the terminal, run go run main.go and visit <a href="http://localhost:4000/home">localhost:4000/home</a> in your favorite browser. You should see a response as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xi4-6pKPopsaS5Qv0iq6HQ.png" /><figcaption>A simple response at /home</figcaption></figure><p>Perfect! As a side note, you might notice some Go tutorials never declare a custom ServeMux. Instead, they use Go’s <strong>DefaultServeMux</strong> as follows.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b02116c112dcfb922614b8b6b69d01c9/href">https://medium.com/media/b02116c112dcfb922614b8b6b69d01c9/href</a></iframe><p>Notice that the home function is registered directly with http.HandleFunc . Behind the scenes, Go creates a DefaultServeMux object in the http package as the default router and adds the handler to it.</p><p>The code above works, and it looks simpler. But, it is <em>not a good practice</em> because DefaultServeMux is a global variable in the http package. Any third-party packages can add malicious handlers to it, which introduces <strong>security risks</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L37uP5pmLXzt0ggPA-m-yw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@art_maltsev?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Artem Maltsev</a> on Unsplash</figcaption></figure><p>Anyway, I digress. Let’s move on.</p><h3>Handler and ServeHTTP</h3><p>Before we start creating Go middleware, it is good for us to know some theories. In particular, w<em>hat exactly is a </em><strong><em>handler</em></strong><em> in Go?</em></p><p>In Go, a handler is just an object (struct) that satisfies the <a href="https://golang.org/pkg/net/http/#Handler">http.Handler</a> interface.</p><pre>type Handler interface {<br>    ServeHTTP(ResponseWriter, *Request)<br>}</pre><p>When we register the home function as a handler using <a href="https://golang.org/pkg/net/http/#ServeMux.HandleFunc">http.HandleFunc</a> , Go transforms our function into a handler with a ServeHTTP method. This ServeHTTP method simply <em>calls our original </em><em>home function</em>.</p><p>To demonstrate, you can imagine Go doing something as follows.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/050f5d877b035100996a98f576542aac/href">https://medium.com/media/050f5d877b035100996a98f576542aac/href</a></iframe><p>Meanwhile, a ServeMux object (be it default or custom) is also a http.Handler . It executes its ServeHTTP method when it receives an HTTP request, which matches the URL request path with the appropriate handler.</p><p>If there is a match, ServeMux proceeds to call the ServeHTTP method of the handler and passes the request to it. The handler then executes its route handling logic and returns a response.</p><p>In a way, you can think of Go’s application routing as <em>a chain of handlers with </em><em>ServeHTTP methods called one after another.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_sG8p2ON5vLFUJOm0BmxrA.png" /><figcaption>A chain of ServeHTTP calls</figcaption></figure><p>Hence, to fit into the chain, a Go middleware must behave like a handler. It performs some logic before passing a request to the <strong>next handler<em> </em></strong>by calling the handler’s ServeHTTP method!</p><h3>The Pattern of Go Middleware</h3><p>By using the ideas presented in the section above, we can finally showcase what a Go middleware looks like. The standard pattern of a Go middleware is as follows.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/761fa37d0152f16dd691c4b71e669406/href">https://medium.com/media/761fa37d0152f16dd691c4b71e669406/href</a></iframe><p>A lot is going on in the snippet above. Let’s break it down.</p><ol><li>goMiddleware is a function that accepts a next parameter of type http.Handler and returns another http.Handler .</li><li>A function f is created inside goMiddleware . It takes in two parameters of type http.ResponseWriter and *http.Request , which are the signatures of a <em>typical handler function</em>.</li><li>The middleware logic is contained inside f .</li><li>f passes the request to the next handler by calling the handler’s ServeHTTP method.</li><li>f is transformed and returned as a http.Handler with the <a href="https://golang.org/pkg/net/http/#HandlerFunc">http.HandlerFunc</a> adapter.</li><li>The returned f forms a <a href="https://tour.golang.org/moretypes/25#:~:text=Go%20functions%20may%20be%20closures,%22bound%22%20to%20the%20variables.&amp;text=Each%20closure%20is%20bound%20to%20its%20own%20sum%20variable."><strong>closure</strong></a> over the next handler. Hence, it can still access the local next variable even after it is returned by goMiddleware .</li></ol><p>In practice, it is uncommon to explicitly create a function f . Instead, we simply pass an <em>anonymous function</em> to http.HandlerFunc and return it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4d074e16b0201ea9d295688f911dbcc6/href">https://medium.com/media/4d074e16b0201ea9d295688f911dbcc6/href</a></iframe><p>Don’t worry if the code snippets above look confusing. It will become clearer as we create and visualize some custom middlewares for our application.</p><p>The main takeaway is that a Go middleware is a function that <strong>accepts the next handler </strong>in the request chain as a parameter. It <strong>returns a handler</strong> that performs some logic before executing the next handler in the chain.</p><h3>Creating Go Middlewares</h3><blockquote>The middlewares we are about to create will be based on the book <a href="https://lets-go.alexedwards.net/">Let’s Go</a> by Alex Edwards.</blockquote><p>Hooray! Now that we know the right concepts and theories, things will be much easier from here onwards. In this section, we will attempt to create two middleware, one to <strong>log HTTP requests </strong>and another to <strong>add basic security response headers</strong>.</p><p>Go back to your terminal, make sure you are still at the project directory and run the command below.</p><pre>$ touch middleware.go</pre><p>This creates middleware.go that will house the two Go middleware we are about to create. Open your code editor and let’s get coding!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d21b787ef3880bb524315243b2ff9b09/href">https://medium.com/media/d21b787ef3880bb524315243b2ff9b09/href</a></iframe><p>logRequestMiddleware logs the network address, protocol version, HTTP method, and request URL to the standard output. This information is available in Go’s <a href="https://golang.org/pkg/net/http/#Request">http.Request</a> object.</p><p>secureHeadersMiddleware sets two security headers in the response (<a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection">X-XSS-Protection</a> and <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options">X-Frame-Options</a>) to defend against XSS and Clickjacking attacks.</p><p>Before we register our middlewares in main.go , it is important to know that the <strong>positioning of your middlewares</strong> affects the behavior of your application.</p><p>In particular, if we want a middleware to <strong>act on every HTTP request</strong>, we should place it <strong>before the ServeMux</strong>. In other words, we need to pass the ServeMux handler as an argument to our middleware.</p><p>An example of such middleware is the one we just created to log requests.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*1pLKJfI7-UrHBh5UnkRcsw.png" /><figcaption>Middleware before ServeMux</figcaption></figure><p>On the other hand, if we want a middleware to <strong>act on specific routes</strong>, we need to place them <strong>after the ServeMux</strong>. We pass individual route handlers as arguments to our middleware to achieve this.</p><p>An example would be an <em>authentication middleware </em>to protect private routes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*DU5o4H4jw9GWv3oWZNFIVg.png" /><figcaption>Middleware after ServeMux</figcaption></figure><p>We want every request to be logged in our simple application and each response to be set with the basic security headers. Hence, both the middlewares should be placed <em>before the ServeMux</em>.</p><p>Let’s register our middlewares in main.go as follows.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/602b4ba7be60d5470f1a7560c0ac3329/href">https://medium.com/media/602b4ba7be60d5470f1a7560c0ac3329/href</a></iframe><p>Notice how we chain the middlewares with ServeMux. mux is a handler and it is passed as an argument into secureHeadersMiddleware . This returns another handler, which is passed into logRequestMiddleware .</p><p>Visually, the request flow looks something as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3gm5uS0_mK0g6oqqvHlqUg.png" /><figcaption>Middlewares to ServeMux to route handler</figcaption></figure><p>To show the second type of middleware positioning, I will use a <em>hypothetical scenario</em>. Let’s say, somewhere in the future, we add a /private route along with a handler function secret .</p><p>A middleware named checkAuthMiddleware is created to protect this route against unauthorized access. To register this middleware to only the /private route, we can do something as follows.</p><pre>mux.Handle(&quot;/private&quot;, checkAuthMiddleware(http.HandlerFunc(secret)))</pre><p>Again, http.HandlerFunc converts the secret function to a http.Handler . This is passed as an argument to checkAuthMiddleware . Notice that we used <a href="https://golang.org/pkg/net/http/#Handle">mux.Handle</a> instead of mux.HandleFunc as we are working directly with a http.Handler .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jdg98bh9UkQSalIlC3CY7w.png" /><figcaption>ServeMux to middleware to secret handler</figcaption></figure><h3>Quick Testing</h3><p>Let us test and see if our application is working correctly as expected.</p><p>I will be using <a href="https://curl.se/"><strong>curl</strong></a><strong> </strong>to send HTTP requests from a terminal and see the responses. It should be pre-installed on Linux and macOS machines. Otherwise, you can download and install it from <a href="https://curl.se/download.html">here</a>.</p><p>If you have been following along, shut down your server with Ctrl+C at the terminal. Then, start it again with the go run command.</p><pre>$ go run .</pre><p>Open another terminal window to run our curl commands. First, we shall test if the HTTP requests are logged properly. You can send a GET request to <a href="http://localhost:4000/home">localhost:4000/home</a> with the command below.</p><pre>$ curl -X GET localhost:4000/home</pre><p>In the terminal window where you started the server, you should see a log as follows. Note that your network address will be different from mine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/518/1*ZJhai5-0tKCvH5MMQg2j1g.png" /><figcaption>A request log</figcaption></figure><p>Perfect! Now, to test our second middleware, we add a -i flag to our curl command above. This makes curl show us the response headers.</p><pre>$ curl -i -X GET localhost:4000/home</pre><p>After you run the command, you should see something as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/605/1*JVm7ZrDPjRRJPI8zQ4x00A.png" /><figcaption>Curl response</figcaption></figure><p>Do you see the X-Frame-Options and X-Xss-Protection headers?</p><h3>Final Thoughts</h3><p>If you made it here, pat yourself on the shoulder! You have learned all the basics you need to start working with Go middlewares.</p><p>Just a final note, you may notice that our application does not scale well with <strong>the number of middlewares</strong>. The chain of middlewares will be harder to maintain as it grows longer.</p><p>Fortunately, there are third-party packages that help us in managing middlewares. One of them is called <a href="https://github.com/justinas/alice">Alice</a>. I’ll leave it to you to check out how it works.</p><p>My name is Jonathan, and I am a backend software engineer in <a href="https://careers.shopee.sg/about">Shopee</a>, one of the fastest-growing e-commerce companies in Southeast Asia. Shopee is expanding at an unprecedented speed, and we are looking for bright engineering talents to join us!</p><p>If you are an engineer who wants to deliver software at scale in a fast-paced environment, then Shopee is the ideal place for you. Feel free to leave a note here or find me on <a href="https://www.linkedin.com/in/jonathanseow5177/">LinkedIn</a> if you wish to learn more about the company and our available positions.</p><p>Peace! ✌️</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da5dc4a3b9aa" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/learn-go-middlewares-by-examples-da5dc4a3b9aa">Learn Go Middlewares by Examples</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learn Encryption to Send Secret Love Letters]]></title>
            <link>https://medium.com/codex/learn-encryption-to-send-secret-love-letters-b5b2c31fe703?source=rss-cb7169f0fc76------2</link>
            <guid isPermaLink="false">https://medium.com/p/b5b2c31fe703</guid>
            <category><![CDATA[software]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[encryption]]></category>
            <category><![CDATA[cryptography]]></category>
            <dc:creator><![CDATA[Jonathan Seow]]></dc:creator>
            <pubDate>Fri, 23 Apr 2021 01:16:36 GMT</pubDate>
            <atom:updated>2021-05-06T08:13:10.852Z</atom:updated>
            <content:encoded><![CDATA[<h4>An introduction to encryption with one-time pad cipher, symmetric and public-key encryption</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NIMOAMbcENASuuOhCV-D2w.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@hudsoncrafted?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Debby Hudson</a> on Unsplash</figcaption></figure><p>I don’t know about you, but I grew up without a smartphone (gosh I’m old).</p><p>I didn’t even have a phone. I had to use my mother’s good old Sony Ericsson to send secret love messages to my crush. It was all fun and games until I had to destroy evidence by deleting over 200 text messages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rRaLb6kcCahOnWdB32rHww.jpeg" /><figcaption>Still better than an iPhone</figcaption></figure><p>Text messages were great, but I preferred the old-school approach of using <strong><em>love letters</em></strong><em>. </em>Ah<em>, </em>how romantic it was to craft a heartfelt letter to win over the heart of a young lady.</p><p>But, there was a problem with love letters. You see, I am a very shy person. I didn’t have the courage to deliver a love letter to my crush physically. I had to rely on my (unreliable) friends to do so, but I couldn’t trust them to not <em>sabotage the letter</em>!</p><p>Sadly, I didn’t have a solution then, but I do now, and it is called <strong><em>encryption</em></strong>.</p><h3>What is encryption?</h3><p>In simple terms, encryption is a way of <em>scrambling data </em>into a<em> </em>seemingly<em> random representation </em>such that only authorized parties can understand it.</p><p>In formal terms, encryption converts <strong>plaintext</strong> into <strong>ciphertext</strong>. Such conversion is done via an <strong>encryption algorithm</strong> with a <strong>cryptographic key</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/654/1*nP-FByGDvhh2dEB31mM4Ug.png" /><figcaption>plaintext + key = ciphertext</figcaption></figure><p>The encryption algorithm is a <strong>mathematical formula </strong>that transforms data into a random-looking ciphertext. It requires a cryptographic key to perform the transformation through <em>a series of definite steps</em>.</p><p>On the other hand, a cryptographic key is a string of characters that acts just like an <strong>actual key of a lock</strong>. Whoever owns it has the ability to <em>lock</em> (encrypt) and <em>unlock</em> (decrypt) the encrypted data.</p><p>Confused? Let’s demonstrate with an example. Looking at the image above, can you guess the algorithm and the key?</p><p>The algorithm I have used above is known as the <strong>One-Time Pad (OTP) cipher</strong>. This cipher <em>substitutes </em>each letter in the plaintext with another letter some positions down the alphabets.</p><p>For example, the letter “h” is moved <strong>7 positions </strong>down to the letter “o” while the letter “e” is moved <strong>5 positions</strong> down to “j”. After singing the classic ABCD song with the remaining letters, you would end up with <strong><em>75342</em></strong>, which is the key to this particular encryption.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/490/1*mEuIsT-dF4smtDdDLSVOfw.png" /><figcaption>OTP cipher</figcaption></figure><p>To add another layer of security, the key of the OTP cipher can be translated into letters in alphabets. For instance, using the letter-to-alphabet table shown below, the key 75342 is equivalent to <strong><em>“hfdec”</em></strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*_jtuaTTJvuYvw7WAVbS4Gg.png" /><figcaption>Letter-to-alphabet</figcaption></figure><p>What about the algorithm? We know that the OTP cipher involves moving and substituting letters, but we can give it a more formal mathematical definition using <em>modular addition</em>.</p><pre>OTPEncrypt(x, n) = (x + n) mod 26  </pre><p>Where x is the integer representation of a → z between 0 → 25 while n is an arbitrary positive integer. Modulo 26 is used to bound the result between 0 to 25 such that any right shifts that fall out of range will be looped back to the start of the alphabet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*gI49MUoG0uOHUvRmY_QJww.png" /><figcaption>Encrypt “x” with a key of 5</figcaption></figure><p>Now, moving letters around and calling it encryption is not at all exciting. In reality, encryption algorithms are far more complex than the OTP cipher with a much longer and random-looking key.</p><p>Good encryption must generate a ciphertext that is <strong>randomized beyond human recognition</strong>. At the same time, the ciphertext must be properly guarded against <a href="https://www.cloudflare.com/learning/bots/brute-force-attack/"><strong>brute-force attacks</strong></a>, which is doable with enough time and computational resources.</p><p>On the other hand, how can we decrypt a ciphertext? In the case of the OTP cipher, we can simply turn the encryption algorithm around to <em>subtract </em>n<em> </em>instead. The decryption formula is as follows:</p><pre>OTPDecrypt(x, n) = (x - n) mod 26</pre><p>If you know the secret key n, you can easily decipher the encrypted text back to plaintext!</p><p>In most cases, the encryption algorithms are the product of quality research, and such information is known to the public. The true secrecy of encryption lies in <em>the key being used</em>, and hence you must guard your secret key at all costs!</p><h3>Symmetric and Asymmetric Encryption</h3><p>In the industry, there are two types of widely used encryption: <strong>symmetric </strong>and <strong>asymmetric </strong>encryption.</p><h4>Symmetric Encryption</h4><p>In symmetric encryption, the <em>same secret key</em> is used for both encryption and decryption. The demonstration I had above with the OTP Cipher is an example of symmetric encryption.</p><p>As an analogy, think of symmetric encryption as a typical box that can be locked and unlocked by one key. The sender can lock a secret item in the box with a key, and the receiver needs the same copy of the key to unlock the box.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*MQfd8ZLcLTH6Xj-EO2O68Q.jpeg" /><figcaption>A box with a lock</figcaption></figure><p>Because symmetric encryption relies only on one key, you need to securely deliver your secret key to the receiving party for decryption. If the secret key is stolen during delivery, the encrypted message would be compromised!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*yT0J2Pr8O6tiZSqVgRAjYA.png" /><figcaption>The downside of symmetric encryption</figcaption></figure><h4>Asymmetric Encryption</h4><p>Unlike symmetric encryption, asymmetric encryption uses two keys for encryption and decryption: the <strong>public key </strong>and the <strong>private key</strong>.</p><p>Data encrypted with the public key can only be decrypted by the private key. At the same time, data encrypted with the private key can only be decrypted by the public key.</p><p>This concept may seem foreign to first-timers. To simplify it, let me provide you with an analogy adapted from the brilliant explanation given by Medium writer <a href="https://medium.com/@vrypan/explaining-public-key-cryptography-to-non-geeks-f0994b3c2d5">Panayotis Vryonis</a>.</p><p>Imagine a box with a special lock that has three states: <strong>two locked states</strong> and <strong>one unlocked state</strong>. The two locked states are on the left (L) and right (R), while the unlocked state is in the middle (M).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*AA-5zsJdqhsthUgXtgegxw.png" /><figcaption>A special lock with three states</figcaption></figure><p>Two different keys are required to work with this lock, and we shall name them Key A and Key B.</p><ul><li>Key A turns the lock from R → M → L (counterclockwise).</li><li>Key B turns the lock from L → M → R (clockwise).</li></ul><p>If the lock is locked by Key A (at L), only Key B can unlock it (turn right to M).</p><p>If the lock is locked by Key B (at R), only Key A can unlock it (turn left to M).</p><p>Now, let’s imagine Alice, being the famous girl in school before phones exist, wants to establish a reliable communication channel with her admirers. This channel needs to meet two criteria:</p><ol><li>Anyone can send a secret message to her, and <strong>only she</strong> can read the message content.</li><li>She can broadcast a message to the public, and her admirers can <strong>confirm with absolute certainty</strong> that the message is from her (instead of an <em>Alice imposter</em>).</li></ol><p>How can she do it? <em>Hint: it involves the box with a special lock, Key A and Key B.</em></p><p>First of all, Alice needs to choose either Key A or Key B as her <em>“private” key</em>, i.e. only she has access to the key. Let’s say she chooses Key A.</p><p>Then, she makes many copies of Key B and distributes them to her admirers along with the box. This makes Key B <em>“public”</em> because anyone can have access and make copies of it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/613/1*gRiK25HYkt1l0vyOb0ZXWg.png" /><figcaption>Key A is private to Alice and Key B is public</figcaption></figure><p>Whoever wishes to speak to her in private can place their message in the box and lock it with the public Key B. Remember, if a box is locked by Key B, only Key A can unlock it, i.e. only Alice can read the secret messages! <strong><em>That’s all there is to meet the first criteria!</em></strong></p><blockquote>🔑 Only the private Key A can unlock a box locked by public Key B.</blockquote><p>What about the second criteria? How can she prevent someone from impersonating her?</p><p>Well, Alice can place her message in boxes locked by her private Key A and distribute them. Again, recall that if a box is locked by Key A, only the public Key B can unlock it.</p><p>So, if I want to make sure the box I receive is from Alice instead of an imposter, all I need is to try and unlock it with my Key B. If I can unlock it, the box must be locked by Key A, i.e. it comes from Alice! If I can’t, the box comes from an imposter. <strong><em>And that settles the second criteria!</em></strong></p><blockquote>🔑 Only the public Key B can unlock a box locked by private Key A.</blockquote><p>This concludes the bare-bone implementation of asymmetric encryption, also known as <strong>public-key encryption</strong>. This encryption is commonly used to securely transfer data over the Internet via HTTPS. An example of asymmetric encryption is the <a href="https://en.wikipedia.org/wiki/RSA_(cryptosystem)">RSA algorithm</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*xoP4I1v1ltsqly-c.png" /><figcaption>Secure Internet browsing with HTTPS</figcaption></figure><h3>Final Thoughts</h3><p>That covers the basics of encryption! The main takeaway from this article is that encryption uses an algorithm with a cryptographic key to scramble data such that they become unrecognizable.</p><p>Symmetric encryption uses only one key to decrypt and encrypt data, hence it must be kept with absolute secrecy. On the other hand, asymmetric encryption uses two keys, one that is kept private while the other can be shared with the public.</p><p>Thank you for reading! Peace ✌️</p><h3>References</h3><ol><li><a href="https://www.cloudflare.com/learning/ssl/what-is-encryption/">What is encryption?</a> by Cloudflare</li><li><a href="https://www.cloudflare.com/learning/ssl/what-is-asymmetric-encryption/">What is asymmetric encryption?</a> by Cloudflare</li><li><a href="https://en.wikipedia.org/wiki/One-time_pad">One-Time Pad Cipher</a> on Wikipedia</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b5b2c31fe703" width="1" height="1" alt=""><hr><p><a href="https://medium.com/codex/learn-encryption-to-send-secret-love-letters-b5b2c31fe703">Learn Encryption to Send Secret Love Letters</a> was originally published in <a href="https://medium.com/codex">CodeX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>