<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Fabricio Pautasso on Medium]]></title>
        <description><![CDATA[Stories by Fabricio Pautasso on Medium]]></description>
        <link>https://medium.com/@fabriciopautasso?source=rss-8fdaec8f00ec------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*qZh6wx_n_Y1_CSPHJEH-TQ.jpeg</url>
            <title>Stories by Fabricio Pautasso on Medium</title>
            <link>https://medium.com/@fabriciopautasso?source=rss-8fdaec8f00ec------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 20:32:29 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@fabriciopautasso/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[7 Useful TypeScript Tips to Level-up your Code]]></title>
            <link>https://javascript.plainenglish.io/7-useful-typescript-tips-to-level-up-your-code-25d0ff545009?source=rss-8fdaec8f00ec------2</link>
            <guid isPermaLink="false">https://medium.com/p/25d0ff545009</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Fabricio Pautasso]]></dc:creator>
            <pubDate>Wed, 14 Apr 2021 12:20:54 GMT</pubDate>
            <atom:updated>2021-04-14T12:20:54.499Z</atom:updated>
            <content:encoded><![CDATA[<h4>Tips and tricks to level-up your TypeScript code</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tums41FARnW23dtyXsi86w.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@mkvandergriff?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Maranda Vandergriff</a> on <a href="https://unsplash.com/s/photos/code?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h4>1. Currying</h4><p>Currying is the process of transforming a function that takes multiple arguments into a series of functions that take one argument at a time.</p><pre>// A curried function<br>let add = (x: number) =&gt; (y: number) =&gt; x + y;</pre><pre>// Simple usage<br>add(321)(345);</pre><pre>// partially applied<br>let add321 = add(321);</pre><pre>// fully apply the function<br>add321(345);</pre><h4>2. String enums</h4><p>Sometimes you need to collect a collection of strings under a common key. Prior to TypeScript 2.4, TypeScript only supported number-based enumerations. If you use a version prior to 2.4, the solution is to use the string literal type to create a string-based enumeration by combining the <a href="https://javascript.plainenglish.io/typescript-advanced-types-b6d0f19bcbf5">union type</a>.</p><p>In a string enum, each member has to be constant-initialized with a string literal, or with another string enum member</p><pre>enum Direction {<br>Up = &quot;UP&quot;,<br>Down = &quot;DOWN&quot;,<br>Left = &quot;LEFT&quot;,<br>Right = &quot;RIGHT&quot;,<br>}</pre><p>Although string enums have no auto-increment behavior, the advantage of string enums is that they can be “serialized” well. In other words, if you are debugging and must read the runtime value of a numeric enum, the value is usually opaque, it cannot convey any useful meaning by itself (although reverse mapping can usually help), string enum allows you to provide meaningful and readable values ​​while your code is running, regardless of the name of the enum member itself.</p><h4>3. Stateful functions</h4><p>A common feature of other programming languages ​​is the use of the static keyword to increase the lifetime (rather than the scope) of function variables to make them beyond the scope of function calls. Here is a C example that achieves this:</p><pre>void addUp() {<br>    static count = 0;<br>    count++;<br>    printf(&quot;Count : %d&quot;, count);<br>}</pre><pre>int main () {<br>    addUp(); // Count : 1<br>    addUp(); // Count : 2<br>    return 0;<br>}</pre><p>Since TypeScript doesn’t have static functions, you can achieve the same thing using various abstractions that wrap over a local variable e.g. using a class:</p><pre>const {addUp} = new class {<br>    count = 0;<br>    addUp = () =&gt; { <br>       this.count++;<br>       console.log(`Count : ${this.count}`); <br>    }<br>};</pre><pre>addUp(); // Count : 1<br>addUp(); // Count : 2</pre><h4>4. Export default can lead to problems</h4><p>If you have a file bar.ts:</p><pre>class Bar {<br>}<br>export default Bar;</pre><p>You would import it in faa.ts using ES6 syntax as follows:</p><pre>import Bar from &quot;./bar&quot;;</pre><p>Here are some maintainability issues:</p><ul><li>If Bar is refactored in bar.ts, it will not be renamed in faa.ts.</li><li>If you eventually need to export more stuff from bar.ts(which is what many of your files have), then you have to juggle the import syntax.</li></ul><p>Therefore, the recommendation is to use simple exports + destructured import:</p><pre>export class Bar{<br>}</pre><p>And then:</p><pre>import {Bar} from &quot;./bar&quot;;</pre><h4>5. Barrel</h4><p>A Barrel is a way to aggregate exports from multiple modules to a single convenient module. The Barrel itself is a module file that can re-export selected exports of other modules. Imagine the following class structure in the library:</p><pre>// tutorial/bar.ts<br>export class Bar {} </pre><pre>// tutorial/bas.ts<br>export class Bas {}</pre><pre>// tutorial/bat.ts<br>export class Bat {}</pre><p>Without a barrel, a consumer would need three import statements:</p><pre>import { Bar } from &#39;../tutorial/bar&#39;; <br>import { Bas } from &#39;../tutorial/bas&#39;; <br>import { Bat } from &#39;../tutorial/bat&#39;;</pre><p>You can instead add a Barrel tutorial/index.ts containing the following:</p><pre>// tutorial/index.ts<br>export * from &#39;./bar&#39;; <br>export * from &#39;./bas&#39;; <br>export * from &#39;./bat&#39;;</pre><p>Now the consumer can import what it needs from the barrel:</p><pre>import { Bar, Bas, Bat } from &#39;../tutorial&#39;;</pre><h4>6. Infer</h4><p>It is the magnifying glass of TypeScript, which is a beautiful inspection tool that can extract types trapped in different types of structures.</p><p>As an example, let’s extract the property’s type from an object:</p><pre>type ObjectInfer&lt;O&gt; = O <br>  extends {a: infer A}<br>  ? A<br>  : never</pre><pre>const object = {a: &#39;hello&#39;};<br>type testResult1 = ObjectInfer&lt;typeof object&gt;; // string<br>type testResult2 = ObjectInfer&lt;string&gt;; // never</pre><h4>7. Truthy</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/0*_rw1KiK3xuHCoFah.jpeg" /><figcaption>Image from <a href="https://samanthaming.com/">https://samanthaming.com</a></figcaption></figure><p>The last tip is to properly understand a basic concept — truthy. Truthy are things that evaluate to true in certain places (such as conditions and boolean &amp;&amp; || operators). The following is the real situation in JavaScript. An example is any number other than 0, such as:</p><pre>if (1) { <br>console.log(&#39;Any number other than 0 is truthy&#39;);<br>}</pre><p>Something that isn’t truthy is called falsy.</p><p><em>More content at </em><a href="http://plainenglish.io/"><em>plainenglish.io</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=25d0ff545009" width="1" height="1" alt=""><hr><p><a href="https://javascript.plainenglish.io/7-useful-typescript-tips-to-level-up-your-code-25d0ff545009">7 Useful TypeScript Tips to Level-up your Code</a> was originally published in <a href="https://javascript.plainenglish.io/">JavaScript in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TypeScript advanced types]]></title>
            <link>https://javascript.plainenglish.io/typescript-advanced-types-b6d0f19bcbf5?source=rss-8fdaec8f00ec------2</link>
            <guid isPermaLink="false">https://medium.com/p/b6d0f19bcbf5</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[coding]]></category>
            <dc:creator><![CDATA[Fabricio Pautasso]]></dc:creator>
            <pubDate>Sun, 11 Apr 2021 05:37:53 GMT</pubDate>
            <atom:updated>2021-04-12T10:19:22.958Z</atom:updated>
            <content:encoded><![CDATA[<h4>TypeScript tricks that will increase your development skills</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0Ca38BL7C5MRI0qsdAQT3Q.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@pankajpatel?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Pankaj Patel</a> on <a href="https://unsplash.com/s/photos/advanced-types-code?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>TypeScript has some advanced language features that can be used when working with basic types and objects. In this article, we will take a quick look at these advanced type features, including:</p><ul><li>Union types</li><li>Type guards</li><li>Type aliases</li><li>Null and undefined</li><li>Object rest and spread</li></ul><h4>Union types</h4><p>TypeScript allows us to represent a type as a combination of two or more other types. This technique is called a union type and uses the pipe symbol (|). Consider the following TypeScript code:</p><pre>var union : string | number;</pre><pre>union = 7;<br>console.log(`union : ${union}`); // Output: union: 7</pre><pre>union = &quot;article&quot;; <br>console.log(`union : ${union}`); // Output: union: article</pre><p>The variable union uses the union type syntax to denote that it can hold either a string or a number .</p><p>A function parameter can also be of union type:</p><pre>function checkType(patientId: (string | number))<br>{<br>    if(typeof(patientId) === &quot;number&quot;)<br>        console.log(&#39;PatientId is a number.&#39;)<br>    else if(typeof(patientId) === &quot;string&quot;)<br>        console.log(&#39;PatientId is a string.&#39;)<br>}<br><br>checkType(9876); // Output: PatientId is a number.<br>checkType(&quot;PI-1234-AZ&quot;); // Output: PatientId is a string.<br>cheyType(true); //Compiler Error: Argument of type &#39;true&#39; is not assignable to a parameter of type string | number</pre><h4>Type guards</h4><p>When using union types, the compiler will still apply its strong typing rules to ensure that we do not mix and match our types. As an example, consider the following code:</p><pre>function add(<br>union1 : string | number,<br>union2 : string | number<br>){<br> return union1 + union2;<br>}</pre><p>The add function accepts two parameters and returns their sum. The union1 and union2arguments are union types, and can therefore be either a stringor a number. Compiling this code, however, will generate the following error:</p><pre><strong>error TS2365: Operator &#39;+&#39; cannot be applied to types &#39;string | number&#39; and &#39;string | number&#39;.</strong></pre><p>What the compiler tells us here is that in the function body, it cannot determine what is the type of union1.</p><p>This is where type guards come in. Type guards is an expression that checks our type and then ensures that the type is within its scope. Consider the following code:</p><pre>function typeGuardAdd(<br>union1 : string | number,<br>union2 : string | number<br>) : string | number {<br> if( typeof union1 ===&quot;string&quot;) {<br>  console.log(&#39;first argument is a string&#39;); <br>  return union1 + union2;<br>}<br> if (typeof union1 === &quot;number&quot; &amp;&amp; typeof union2 === &quot;number&quot;) {<br>  console.log(&#39;both arguments are numbers&#39;);<br>  return union1 + union2;<br>}<br>console.log(&#39;default return&#39;);<br>return union1.toString() + union2.toString();<br>}</pre><p>In the main body of the code, we have two if statements. The first if statement checks whether the type of union1is a string. If it is a string, the type of union1 is treated as a string within the body of the if statement. The second ifstatement checks whether union1and union2are both numeric types. Within the body of the second if statement, both union1 and union2are treated as numbers. These two if statements are our type guards.</p><p>Note that our final return statement is calling the toString function on union1and union2. By default, all basic JavaScript types have a toStringfunction, so in fact, we treat both parameters as strings and return the result.</p><h4>Type aliases</h4><p>Sometimes, when using union types, it may be difficult to remember which types are allowed. For this reason, TypeScript introduced the concept of type aliases, where we can create special named types for type unions. Therefore, type aliases are a convenient naming convention for union types.</p><p>Type aliases can be used anywhere common types are used, and are represented by the type keyword. Therefore, we can simplify the code as follows:</p><pre>type StringOrNumber = string | number;</pre><pre>function aliasesAdd(<br>union1 : StringOrNumber,<br>union2 : StringOrNumber<br>){<br> return union1.toString() + union2.toString();<br>}</pre><p>Type aliases can also be used for function signatures, as follows:</p><pre>type CallbackWithString = (string) =&gt; void;</pre><pre>function usingCallbackWithString(<br>callback: CallbackWithString) {<br>  callback(&quot;this is a string&quot;);<br>}</pre><h4>Null and undefined</h4><p>In JavaScript, if a variable has been declared but no value is assigned, the query for its value will return undefined. JavaScript also includes the keyword nullto distinguish between the case where the variable is known but has no value (empty) and the case where the variable is not defined in the current scope (undefined). Consider the following JavaScript code:</p><pre>function testUndefined(test) {<br>  console.log(&#39;test parameter :&#39; + test);<br>}<br>testUndefined(); // test parameter :undefined</pre><pre>testUndefined(null); // test parameter :null</pre><p>The testUndefined function is called for the first time without any parameters. In fact, this is calling the function without knowing or caring about what parameters it needs. JavaScript allows this kind of function call syntax.</p><p>TypeScript has included two keywords for these cases, named null and undefined. Let’s re-write this function in TypeScript, as follows:</p><pre>function testUndefined(test : null | number) { <br>  console.log(&#39;test parameter :&#39; + test);<br>}</pre><p>Here, we have defined the testUndefined function to allow for the function to be called with either a number or a null value. If we try to call this function in TypeScript without any arguments (testUndefined();), as we did in JavaScript, TypeScript will generate an error:</p><pre><strong>error TS2346: Supplied parameters do not match any signature of call target.</strong></pre><p>This ability to specify that a function can be called with a null value allows us to ensure that the correct use of our function is known at compile time.</p><h4>Object rest and spread</h4><p>When using basic JavaScript objects, we often need to copy the properties of one object to another, or mix and match the properties of various objects. In order to meet these requirements, TypeScript adopted ES7 proposals and language grammar, called Object Rest and Spread. Consider the following TypeScript code:</p><pre>let firstPerson = { id: 1, name : &quot;Rachel&quot;};<br>let secondPerson = { …firstPerson };</pre><pre>console.log(`secondPerson.id : ${secondPerson.id}`); // secondPerson.id : 1<br>console.log(`secondPerson.name : ${secondPerson.name}`); // secondPerson.name : Rachel</pre><p>Here, we first define a simple JavaScript object named firstPerson, which has idand nameattributes. Then, by specifying {… firstPerson}, use the new ES7 syntax to copy all the attributes of firstPersonto another object named secondPerson. To test whether it actually copied all the properties, we then log the values ​​of secondPerson.id and secondPerson.name to the console.</p><p>We can also use this syntax to combine multiple objects together, as follows:</p><pre>let name = { name : &quot;Rachel&quot;};<br>let id = { id : 2};</pre><pre>let compose = { ...name, ...id };</pre><pre>console.log(`compose.id : ${compose.id}`);<br>console.log(`compose.name : ${compose.name}`);</pre><p><em>The rest and spread syntax to copy properties will apply these properties incrementally. In other words, if two objects both have a property with the same name, then the object property that was specified last will take precedence.</em></p><h4><strong>Conclusion</strong></h4><p>In this article, we reviewed a couple of advanced TypeScript types and techniques. Some takeaways are:</p><ul><li>When we need to use union types often within our code, type aliases provide an easier and more intuitive way of declaring named union types.</li><li>Rest and spread syntax could be useful when you need to combine two objects</li><li>TypeScript compiler is usually protecting our code from being used in not intended ways. For example, calling a function with an undefined value. Something that you can do in JS.</li><li>Type guards allow you to check the type of a variable within your code and guarantee that the variable is of the type you expect within your code block.</li></ul><p><em>More content at </em><a href="https://plainenglish.io/"><strong><em>plainenglish.io</em></strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b6d0f19bcbf5" width="1" height="1" alt=""><hr><p><a href="https://javascript.plainenglish.io/typescript-advanced-types-b6d0f19bcbf5">TypeScript advanced types</a> was originally published in <a href="https://javascript.plainenglish.io/">JavaScript in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Improving the Integration with Remote Services]]></title>
            <link>https://levelup.gitconnected.com/improving-the-integration-with-remote-services-742e093eca6b?source=rss-8fdaec8f00ec------2</link>
            <guid isPermaLink="false">https://medium.com/p/742e093eca6b</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[software-patterns]]></category>
            <category><![CDATA[facade-pattern]]></category>
            <category><![CDATA[services]]></category>
            <dc:creator><![CDATA[Fabricio Pautasso]]></dc:creator>
            <pubDate>Fri, 26 Mar 2021 13:07:26 GMT</pubDate>
            <atom:updated>2021-03-26T13:07:26.881Z</atom:updated>
            <content:encoded><![CDATA[<h4>Remote Façade Pattern</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aUsxHj14Zx9OYC2v7Ypj-Q.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@dialex?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Diogo Nunes</a> on <a href="https://medium.com/s/photos/fa%C3%A7ade-pattern?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>In an object-oriented model, it is better to use small objects with small methods. This provides you with many opportunities to control and replace behavior, and can use good intention revealing naming to make the application easier to understand. One consequence of this fine-grained behavior is that there is usually a lot of interaction between objects, and this interaction usually requires a lot of method calls.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/441/1*-0iQoMDUZVQoukrWWTDrUA.png" /></figure><p>Within a single address space, fine-grained interactions can work well, but when you make calls between processes, this happy state does not exist. The cost of remote calls is much higher because there is still a lot to do: data may have to be marshaled, security may need to be checked, and data packets may need to be routed through a switch. If these two processes are running on a machine on the other side of the earth, the speed of light may be a factor. The cruel fact is that even if two processes are on the same computer, any inter-process call is orders of magnitude more expensive than an intra-process call. Even for believers of lazy optimization, this performance impact cannot be ignored.</p><p>As a result, any object intended to be used as a remote object needs a coarse-grained interface to minimize the number of calls required to accomplish something. This will not only affect your method calls but also your objects. For example, there is no need to request an order and its order lines separately, but the order and order lines need to be accessed and updated in one call. This will affect the entire object structure and programming becomes more difficult and productivity decreases.</p><h4>Remote Façades</h4><p>Remote Facade tackles the distribution problem which the standard OO approach of separating distinct responsibilities into different objects; and as a result, it has become the standard pattern for this problem.</p><p>In a simple example, like integrating an application to a wearable data provider, a Remote Facade replaces all the getting methods of the external provider API with one getter, often referred to as bulk accessor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*uY7b_fU_DifHr1Ql6OEQ9A.png" /></figure><p>In more complex situations, a single Remote Facade can act as a remote gateway for many fine-grained objects. For example, the Order Facade can be used to obtain and update order information, all order lines, and some customer data. When transferring information in large quantities like this, you need it in a form that is easy to move on the network. If there are fine-grained classes on both sides of the connection and they are serializable, they can be transferred directly by copying.</p><p>However, you usually cannot do this. You may not want to replicate the domain class on multiple processes, or it may be difficult to serialize part of the domain model due to the complex relational structure of the domain model. Clients may not want the entire model, but just a simplified subset of it. In these cases, it makes sense to use Data Transfer Objects (<a href="https://en.wikipedia.org/wiki/Data_transfer_object">DTO</a>) as the basis of the transfer.</p><h4>Granularity and responsibilities</h4><p>Granularity is one of the most difficult problems of Remote Facade. Some people like to make fairly small remote facades, such as one for each use case. Others prefer the coarse-grained structure, and the remote facades are much less. When integrating with external services like Twilio for SMS, Chats, and Video Services or SendGrid for email delivery, I usually create a Facade for each external integration.</p><p>In addition to providing a coarse-grained interface, other responsibilities can also be added to the Remote Facade. For example, its method is the natural point to add application security. An access control list can tell which users can call which methods. The Remote Facade method is also the natural point of application transaction control. The method can start a transaction, complete all internal work, and then commit the transaction at the end.</p><p>One of the biggest mistakes in implementing Remote Facade is putting domain logic into it. Any Remote Facade should be thin skin, with very few responsibilities. If you need domain logic for workflow or coordination, you can put it in a fine-grained object, or create a separate non-remote <a href="https://dzone.com/articles/transaction-script-pattern">Transaction Script</a> to contain it. You should be able to run the entire application locally without using Remote Facades or copying any code.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=742e093eca6b" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/improving-the-integration-with-remote-services-742e093eca6b">Improving the Integration with Remote Services</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Access Host Resources from a Docker Container]]></title>
            <link>https://levelup.gitconnected.com/how-to-access-host-resources-from-a-docker-container-317e0d1f161e?source=rss-8fdaec8f00ec------2</link>
            <guid isPermaLink="false">https://medium.com/p/317e0d1f161e</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Fabricio Pautasso]]></dc:creator>
            <pubDate>Thu, 18 Mar 2021 00:18:15 GMT</pubDate>
            <atom:updated>2021-03-18T00:18:15.222Z</atom:updated>
            <content:encoded><![CDATA[<h4>Bypassing Docker abstractions</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sssMt3-edaVd73BoiBYVhg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@goosegrease?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Derek Oyen</a> on <a href="https://medium.com/s/photos/whale?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Although Docker tries to abstract the application from the host on which it is running, it can never be completely ignored. In order to provide its abstraction, Docker must add a layer of indirection. These layers may have an impact on the system you are running, and sometimes you need to understand them to solve operational problems.</p><p>In this article, we will study how to circumvent some of these abstractions, and finally, get a Docker container with almost no Docker.</p><h4>Namespaces and Docker</h4><p>Docker volumes are convenient for sharing files from the host and keeping larger files out of image layers. They can also be significantly faster for filesystem access than the container filesystem, as some storage backends impose significant overheads for certain workloads.</p><p>Docker’s setting to provide each container with its own network interface can cause another performance drop. Like file system performance, network performance is definitely not everyone’s bottleneck, but you may want to benchmark yourself. In addition, you may have other reasons to bypass the Docker network completely.</p><p>For whatever reason, sometimes Docker abstraction hinders your work, and Docker does provide the ability to opt-out as needed.</p><p>Docker offers several ways to bypass the kernel namespace functionality that Docker uses.</p><blockquote><strong>WHAT IS A NAMESPACE?</strong> A namespace is a service provided by the kernel to programs that allows them to obtain a view of the global resource in a way that appears to have a separate instance of the resource. For example, a program can request a network namespace, which will provide you with a complete network stack. Docker uses and manages these namespaces to create its containers.</blockquote><p>The following table summarizes how Docker uses namespaces, and how you can effectively switch them off.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*xwAvDtURc1dTXfTnnZV7Tw.png" /></figure><p>For example, if your application is a heavy user of shared memory, and you want the container to share this space with the host, you can use the --ipc=hostflag to achieve this. Docker currently does not use the user namespace function of the Linux kernel, despite continuous efforts in this area.</p><h4>Networking</h4><p>To use the host’s network, you run your container with the--netflag set to host, like this:</p><pre>fabripautasso@Fabricios-MacBook-Pro ~ $ docker run -it --net=host ubuntu /bin/bash</pre><pre>root@docker-desktop:/#</pre><p>You’ll notice that this immediately differs from a network-namespaced container in that the hostname within the container is the same as the host’s. On a practical level, this can cause confusion, as it’s not so obvious that you’re in a container.</p><p>In a network-isolated container, a quick netstat will show that there are no connections on startup:</p><pre>root@3f17f5f70226:/# netstat</pre><pre>Active Internet connections (w/o servers)</pre><pre>Proto Recv-Q Send-Q Local Address           Foreign Address         State</pre><pre>tcp        0      0 3f17f5f70226:52242      actiontoad.canonical:80 TIME_WAIT</pre><pre>tcp        0      0 3f17f5f70226:52240      actiontoad.canonical:80 TIME_WAIT</pre><pre>tcp        0      0 3f17f5f70226:50128      kazooie.canonical.co:80 TIME_WAIT</pre><pre>Active UNIX domain sockets (w/o servers)</pre><pre>Proto RefCnt Flags       Type       State         I-Node   Path</pre><p>A similar run using the host’s network shows the usual network-busy host of a similarly busy technical author:</p><pre>root@docker-desktop:/# netstat -nap | head</pre><pre>Active Internet connections (servers and established)</pre><pre>Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name</pre><pre>tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      -</pre><pre>tcp        0      0 0.0.0.0:65101           0.0.0.0:*               LISTEN      -</pre><pre>tcp        0      0 192.168.65.3:62732      91.189.88.152:80        TIME_WAIT   -</pre><pre>tcp        0      0 192.168.65.3:62730      91.189.88.152:80        TIME_WAIT   -</pre><pre>tcp        0      0 192.168.65.3:62734      91.189.88.152:80        TIME_WAIT   -</pre><pre>tcp6       0      0 :::111                  :::*                    LISTEN      -</pre><pre>tcp6       0      0 :::62141                :::*                    LISTEN      -</pre><pre>udp        0      0 0.0.0.0:64167           0.0.0.0:*                           -</pre><p>For several reasons, the —-net=hostflag is the most commonly used flag. First, it can make connecting containers easier. However, you will lose the benefits of container port mapping. For example, if you have two containers listening on port 80, you cannot run them on the same host in this way.</p><p>The second reason is that when this flag is used, the network performance is much better than Docker. While the native network only needs to run through the host’s TCP/IP stack to the network interface card (NIC), Docker must additionally maintain a virtual Ethernet pair (also known as a veth pair, which is a virtual representation of a physical connection through an Ethernet cable), a network bridge between this pair of network and host network and a layer of network address translation (NAT). Under normal usage, this overhead may cause the Docker network speed to be only half of the local host network speed.</p><h4>PID</h4><p>The PID namespace flag looks like:</p><pre>fabripautasso@Fabricios-MacBook-Pro ~ $ docker run ubuntu ps -p 1</pre><pre>PID TTY          TIME CMD</pre><pre>1 ?        00:00:00 ps</pre><pre>fabripautasso@Fabricios-MacBook-Pro ~ $ docker run --pid=host ubuntu ps -p 1</pre><pre>PID TTY          TIME CMD</pre><pre>1 ?        00:00:07 init</pre><p>The preceding example demonstrates that the init process of the host has process ID 1 in the container that has a view of the host PIDs, whereas without that view the only process seen is the ps command itself.</p><h4>Mount</h4><p>If you want access to the host’s devices, you can use the--device flag if you want to use a specific device, or you can mount the entire host’s filesystem with the --volumeflag:</p><pre>docker run -ti —- volume /tmp:/ctmp ubuntu /bin/bash</pre><p>This command mounts the host’s /tmpdirectory to the container’s ctmp directory. Mounting the host’s / directory to the container’s / directory is explicitly disallowed by the docker command.</p><h4>Transparent host</h4><p>You can use these flags to create a container with almost a host view:</p><pre>fabripautasso@Fabricios-MacBook-Pro ~ $ <br>docker run -ti --net=host --pid=host --ipc=host \             1<br>--volume /:/tmp \                                             2<br>busybox chroot /tmp                                           3</pre><ul><li>1 — Runs container with 3 of the flags discussed over the article --net , --ipc , and --pid</li><li>2 — Mounts the root file system of the host to a directory /tmp on the container</li><li>3 — Starts up a busybox container, and executes the chroot command that will make the mounted filesystem appear as the root.</li></ul><h4>Final thoughts</h4><p>Using these flags to bypass Docker’s abstractions will expose you to more security risks. In terms of security, using them should be equivalent to running the containers with the --privilegedflag. So, be careful if you need to use them.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=317e0d1f161e" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/how-to-access-host-resources-from-a-docker-container-317e0d1f161e">How to Access Host Resources from a Docker Container</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Safe Proxies]]></title>
            <link>https://levelup.gitconnected.com/safe-proxies-1008b12c54ea?source=rss-8fdaec8f00ec------2</link>
            <guid isPermaLink="false">https://medium.com/p/1008b12c54ea</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[proxy]]></category>
            <dc:creator><![CDATA[Fabricio Pautasso]]></dc:creator>
            <pubDate>Wed, 10 Mar 2021 13:58:14 GMT</pubDate>
            <atom:updated>2021-03-10T13:58:14.228Z</atom:updated>
            <content:encoded><![CDATA[<h4>Securing your internal systems</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LDMv3nzRaYsgklB02Z0t8Q.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@jdent?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Jason Dent</a> on <a href="https://unsplash.com/s/photos/safe-proxi?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Imagine an attacker who wants to deliberately disrupt your system. Or, an engineer with a privileged account made a profound change by mistake and shut down an entire server fleet. Because your understanding of the system is very clear, and it was designed to provide minimal privileges, the impact on the environment is limited. When investigating and performing incident response, you can determine the root cause of the problem and take appropriate measures.</p><p>Maybe not all of your systems are suitable for this situation, but you need a way to make the running system safer and less prone to downtime. A safe proxy is one way to do this.</p><h4>Safe Proxies</h4><p>Generally, proxies provide a way to address new reliability and security requirements without requiring substantial changes to the deployed system. Without modifying the existing system, you can simply use a proxy to route connections that could otherwise be directly connected to the system. Proxies can also contain controls that meet your new security and reliability requirements.</p><p>A security proxy is a framework that allows authorized personnel to access or modify the state of physical servers, virtual machines, or specific applications. Google uses a secure proxy to view, approve and run risky commands without having to establish an SSH connection with the system. Using these proxies, enable them to grant fine-grained access to debugging problems, or limit the rate at which the computer restarts. The security proxy represents a single point of entry between networks and is a key tool that enables the following operations:</p><ul><li>Audit every operation in the fleet</li><li>Control access to resources</li><li>Protect production from human mistakes at scale</li></ul><p><a href="https://www.usenix.org/conference/srecon19emea/presentation/czapinski">Zero Touch Prod</a> is a Google project that requires that every change in production must be automated (not human), pre-verified by software, or triggered by an audited <em>breakglass mechanism</em>.</p><blockquote>In computing “<strong>Break Glass</strong>” is the act of checking out a system account password to bypass normal access controls <strong>procedures</strong> for a critical emergency. This provides the user immediate access to an account that they may not normally be authorized to access.</blockquote><p>The security proxy is one of a series of tools that Google uses to implement these principles. They estimate that zero-touch products can prevent or alleviate about 13% of all outages evaluated by Google.</p><h4>Safe Proxy Model</h4><p>In the security proxy model shown below, the client talks to the proxy instead of directly talking to the target systems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/691/1*nrNNDyCOAh_fo_wB4-y05g.png" /></figure><p>Google implemented this behavior by restricting the target system to accept calls from proxies only. This configuration specifies which client roles can perform which application layer remote procedure calls (RPC) through an access control list (ACL). After checking the access rights, the proxy sends the request to be executed via RPC to the target system. Usually, each target system has an application layer program that can receive the request and execute it directly on the system. The proxy records all requests and commands issued by the system with which it interacts.</p><p>Proxies provide the following benefits:</p><ul><li>A central point to enforce multi-party authorization (MPA), where the access decisions for requests that interact with sensitive data can be made</li><li>Administrative usage auditing</li><li>Rate limiting</li><li>Compatibility with closed-source third-party target systems</li><li>Continuous improvement integration, where security and reliability enhancements can be added to the central proxy point</li></ul><p>Proxies also have some downsides and potential pitfalls:</p><ul><li>Increased cost, in terms of maintenance and operational overhead.</li><li>A single point of failure, if either the system itself or one of its dependencies is unavailable. This can be mitigated by running multiple instances to increase redundancy.</li><li>A policy configuration for access control, which can be a source of errors itself.</li><li>A central machine that an attacker could take control of.</li><li>Resistance to change, as users may wish to connect directly to production systems.</li></ul><p>Since the main use case of a security proxy is to add security and reliability functions related to access control, the interface exposed by the proxy should use the same external API as the target system. Therefore, the proxy does not affect the overall user experience. Assuming that the safe proxy is transparent, it can simply forward traffic after performing some pre and post-processing for authentication and recording.</p><h4>Final thoughts</h4><p>Using a safe proxy is a way to add logging and multi-party authorization to the system. The proxy can therefore help you improve the security and reliability of the system. For existing systems, this method can be a cost-effective option.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1008b12c54ea" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/safe-proxies-1008b12c54ea">Safe Proxies</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Analytics vs Transactional Processing]]></title>
            <link>https://levelup.gitconnected.com/analytics-vs-transactional-processing-1118915a0ea7?source=rss-8fdaec8f00ec------2</link>
            <guid isPermaLink="false">https://medium.com/p/1118915a0ea7</guid>
            <category><![CDATA[transactions]]></category>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[data-warehouse]]></category>
            <dc:creator><![CDATA[Fabricio Pautasso]]></dc:creator>
            <pubDate>Wed, 03 Mar 2021 16:42:27 GMT</pubDate>
            <atom:updated>2021-03-03T16:42:27.988Z</atom:updated>
            <content:encoded><![CDATA[<h4>OLTP, OLAP, stars, and snowflakes</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7uJyOAz_TH0zIWa-tO5WfQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@zmachacek?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Zdeněk Macháček</a> on <a href="https://unsplash.com/s/photos/snowflake-star?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>In the early days of business data processing, write operations to the database usually correspond to commercial transactions that occur: selling, placing orders with suppliers, paying employees’ salaries, etc. Transaction does not involve currency exchange, but the term transaction is stagnant, referring to a set of reads and writes that constitute a logical unit.</p><p>Even if the database started to be used for many different types of data annotations such as blog posts, actions in games, contacts in the address book, etc., the basic access pattern is still similar to processing business transactions. Applications usually use indexes to find a small number of records by certain keys. Insert or update records based on user input. Since these applications are interactive, the access mode is called <strong><em>online transaction processing</em></strong> (OLTP).</p><p>However, databases also started to be used more and more for data analysis, which has very different access patterns. Generally, analytical queries need to scan a large number of records, read only a few columns of each record, and then calculate summary statistics (such as count, sum, or average) instead of returning the raw data to the user. For example, if your data is a sales transaction table, the analytical query might be:</p><ul><li>What was the total revenue of each of our stores in February?</li><li>How many more apples than usual did we sell during our latest promotion?</li><li>Which brand of food is most often purchased together with mustard?</li></ul><p>These queries are usually written by business analysts and fed into reports that help company management make better decisions (business intelligence). In order to distinguish this mode of using a database from transaction processing, it is called <strong><em>online analytical processing</em> </strong>(OLAP)</p><p>At first, the same databases were used for both transaction processing and analytic queries. Facts have proved that SQL is very flexible in this regard: it is applicable to both OLTP-type queries and OLAP-type queries. However, in the late 1980s and early 1990s, there was a trend for companies to stop using their OLTP system for analysis and instead run analysis on a separate database. This separate database is called a data warehouse.</p><h4><strong>Data Warehousing</strong></h4><p>Companies may have dozens of different transaction processing systems: systems that power customer-facing websites, point-of-sale (checkout) systems in physical stores, inventory tracking in warehouses, route planning of vehicles, management of suppliers, employees, etc. Each of these systems is complex and needs a team of people to maintain it, so the systems end up operating mostly autonomously from each other.</p><p>These OLTP systems are generally expected to be highly available and able to process transactions with low latency, as they are often critical to business operations. Therefore, database administrators closely protect their OLTP databases. They are usually reluctant to let business analysts run ad hoc analytical queries on OLTP databases because these queries are usually expensive and scan most of the data set, which may harm the performance of concurrently executing transactions.</p><p>In contrast, a <strong><em>data warehouse</em></strong> is a separate database, and analysts can query its inner content without affecting OLTP operations. The data warehouse contains read-only copies of data in all the company’s various OLTP systems. Extract data from the OLTP database (using periodic data dumps or continuous update streams), transform it into an easy-to-analyze mode, clean it, and then load it into the data warehouse. The process of putting data into the warehouse is called <strong><em>extract-transform-load </em></strong>(ETL).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1021/1*NJzLr0DcOp3Lu1yNR76F5g.jpeg" /><figcaption>Image from Wikipedia</figcaption></figure><p>Now, almost all large enterprises have data warehouses, but in small enterprises, they are almost unheard of. This may be because most small companies do not have many different OLTP systems, and most small companies have a small amount of data. Small enough to be queried in a regular SQL database or even analyzed in a spreadsheet. In a large company, it takes a lot of heavy work to do something that is simple in a small company.</p><p>A big advantage of using a separate data warehouse instead of directly querying the OLTP system for analytics is that the data warehouse can be optimized for analytics access patterns.</p><h4>The difference between OLTP database and data warehouse</h4><p>The data model of a data warehouse is usually relational because SQL is usually very suitable for analytics queries. There are many graphical data analytics tools that can generate SQL queries, visualize the results and allow analysts to browse the data (through operations such as drill down, slice, and dice).</p><p>On the surface, data warehouses and relational OLTP databases look similar because they both have SQL query interfaces. However, the internals of the system may look completely different because they are optimized for very different query patterns. Many database vendors now focus on supporting transaction processing or analytical workloads, but they cannot support both at the same time.</p><p>Certain databases (such as Microsoft SQL Server and SAP HANA) support transaction processing and data warehousing in the same product. However, they are increasingly becoming two independent storage and query engines, which happen to be accessible through a public SQL interface.</p><p>Data warehouse vendors (such as Teradata, Vertica, SAP HANA, and ParAccel) usually sell their systems under expensive commercial licenses. Amazon Redshift is the hosted version of ParAccel. Recently, there have been many open-source SQL-based Hadoop projects. They are young but aim to compete with commercial data warehouse systems. These include Apache Hive, Spark SQL, Cloudera Impala, Facebook Presto, Apache Tajo, and Apache Drill. Some of them are based on the ideas of Google Dremel.</p><h4>Stars and Snowflakes Schemas</h4><p>Many data warehouses are used in a fairly formulaic style, known as a <strong><em>star schema</em></strong> (also known as dimensional modeling).</p><p>The example schema below was modeled using the start schema pattern. At the center of the schema is a so-called <em>fact table</em>. Each row of the fact table represents an event that occurred at a particular time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/528/0*jND8rwhngtv2_21s.png" /><figcaption>Image from guru99.com</figcaption></figure><p>Usually, the facts are captured as a single event, because this allows the greatest flexibility for later analysis. However, this means that the fact table can become very large. Large companies like Apple, Walmart, or eBay may have dozens of petabytes of transaction history in their data warehouse, most of which are actually in fact tables.</p><p>Some of the columns in the fact table are attributes, such as the units sold of the product and the revenue. The other columns in the fact table are foreign key references to other tables (called dimension tables). Since each row in the fact table represents an event, the dimensions represent the person, event, location, time, method, and reason of the event.</p><p>Even dates and times are usually represented by dimension tables because this can encode other information about dates (such as public holidays) so that the query can distinguish sales during holidays and non-holidays.</p><p>The name “star schema” comes from the fact that when visualizing table relationships, the fact table is in the middle and surrounded by its dimension tables; the connection with these tables is like the light of stars.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/987/0*J4ewDDOLiJps-20N.png" /><figcaption>Image from guru99.com</figcaption></figure><p>A variation of this template is called the <strong><em>snowflake</em></strong> pattern, where the size is further subdivided into multiple sub-dimensions. For example, there may be separate tables for brands and product categories, and each row in the dimension product table can reference the brand and category as foreign keys instead of storing them as strings in the table. Snowflake schemas are more normalized than star schemas, but star schemas are often preferred because they are simpler for analysts to work with.</p><p>In a typical data warehouse, tables are usually very wide: fact tables usually have more than 100 columns, sometimes hundreds. The dimension tables may also be very wide because they contain all metadata that may be relevant to the analysis.</p><h4>Final thoughts</h4><p>OLTP systems are usually user-oriented, which means they may see a large number of requests. To handle the load, the application usually touches only a few records in each query. The application uses a certain key to request records, and the storage engine uses an index to find the data of the requested key. Disk seek time is usually the bottleneck here.</p><p>Data warehouses and similar analytics systems are little known because they are mainly used by business analysts rather than end-users. Compared with OLTP systems, they handle much fewer queries, but the requirements for each query are usually high, requiring millions of records to be scanned in a short time. Disk bandwidth (rather than seek time) is usually the bottleneck here.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1118915a0ea7" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/analytics-vs-transactional-processing-1118915a0ea7">Analytics vs Transactional Processing</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Observability]]></title>
            <link>https://levelup.gitconnected.com/kubernetes-observability-425fe60bcdc1?source=rss-8fdaec8f00ec------2</link>
            <guid isPermaLink="false">https://medium.com/p/425fe60bcdc1</guid>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[metrics]]></category>
            <category><![CDATA[logs]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Fabricio Pautasso]]></dc:creator>
            <pubDate>Wed, 24 Feb 2021 17:52:37 GMT</pubDate>
            <atom:updated>2021-02-24T17:52:37.987Z</atom:updated>
            <content:encoded><![CDATA[<h4>Logs and metrics tools</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2EmPTT2DZXy7H93q6xyk0w.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@shanehauser?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Shane Hauser</a> on <a href="https://unsplash.com/s/photos/binoculars?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>The ability to identify and troubleshoot an issue in a production system is critical. In this article, we will talk about a couple of tools that can help us understand what’s going on in a production Kubernetes cluster.</p><h3>Observability</h3><p>Observability is the ability to understand what is happening in a production system. We can deal with low-observability systems (inability to understand what is happening) or high-observability systems (where events and internal states can be inferred from the outside through tools).</p><p>Observability is an attribute of the system itself. Generally, monitoring is the operation of obtaining information about the current or past state of the system. Everything is a naming debate, but you monitor the system to collect the observable parts of it. In most cases, monitoring is easy. There are many great tools that can help us capture and analyze information and present it in various ways. However, the system needs to disclose relevant information so that it can be collected. It is very difficult to disclose the correct amount of information. Too much information will generate a lot of noise, thereby masking the relevant signal. Too little information would not be enough to detect the problem.</p><p>Distributed systems (such as systems that follow a microservice architecture) also have problems because the complexity of the system may make it difficult to understand its internal state. In some cases, the behavior is also unpredictable. Such systems will never be completely healthy in nature; there will be small problems everywhere. You need to develop a priority system to determine which issues require immediate action and which issues can be resolved later.</p><p>The main tools for microservice observability are <strong>logs</strong> and <strong>metrics</strong>. They are well understood and used by the community, and there are many tools that can greatly simplify their use. They can be used as locally installed software packages or as cloud services that help retain data and reduce maintenance costs.</p><p>In Kubernetes, the most important details of the cluster status can be checked through kubectl. This will include detailed information such as deployed version, reboot, pull image, etc. Logs and metrics have different goals, and both can be complex.</p><h4>Logs</h4><p>Logs track the only events that occur in the system. Each log stores a message, which is generated when a specific part of the code is executed. The log can be completely general (call function X) or it can contain specific details (call function X using parameter A). The most common format for logs is to generate them as pure strings. This is very flexible, and usually, tools related to logs can be used for text searches. Each log contains some metadata about who generated the log, when it was created, etc.</p><p>The log also includes the severity level. This allows classification so that we can capture the importance of the message. The severity level can be <em>DEBUG</em>, <em>INFO</em>, <em>WARNING</em>, or <em>ERROR</em> in order of importance. This severity allows us to filter out unimportant logs and determine what actions should be taken. The logging tool can be configured to set thresholds. Less serious logs will be ignored.</p><p>In a web service environment, most logs will be generated as part of the web request-response. This means that the request will reach the system, be processed, and return a value. Several logs will be generated along the way. Remember that in a system under load, multiple requests will occur at the same time, so logs from multiple requests will also be generated at the same time. You can add a common request ID to group all related logs generated for a single request.</p><p>Each individual log may be relatively large, and the total will take up a lot of disk space. Logs may quickly exceed the ratio in a system under load. Different log systems allow us to adjust their retention time, which means we only keep them for a certain amount of time. It is important to find a balance between keeping logs to see what happened in the past and using a reasonable amount of space.</p><p>Some tools allow us to use raw logs to generate aggregate results. They can count the number of occurrences of a particular log and generate an average number of times per minute or other statistics. However, this is expensive because each log takes up space. To observe this aggregation behavior, it is best to use a specific metrics system.</p><h4>Metrics</h4><p>Metrics deals with aggregated data. The information they display is not a single event, but a group. This allows us to check the general status of the cluster in a better way than using logs.</p><p>Where information about each event is kept in the log, metrics reduce the information to the number of times the event has occurred, or reduce it to a value that can be averaged or aggregated in some way. This makes the metrics much lighter than logs and allows us to plot them based on time.</p><p>Metrics display information, such as the number of requests per minute, the average time of a request in one minute, the number of queued requests, the number of errors per minute, and so on. Capturing and analyzing performance-related information, such as average request time, enables us to discover possible bottlenecks and take quick actions to improve system performance. On average, this is easier to handle using metrics than logs because a single request may not capture enough information for us to see the big picture. It also helps us predict future bottlenecks.</p><h3>Kubernetes and Observability</h3><p>Kubernetes is the main platform for deploying and maintaining containers today. As it rises to the top of the industry’s consciousness, new technologies have also emerged to provide effective observability tools.</p><h4>Prometheus</h4><p>When you want to monitor the Kubernetes infrastructure and the services running in the cluster at the same time, <a href="https://prometheus.io/">Prometheus</a> is the best starting point. It provides everything you need, including the client tool library, storage backend, visual UI, and alert framework. Running Prometheus can also provide a large number of infrastructure metrics immediately. Although data exchange is not bidirectional in all cases, it also provides integration with third-party providers for storage.</p><h4>OpenCensus</h4><p><a href="https://opencensus.io/">OpenCensus</a> is an open-source version of the Census library developed by Google. It provides measurement and tracking detection functions, and supports many backends to export metrics to Prometheus. Please note that OpenCensus does not monitor your infrastructure, if you choose to use OpenCensus for custom metric telemetry, you still need to determine the best method.</p><h4>Graphite</h4><p><a href="https://graphiteapp.org/">Graphite</a> originated from Orbitz’s internal development work and is now positioned as a surveillance tool that can be used in enterprises. It provides metrics storage and retrieval mechanism but does not provide a detection function. Therefore, you still need to implement Prometheus or OpenCensus tools to collect metrics.</p><h4>InfluxDB</h4><p><a href="https://www.influxdata.com/get-influxdb/">InfluxDB</a> is another open-source database dedicated to storing and retrieving time-series metrics. Unlike Graphite, InfluxDB is supported by a company called InfluxData, which provides InfluxDB software and a cloud-hosted version called InfluxDB Cloud.</p><h4>fluentd</h4><p>In the Kubernetes ecosystem, fluentd is the de-facto open source standard for collecting logs emitted in the cluster and forwarding them to the designated backend. You can use configuration mapping to modify the behavior of fluentd.</p><h4>Zipkin</h4><p><a href="https://zipkin.io/">Zipkin</a> is a complete distributed tracing solution, including detection, storage, and visualization. It is a set of real tools that have been tested in practice. It has existed for many years and has a strong community of users and developers. It can also be used as a backend for other tool options such as OpenCensus.</p><h4>Jaeger</h4><p><a href="https://www.jaegertracing.io/">Jaeger</a> is another open-source tracking solution that contains all the components you need. This is a new project that is being incubated by the Cloud Native Computing Foundation (CNCF). The final choice of whether to use Zipkin or Jaeger may ultimately depend on your experience with them and support for the language you use to write services.</p><h4>Grafana</h4><p><a href="https://grafana.com/">Grafana</a> is multi-platform open-source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources. It&#39;s currently the de facto standard for open-source visualization.</p><h3>Final thoughts</h3><p>Observability on Kubernetes has many parts and many options for each type of demand. Metrics, logging, and tracing tools provide the basis of information needed to make service decisions.</p><p>Finding the right balance for the metrics in the system is not an easy task and requires time and trial and error. However, four indicators of online services are always important. These are as follows:</p><ul><li>Latency</li><li>Traffic</li><li>Errors</li><li>Saturation</li></ul><p>These metrics can be found in the <a href="https://sre.google/sre-book/monitoring-distributed-systems/#:~:text=The%20four%20golden%20signals%20of,system%2C%20focus%20on%20these%20four.&amp;text=The%20time%20it%20takes%20to%20service%20a%20request.">Google SRE Book</a> as the <em>Four Golden Signals</em> and are recognized as the most important high-level elements for successful monitoring.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=425fe60bcdc1" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/kubernetes-observability-425fe60bcdc1">Kubernetes Observability</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building heavy-duty containers]]></title>
            <link>https://medium.com/data-science/building-heavy-duty-containers-204354a67036?source=rss-8fdaec8f00ec------2</link>
            <guid isPermaLink="false">https://medium.com/p/204354a67036</guid>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[supervisord]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[script]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Fabricio Pautasso]]></dc:creator>
            <pubDate>Wed, 17 Feb 2021 13:18:54 GMT</pubDate>
            <atom:updated>2021-02-17T13:18:54.670Z</atom:updated>
            <content:encoded><![CDATA[<h4>Restart flag, init, and supervisor processes</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X1X59XPEvFMZLxJ3jGiyHA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@patwhelen?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Pat Whelen</a> on <a href="https://unsplash.com/s/photos/heavy-duty?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>There are cases where software fails in rare conditions that are temporary in nature. Although it’s important to be made aware when these conditions arise, it’s usually at least as important to restore the service as quickly as possible.</p><p>When all the processes in a container have exited, that container will enter the exited state. A Docker container can be in one of four states:</p><ul><li>Running</li><li>Paused</li><li>Restarting</li><li>Exited (also used if the container has never been started)</li></ul><p>A basic strategy for recovering from temporary failures is that when a process exits or fails, it will automatically restart. Docker provides some options for monitoring and restarting containers.</p><h4>Restarting containers automatically</h4><p>Docker provides support for this feature through a restart strategy. Using the —- restartflag when the container is created, you can tell Docker to do any of the following:</p><ul><li>Never restart (default)</li><li>Attempt to restart when a failure is detected</li><li>Attempt for some predetermined time to restart when a failure is detected</li><li>Always restart the container regardless of the condition</li></ul><p>Docker does not always try to restart the container immediately. If it did, it will cause more problems than solved. Imagine a container that does nothing but prints the time and then exits. If the container is configured to always restart, and Docker always restarts it immediately, the system will not perform any operation and only restart the container. Instead, Docker uses an exponential backoff strategy to time restart attempts.</p><p>The backoff strategy determines how much time should elapse between successive restart attempts. The exponential backoff strategy will double the time spent waiting for each successive attempt. For example, if the container needs to be restarted the first time, Docker waits for 1 second, then on the second attempt, it will wait 2 seconds, the third attempt will wait 4 seconds, the fourth time will wait 8 seconds, and so on. The exponential backoff strategy with a short initial waiting time is a common service recovery technique. You can see that Docker itself adopted this strategy by building a container that always restarts and only prints the time</p><pre>docker run -d —-name backoff-sample --restart always busybox date</pre><p>Then after a few seconds check out the logs to watch it back off and restart</p><pre>docker logs -f backoff-sample</pre><p>The only reason you might not want to directly adopt this feature is that the container is not running during the backoff period. Containers waiting to be restarted are in a restart state. To demonstrate, try running another process in the backoff sample container</p><pre>docker exec backoff-sample echo testing</pre><p>Running that command should result in an error message</p><pre>Error response from daemon: Container 4affc02f445dd426c0b7daf79efc31f83b1e6e3c7397323bd235a8cd80453bb1 is restarting, wait until the container is running</pre><p>This means you cannot perform any operations that require the container to be running, such as executing other commands in the container. If you need to run diagnostics in a damaged container, that might be a problem. A more complete strategy is to use containers running init or supervisor processes.</p><h4>Supervisor and startup processes</h4><p>The supervisor program or initialization program is a program used to start and maintain the state of other programs. On Linux systems, PID#1 is an initialization process. It starts all other system processes and restarts them if they fail unexpectedly. It is a common practice to use a similar pattern to start and manage processes within the container.</p><p>If the target process (such as a web server) fails and restarts, using a supervisor inside the container will keep the container running. Several programs may be used in the container. The most popular ones include init, systemd, runit, upstart and supervisord.</p><p>There are a lot of Docker images in DockerHub that produce a full LAMP (Linux, Apache, MySQL PHP) stack inside a single container. Containers created this way use supervisord to make sure that all the related processes are kept running. Start an example container</p><pre>docker run -d -p 80:80 --name lamp-test mattrayner/lamp</pre><p>You can see what processes are running inside this container by using the docker topcommand</p><pre>docker top lamp-test</pre><pre>PID                 USER                TIME                COMMAND</pre><pre>5833                root                0:00                {supervisord} /usr/bin/python3 /usr/local/bin/supervisord -n</pre><pre>6462                root                0:00                apache2 -D FOREGROUND</pre><pre>6463                root                0:00                {pidproxy} /usr/bin/python3 /usr/local/bin/pidproxy /var/run/mysqld/mysqld.pid /usr/bin/mysqld_safe</pre><pre>6464                1000                0:00                apache2 -D FOREGROUND</pre><pre>6465                1000                0:00                apache2 -D FOREGROUND</pre><pre>6466                1000                0:00                apache2 -D FOREGROUND</pre><pre>6467                1000                0:00                apache2 -D FOREGROUND</pre><pre>6468                1000                0:00                apache2 -D FOREGROUND</pre><pre>6469                root                0:00                {mysqld_safe} /bin/sh /usr/bin/mysqld_safe</pre><pre>6874                1000                0:00                /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --user=www-data --log-error=/var/log/mysql/error.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/run/mysqld/mysqld.sock --port=3306 --log-syslog=1 --log-syslog-facility=daemon --log-syslog-tag=</pre><p>The top subcommand will display the host PID of each process in the container. You will see supervisord, mysql and apache in the list of running programs. Now that the container is running, you can test the supervisord restart function by manually stopping one of the processes inside the container.</p><p>First, let’s get the PID of each process</p><pre>docker exec lamp-test ps</pre><pre>PID TTY          TIME CMD</pre><pre>1 ?        00:00:00 supervisord</pre><pre>503 ?        00:00:00 apache2</pre><pre>504 ?        00:00:00 pidproxy</pre><pre>510 ?        00:00:00 mysqld_safe</pre><pre>943 ?        00:00:00 ps</pre><p>Let’s kill the apache2 process</p><pre>docker exec lamp-test kill 503</pre><p>Running this command will run the Linux kill program inside the lamp-test container and tell the apache2 process to shut down. When apache2 stops, the supervisord process will log the event and restart the process. The container logs will clearly show these events:</p><pre>docker logs lamp-test</pre><pre>Updating for PHP 7.4</pre><pre>---------</pre><pre>2021-02-16 12:58:42,878 INFO supervisord started with pid 1</pre><pre><strong>2021-02-16 12:58:43,882 INFO spawned: &#39;apache2&#39; with pid 503</strong></pre><pre>2021-02-16 12:58:43,885 INFO spawned: &#39;mysqld&#39; with pid 504</pre><pre>2021-02-16 12:58:45,332 INFO success: apache2 entered RUNNING state, process has stayed up for &gt; than 1 seconds (startsecs)</pre><pre>2021-02-16 12:58:45,332 INFO success: mysqld entered RUNNING state, process has stayed up for &gt; than 1 seconds (startsecs)</pre><pre><strong>2021-02-16 13:08:42,334 INFO exited: apache2 (exit status 0; expected)</strong></pre><pre><strong>2021-02-16 13:08:43,345 INFO spawned: &#39;apache2&#39; with pid 956</strong></pre><pre><strong>2021-02-16 13:08:44,385 INFO success: apache2 entered RUNNING state, process has stayed up for &gt; than 1 seconds (startsecs)</strong></pre><p>A common alternative to using init or supervisor programs is to use a startup script that at least checks the prerequisites for the successful startup of the included software. These are sometimes used as default commands for containers. Docker containers run something called an <em>entrypoint</em> before executing the commands. Entrypoints are perfect places to put code that validates the preconditions of a container.</p><p><a href="https://docs.docker.com/config/containers/multi-service_container/">Startup scripts</a> are an essential part of building durable containers and can always be combined with Docker restart policies to take advantage of the strengths of each.</p><h4>Final cleanup</h4><p>Ease of cleanup is one of the most important reasons to use containers and Docker. The isolation provided by the container simplifies all the steps you need to perform to stop the process and delete files. With Docker, the entire cleaning process is reduced to one of a few simple commands. In any cleanup task, you must first determine which container to stop and/or delete.</p><p>Let’s list all the containers first</p><pre>docker ps -a</pre><p>Since the containers created for the examples in this article will no longer be used, you should be able to safely stop and delete all listed containers. If you created a container for your activity, make sure to pay attention to the container to clean up.</p><p>All containers use hard drive space to store logs, container metadata, and files that have been written to the container file system. All containers also consume resources in the global namespace, such as container name and host port mapping. In most cases, containers that are no longer used should be deleted.</p><p>Let’s delete the lamp-test container:</p><pre>docker rm -f lamp-test</pre><p>As this container was running, if we try to run the previous command without the -f flag Docker will throw an error. Docker containers need to be stopped before trying to remove them. You can do this running docker stop command or instead of running two commands add the <em>force </em>flag to the remove command.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=204354a67036" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/building-heavy-duty-containers-204354a67036">Building heavy-duty containers</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Saga pattern in Microservices]]></title>
            <link>https://levelup.gitconnected.com/the-saga-pattern-in-microservices-fbed0b90bf92?source=rss-8fdaec8f00ec------2</link>
            <guid isPermaLink="false">https://medium.com/p/fbed0b90bf92</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[saga]]></category>
            <category><![CDATA[patterns]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Fabricio Pautasso]]></dc:creator>
            <pubDate>Thu, 11 Feb 2021 05:46:20 GMT</pubDate>
            <atom:updated>2021-02-11T05:46:20.913Z</atom:updated>
            <content:encoded><![CDATA[<h4>Handling distributed transactions</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6c51Eng452hyANIS7Cy9CA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@pawel_czerwinski?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Paweł Czerwiński</a> on <a href="https://unsplash.com/s/photos/saga-pattern?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>When everything is normal, the API composition and CQRS patterns can provide appropriate solutions for distributed queries.</p><p><a href="https://levelup.gitconnected.com/running-distributed-queries-in-micro-services-157b12b93154">Running Distributed Queries in Microservices</a></p><p>However, maintaining the integrity of distributed data is complex. If you store all data in a single relational database and specify the appropriate constraints in the schema, you can rely on the database engine to maintain data integrity.</p><p>The situation is very different when multiple microservices keeps the data in isolated data stores (relational or non-relational). Data integrity is critical, but it must be maintained by code. The saga pattern addresses this concern.</p><h4>ACID</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*CZBEDCA3Gvzsf8TNNQ6E1g.jpeg" /></figure><p>A common measure of data integrity is that all transactions that modify data have the ACID properties:</p><ul><li><strong>Atomic:</strong> All operations in the transaction succeed or all fail.</li><li><strong>Consistent:</strong> The data state meets all constraints before and after the transaction</li><li><strong>Isolated:</strong> Concurrent transactions behave as if they are serialized.</li><li><strong>Durable:</strong> When a transaction completes successfully, the results are persisted.</li></ul><p>ACID attributes are not specific to relational databases but are often used in this context, mainly because the relational schema and its formal constraints provide a convenient measure of consistency. Isolation properties usually severely affect performance and might be relaxed by some systems that prefer high-performance and eventual consistency.</p><h4>The CAP Theorem</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*7w5OpqT1M7YJ9KNbCC2smQ.jpeg" /></figure><p>The CAP theorem states that a distributed system cannot have all the following three properties at the same time:</p><ul><li><strong>Consistency: </strong>means that no matter what path the client requests comes from, all clients will see the same data at the same time. This is essential for frequently updated applications.</li><li><strong>Availability: </strong>means that all functioning application components will return a valid response, even if they are down. This is particularly important if an application’s user population has a low tolerance for outages (such as a retail portal).</li><li><strong>Partition resiliency</strong>: means that the application will operate even during a network failure that results in lost or delayed messages between services. This comes into play for applications that integrate with a large number of distributed, independent components.</li></ul><p>In practice, you need to choose whether you implement a CP system or an AP system. The CP system (consistent and partition resilient) is always consistent, and if there is a network partition between components, the CP system will not provide queries or make changes. It can only run after the system is fully connected. Obviously, this means you don’t have availability.</p><p>On the other hand, the AP system (available and with partition resilient) is always available and can operate in a split-brain manner. When the system is split, each part may continue to operate normally, but the system will become inconsistent because each part does not know what is happening in the other parts.</p><p>AP systems are often referred to as eventually consistent systems because when the connection is restored, certain coordination procedures can ensure that the entire system is synchronized again. An interesting variant is a frozen system, where when a network partition occurs, they degrade normally, and both parts continue to process queries, but refuse all modifications to the system. Please note that since some transactions in the partition may still not be replicated to another part, there is no guarantee that the two parts are consistent during the partition. Generally, it is good enough because the difference between the split parts is small and does not increase over time due to the rejection of new changes.</p><h4>The Saga Pattern</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/779/0*5TmAgQqwZK62HlDt.png" /><figcaption>Image from microservices.io</figcaption></figure><p>The basic idea of ​​the saga pattern is to centrally manage the operations of all microservices, and for each operation, if the entire transaction cannot be completed for some reason, a compensation operation will be performed. This achieves the atomicity of ACID. However, changes on each microservice are immediately visible, not only at the end of the entire distributed transaction. This violates the consistency and isolation properties. This is not a problem if you design your system as AP (eventually consistent). However, it requires your code to be aware of this and be able to use data that may be partially inconsistent or out of date. In many cases, this is an acceptable compromise.</p><p>How does the Saga Pattern work? Saga is a set of operations on microservices and corresponding compensation operations. When an operation fails, its compensation operation and all previous compensation operations will be called in the reverse order to roll back the entire state of the system.</p><p>The implementation of Sagas is not easy, because the compensation operation may also fail. Generally, the transition state must be persistent and must be marked as such, and a large amount of metadata must be stored to achieve a reliable rollback. A good practice is to frequently run out-of-band processes and clean up faulty Sagas that cannot complete all of its compensation operations in real-time. A good way to think about Sagas is as a workflow. Workflows are cool because they can implement long processes that even involve people and not just software.</p><p>There are two types of Sagas:</p><ul><li><strong>Orchestration-Based Saga:</strong> In this approach, there is a Saga orchestrator that manages all the transactions and directs the participant services to execute local transactions based on events. This orchestrator can also be thought of as a Saga Manager.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*l6Rs8BIorqEV-b1L.png" /><figcaption>Image from microservices.io</figcaption></figure><ul><li><strong>Choreography-Based Saga: </strong>In this approach, there is no central orchestrator. Each service participating in the Saga performs their transaction and publish events. The other services act upon those events and perform their transactions. Also, they may or not publish other events based on the situation.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1018/0*NLPXcFzSYDlgerpy.png" /><figcaption>Image from microservices.io</figcaption></figure><h4>Conclusion</h4><p>The main advantage of the Saga pattern is that it can help maintain data consistency between multiple services without tight coupling. This is an extremely important aspect of microservice architecture.</p><p>However, the main disadvantage of the Saga pattern is the obvious complexity from a programming perspective. Another challenge is that compensation transactions must also be designed to make Sagas work properly.</p><p>Choose your database wisely. Then, design your microservice workflow and framework to ensure that you don’t undermine your goals</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fbed0b90bf92" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/the-saga-pattern-in-microservices-fbed0b90bf92">The Saga pattern in Microservices</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes is deprecating Docker in 2021]]></title>
            <link>https://levelup.gitconnected.com/kubernetes-is-deprecating-docker-in-2021-fa8317f9f070?source=rss-8fdaec8f00ec------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa8317f9f070</guid>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Fabricio Pautasso]]></dc:creator>
            <pubDate>Fri, 05 Feb 2021 00:14:57 GMT</pubDate>
            <atom:updated>2021-02-05T00:14:57.031Z</atom:updated>
            <content:encoded><![CDATA[<h4>Is the end of Docker?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XF4kX2toI9_AiC_W1ZnXLA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@usmanyousaf?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Usman Yousaf</a> on <a href="https://unsplash.com/s/photos/panic?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Kubernetes is deprecating Docker as a container runtime after version <a href="https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.20.md#deprecation">1.20</a>.</p><p>Docker as an underlying runtime is being deprecated in favor of runtimes that use the <a href="https://kubernetes.io/blog/2016/12/container-runtime-interface-cri-in-kubernetes/">Container Runtime Interface (CRI)</a> created for Kubernetes.</p><p>If you are an end-user of Kubernetes, it will not bring you much change. This does not mean the demise of Docker, nor does it mean that you cannot or should no longer use Docker as a development tool. Docker is still a useful tool for building containers, and the images produced by running docker build can still be run in your Kubernetes cluster.</p><p>In case you are using a managed Kubernetes service, such as GKE, EKS, or AKS (containerized by default), you need to ensure that your worker nodes are using a supported container runtime before removing Docker support in future Kubernetes releases. If you have node customizations, you may need to update them based on your environment and runtime requirements.</p><p>If you want to roll your own cluster, you also need to make changes to avoid cluster breakdown. In v1.20, you will receive a Docker deprecation warning. If Docker runtime support is removed in a future Kubernetes version (<strong>currently planned to be version 1.22 in the second half of 2021</strong>), it will no longer be supported and you will need to switch to one of the other compatible container runtimes, such as containerd or CRI -O. Just make sure that the runtime you choose supports the configuration of the docker daemon currently in use (such as logging).</p><h4>So, why people are panicking when they see this announcement?</h4><p>Two environments are being discussed, and that’s creating confusion. Inside a Kubernetes cluster, there’s a thing called a container runtime that’s responsible for pulling and running container images. Docker is a popular choice for that runtime, but Docker was not designed to be embedded inside Kubernetes, and that causes problems.</p><p><em>Docker</em> is an entire tech stack, and one part of it is a thing called “containerd” which is a high-level container runtime by itself. Docker is cool and useful because it has many UX enhancements that make it really easy to interact with people when doing development work, but for Kubernetes, these UX enhancements are not necessary because it is not human.</p><p>Because of this user-friendly abstraction layer, Kubernetes must use another tool called Dockershim to get what it really needs, which is containerd. That’s not great, because it provides another thing that must be maintained and may be damaged. In fact, what is happening here is that Dockershim will be removed from Kubelet as early as the v1.23 release, thereby removing support for Docker as a container runtime.</p><h4>Why Dockershim is needed?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4H3hV0Y_ov9mMUFwKey4Mg.png" /></figure><p>Docker isn’t compliant with CRI, the <a href="https://kubernetes.io/blog/2016/12/container-runtime-interface-cri-in-kubernetes/">Container Runtime Interface</a>. If it were, we wouldn’t need the shim, and this wouldn’t be a thing.</p><p>Dockershim was a temporary solution(hence the name: shim) proposed by the Kubernetes community to increase support for Docker so that it can be used as its container runtime. The deprecation of Dockershim only means that the code maintenance of Dockershim in the Kubernetes code base will be stopped.</p><p>This is because Dockershim has become a heavy burden for Kubernetes maintainers. With this change, the Kubernetes community will only be able to maintain the Kubernetes Container Runtime Interface (CRI).</p><h4>Do we keep using Dockerfiles?</h4><p>This change addresses a different environment than most developers use to interact with Docker. The Docker installation that is used in development is unrelated to the Docker runtime inside the Kubernetes cluster.</p><p>As a developer, Docker is still useful in all the ways it was before this change was announced. The image that Docker produces isn’t really a Docker-specific image — it’s an OCI (<a href="https://opencontainers.org/">Open Container Initiative</a>) image. Any OCI-compliant image, regardless of the tool you use to build it, will look the same to Kubernetes. Both <a href="https://containerd.io/">containerd</a> and <a href="https://cri-o.io/">CRI-O</a> know how to pull those images and run them.</p><h4><strong>What should we verify when changing CRI implementation?</strong></h4><p>Although the underlying containerization code is the same between Docker and most CRIs, there are some differences at the edge. Some common things to consider when migrating are:</p><ul><li>Logging configuration</li><li>Runtime resource limitations</li><li>Node provisioning scripts that call docker or use docker via it’s control socket</li><li>Kubectl plugins that require docker CLI or the control socket</li><li>Kubernetes tools that require direct access to Docker (e.g. kube-imagepuller)</li><li>Configuration of functionality like registry-mirrors and insecure registries</li><li>Other support scripts or daemons that expect Docker to be available and are run outside of Kubernetes (e.g. monitoring or security agents)</li><li>GPUs or special hardware and how they integrate with your runtime and Kubernetes</li></ul><h4>Conclusion</h4><p>Therefore, this change is coming. It can cause problems for some people, but it is not catastrophic, generally, it is a good thing. Depending on how you interact with Kubernetes, this may not make any sense to you, or it may mean some work. In the long run, it will make things easier.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa8317f9f070" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/kubernetes-is-deprecating-docker-in-2021-fa8317f9f070">Kubernetes is deprecating Docker in 2021</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>