<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aaron Zhuo on Medium]]></title>
        <description><![CDATA[Stories by Aaron Zhuo on Medium]]></description>
        <link>https://medium.com/@aaronzhuo?source=rss-9175e0dbe19d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*f4XHaJOIyiBpaota.jpg</url>
            <title>Stories by Aaron Zhuo on Medium</title>
            <link>https://medium.com/@aaronzhuo?source=rss-9175e0dbe19d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 12:34:33 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@aaronzhuo/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Interfaces in Go]]></title>
            <link>https://aaronzhuo.medium.com/interfaces-in-go-3861ec1b75ae?source=rss-9175e0dbe19d------2</link>
            <guid isPermaLink="false">https://medium.com/p/3861ec1b75ae</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[design-patterns]]></category>
            <category><![CDATA[go]]></category>
            <dc:creator><![CDATA[Aaron Zhuo]]></dc:creator>
            <pubDate>Mon, 07 Dec 2020 04:48:32 GMT</pubDate>
            <atom:updated>2020-12-07T22:44:39.869Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*0aMKg-0bcH95UY0h" /></figure><h3>Preface</h3><p>Interfaces is a very important feature in Go. It is a key to implement polymorphism and dependency injection in Go. In spite of its importance, there are some serious arguments about the best practice of Go interfaces, which may lead to another <a href="https://www.ling.upenn.edu/courses/Spring_2003/ling538/Lecnotes/ADfn1.htm">little-endian v.s. big-endian war</a> in the future. The purpose of this blog to is to discuss these arguments in a peaceful way and find a way to avoid this potential war (if possible).</p><h3>Introduction to Go Interfaces</h3><p>The following shows an example of Go interface:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/926a44305d2496b7d894d4737023d68e/href">https://medium.com/media/926a44305d2496b7d894d4737023d68e/href</a></iframe><p>From the above example, you can see that:</p><ul><li>A Go interface defines one or more methods.</li><li>Unlike other languages, you don’t have to explicitly declare that a type implements an interface. A struct S is a thing which is defined by an interface I as long as the struct S implements all the methods defined by the interface I. In this example,FlyWithWings and FlyWithSuperPower are both FlyBehaviour as they all implement the methods defined in the interface FlyBehaviour.</li></ul><h3>Best Practice (a.k.a. Arguments)</h3><p>The concept and usage of Go interfaces is simple. However, there are some arguments regarding its best practice. Now let us discuss those arguments and find some agreement regarding the best practice of Go interfaces.</p><h4><strong>“The smaller the interface，the stronger the abstraction”</strong></h4><p>There is no argument about this best practice as this is basically another expression of <a href="https://en.wikipedia.org/wiki/Interface_segregation_principle">Interface Segregation Principle</a>.</p><p>The following shows an example that follows this principle, in which the interface user.Manager is split into multiple smaller interfaces.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/14dca44e7ced3ac71c226c9f44970863/href">https://medium.com/media/14dca44e7ced3ac71c226c9f44970863/href</a></iframe><h4><strong>&quot;Accept interfaces but return structs”</strong></h4><p>We all agree on accepting interfaces other than concrete implementations as this is the key to follow <a href="https://en.wikipedia.org/wiki/SOLID">SOLID principles</a> (e.g. <a href="https://en.wikipedia.org/wiki/Dependency_inversion_principle">Dependency Inversion Principle</a>) and realize some design patterns (e.g. <a href="https://en.wikipedia.org/wiki/Decorator_pattern">Decorator Pattern</a>) in Go. However, there is no solid answer to returning interfaces (abstraction) or structs (implementation).</p><p>The benefit of returning structs is to give consumers the freedom to define interfaces on their side. Take the above interface user.ManagerInterface as an example, consumers can define this interface based on the struct user.Manager (the implementation of user manager) and their need in their own packages.</p><p>The benefit of returning interfaces is to free consumers from defining interfaces on their own, as long as the returning interfaces provide strong abstraction. Moreover, some design patterns, such as <a href="https://en.wikipedia.org/wiki/Factory_method_pattern">Factory Method Pattern</a>, require to return interfaces other than concrete implementation. For example, the method <a href="https://github.com/golang/go/blob/dcd3b2c173b77d93be1c391e3b5f932e0779fb1f/src/crypto/aes/cipher_asm.go#L33-L54">aes.newCipher</a> returns the interface <a href="https://github.com/golang/go/blob/dcd3b2c173b77d93be1c391e3b5f932e0779fb1f/src/crypto/cipher/cipher.go#L15">cipher.Block</a> with different structs in different circumstances:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4c1081574dc77758f4ecd6ebd02a7140/href">https://medium.com/media/4c1081574dc77758f4ecd6ebd02a7140/href</a></iframe><p>Normally, the smaller the interface，the stronger the abstraction. The stronger the abstraction, the more acceptable it is to return interfaces.</p><h4><strong>“Go interfaces generally belong in the package that uses values of the interface type, not the package that implements those values”</strong></h4><p>The above sentence comes from <a href="https://github.com/golang/go/wiki/CodeReviewComments#interfaces">Go Code Review Comments, Interface Section</a>, which totally conflicts with what it is said in <a href="https://golang.org/doc/effective_go.html#generality">Effective Go</a>:</p><p>“If a type exists only to implement an interface and will never have exported methods beyond that interface, there is no need to export the type itself. Exporting just the interface makes it clear the value has no interesting behavior beyond what is described in the interface.”</p><p>So where should we define interfaces? Consumer side or producer side? I think this depends on whether the interface that you define has good abstraction or not. If an interface provides very strong abstraction, then it makes sense to put it in an individual shared package. The interface io.Reader in the <a href="https://github.com/golang/go/blob/c170b14c2c1cfb2fd853a37add92a82fd6eb4318/src/io/io.go#L77-L92">package io</a> is a perfect example of interfaces with strong abstraction. It only has one method Read and everyone agrees what this method should look like. Therefore, everyone is fine with putting the interface Reader in the package io, which is a producer that provides implementations of this interface.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0fa2a9a898e6638387410ec45486b0ac/href">https://medium.com/media/0fa2a9a898e6638387410ec45486b0ac/href</a></iframe><p>The opposite example of putting Go interfaces on the consumer side is <a href="https://github.com/google/go-github">Go client of Github</a>. It is apparently not a good idea to define a github.Client interface as a Github client needs to provide a lot of methods, which leads to weak abstraction for making a github.Client interface. In this case, it makes more sense for consumers to define their own github.Client interfaces, which may vary from consumer to consumer and may only have very few methods.</p><h4><strong>“It can be useful to define a default abstraction on the producer side”</strong></h4><p>It is easy to determine where to put an interface when the interface provides either very strong or very week abstraction. But we are not always that lucky in reality. Take the above interface user.ManagerInterface as an example, it makes sense to split it into multiple smaller interfaces. But do these interfaces provide strong abstraction so that it is acceptable to define them at the producer side (the package user)? Or should only this producer provide the concrete implementation of user manager?</p><p>Suppose the package user is an internal package for a project and it has five consumers (five packages are using this package). The interface user.ManagerInterface needs to be defined five times in these packages for programming to interfaces, if there is no default interface user.ManagerInterface and it is very likely these packages may define the same interface user.ManagerInterface. Therefore, it can be very useful for the package user (producer) to provide a default interface and its mocks in this case. Here is the pseudo-code:</p><p>From the above example, you can see that:</p><ul><li>The implementation user.Manager is exposed and returned, which gives consumers the freedom to define the abstraction of user.ManagerInterface base on this implementation.</li><li>The package user provides a default abstraction of user.Manager and its mocks through the interface user.ManagerInterface and the struct user.ManagerMock, which frees some consumers from defining the abstraction ofuser.Manager and its mocks.</li><li>This pattern is a trade-off of the argument about where to define interfaces. That is, provide default abstraction on the producer side but also give consumers the freedom to define their own abstraction. And most importantly, let consumers decide what they want to use.</li></ul><h3>Summary</h3><p>This blog talks about several aspects of interfaces, including the size of interfaces, where to define interfaces, return structs or interfaces, and the benefit of providing a default abstraction for an implementation. Like uncertainty of life, there is no solid answer to these arguments/questions. But I do hope this blog help you clean up some puzzle about Golang interfaces. And most importantly, may the world piece forever.</p><h3>Reference</h3><ul><li><a href="https://en.wikipedia.org/wiki/Interface_segregation_principle">Interface Segregation Principle</a></li><li><a href="https://en.wikipedia.org/wiki/SOLID">SOLID principles</a></li><li><a href="https://en.wikipedia.org/wiki/Dependency_inversion_principle">Dependency Inversion Principle</a></li><li><a href="https://en.wikipedia.org/wiki/Decorator_pattern">Decorator Pattern</a></li><li><a href="https://en.wikipedia.org/wiki/Factory_method_pattern">Factory Method Pattern</a></li><li><a href="https://github.com/golang/go/wiki/CodeReviewComments#interfaces">Go Code Review Comments, Interface Section</a></li><li><a href="https://golang.org/doc/effective_go.html#generality">Effective Go</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3861ec1b75ae" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Encapsulation in Go]]></title>
            <link>https://aaronzhuo.medium.com/encapsulation-in-go-328c44dcb362?source=rss-9175e0dbe19d------2</link>
            <guid isPermaLink="false">https://medium.com/p/328c44dcb362</guid>
            <category><![CDATA[mídium]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[encapsulation]]></category>
            <dc:creator><![CDATA[Aaron Zhuo]]></dc:creator>
            <pubDate>Wed, 02 Dec 2020 04:09:40 GMT</pubDate>
            <atom:updated>2020-12-07T05:06:51.222Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/669/0*f94CUADnkA2nPi_b" /></figure><h3>Preface</h3><p>Encapsulation, as known as information hiding, is a key aspect of object-oriented programming. An object’s field or method is said to be encapsulated if it is inaccessible to users of the object. Unlike classical objected programming languages like Java, Go has very specific encapsulation rules. This blog is going to explore these “interesting” rules.</p><h3><strong>Encapsulation Rules in Go</strong></h3><p>Go has only one rule to set up encapsulation: capitalized identifiers are exported from the package where they are defined and un-capitalized ones are not. <strong>A field/method of a struct/interface is exported only when the names of the field/method and struct/interface are both capitalized (AND condition).</strong></p><p>Let’s go through an example to discuss the difference between Java and Go in terms of encapsulation.</p><p>Suppose we want to define a simple counter (without consideration of race condition), we can realize it in Go and Java in the following way:</p><p>Go:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/147cf5dc911fa280abf2fa4f0719b903/href">https://medium.com/media/147cf5dc911fa280abf2fa4f0719b903/href</a></iframe><p>Java:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dfd92e117c03803084ea9a48910faa39/href">https://medium.com/media/dfd92e117c03803084ea9a48910faa39/href</a></iframe><p>In Java, the field counter of the class Counter can only be accessed within the class due to the private directive. Clients of the classCounter can only access its public methods and fields. <strong>In Go, the field</strong><strong>counter of the struct</strong><strong>Counter can be directly accessed by other structs or functions that are defined in the same package</strong>, like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f7db8448a32cbfd01a1cdc98832662b0/href">https://medium.com/media/f7db8448a32cbfd01a1cdc98832662b0/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*WOW1A1oQTAmhfqeE" /></figure><p><strong>From the above example, you can see: unlike Java or other object-oriented programming languages that control the visibility of names on the class level, Go controls encapsulation at the package level.</strong> The workaround to fix this is to define an interfaceCounter and use it to replace usage of structSimpleCounter. In this way, only the struct SimpleCounter can access its private fields and methods (see the following example). In other words, Go interface can help you achieve information hiding on the interface/struct level.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a503ceaf041089ed12511d4f8d7a2649/href">https://medium.com/media/a503ceaf041089ed12511d4f8d7a2649/href</a></iframe><h3><strong>Encapsulation in Internal Packages</strong></h3><p>With the above encapsulation rules, <a href="https://docs.google.com/document/d/1e8kOo3r51b2BWtTs_1uADIA5djfXhPT36s6eHVRIvaU/edit">Go internal packages</a> have an extra rule: “An import of a path containing the element “internal” is disallowed if the importing code is outside the tree rooted at the parent of the “internal” directory.” — <a href="https://docs.google.com/document/d/1e8kOo3r51b2BWtTs_1uADIA5djfXhPT36s6eHVRIvaU/edit">Design Document of Go Internal Package</a></p><p>Here is an example:</p><pre>foo:    -&gt; repo<br>   cmd:<br>       server:<br>           main.go<br>   internal:<br>       module:<br>           module1:<br>               service:<br>                   service.go<br>                   internal:<br>                       repo:<br>                           repo.go<br>       pkg:<br>           pkg1:<br>               code.go<br>           pkg2:<br>               code.go<br>   pkg:<br>       pkg1:<br>           code.go</pre><p>In this case:</p><ul><li>Packages defined in the directoryfoo/internal/ can be imported by packages defined in a directory rooted at foo/ no matter how deep the directory layout is. For example, the packagefoo/cmd/server can import the foo/internal/pkg/pkg2 package.</li><li>The deepest internal dominates encapsulation rules when there are multiple internals in a package&#39;s import path. For example, the packagefoo/internal/module1/service/internal/repo can only be imported by packages in the directory tree rooted at foo/internal/module1/service/ (other than foo/), which is only the package foo/internal/module1/service in this case.</li></ul><h4>When to Use Internal Packages</h4><p>When to use internal packages? We only need to remember one rule: Define a package in the internal folder when you want it to be shared only among packages rooted at the parent of the &quot;internal&quot; directory. Take the above project layout of foo project (microservice) as an example, there are two typical use cases of internal packages:</p><ol><li>Define a project’s internal packages: the directoryfoo/internal/ in the above example saves all the packages that can only be used in this project. This is because the directoryfoo/internal is rooted in the root folder of the projectfoo and all the packages defined in this project are also rooted at the root folder of this project. Therefore, any package defined in the project foo can access packages defined in the folderfoo/internal.</li><li>Define a package’s exclusive packages: the packagefoo/internal/module1/service/internal/repo can only be used by the package foo/internal/module1/service according to rules of internal packages and this makes sense in terms of domain-driven design. Normally a domain-driven &quot;module&quot; consists of three packages:api-server, service and repository and only service can access repository. In other words, the packageservice should own the repository package regarding design pattern and code organization. Therefore, in the example, it makes sense to define the packagerepository under the internal folder of the package foo/internal/module1/service.</li></ol><h3><strong>Some Interesting Study cases</strong></h3><p>The mystery of Go encapsulation rules sometimes can make you confused and lost. For example, here are some “interesting” examples of Go encapsulation.</p><h4>Public Data Structs with Private Fields.</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6482c61a683ec0218f76e75070cda137/href">https://medium.com/media/6482c61a683ec0218f76e75070cda137/href</a></iframe><p>In this example, CreateUserRequest struct allows UserRepo to control what to expose to users: When creating a user, a caller uses public fields CreateUserRequest struct to pass exposed parameters while UserRepo uses private fields of CreateUserRequest struct to set up internal parameters. This prevents callers from setting some metadata that is exclusively controlled by UserRepo.</p><h4>Private Interface with Private Methods.</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d298bd8ffe472d60cc4e2feb37afcbc6/href">https://medium.com/media/d298bd8ffe472d60cc4e2feb37afcbc6/href</a></iframe><p>You can define a private interface with some private methods for the purpose of <strong>dependency injection</strong> other than <strong>abstraction</strong>. For example, the interface helper in the above example makes the method defaultHelper.doSomething method replace-able by the method helpMock.doSomething.</p><p>Should a private interface own some public methods? No, it SHOULD NOT as public methods in a private interface never get a chance to be exported.</p><h4>Private Data Structs with Public Fields.</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/65cd26c26c77d2d3cd14f61321688762/href">https://medium.com/media/65cd26c26c77d2d3cd14f61321688762/href</a></iframe><p>A private data struct with public fields means it can only be used within the package where it is defined and those public fields are for marshal/unmarshal purposes. A field must be capitalized if it wants to be marshaled/unmarshalled.</p><h4>Private Object Structs with Public Methods.</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/66810315f95de04487d598f1396f0bf9/href">https://medium.com/media/66810315f95de04487d598f1396f0bf9/href</a></iframe><p>A private object struct with public methods means it implements a public interface and has no interest in exposing itself. This follows the <a href="https://golang.org/doc/effective_go.html#generality">Generality principle</a> defined in <a href="https://golang.org/doc/effective_go.html">Effective Go</a>: “If a type exists only to implement an interface and will never have exported methods beyond that interface, there is no need to export the type itself. Exporting just the interface makes it clear the value has no interesting behavior beyond what is described in the interface. It also avoids the need to repeat the documentation on every instance of a common method.”</p><p>The above code is copied from the built-in <a href="https://golang.org/pkg/io/">Go io package</a>. You can see that the multiReader struct only exposes the interfaceReader.</p><h3><strong>Summary</strong></h3><p>In summary, Go has the following encapsulation rules:</p><ul><li>Go controls the visibility of names at the package level. A field/method of a struct/interface is exported only when the names of the field/method and struct/interface are both capitalized (AND condition).</li><li>An import of a path containing the element “internal” is disallowed if the importing code is outside the tree rooted at the parent of the “internal” directory.</li><li>A field must be capitalized if it wants to be JSON marshal/unmarshal, no matter whether the struct it belongs to is capitalized or not.</li></ul><h3><strong>Reference</strong></h3><ul><li><a href="https://github.com/KeKe-Li/book/blob/master/Go/The.Go.Programming.Language.pdf">The Go Programming Language, Chapter 6, Section 6: Encapsulation</a></li><li><a href="https://docs.google.com/document/d/1e8kOo3r51b2BWtTs_1uADIA5djfXhPT36s6eHVRIvaU/edit">Design Document of Go Internal Package</a></li><li><a href="https://golang.org/doc/effective_go.html">Effective Go</a></li><li><a href="https://golang.org/doc/effective_go.html#generality">Generality Principle About Interface and Encapsulation</a></li><li><a href="https://golang.org/pkg/io/">Go io package</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=328c44dcb362" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Go Modules]]></title>
            <link>https://aaronzhuo.medium.com/go-modules-d51fa19cf715?source=rss-9175e0dbe19d------2</link>
            <guid isPermaLink="false">https://medium.com/p/d51fa19cf715</guid>
            <category><![CDATA[mídium]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[go-modules]]></category>
            <dc:creator><![CDATA[Aaron Zhuo]]></dc:creator>
            <pubDate>Wed, 02 Dec 2020 03:28:12 GMT</pubDate>
            <atom:updated>2020-12-02T03:28:12.524Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QixU2nJcoYZNCo6JLsvzKQ.png" /></figure><h3>Preface</h3><p>What is included in this blog:</p><ul><li>A brief introduction of Go modules and Semantic Import Versioning</li><li>A discussion about how to convert multiple Go libraries in the same repository to Go modules</li><li>A discussion about how to utilize Go Modules in microservices</li></ul><h3><strong>prerequisites</strong></h3><h4>Go Modules</h4><p><a href="https://blog.golang.org/modules2019">Go Modules</a> is an experimental opt-in feature in Go 1.11 with the plan of finalizing feature for Go 1.13. The definition of a Go module from <a href="https://go.googlesource.com/proposal/+/master/design/24301-versioned-go.md">this proposal</a> is “a group of packages that share a common prefix, the module path, and are versioned together as a single unit”. It is designed for resolving <a href="https://en.wikipedia.org/wiki/Dependency_hell">dependency hell</a> problems in Go, like conflicting dependencies and diamond dependency.</p><h4><strong>An Example</strong></h4><p>Here is an example of Go Modules:</p><pre>path/to/my-repo:<br>  bar:<br>      go.mod<br>      bar-file1.go<br>      bar-file2.go<br>      foo:<br>          foo-file1.go<br>          foo-file2.go<br>  mixi:<br>      go.mod<br>      mixi-file1.go<br>      mixi-file2.go</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lik0lmnNwW9gtHcA.png" /></figure><p>As shown in the picture, the repository my-repo has two modules bar and mixi. Take the module bar as an example, it contains two packages: the package bar and the packagefoo. <strong>The file </strong><strong>go.mod under the directory </strong><strong>path/to/my-repo/bar defines the module&#39;s path and its dependencies:</strong></p><pre>module path/to/my-repo/bar<br><br>require (<br> golang.org/x/text v0.3.0<br> rsc.io/sampler v1.99.99<br> // Other dependencies<br>)</pre><p>The filego.mod bundles the package bar and the packagefoo together as a unit. For example, the import statement in the following code will import the module path/to/my-repo/bar (which includes the package foo) rather than the package path/to/my-repo/bar/foo when Go Modules is enabled. <strong>Even though the code looks the same, the path in the import statement is recognized as the module path, not the package path, once Go Modules is used</strong></p><pre>import &quot;path/to/my-repo/bar/foo&quot;<br><br>func main () {<br>  foo.DoSomething()<br>}</pre><h4><strong>How to Enable Go Modules</strong></h4><p>In order to use Go Modules, you need to upgrade your Go to v1.11 or any later version and set the environment variable export GO111MODULE=on.</p><h4><strong>When to Use Go Modules</strong></h4><p><strong>The major purpose of Go Modules is to let one or more packages be versioned, released, and retrieved together as a single unit. Therefore, the public packages, for example, Go libraries and SDKs, are major targets of Go Modules as they need to be published properly for public use.</strong> You do not need to convert internal packages or any internal-used-only packages within a microservice repository to Go modules. These packages can directly import and use modules once Go Modules feature is enabled, even if they are not converted to modules.</p><h4>Semantic Import Versioning</h4><p><a href="https://github.com/azhuox/blogs/blob/master/golang/semantic_import_versioning/blog.md">Semantic Import Versioning</a> is a method proposed for adopting <a href="https://semver.org/">Semantic Versioning</a> in Go packages and modules. The idea behind it is embedding the major version (say v2) in the package path (for packages) or the module path (for modules) with the following rules:</p><ul><li>v1 must be omitted from the module path. <a href="https://github.com/golang/go/issues/24301#issuecomment-371228664">This post</a> explains the reason. You may need to follow this rule in your packages if you are thinking of converting your packages to modules one day.</li><li>The Major versions which are higher than v1 must be embedded in the package path or the module path so that Semantic Versioning can be applied to Go packages and modules.</li></ul><p>The following picture demonstrates the rules above:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*URL-uxjr49Tp4KLI.png" /></figure><h4><strong>Releasing</strong></h4><p><strong>With Go Modules and Semantic Import Versioning, you can release your modules by creating git tags. A tag corresponds to a version.</strong> For example, the following git command releases the bar module v2.3.3:</p><pre>git tag bar/v2.3.3 &amp;&amp; git push -q origin master bar/v2.3.3</pre><p>You can read <a href="https://github.com/azhuox/blogs/blob/master/golang/semantic_import_versioning/blog.md">my last blog</a> for more details about how to releases modules with Semantic Import Versioning.</p><p><strong>All in all, Go Modules provides a way to group one or more packages as a single retrievable unit, while Semantic Import Versioning is a method for applying Semantic Versioning in Go packages and modules to make them versioned. These two things are designed for breaking a repository into multiple retrievable units (modules) so that Go can grab dependencies at the module granularity rather than the repository granularity.</strong></p><h3><strong>Utilizing Go Modules</strong></h3><h4>General Guide of Converting Go Packages to Go Modules</h4><p>I wrote <a href="https://github.com/azhuox/blogs/tree/master/golang/go_modules/example/module">a dummy package</a> called module for demonstrating how to convert one or more Go packages to a Go module.</p><h4><strong>Converting</strong></h4><p>It is very easy to convert one or more Go packages to a Go module. Take the module package as an example, here are the steps of converting it to a Go module:</p><ol><li>Cd to the root directory of the module package: cd path/to/module</li><li>Convert the package to a module: go mod init github.com/azhuox/blogs/golang/go_modules/example/module</li><li>Compile the module and its dependencies: go build</li><li>Commit the changes automatically generated by Go: git add ./go.mod ./go.sum &amp;&amp; git commit -q -m &quot;Convert the package to a module&quot; &amp;&amp; git push origin master -q</li><li>(Optional) you can run go mod vendor to reset the module&#39;s vendor directory to include all the packages and modules which are required for building and testing all of the module&#39;s packages. This is the way to provide dependencies for the older versions of Go that do not fully understand Go modules. Any version of Go &gt;= v1.11 does not need this.</li></ol><p>Here are the contents of the go.mod file automatically generated by Go. You can see that it defines the module&#39;s path, glues anything under the path/to/example/module directory as a single unit and lists all of its dependencies.</p><pre>module github.com/azhuox/blogs/golang/go_modules/example/module<br><br>go 1.12<br><br>require (<br> golang.org/x/net v0.0.0-20190328230028-74de082e2cca<br> rsc.io/quote v1.5.2<br>)</pre><p>Go utilizes the following roles to grab the module’s dependencies:</p><ol><li><strong>It grabs the latest version for the packages that have been converted to modules. For example, </strong><strong>rsc.io/quote v1.5.2.</strong></li><li><strong>It grabs the latest commit for the packages that have not been converted to modules with the format </strong><strong>v0.0.0-{date}-{first_12_characters_of_commit_id}. For example, </strong><strong>golang.org/x/net v0.0.0-20190328230028-74de082e2cca.</strong></li></ol><h4><strong>Releasing</strong></h4><p>A module can only be used as a module after it is released. A module is released by creating git tags and each tag corresponds to a version. However, there are two problems we need to solve before releasing a module.</p><p>The first problem is how to release v2 or higher Major versions. Go utilizes two methods, Major Branch and Major Subdirectory, which are provided by <a href="https://research.swtch.com/vgo-module#from_repository_to_modules">this proposal</a> to solve this problem. <a href="https://blog.golang.org/v2-go-modules">This blog</a> demonstrates these two methods and compares their advantages and disadvantages. In this blog, Major Subdirectory is used for all the examples as it does not require to duplicate any code.</p><p>The second problem is we need to figure out whether to consider the conversion from Go package(s) to a Go module a breaking change or not. If so, we need to upgrade the Major version using <a href="https://semver.org/">Semantic Versioning</a>. If not, we need to decide what versions we need to release. I prefer to just release the latest version of the package(s) listed in the CHANGELOG.md file for the following reasons:</p><ol><li><strong>The conversion from Go package(s) to a Go module is not a breaking change as the package(s) can still work with older versions of Go even if the package(s) are converted to a module. So it does not make sense to upgrade the Major version for this kind of change.</strong></li><li><strong>The conversion from Go package(s) to a Go module does not add any new feature or fix any bug. So upgrading the Minor or Patch version, in this case, does not make sense either.</strong></li></ol><p>Now let us come back to the module example and release its latest version. Here is what I did:</p><ol><li>Appended v2 to the end of the module path (module github.com/azhuox/blogs/golang/go_modules/example/module/v2) as the latest version of the module package is v2.0.1.</li><li>Add a note under the v2.0.1 release note in the CHANGELOG.md file to indicate that the package is converted to a module in and after this version.</li><li>Release v2.0.1 by creating a git tag: git tag golang/go_modules/example/module/v2.0.1 &amp;&amp; git push -q origin master golang/go_modules/example/module/v2.0.1</li></ol><h4><strong>Consuming A Module</strong></h4><p>You can still use this package, without Go Modules enabled, by using some Go dependency management tool (e.g. dep) with the following specification. This will grab the whole repository which includes the module module for your build.</p><pre>[[constraint]]<br>name = &quot;github.com/azhuox/blogs&quot;<br>branch = &quot;master&quot;</pre><p>With Go Modules, what you need to do is import and use the module in your Go program and run go build. It will automatically grab the golang/go_modules/example/module/v2.0.1 module other than the whole repository for your build.</p><h4>Converting Go Libraries to Go Modules</h4><p>The section above already demonstrates how to convert one or more Go packages to a Go module. This section majorly talks about how to convert all the Go packages (libraries) within the same repository to Go modules.</p><p>I wrote three packages liba libb and libc under the github.com/azhuox/blogs/golang/go_modules/example/libs/ directory for the demo purpose. Among these three packages, the package libb depends on the package libawhile the package libc depends on the packagelibb and libc.</p><p><strong>A principle that we need to follow in this case is to convert the packages that have no dependency on other packages within the same repository, and then convert the packages which dependencies have been converted Go modules.</strong> This indicates that we need to convert the package liba first, then the package libb and then the package libc in this case.</p><p>Let us see what will happen if we convert libc first:</p><pre>go mod init github.com/azhuox/blogs/golang/go_modules/example/libs/libc<br>go: creating new go.mod: module github.com/azhuox/blogs/golang/go_modules/example/libs/libc<br>go build:<br><br>can&#39;t load package: package github.com/azhuox/blogs/golang/go_modules/example/libs/libc: unknown import path &quot;github.com/azhuox/blogs/golang/go_modules/example/libs/libc&quot;: ambiguous import: found github.com/azhuox/blogs/golang/go_modules/example/libs/libc in multiple modules:<br>      github.com/azhuox/blogs/golang/go_modules/example/libs/libc (/Users/achuo/go/src/github.com/azhuox/blogs/golang/go_modules/example/libs/libc)<br>      github.com/azhuox/blogs v0.0.0-20190330175117-09a7dbd4a3ce (/Users/achuo/go/pkg/mod/github.com/azhuox/blogs@v0.0.0-20190330175117-09a7dbd4a3ce/golang/go_modules/example/libs/libc)</pre><p>The cause of this ambiguous import problem is Go grabs the whole repository github.com/azhuox/blogs v0.0.0-20190330175117-09a7dbd4a3ce to get the liba and libb package for satisfying the dependencies of the libc module. However, github.com/azhuox/blogs v0.0.0-20190330175117-09a7dbd4a3ce also includes a copy of the libc package, which confuses the Go compiler. To fix this, we need to convert the liba and libb package to Go modules and release them, so that they can be retrieved and parsed properly as two individual modules by Go.</p><p>Now let us convert these three libs in the correct order.</p><p>Convert the packageliba to a module:</p><pre>cd path/to/libs/liba<br>go mod init github.com/azhuox/blogs/golang/go_modules/example/libs/liba<br>  go: creating new go.mod: module github.com/azhuox/blogs/golang/go_modules/example/libs/liba<br>go build<br>  go: finding golang.org/x/net/context latest<br>  go: finding golang.org/x/net latest<br><br># Commit changes<br>#<br>git add ./go.mod ./go.sum<br>git commit ./go.mod ./go.sum -q -m &quot;Convert liba  to a module&quot; &amp;&amp; git push origin master -q<br><br># Release the latest version (v1.1.0):<br>#<br>git tag golang/go_modules/example/libs/liba/v1.1.0 &amp;&amp; git push -q origin master golang/go_modules/example/libs/liba/v1.1.0</pre><p>convert the packagelibb to a module:</p><pre>go mod init github.com/azhuox/blogs/golang/go_modules/example/libs/libb<br>  go: creating new go.mod: module github.com/azhuox/blogs/golang/go_modules/example/libs/libb<br>go build<br>  go: downloading github.com/azhuox/blogs/golang/go_modules/example/libs/liba v1.1.0<br>  go: extracting github.com/azhuox/blogs/golang/go_modules/example/libs/liba v1.1.0<br>  ...<br><br>git add ./go.mod ./go.sum<br>git commit ./go.mod ./go.sum -q -m &quot;Convert libb  to a module&quot; &amp;&amp; git push origin master -q<br>git tag golang/go_modules/example/libs/libb/v1.0.0 &amp;&amp; git push -q origin master golang/go_modules/example/libs/libb/v1.0.0</pre><p>Convert the package libc to a module:</p><pre>go mod init github.com/azhuox/blogs/golang/go_modules/example/libs/libc<br>go build<br>  go: downloading github.com/azhuox/blogs/golang/go_modules/example/libs/libb v1.0.0<br>  go: extracting github.com/azhuox/blogs/golang/go_modules/example/libs/libb v1.0.0<br>  ...<br><br>git add ./go.mod ./go.sum<br>git commit ./go.mod ./go.sum -q -m &quot;Convert libc  to a module&quot; &amp;&amp; git push origin master -q<br>git tag golang/go_modules/example/libs/libc/v1.0.0 &amp;&amp; git push -q origin master golang/go_modules/example/libs/libc/v1.0.0</pre><p>You can see the package libc is converted to a module correctly and it can retrieve the modules liba and libb in its build without any problem.</p><h3>Go Modules and Microservices</h3><p>I wrote <a href="https://github.com/azhuox/blogs/tree/master/golang/go_modules/example/micro-service">a dummy micro-service</a> for demonstrating how to utilize Go Modules in a microservice. Here is its project layout:</p><pre>github.com/azhuox/blogs/tree/master/golang/go_modules/example/micro-service:<br>  - sdks<br>      - go<br>  - internal<br>      - api<br>      - pkga<br>      - pkgb<br>  - server<br>      - main.go<br>  - vendor<br>  - Gopkg.toml<br>  - Gopkg.lock<br>  - Dockerfile</pre><p>I want to mention that the package internal/pkgb is using the package libc that we just converted to a Go module above. In this case, libc is retrieved together with liba and libb from the github.com/azhuox/blogs repository when Go Modules is not enabled. But it is retrieved individually as a single unit when Go Modules is enabled.</p><p>From the project layout, you can also see that the microservice is built as a docker image with the following Dockerfile:</p><pre>FROM golang:1.12-alpine3.9<br><br>RUN apk add --update \<br>  ca-certificates \<br>  git<br><br>COPY . $GOPATH/src/github.com/azhuox/blogs/golang/go_modules/example/micro-service<br>RUN go build -o /usr/bin/micro-service github.com/azhuox/blogs/golang/go_modules/example/micro-service/server &amp;&amp; rm -rf $GOPATH/*<br><br>ENTRYPOINT [&quot;/usr/bin/micro-service&quot;]</pre><p>As mentioned in the <a href="https://gist.github.com/azhuox/8c999710447d2c06eb2458897ad43110#when-to-use-go-modules">When to Use Go Modules</a> section, only public packages need to be converted to modules. In this case, the package sdks/go is the only package that gets publicly used. Therefore, we only need to convert this package to a module and releases its latest version:</p><pre>go mod init github.com/azhuox/blogs/golang/go_modules/example/micro-service/sdks/go<br>go build<br>  ...<br>git add ./go.mod ./go.sum<br>git commit ./go.mod ./go.sum -q -m &quot;Convert micro-service/sdks/go to a module&quot; &amp;&amp; git push origin master -q<br>git tag golang/go_modules/example/micro-service/sdks/go/v1.0.2 &amp;&amp; git push -q origin master golang/go_modules/example/micro-service/sdks/go/v1.0.2</pre><p>Go Modules, in this case, refers to the new Go package management tool called <a href="https://github.com/golang/go/wiki/vgo">vgo</a> which is integrated into go tools like go get and go mod. The following steps demonstrate how to use it to manage the dependencies for the microservice:</p><ol><li>Launch a terminal and then enable Go Modules in the terminal: export GO111MODULE=on.</li><li>Cd the root directory of the microservice.</li><li>Add a go.mod file to the root directory of the microservice: go mod init github.com/azhuox/blogs/golang/go_modules/example/micro-service.</li><li>Run or test the microservice to ensure that everything works fine: go run ./server/main.go. This will generate a file called go.sum if everything goes well.</li><li>Remove the files for the old dependency management tool, which is Gopkg.toml and Gopkg.lock in this case.</li><li>Commit the changes.</li></ol><p>Now we successfully replace the old dependency management tool with Go Modules. However, there are two cases we need to deal with in the Continuous Integration (CI) process: with a vendor or without a vendor.</p><h4>CI Without Vendor</h4><p>Without vendors means utilizing Go Modules to dynamically grab dependencies when building docker images during the CI process. To do this, we need to do the following steps:</p><ol><li>Add an environment variable ENV GO111MODULE=on in the Dockerfile to enable Go Modules.</li><li>Remove the vendor directory since we don&#39;t need it anymore.</li><li>Commit the changes.</li></ol><h4>CI With Vendor</h4><p>With vendor means we want to dump all the dependencies into the vendor directory and let the CI build the docker image based on the vendor directory. The following steps demonstrate how to do it:</p><ol><li>Dump all the dependencies into the vendor directory: go mod vendor.</li><li>Commit the changes.</li><li>If Go Modules is enabled in the CI tool, add the -mod=vendor in the go build step in the Dockerfile: go build -mod=vendor -o /usr/bin/micro-service github.com/azhuox/blogs/golang/go_modules/example/micro-service/server &amp;&amp; rm -rf $GOPATH/*.</li></ol><h4>Update A Dependency in the vendor Directory</h4><p>Suppose we want to build docker images with the vendor directory and use the latest version of libc (say v1.5.0) in the microservice. The following steps demonstrate the update process:</p><ol><li>Get the version: go get github.com/azhuox/blogs/golang/go_modules/example/libs/libc@v1.5.0.</li><li>Update the vendor directory: go mod vendor.</li></ol><p>This may not work when the microservice is not using any new feature released after the current version of libc (v1.0.0 in this case). To force update it, we need to add a replace statement in the go.mod file and then run go mod vendor:</p><pre>replace (<br>    github.com/azhuox/blogs/golang/go_modules/example/libs/libc v1.0.0 github.com/azhuox/blogs/golang/go_modules/example/libs/libc v1.5.0<br>)</pre><h3><strong>Summary</strong></h3><ul><li>Go Modules allows you to group one or more packages to a single unit which is released and retrieved together.</li><li>Semantic Import Versioning is a method for applying Semantic Versioning to Go packages and modules to make them versioned.</li><li>Only the publicly-used packages, for example, Go libraries and SDKs, need to convert to Go modules (which produces the modules).</li><li>It is very easy to replace a legacy Go package management tool (e.g. dep) with Go modules (which consumes the modules).</li></ul><h3>Reference</h3><ul><li><a href="https://blog.golang.org/modules2019">Go Modules</a></li><li><a href="https://go.googlesource.com/proposal/+/master/design/24301-versioned-go.md">Proposal: Versioned Go Modules</a></li><li><a href="https://en.wikipedia.org/wiki/Dependency_hell">dependency hell</a></li><li><a href="https://research.swtch.com/vgo-import">Semantic Import Versioning</a></li><li><a href="https://semver.org/">Semantic Versioning</a></li><li><a href="https://github.com/azhuox/blogs/blob/master/golang/semantic_import_versioning/blog.md">Semantic Import Versioning in Go</a></li><li><a href="https://research.swtch.com/vgo-module#from_repository_to_modules">Defining Go Modules</a></li><li><a href="https://github.com/golang/go/wiki/vgo">vgo</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d51fa19cf715" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Nil in Go]]></title>
            <link>https://aaronzhuo.medium.com/nil-in-go-52ca66c83e46?source=rss-9175e0dbe19d------2</link>
            <guid isPermaLink="false">https://medium.com/p/52ca66c83e46</guid>
            <category><![CDATA[mídium]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Aaron Zhuo]]></dc:creator>
            <pubDate>Tue, 01 Dec 2020 05:13:47 GMT</pubDate>
            <atom:updated>2020-12-07T05:27:36.405Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CAfgIJGs0D4i4hGUeGYAJA.png" /></figure><h3>What Is nil in Go</h3><p>nil in Go has several meanings:</p><ul><li>It represents “null” in Go. This means two things: 1. It does not have a type. 2. Its value is “null”.</li><li>It is a predeclared identifier in Go, which means you can use it without declaring it.</li><li>It represents zero values (and default values) of some types in Go, including interface types, pointer types, slice types, map types, channel types, function types.</li></ul><h3>Using nil as Zero Values</h3><p>nil represents zero values (and default values) of some types in Go.</p><p>Example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/563e7ff7a730934f1399a7e0b9903734/href">https://medium.com/media/563e7ff7a730934f1399a7e0b9903734/href</a></iframe><h3>Using nil in Comparison</h3><h4>Two nil Values of Two Different Types Are Not Comparable</h4><p>Example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/30120330605189fa6b33ad7da193b18f/href">https://medium.com/media/30120330605189fa6b33ad7da193b18f/href</a></iframe><p>This code will fail to compile as they are trying to compare nil values of two different types.</p><h4>Two nil Values of The Same Type May Not Be Comparable</h4><p>Example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/975195e36923362f128d09148ecbec32/href">https://medium.com/media/975195e36923362f128d09148ecbec32/href</a></iframe><p>Take var sb = (map[string]bool)(nil) == (map[string]bool)(nil) as an example, the reason why two nil values of the same type ( map[string]bool) are not comparable is because Go does not support comparison in slice, map, and function types. <strong>You can see that we are comparing two values of a non-comparable type in this case, thus causing the failure.</strong></p><p>But the following code works and the results are true:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1a78637855ea956d79e6ba76aa7101be/href">https://medium.com/media/1a78637855ea956d79e6ba76aa7101be/href</a></iframe><p>Take var sb = (map[string]bool)(nil) == nil as an example, (map[string]bool)(nil) declares a temporary variable map[string]bool which value is nil and (map[string]bool)(nil) == nil detects whether the variable&#39;s value is nil and then assigns the results to sb. <strong>You can see that we are comparing the value of a non-comparable type with its zero value (</strong><strong>nil) in this case. That&#39;s why it works.</strong></p><h4>Two nil Values of The Same Type Can Be Comparable Only When This Type Supports Comparision</h4><p>Example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a12437e91d534943a9504ffce97fe7fe/href">https://medium.com/media/a12437e91d534943a9504ffce97fe7fe/href</a></iframe><h4>Be Careful in nil Comparison When Interface Values Are Involved</h4><p>The following code will not cause any compiler failure but the result is false other than true.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4663391b4175a954500794a22ed4d359/href">https://medium.com/media/4663391b4175a954500794a22ed4d359/href</a></iframe><p>Explanation:</p><ul><li><strong>An interface value consists of a dynamic type and a dynamic value.</strong> interface{}(nil) declares an interface value with {type: nil, value: nil}.</li><li>The non-interface value is converted to the type of the interface value before making the comparison with an interface value. In this example, (*int)(nil) is converted to an interface value with {type: *int, value: nil}.</li><li>Two nil interface values are equivalent only when they carry the same type. In this case, the converted interface value {type: *int, value: nil} has a concrete dynamic type but the other interface value has not. That is why the comparison result is false.</li></ul><p>A more interesting example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b8e0646ab9cbc349eb80b4bcd6590599/href">https://medium.com/media/b8e0646ab9cbc349eb80b4bcd6590599/href</a></iframe><p>Explanation:</p><ul><li><strong>An interface value equals to </strong><strong>nil only when its type and value are both </strong>`<strong>nil</strong>`<strong>.</strong> In this example, w is an io.Writer interface value with {type: *bytes.Buffer, value: nil} after the w=b assignment. Therefore, w==nil is false as it carries *byte.Buffer other than nil as its concrete dynamic type.</li></ul><h3>Summary</h3><ul><li>nil is and a pre-declared identifier that can be used to represent the zero values of some types in Go.</li><li>Be careful when using nil in comparison, especially when interface values are involved. You need to understand what you are comparing: types, or values, or both.</li><li>(a thing)(nil) may not equal to nil, depends on what that thing is (a pointer or an interface). This means nil is strong-typed even though nildoes not have a default type (<strong>sarcasm</strong>).</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=52ca66c83e46" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to Kubernetes]]></title>
            <link>https://aaronzhuo.medium.com/introduction-to-kubernetes-f119cf3073ad?source=rss-9175e0dbe19d------2</link>
            <guid isPermaLink="false">https://medium.com/p/f119cf3073ad</guid>
            <category><![CDATA[entry-level]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Aaron Zhuo]]></dc:creator>
            <pubDate>Mon, 30 Nov 2020 17:33:12 GMT</pubDate>
            <atom:updated>2020-11-30T17:35:35.735Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*rfpMHme7_yy8ZYoQZAeRbQ.png" /></figure><h3>Preface</h3><p>This series of blogs is an introduction to Kubernetes (k8s) explaining its basic concepts and commonly-used components for building and deploying applications in Kubernetes. The purpose of this series of blogs is for you to quickly learn the basic knowledge of Kubernetes in an hour. However, I highly recommend you check <a href="https://kubernetes.io/docs/home/">official Kubernetes documentation</a> if you want to dive deeply into Kubernetes.</p><h3>Index</h3><ul><li><a href="https://azhuox.medium.com/kubernetes-overview-c6f2d4ca0524">Kubernetes Overview</a></li><li><a href="https://azhuox.medium.com/kubernetes-pods-e90845fd67c8">Kubernetes Pods</a></li><li><a href="https://azhuox.medium.com/kubernetes-deployments-d04263c67b24">Kubernetes Deployments</a></li><li><a href="https://azhuox.medium.com/kubernetes-services-42a8b971e5ed">Kubernetes StatefulSets</a></li><li><a href="https://azhuox.medium.com/kubernetes-services-1f4d3e43db67">Kubernetes Services</a></li><li><a href="https://azhuox.medium.com/persistent-volumes-and-persistent-volume-claims-1b2f1fbdda43">Kubernetes Persistent Volumes and Persistent Volume Claims</a></li></ul><h3>Reference</h3><ul><li><a href="https://kubernetes.io/docs/home/">Kubernetes Documentation Center</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f119cf3073ad" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Overview]]></title>
            <link>https://aaronzhuo.medium.com/kubernetes-overview-c6f2d4ca0524?source=rss-9175e0dbe19d------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6f2d4ca0524</guid>
            <category><![CDATA[entry-level]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Aaron Zhuo]]></dc:creator>
            <pubDate>Mon, 30 Nov 2020 17:22:02 GMT</pubDate>
            <atom:updated>2020-11-30T23:05:59.749Z</atom:updated>
            <content:encoded><![CDATA[<p>Kubernetes is a portable, extensible system for running and coordinating containerized workloads and services across a cluster of machines. It is designed to manage the life cycle of containerized applications with the guarantee of stability, scalability, and high availability.</p><p>The following picture shows a Kubernetes cluster running on <a href="https://cloud.google.com/kubernetes-engine">Google Kubernetes Engine (GKE)</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H4SyZE8ZQcWE_s2lopwy9Q.png" /></figure><p>From the picture, you can see that:</p><ul><li>A Kubernetes cluster has one or more master nodes (for high availability purposes). A master node consists of the following core components:</li><li>→ etcd is used as a backing store for all the Kubernetes&#39;s cluster data.</li><li>→ controller manager is in charge of maintaining the state of the cluster, such as rolling updates, fault detection, auto-scaling, etc.</li><li>→ scheduler is responsible for scheduling Pods to proper worker nodes according to a predetermined scheduling policy.</li><li>→ apiserver provides a uniform entry for authentication, authorization, API registration, and resource operations. As a developer, this is the only component you need to interact in a Kubernetes master. When you run a Kubernetes CLI ( kubectl) command in a terminal, such as kubectl get pods, what happens is kubectl converts the command to an API request, sends it to apiserver and displays the results in the terminal.</li><li>A Kubernetes cluster has one or more worker nodes that are running in multiple zones. Each worker node is a <a href="https://cloud.google.com/compute">Google Compute Engine (GCE)</a> virtual machine and has the following core components:</li><li>→ kubele is responsible for maintaining the life cycle of Pods and containers.</li><li>→ kube-proxy provides service discovery and load balancing functionality for Kubernetes Services.</li></ul><p>There are actually more components in a Kubernetes cluster. But as a developer, you do not need to dive deeply to be familiar with any of these components. It is enough for you to know that a Kubernetes cluster is a distributed system that consists of multiple machines running across multiple zones.</p><p>While Kubernetes utilizes containers ( <a href="https://www.docker.com/resources/what-container">docker containers</a> as default for GKE clusters) as the underlying mechanism to deploy &amp; run applications, it builds additional layers of abstraction on the top of the container interface to make it easier for developers to containerize their applications. Instead of managing and interacting with containers directly, developers use objects and workloads that Kubernetes provides to construct their applications.</p><p>Here are the main objects and workloads that Kubernetes provides:</p><ul><li><a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/">Namespaces</a>: A Namespace is considered a virtual cluster in a Kubernetes cluster. It is used to separate projects and teams. In our system, each of our applications (a.k.a microservices) has two Namespaces, one for the demo environment and another one for the prod environment.</li><li><a href="https://kubernetes.io/docs/concepts/configuration/configmap/">ConfigMaps</a>: A ConfigMap is a Kubernetes object for storing non-confidential data in key-value pairs. It allows you to decouple environment-specific configuration from containers in your application, which makes your application easily portable. A Pod can consume data in a ConfigMap as environment variables or mount a ConfigMap as a Volume.</li><li><a href="https://kubernetes.io/docs/concepts/configuration/secret/">Secrets</a>: A Secret is a Kubernetes object for storing sensitive configuration, such as passwords, API keys, and OAuth tokens, in key-value pairs.</li><li><a href="https://kubernetes.io/docs/concepts/workloads/pods/">Pods</a>: A Pod is the smallest deployable unit that you can create in Kubernetes. A Pod consists of one or more containers. These containers have their own CPU &amp; memory resources but need to share other computing resources, including storage and network. When a Pod is created, it will be scheduled to a proper Kubernetes worker node based on its predetermined policy. You can consider a Pod a virtual machine running in a Kubernetes cluster. We will take about this in the next chapter.</li><li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/">ReplicaSets</a>: A ReplicaSet is used to maintain a stable set of replica Pods running at the same state at any given time. It is also used to guarantee the availability of a specified number of replica Pods.</li><li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">Deployments</a> &amp; <a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/">Stateful Sets</a>: A Deployment provides declarative updates, such as rolling updates and rolling back, for Pod ReplicaSets.<br> In other words, A Deployment ensures a group of identical Pods to achieve the desired state at any time. StatefulSets provides the same functionality. However, Deployments are used to manage stateless applications while StatefulSets are used to manage stateful applications.</li><li><a href="https://kubernetes.io/docs/concepts/services-networking/service/">Services</a>: A Service is an object that exposes an application running as a set of Pods as a network service. A Service is responsible for load-balancing requests to your application’s Pods.</li></ul><p>When working with Kubernetes, you need to know how to utilize its CLI tool kubectl to interact with Kubernetes clusters. Here are some frequently used kubectl commands that you may use in your daily work.</p><ul><li>Connecting a GKE cluster: gcloud container clusters get-credentials &lt;your_cluster&gt; --region &lt;region&gt; --project &lt;your_project&gt; is the command for connecting to the given GKE cluster. You may need to check official docs for more details about connecting to a Kubernetes cluster with a specific cloud provider.</li><li>Switching current namespace: kubectl config set-context --current --namespace=&lt;your_namespace&gt; switches your current context (Namespace) to the given Namespace. This command is useful when you need to switch between the demo and prod environment for a microservice.</li><li>Create one or more Kubernetes objects: kubectl apply -f &lt;a_yaml_or_json_file&gt; can help you create one or more Kubernetes objects. The given file must be YAML or JSON formatted. You can put the specification of many Kubernetes objects into a single file as long as each object is separated with ---.</li><li>Get Kubernetes objects: kubectl get &lt;object_type&gt; can help you get a certain type of Kubernetes objects. Here are two examples of this command:</li></ul><p>I recommend you read <a href="https://azhuox.medium.com/kubernetes-pods-e90845fd67c8">this blog</a> if you are curious about the minimal deploy-able unit in Kubernetes: Pods.</p><h3>Reference</h3><ul><li><a href="https://cloud.google.com/kubernetes-engine">Google Kubernetes Engine (GKE)</a></li><li><a href="https://cloud.google.com/compute">Google Compute Engine (GCE)</a></li><li><a href="https://www.docker.com/resources/what-container">Docker Containers</a></li><li><a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/">Kubernetes Namespaces</a></li><li><a href="https://kubernetes.io/docs/concepts/configuration/configmap/">Kubernetes ConfigMaps</a></li><li><a href="https://kubernetes.io/docs/concepts/configuration/secret/">Kubernetes Secrets</a></li><li><a href="https://kubernetes.io/docs/concepts/workloads/pods/">Kubernetes Pods</a></li><li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/">Kubernetes ReplicaSets</a></li><li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">Kubernetes Deployments</a></li><li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/">Kubernetes Stateful Sets</a></li><li><a href="https://kubernetes.io/docs/concepts/services-networking/service/">Kubernetes Services</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6f2d4ca0524" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Pods]]></title>
            <link>https://aaronzhuo.medium.com/kubernetes-pods-e90845fd67c8?source=rss-9175e0dbe19d------2</link>
            <guid isPermaLink="false">https://medium.com/p/e90845fd67c8</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[entry-level]]></category>
            <dc:creator><![CDATA[Aaron Zhuo]]></dc:creator>
            <pubDate>Mon, 30 Nov 2020 16:53:07 GMT</pubDate>
            <atom:updated>2020-11-30T16:53:07.187Z</atom:updated>
            <content:encoded><![CDATA[<h3>Preface</h3><p>This blog talks about the basic knowledge of Kubernetes <a href="https://kubernetes.io/docs/concepts/workloads/pods/">Pods</a>. But before exploring Kubernetes Pods, let us first go through what a <a href="https://docs.docker.com/get-started/overview/">Docker container</a> is as it is the major container technology that we use to run our applications.</p><h3>Docker Containers</h3><p>Docker is an open platform that allows you to package and run applications in a loosely isolated environment called a container. A Docker container is a runnable instance of a Docker image and a Docker read-only template for creating a Docker container. In other words, we need to construct a Docker image before running a Docker container. Take the service foo as an example, here is the Docker file for creating its Docker images:</p><pre># Suppose the CI tool already build the `foo` service as a executable binary file and saved it in `./go-server`.<br><br>FROM alpine:3.5<br><br>RUN apk update &amp;&amp; apk add ca-certificates &amp;&amp; rm -rf /var/cache/apk/*<br><br>COPY ./go-server /usr/bin/server<br><br>ENTRYPOINT [&quot;/usr/bin/server&quot;]</pre><p>From the file, you can see a minimal Docker file can be as simple as only containing the following directives:</p><ul><li>FROM directive specifies alpine:3.5, which essentially is a mini Linux system, as its parent container.</li><li>RUN apk update… updates container’s dependencies.</li><li>COPY command copies an executable binary file that contains all the business logic of the service foo from ./go-server to /user/bin/server.</li><li>ENTRYPOINT tells Docker to executes /user/bin/server as this container’s entry point.</li></ul><p>With this Docker file, our continuous integration (CI) tool is able to build and push a new image to the servicefoo’s Docker image warehouse every time when there is a new change in foo service&#39;s repo. Then with this Docker image, we can run and deploy the service foo as a containerized application in Kubernetes.</p><h3>Pod Overview</h3><p>A Pod is the smallest deployable unit in Kubernetes. It consists of one or more containers. These containers have their own CPU and memory resources but need to share other resources, including storage and network. As shown in the following picture, a Pod is very similar to an application-specific “host” running one or more “processes” (containers). These “processes” work together to construct a containerized workload or service.c</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*473XwpPeJOC5A17p" /></figure><h3>Use of Pods</h3><p>The following example demonstrates how to use a K8s Pod to construct a single-replica microservice. This Pod consists of three containers: the container user-usvc has all the microservice&#39;s business logic, the container cloudsql-proxy proxies all the MySQL requests to a Google Cloud SQL instance, while the container datadog-agent sends the logs to datadog server.</p><pre># ConfigMap for user-msvc<br>apiVersion: v1<br>kind: ConfigMap<br>metadata:<br>  name: user-msvc<br>  namespace: user-msvc<br>data:<br>  mysql-host: 127.0.0.1:3306<br><br>---  <br># Secret of user-msvc<br>apiVersion: v1<br>kind: Secret<br>metadata:<br>  name: user-msvc<br>  namespace: user-msvc<br>type: Opaque<br>data:<br>  datadog-api-key: XKSSAKCKANmOWQ5NDU1NWU2MWE2ZDI=<br>  github-access-token: Yzk4MDZkNzAxNTQwMjkwOA==<br>  # All other configs are omitted.  <br><br>---<br># Pod for user-msvc<br>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: user-msvc<br>  namespace: user-msvc<br>  labels:<br>    app: user-msvc<br>spec:<br>  # Storage<br>  volumes:<br>  - name: datadog-data<br>    emptyDir: {}<br>  - name: user-msvc-secret<br>    secret:<br>      defaultMode: 420<br>      secretName: user-msvc-secret<br>    <br>  # &quot;Processes&quot;<br>  containers:<br>  - name: user-msvc<br>    image: gcr.io/path/to/user-msvc:1.0.0<br>    ports:<br>    - name: secure-port<br>      containerPort: 443<br>      protocol: TCP<br>    env:<br>    - name: MYSQL_HOST<br>      valueFrom:<br>        configMapKeyRef:<br>          name: user-msvc<br>          key: mysql-host<br>    volumeMounts:<br>      - name: ndatadog-data<br>        mountPath: /var/log/user-msvc<br>      - name: user-msvc-secret<br>        mountPath: /etc/user-msvc/secret<br>    requests:<br>      cpu: 100m<br>      memory: 100Mi<br>    limits:<br>        cpu: 200m<br>        memory: 200Mi<br><br>  - name: cloudsql-proxy<br>    image: gcr.io/cloudsql-docker/gce-proxy:1.12<br>    ports:<br>    - name: connect-port<br>      containerPort: 3306<br>      protocol: TCP<br>    requests:<br>      ...<br><br>  - name: datadog-agent<br>    image: gcr.io/path/to/monitor:1.0.0<br>    volumeMounts:<br>      - name: datadog-data<br>        mountPath: /var/log/monitor<br>      - name: user-msvc-secret<br>        mountPath: /etc/datadog-agent/secret<br>    requests:<br>      ...<br>---<br><br><br># External Service for user-msvc<br>kind: Service<br>apiVersion: v1<br>metadata:<br>  name: user-msvc<br>spec:<br>  ports:<br>  - name: secure-port<br>    port: 443<br>    targetPort: 443<br>    protocol: TCP<br>  selector:<br>    app: user-msvc<br>  type: LoadBalancer<br>  loadBalancerIP: 12.34.56.78 # This is a fake IP</pre><p>The following picture shows the topology of the Pod user-msvc in Kubernetes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/0*B-JQjAvQlso5yfoz" /></figure><p>From the above Pod configuration, we can see that:</p><ul><li>The ConfigMap user-msvc and Secret user-msvc are created for storing configurations and sensitive data.</li><li>The field spec.containers defines all the containers of the Pod. For each container, you need to configure which image it is going to run, all the environmental variables, and computing resources it needs, including storage, CPU, memory, and network.</li><li>Containers within the same Pod share the network, which means these containers reach each other through 127.0.0.1. However, a port can only be exclusively occupied by a container. In this case, the container cloudsql-proxy exposes itself by opening the port 3306. Therefore, the container user-msvc is able to connect to it through 127.0.0.1:3306. Moreover, the container user-msvc opens the 443 port for processing incoming requests from the container user-msvc <a href="https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer">Kubernetes LoadBalancer Service</a>.</li><li>The field spec.volumes specifies the shared storage resources for all the containers of the Pod. Kubernetes supports many types of Volumes. you can check <a href="https://kubernetes.io/docs/concepts/storage/volumes/">this doc</a> for more details about Kubernetes Volumes.</li></ul><p>In this example, an <a href="https://kubernetes.io/docs/concepts/storage/volumes/#emptydir">emptyDir</a> volume is created when the Pod is created. The volume is respectively mounted to the path/var/log and /var/log/monitor in the container user-msvc and monitor. Both containers share the data in this Volume. Because of this, the container user-msvc can create a file called /var/log/user-msvc-error.log and writes logs to this file, while the container datadog-agent can read the logs from the file /var/log/monitor/user-msvc-error.log and then sends them to datadog.</p><p>The Secret user-msvc is also used as Volume in this example. Then it is mounted in the path/etc/user-msvc/secret in the container user-msvc and the path /etc/datadog-agent/secret in the container datadog-agent. When a Secret is mounted into a directory in a container, each of its data will be created as an individual file in that directory. Moreover, a Secret Volume should be read-only. A ConfigMap can be directly used in a Pod’s environmental variables or can be used as a Volume as well.</p><p>All in all, this example demonstrates that a Pod is like an application-specific “host” that coordinates one or more “processes” (containers) to work together to provide some kind of service.</p><h3>What Is Next</h3><p>It is not a good idea to directly utilize K8s Pods to run applications as Pods are mortal. They cannot be resurrected when they are killed for whatever reason. Because of this, you should use K8s Deployments to run stateless applications and K8s StatefulSets to run stateful applications.</p><p>I recommend you read <a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/">this document</a> if you want to know more details about Kubernetes Pods, such as pod lifetime, pod phase, pod conditions, etc.</p><p>I recommend you read <a href="https://azhuox.medium.com/kubernetes-deployments-d04263c67b24">this blog</a> if you are curious about how to utilize K8s Deployments to run stateless applications in Kubernetes.</p><p>I recommend you read <a href="https://azhuox.medium.com/kubernetes-services-42a8b971e5ed">this blog</a> if you are curious about how to utilize K8s StatefulSets to run stateful applications in Kubernetes.</p><h3>Reference</h3><ul><li><a href="https://kubernetes.io/docs/concepts/workloads/pods/">Kubernetes Pods</a></li><li><a href="https://docs.docker.com/get-started/overview/">Docker Containers</a></li><li><a href="https://kubernetes.io/docs/concepts/storage/volumes/">Kubernetes Volumes</a></li><li><a href="https://kubernetes.io/docs/concepts/storage/volumes/#emptydir">Kubernetes EmptyDir</a></li><li><a href="https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer">Kubernetes LoadBalancer Service</a></li><li><a href="https://www.aaronzhuo.com/kubernetes-deployments/">Kubernetes Deployments</a></li><li><a href="https://www.aaronzhuo.com/kubernetes-statefulsets/">Kubernetes StatefulSets</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e90845fd67c8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Deployments]]></title>
            <link>https://aaronzhuo.medium.com/kubernetes-deployments-d04263c67b24?source=rss-9175e0dbe19d------2</link>
            <guid isPermaLink="false">https://medium.com/p/d04263c67b24</guid>
            <category><![CDATA[entry-level]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Aaron Zhuo]]></dc:creator>
            <pubDate>Mon, 30 Nov 2020 16:44:31 GMT</pubDate>
            <atom:updated>2020-11-30T16:59:10.920Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Prerequisites</strong></h3><p>I recommend you know the basic knowledge of Kubernetes Pods before reading this blog. You can check <a href="https://azhuox.medium.com/kubernetes-pods-e90845fd67c8">this blog</a> for details about Kubernetes Pods.</p><h3><strong>What Is A Deployment</strong></h3><p>Normally, when working with Kubernetes, rather than directly managing a group of replicated Pods, you would like to leverage higher-level Kubernetes objects &amp; workloads to manage those Pods for you. <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">Kubernetes Deployments</a> is one of the most common workloads in Kubernetes that provides flexible life cycle management for a group of replicated Pods.</p><p>A Deployment is a Kubernetes object that provides declarative updates, such as scaling up/down, rolling updates, and rolling back, for a group of identical Pods. In other words, A Deployment ensures a group of identical Pods to achieve the desired state.</p><h3><strong>ReplicaSets v.s. Deployments</strong></h3><p>Rather than directly managing Pods, Deployment utilizes ReplicaSets to perform declarative updates for a group of replicated Pods. The following picture demonstrates the relationship between ReplicaSets and Deployment:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0ZRnWHPOxjm_nB5k.png" /></figure><p>ReplicaSet ensures that a specific number of pod replicas are running at a given time, based on replica number defined in a ReplicaSet manifest. Although it provides an easy way to replicates Pods, it lacks the ability to do rolling updates on pods.</p><p>Deployments are built on the top of ReplicaSets. A Deployment essentially is a set of ReplicaSets. It rolls out a new ReplicaSet with the desired number of Pods and smoothly terminates Pods in the old ReplicaSet when a rolling update occurs. In other words, a Deployment performs the rolling update by replacing the current ReplicaSet with a new one. You can check this doc for more details about rolling update or rolling back Deployments.</p><h3><strong>A Deployment Example</strong></h3><p>The following is an example of a Deployment configuration for creating an Nginx server with three replicated Pods.</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br> name: nginx-deployment-demo<br>spec:<br> selector:<br>   matchLabels:<br>     app: nginx<br>     env: demo<br> replicas: 3<br> strategy:<br>   rollingUpdate:<br>     maxUnavailable: 0<br> template:<br>   metadata:<br>     labels:<br>       app: nginx<br>       env: demo<br>   spec:<br>     affinity:<br>       nodeAffinity:<br>         requiredDuringSchedulingIgnoredDuringExecution:<br>           nodeSelectorTerms:<br>           - matchExpressions:<br>             - key: failure-domain.beta.kubernetes.io/zone<br>               operator: In<br>               values:<br>               - us-central1-a<br>               - us-central1-b<br>               - us-central1-c<br>       podAntiAffinity:<br>         preferredDuringSchedulingIgnoredDuringExecution:<br>         - weight: 100<br>           podAffinityTerm:<br>             labelSelector:<br>               matchExpressions:<br>               - key: app<br>                 operator: In<br>                 values:<br>                 - nginx<br>             topologyKey: kubernetes.io/hostname<br>     containers:<br>     - name: nginx<br>       image: nginx:1.15.3<br>       ports:<br>       - containerPort: 80</pre><h4>Metadata</h4><p>The field metadata contains metadata of this Deployment, which includes the name of the Deployment and the Namespace it belongs to. You can also put <a href="https://kubernetes.io/docs/reference/kubernetes-api/labels-annotations-taints/">labels and annotations</a> in the field metadata.</p><h4><strong>Deployment Spec and Pod Template</strong></h4><p>The field spec defines the specification of this Deployment and the field spec.template defines a template for creating the Pods this Deployment manages.</p><h4><strong>Pod Selector</strong></h4><p>The field spec.selector is used for the Deployment to find which pods to manage. In this example, the Deployment uses app: nginx &amp;&amp; env: demo defined in the field sepc.selector.matchLabels to find the pods that have labels{app: nginx, env: demo} (defined in the field spec.template.metadata.labels). The field sepc.selector.matchLabels defines a map of key-value pairs and match requirements are ANDed.</p><p>Instead of using the field sepc.selector.matchLabels, you can use the field sepc.selector.matchExpressions to define more sophisticated match roles. You can check <a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#labelselector-v1-meta">this doc</a> for more details about the usage of the field sepc.selector.matchExpressions.</p><p>As you can see, a Deployment relies on pod labels and pod selector to find its pods. Therefore, it is recommended to put some unique pod labels for a Deployment. Otherwise, Deployment A may end up managing the pods that belong to Deployment B.</p><h4>Replica</h4><p>The field spec.replica specifies the desired number of Pods for the Deployment. Kubernetes guarantees that there are alwaysspec.replica number of Pods that are up and running.</p><p>It is highly recommended to run at least two replicas for any Deployment in Production. This is because having at least two replicas at the beginning can help you keep your Deployments stateless, as the problem can be easily detected when you are trying to introduce “stateful stuff” to a Deployment with at least two replicas. For example, you will quickly realize the problem when you are trying to add a cron job to a two-replicas Deployment to process some data on a daily base: the data will be processed twice a day as all replicas will execute this cron job every day, which may cause some unexpected behavior. In addition, a singleton pod may cause some downtime in some cases. For example, a single-replica deployment will not be available for a moment when the single Pod is triggered to restart for whatever reason.</p><h4><strong>Rolling Update Strategies</strong></h4><p>The field spec.strategy defines the strategy for replacing old pods with new ones when a rolling update occurs. The field spec.strategy.type can be Recreate or RollingUpdate. The default value is RollingUpdate.</p><p>In general, it is not recommended use Recreate in Production based on the consideration of availability. This is because Recreate will introduce downtime when a rolling update occurs: All the existing pods need to be terminated before new ones are created when spec.strategy.type is Recreate.</p><p>You can use maxUnavailable and maxSurge to control the update process when you set spec.strategy.type == RollingUpdate. The field maxUnavailable sets the maximum number of Pods that can be unavailable during an update process, while the field maxSurge specifies the maximum number of Pods that can be created over the desired number of Pods. The default value is 25% for these two fields. Moreover, they cannot be set 0 at the same time, as this stops the Deployment from performing the rolling update. you can set the field maxUnavailable 0 as this is the most effective way to prevent your old pods from being terminated while there are some problems spinning up new pods.</p><h4><strong>Pod Affinity</strong></h4><p>The field affinity inside the field spec.template.spec allows you to specify on which zones/nodes you want to run your Deployment&#39;s Pods. As shown in the following picture, the ideal scenario of running a Deployment is running multiple replicas in different nodes in different zones, and avoid running multiple replicas in the same node. You can check <a href="https://kubernetes.io/docs/concepts/configuration/assign-pod-node/">this doc</a> for more details about how to assign your Pods to proper nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vWo9ScMwt7Izcmdy.png" /></figure><h3>What Is Next</h3><p>I recommend you read <a href="https://azhuox.medium.com/kubernetes-services-42a8b971e5ed">this blog</a> if you are curious about how to utilize Kubernetes StatefulSets to run stateful applications in Kubernetes.</p><p>I recommend you read <a href="https://azhuox.medium.com/kubernetes-services-1f4d3e43db67">this blog</a> if you are curious about how to utilize Kubernetes Services to load balance traffic to your applications in Kubernetes.</p><h3>Reference</h3><ul><li><a href="https://azhuox.medium.com/kubernetes-pods-e90845fd67c8">Kubernetes Pods</a></li><li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">Kubernetes Deployments</a></li><li><a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#labelselector-v1-meta">LabelSelector in Kubernetes</a></li><li><a href="https://kubernetes.io/docs/concepts/configuration/assign-pod-node/">Assigning Pods to Nodes</a></li><li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment">Updating a Kubernetes Deployment</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d04263c67b24" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes StatefulSets]]></title>
            <link>https://aaronzhuo.medium.com/kubernetes-services-42a8b971e5ed?source=rss-9175e0dbe19d------2</link>
            <guid isPermaLink="false">https://medium.com/p/42a8b971e5ed</guid>
            <category><![CDATA[entry-level]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Aaron Zhuo]]></dc:creator>
            <pubDate>Mon, 30 Nov 2020 00:50:42 GMT</pubDate>
            <atom:updated>2020-11-30T16:57:39.836Z</atom:updated>
            <content:encoded><![CDATA[<h3>prerequisites</h3><p>I recommend you know the basic knowledge of Kubernetes Pods before reading this blog. You can check <a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/">this blog</a> for details about Kubernetes Pods.</p><h3>What Is A StatefulSet</h3><p>StatefulSet is a Kubernetes object designed to manage stateful applications. Like a Deployment, a StatefulSet scales up a set of pods to the desired number that you define in a config file. Pods in a StatefulSet runs the same containers that are defined in the Pod spec inside the StatefulSet spec. Unlike a Deployment, every Pod of a StatefulSet owns a sticky and stable identity. A StatefulSet also provides the guarantee about ordered deployment, deletion, scaling, and rolling updates for its Pods.</p><h3>A StatefulSet Example</h3><p>A complete StatefulSet consists of two components:</p><ul><li>A Headless Service used to control the network ID of the StatefulSet’s Pods</li><li>A StatefulSet object used to create and manage its Pods.</li></ul><p>The following example demonstrates how to use a StatefulSet to create a ZooKeeper Server. Please note that the following StatefulSet Spec is simplified for demo purposes. You can check <a href="https://github.com/kubernetes/contrib/blob/master/statefulsets/zookeeper/zookeeper.yaml">this YAML file</a> for the complete configuration of this ZooKeeper Service.</p><h3>ZooKeeper Service</h3><p>A <a href="https://zookeeper.apache.org/">Zookeeper service</a> is a distributed coordination system for distributed applications. It allows you to read, write data, and observe data updates. Data is stored and replicated in each ZooKeeper server and these servers work together as a ZooKeeper Ensemble.</p><p>The following picture shows the overview of a ZooKeeper service with five ZooKeeper servers. You can see each server in a ZooKeeper service has a stable network ID for potential leader elections. Moreover, one of the ZooKeeper servers needs to be selected as a leader for managing the service topology and processing write requests. StatefulSets is suitable for running such an application as it guarantees uniqueness for Pods.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xFTTNUaC4X5uJwkh.jpeg" /></figure><h3>Headless Service</h3><p>A Headless Service is responsible for controlling the network domain for a StatefulSet. The way to create a headless service is to specify clusterIP == None.</p><p>The following spec is for creating a Headless Service for the ZooKeeper service. This Headless Service is used to manage <a href="https://gist.github.com/azhuox/d6396d1d13c94f9476dc740f0045df85#pod-identify">Pod Identify</a> for the following StatefulSet.</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  namespace: default<br>  name: zk-hs<br>  labels:<br>    app: zk-hs<br>spec:<br>  ports:<br>  - port: 2888<br>    name: server<br>  - port: 3888<br>    name: leader-election<br>  clusterIP: None<br>  selector:<br>    app: zk</pre><p>Unlike a <a href="https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types">ClusterIP Service</a> or a<a href="https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer">LoadBalancer Service</a>, a Headless Service does not provide load-balancing. Based on my experience, any request to zk-hs.default.svc.cluster.local is always redirected to the first StatefulSet Pod ( zk-0 in the example). Therefore, A Kubernetes Service that provides load balancing or an Ingress is required if you need to load-balance traffic for your StatefulSet.</p><h3>StatefulSet Spec</h3><p>The following spec demonstrates how to use a StatefulSet to run a ZooKeeper service:</p><pre>apiVersion: apps/v1<br>kind: StatefulSet<br>metadata:<br>  namespace: default<br>  name: zk<br><br># StatefulSet spec<br>spec:<br>  serviceName: zk-hs<br>  selector:<br>    matchLabels:<br>      app: zk # It has to match .spec.template.metadata.labels<br>  replicas: 5<br>  podManagementPolicy: OrderedReady<br>  updateStrategy:<br>    type: RollingUpdate<br><br>  # volumeClaimTemplates creates a Persistent Volume for each StatefulSet Pods.<br>  volumeClaimTemplates:<br>  - metadata:<br>      name: datadir<br>    spec:<br>      accessModes: [ &quot;ReadWriteOnce&quot; ]<br>      storageClassName: standard<br>      resources:<br>        requests:<br>          storage: 10Gi<br><br>  # Pod spec<br>  template:<br>    metadata:<br>      labels:<br>        app: zk<br>    spec:<br>      affinity:<br>        nodeAffinity:<br>          ...<br>        podAntiAffinity:<br>          ...<br>      # Containers running in each Pod<br>      containers:<br>      - name: k8szk<br>        image: gcr.io/google_samples/k8szk:v3<br>        ports:<br>        - containerPort: 2181<br>          name: client<br>        - containerPort: 2888<br>          name: server<br>        - containerPort: 3888<br>          name: leader-election<br>        env:<br>        ...<br>        readinessProbe:<br>          exec:<br>            command:<br>            - &quot;zkOk.sh&quot;<br>          initialDelaySeconds: 10<br>          timeoutSeconds: 5<br>        livenessProbe:<br>          ...<br>        volumeMounts:<br>        - name: datadir<br>          mountPath: /var/lib/zookeeper</pre><h4>Metadata</h4><p>The field metadata contains metadata of this StatefulSet, which includes the name of this StatefulSet and the Namespace it belongs to. You can also put <a href="https://kubernetes.io/docs/reference/kubernetes-api/labels-annotations-taints/">labels and annotations</a> in this field.</p><h4><strong>Stateful Set Spec and Pod Template</strong></h4><p>The field spec defines the specification of this StatefulSet and the field spec.template defines a template for creating the Pods this StatefulSet manages.</p><h4><strong>Pod Selector</strong></h4><p>Like a Deployment, a StatefulSet uses the field spec.selctor to find which Pods to manage. You can check <a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#labelselector-v1-meta">this doc</a> for details about the usage of Pod Selector.</p><h4><strong>Replica</strong></h4><p>The field spec.replica specifies the desired number of Pods for the StatefulSet. It is recommended to run an odd number of Pods for some stateful applications like ZooKeepers, based on the consideration of the efficiency of some operations. For example, a ZooKeeper service marks a data write complete only when more than half of its servers send an acknowledgment back to the leader. Take a six pods ZooKeeper service as an example. The service remains available as long as at least four servers (ceil(6/2 + 1)) are available, which means your service can tolerate the failure of two servers. Nevertheless, it can still tolerate two-servers failure when the server number is lowered down to five. Meanwhile, this also improves write efficiency as now it only needs 3 servers&#39; acknowledgment to complete a write request. Therefore, having the sixth server, in this case, does not give you any additional advantage in terms of write efficiency and server availability.</p><h4><strong>Pod Identify</strong></h4><p>A StatefulSet Pod is assigned a unique ID (aka. Pod Name) from its Headless Service when it is created. This ID sticks to the Pod during the life cycle of the StatefulSet. The pattern of constructing ID is ${statefulSetName}-${ordinal}. For example, Kubernetes will create five Pods with five unique IDs zk-0, zk-1, zk-2, zk-3 and zk-4 for the above ZooKeeper service.</p><p>The ID of a StatefulSet Pod is also its hostname. The subdomain takes the form ${podID}.${headlessServiceName}.{$namespace}.svc.cluster.local where cluster.local is the cluster domain. For example, the subdomain of the first ZooKeeper Pod is zk-0.zk-hs.default.svc.cluster.local. It is recommended to use a Stateful Pod&#39;s subdomain other than its IP to reach the Pod as the subdomain is unique within the whole cluster.</p><h4><strong>Pod Management Policy</strong></h4><p>You can choose whether to create/update/delete a StatefulSet’s Pod in order or in parallel by specifying spec.podManagementPolicy == OrderedReady or spec.podManagementPolicy == Parallel. OrderedReady is the default setting and it controls the Pods to be created with the order 0, 1, 2, ..., N and to be deleted with the order N, N-1, ..., 1, 0. In addition, it has to wait for the current Pod to become Ready or terminated prior to terminating or launching the next Pod. Parallel launches or terminates all the Pods simultaneously. It does not rely on the state of the current Pod to lunch or terminate the next Pod.</p><h4><strong>Update Strategy</strong></h4><p>There are several rolling update strategies available for StatefulSets. RollingUpdate is the default strategy and it deletes and recreates each Pod for a StatefulSet when a rolling update occurs.</p><p>Doing rolling updates for the stateful applications like ZooKeepers is a little bit tricky: Other Pods need enough time to elect a new leader when the StatefulSet Controller is recreating the leader. Therefore, You should consider configuring <a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#define-readiness-probes">readnessProbe</a> and readnessProbe.initialDelaySeconds for the containers inside a StatefulSet Pods to delay the new Pod to be ready, thus delaying the rolling update of the next Pod and giving other running Pods enough time to update the service topology. This should give your stateful applications, for example, a ZooKeeper service, enough time to handle the case where a Pod is lost and back.</p><h4><strong>Pod Affinity</strong></h4><p>Like a Deployment, the ideal scenario of running a StatefulSet is to distribute its Pods to different nodes in different zones and avoid running multiple Pods in the same node. The spec.template.spec.affinity field allows you to specify node affinity and inter-pod affinity (or anti-affinity) for the SatefulSet Pods. You can check <a href="https://kubernetes.io/docs/concepts/configuration/assign-pod-node/">this doc</a> for details about using node/pod affinity in Kubernetes</p><h4><strong>volumeClaimTemplates</strong></h4><p>The field spec.volumeClaimTemplates is used to provide stable storage for StatefulSets. As shown in the following picture, the field spec.volumeClaimTemplates creates a Persistent Volume Claim ( datadir-zk-0), a Persistent Volume ( pv-0000), and a 10 GB standard persistent disk for Pod zk-0. These storage settings have the same life cycle as the StatefulSet, which means the storage for a Stateful Pod is stable and persistent. Any StatefulSet Pod will not lose its data whenever it is terminated and recreated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NxQikd85lLw7ioUn.jpeg" /></figure><h3>What Is Next</h3><p>I recommend you read <a href="https://azhuox.medium.com/kubernetes-deployments-d04263c67b24">this blog</a> if you are curious about how to utilize Kubernetes Deployments to run stateless applications in Kubernetes.</p><p>I also recommend you read <a href="https://azhuox.medium.com/kubernetes-services-1f4d3e43db67">this blog</a> if you are curious about how to utilize Kubernetes Services to load balance traffic to your applications in Kubernetes.</p><h3>Reference</h3><ul><li><a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/">Kubernetes Pods</a></li><li><a href="https://github.com/kubernetes/contrib/blob/master/statefulsets/zookeeper/zookeeper.yaml">The StatefulSet Configuration for Setting Up A ZooKeeper Service</a></li><li><a href="https://zookeeper.apache.org/">Official Website of ZooKeeper</a></li><li><a href="https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types">ClusterIP Services in Kubernetes</a></li><li><a href="https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer">LoadBalancer Services in Kubernetes</a></li><li><a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#labelselector-v1-meta">LabelSelector in Kubernetes</a></li><li><a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#define-readiness-probes">Readiness Probe in Kubernetes</a></li><li><a href="https://kubernetes.io/docs/concepts/configuration/assign-pod-node/">Assigning Pods to Nodes</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=42a8b971e5ed" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Services]]></title>
            <link>https://aaronzhuo.medium.com/kubernetes-services-1f4d3e43db67?source=rss-9175e0dbe19d------2</link>
            <guid isPermaLink="false">https://medium.com/p/1f4d3e43db67</guid>
            <category><![CDATA[entry-level]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Aaron Zhuo]]></dc:creator>
            <pubDate>Sun, 29 Nov 2020 23:07:16 GMT</pubDate>
            <atom:updated>2020-11-30T17:01:20.175Z</atom:updated>
            <content:encoded><![CDATA[<h3>What Is A Kubernetes Service</h3><p>A <a href="https://kubernetes.io/docs/concepts/services-networking/service/">Service</a> is a Kubernetes object that exposes a set of Pods as a network service. Moreover, it provides a service discovery mechanism that dynamically adds or removes IP addresses of Pods to its endpoint list based on the creation or deletion of these Pods.</p><h3>Service Types</h3><p>Kubernetes provides many types of services but here only those frequently used ones are introduced. You can check <a href="https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types">this document</a> for more details. Here only commonly used Services will be introduced.</p><h4>LoadBalancer</h4><p>A LoadBalancer exposes a set of Pods externally. A LoadBalancer is an L4 (Layer 4) load balancer, which means it can only utilize the information at the <a href="https://en.wikipedia.org/wiki/Transport_layer">transport layer</a> (Layer 4) to determine how to distribute client requests across a group of Pods.</p><p>Here is an example of LoadBalancer that makes the Kubernetes application foo public in the demo environment:</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: foo-service<br>  namespace: foo-demo<br>spec:<br>  type: LoadBalancer<br>  selector:<br>    app: foo<br>    environment: demo<br>  ports:<br>  - protocol: TCP<br>    port: 443<br>    targetPort: 8443<br>status:<br>  loadBalancer:<br>    ingress:<br>    - ip: 10.254.2.127</pre><p>From the spec, you can see that:</p><ul><li>It relies on the field spec.selector to select Pods.</li><li>The field status.loadBalancer shows the external IP address that is automatically assigned by Kubernetes.</li><li>The field spec.ports defines the ports that this service opens for the foo application.</li><li>With the external IP address and then open port, the service foo in the demo environment can be accessed with the address 10.254.2.127:443. When a request is sent to this address, the LoadBalancer will redirect it to port 8443 of one of the foo Pods.</li></ul><h4>ClusterIP</h4><p>A ClusterIP is a Service that exposes a set of Pods on a cluster-internal IP, which means this Service is only reachable from within the cluster. It is also an L4 load balancer that can only provide simple load balancing functionality based on information at the transport layer.</p><p>Here is an example of ClusterIP for the application foo:</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: default-grpc<br>  namespace: foo-demo<br>spec:<br>  type: ClusterIP<br>  selector:<br>    app: foo<br>    environment: demo<br>  clusterIP: 10.0.54.223<br>  ports:<br>  - name: grpc<br>    port: 8443<br>    protocol: TCP<br>    targetPort: 8443</pre><p>From the spec, you can see that:</p><ul><li>Like a LoadBalancer Service, a ClusterIP Service also relies on the field spec.selector to select Pods.</li><li>The field spec.clusterIP shows the internal IP address that is automatically allocated by Kubernetes. Only workloads within the same cluster can utilize this Service to access the application foo.</li><li>The field spec.ports defines the ports that this service opens for the application foo.</li></ul><p>Kubernetes will allocate a unique DNS address to a Service when it is created. The format of the DNS address is service-name.namespace.svc.cluster.local. For example, the DNS address for the above ClusterIP Service is default-grpc.foo-demo.svc.cluster.local.</p><h4>Ingress</h4><p>An Ingress is an object that manages external access to one or more Kubernetes applications in a cluster. It is not a Kubernetes Service, but it does provide load balancing, SSL termination, and name-based virtual hosting.</p><p>Unlike a Kubernetes Service which is L4 load balancer and can only manage one Kubernetes applications, an Ingress is a L7 (application layer) load balancer and can manage multiple Kubernetes applications based on path or hostnames. For example, the following shows an example of path-based Ingress. With this Ingress, requests with the URL foo.bar.com/foo will be redirected to service1 (with the 8000 port) while requests with the URL foo.bar.com/bar will be redirected to service2 (with the 9000 port). service1 and service2 can either be ClusterIP or NodePort Services.</p><pre>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br>  name: simple-fanout-example<br>spec:<br>  rules:<br>  - host: foo.bar.com<br>    http:<br>      paths:<br>      - path: /foo<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: service1<br>            port:<br>              number: 8000<br>      - path: /bar<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: service2<br>            port:<br>              number: 9000</pre><h3>What is Next</h3><p>I recommend you read <a href="https://azhuox.medium.com/persistent-volumes-and-persistent-volume-claims-1b2f1fbdda43">this blog</a> if you are curious about how to utilize Kubernetes Persistent Volumes and Persistent Volume Claims to provision persistent storage for your applications in Kubernetes.</p><h3>Reference</h3><ul><li><a href="https://kubernetes.io/docs/concepts/services-networking/service/">Kubernetes Services</a></li><li><a href="https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types">Kubernetes Service Types</a></li><li><a href="https://en.wikipedia.org/wiki/Transport_layer">transport layer</a></li><li><a href="https://kubernetes.io/docs/concepts/overview/">Kubernetes Overview</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1f4d3e43db67" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>