<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Engineering Exception - Medium]]></title>
        <description><![CDATA[Engineering Exception - Medium]]></description>
        <link>https://medium.com/engineeringexception?source=rss----cbb0b91da30b---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Engineering Exception - Medium</title>
            <link>https://medium.com/engineeringexception?source=rss----cbb0b91da30b---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 20:27:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/engineeringexception" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Managing secrets in Serverless]]></title>
            <link>https://medium.com/engineeringexception/managing-secrets-in-serverless-6da296cfbfb3?source=rss----cbb0b91da30b---4</link>
            <guid isPermaLink="false">https://medium.com/p/6da296cfbfb3</guid>
            <category><![CDATA[google-kms]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[berglas]]></category>
            <dc:creator><![CDATA[Random Exception]]></dc:creator>
            <pubDate>Sat, 07 Sep 2019 18:16:44 GMT</pubDate>
            <atom:updated>2019-09-07T18:16:44.520Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EDpTwQOv1RSBt3epxsnykQ.jpeg" /><figcaption>unsplash.com</figcaption></figure><p>Your serverless function often needs to talk with external API or database and you have to manage secrets in a way.</p><p>Let&#39;s define what secret is</p><ul><li>Api keys</li><li>Database credentials</li><li>Passwords</li></ul><p>and more…</p><p>I am using public github repository and I don&#39;t want to share my secrets in public and it is not best practice either.</p><p><strong>Solution 1:</strong></p><p>You could use environment variable to share secret between serverless functions</p><pre>gcloud functions deploy <strong>FUNCTION_NAME</strong> --set-env-vars DB_KEY=SECRET</pre><p>But it creates a big security leak in your project because your environment variables are reachable by all other third party libraries.</p><p><strong>Solution 2:</strong></p><p>You could use Google KMS(Key Management Service) which is cloud-hosted key management service. You could easily bind specific users to encrypt/decrypt any file and so you don’t need to define any password which is already stored in cloud. I think that’s the point, it is more secure.</p><p>Let’s say we have secrets.properties file which is included in .gitignore file so don’t commit that file. We are keeping as secret.</p><pre><strong># First create keyring that will hold all keys</strong></pre><pre>gcloud kms keyrings create mysecrets --location global</pre><pre><strong># This is the key that will encrypt secrets.properties</strong></pre><pre>gcloud kms keys create mykey --location global \<br>  --keyring mysecrets --purpose encryption</pre><pre><strong># Now encrypt secrets.properties file</strong></pre><pre>gcloud kms encrypt --location global \<br>  --keyring mysecrets --key mykey \<br>  --plaintext-file secrets.properties \<br>  --ciphertext-file secrets.properties.encrypted</pre><pre><strong># You could commit secrets.properties.encrypted file<br># To decrypt the file </strong></pre><pre>gcloud kms decrypt --location global \<br>  --keyring mysecrets --key mykey \<br>  --ciphertext-file secrets.properties.encrypted \<br>  --plaintext-file secrets.properties</pre><p>This solution is better than the first one but still you have to manage the file and even if you update any password in secrets.properties file, you have to do all process from the beginning.</p><p><strong>Solution 3:</strong></p><p>It is the last and best solution for me. I have found <a href="https://github.com/GoogleCloudPlatform/berglas">Berglas </a>project.</p><p>On the background Berglas uses Google KMS but it helps you to manage your secrets.</p><p>In this case we don’t need secrets.properties file in the project because all encrypted secrets are stored in Google Storage.</p><pre># Create bucket to store secrets in Google Storage and<br># Initialize keyrings and keys in Google KMS</pre><pre>berglas bootstrap --bucket SECRET_BUCKET</pre><p>It will create the key under projects/${PROJECT_ID}/locations/global/keyRings/berglas/cryptoKeys/berglas-key</p><p>Define your secrets</p><pre>berglas create SECRET_BUCKET/DB_KEY SECRET \<br>  --key projects/${PROJECT_ID}/locations/global/keyRings/berglas/cryptoKeys/berglas-key</pre><p>Now you could access secrets from your code.</p><pre>import (<br>	&quot;context&quot;<br>	&quot;log&quot;<br><br>	&quot;github.com/GoogleCloudPlatform/berglas/pkg/berglas&quot;<br>)<br><br>func main() {<br>	ctx := context.Background()<br>        plaintext, err := berglas.Resolve(ctx, &quot;DB_KEY&quot;)<br>	if err != nil {<br>	   log.Fatal(err)<br>	}</pre><pre>        fmt.Printf(&quot;%s&quot;, plaintext)</pre><pre>}</pre><p>Berglas uses your default google account in the environment. So be sure you logged in correct account and set it as default.</p><pre>gcloud auth application-default login</pre><p>After this setup, I don’t need to remember or setup any credential. It automatically fetches secrets from Google KMS.</p><p><em>Sources</em></p><ul><li><a href="https://cloud.google.com/functions/docs/env-var">Using Environment Variables | Cloud Functions Documentation | Google Cloud</a></li><li><a href="https://cloud.google.com/kms/">Cloud Key Management Service | Cloud KMS | Google Cloud</a></li><li><a href="https://github.com/GoogleCloudPlatform/berglas">GitHub - GoogleCloudPlatform/berglas: A tool for managing secrets on Google Cloud</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6da296cfbfb3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/engineeringexception/managing-secrets-in-serverless-6da296cfbfb3">Managing secrets in Serverless</a> was originally published in <a href="https://medium.com/engineeringexception">Engineering Exception</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Go Context]]></title>
            <link>https://medium.com/engineeringexception/go-context-c44d681da2e8?source=rss----cbb0b91da30b---4</link>
            <guid isPermaLink="false">https://medium.com/p/c44d681da2e8</guid>
            <category><![CDATA[context]]></category>
            <category><![CDATA[go]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Random Exception]]></dc:creator>
            <pubDate>Tue, 21 Aug 2018 19:11:56 GMT</pubDate>
            <atom:updated>2018-08-21T19:10:52.259Z</atom:updated>
            <content:encoded><![CDATA[<p>There are still many discussions ongoing on <a href="https://www.reddit.com/r/golang/comments/6s5a7q/context_should_go_away_for_go_2/">reddit</a> and <a href="https://github.com/golang/go/issues/20280">github</a> how bad <em>Context</em> is but I want to concentrate on this post how we can use it.</p><blockquote><em>Context</em> is developed for making it easy to pass request-scoped values, cancelation signals, and deadlines across API boundaries to all the goroutines involved in handling a request. (<a href="https://blog.golang.org/context">https://blog.golang.org/context</a>)</blockquote><p>The developers describe <em>Context</em> as above. It is simply request object that flows through your program and interact (sending signal like cancelation or timeout).</p><p>Let’s first check <a href="https://golang.org/pkg/context/">Context interface</a>.</p><pre>type Context <strong>interface</strong> {</pre><pre>  Deadline() (deadline time.Time, ok <strong>bool</strong>)</pre><pre>  Done() &lt;-chan <strong>struct</strong>{}</pre><pre>  Err() error</pre><pre>  Value(key <strong>interface</strong>{}) <strong>interface</strong>{}<br>}</pre><p><strong>Deadline</strong></p><p>Deadline returns the time when work done on behalf of this context should be canceled. Deadline returns ok==false when no deadline is set. Successive calls to Deadline return the same results.</p><p><strong>Done</strong></p><p>Done returns a channel that&#39;s closed when work done on behalf of this context should be canceled. Done may return nil if this context can never be canceled. Successive calls to Done return the same value.</p><ul><li><strong>WithCancel</strong> arranges for Done to be closed when cancel is called</li><li><strong>WithDeadline</strong> arranges for Done to be closed when the deadline expires</li><li><strong>WithTimeout</strong> arranges for Done to be closed when the timeout elapses.</li></ul><p><strong>Err</strong></p><ul><li>If Done is not yet closed, Err returns nil.</li><li>If Done is closed, Err returns a non-nil error explaining why:</li></ul><p>There are two options on this condition.</p><ol><li>Canceled if the context was canceled</li><li>DeadlineExceeded if the context&#39;s deadline passed.</li></ol><p>After Err returns a non-nil error, successive calls to Err return the same error.</p><p><strong>Value</strong></p><p>Value returns the value associated with this context for a key, or nil if no value is associated with the key. Successive calls to Value with the same key return the same result.</p><blockquote>Use context values only for request-scoped data that transitions processes and API boundaries, not for passing optional parameters to functions.</blockquote><p>A key identifies a specific value in a Context. Functions that wish to store values in Context typically allocate a key in a global variable and use that key as the argument to context. WithValue() and Context.Value(). A key can be any type that supports equality.</p><p>Packages should define keys as an unexported type to avoid collisions.</p><p>So how to use context in your program.</p><blockquote>At Google, we require that Go programmers pass a Context parameter as the first argument to every function on the call path between incoming and outgoing requests. (<a href="https://blog.golang.org/context">https://blog.golang.org/context</a>)</blockquote><p>Let’s say you need to make http request and it should call fallback function after 5 seconds timeout.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8408e1856e56f321cb227fd2428f306c/href">https://medium.com/media/8408e1856e56f321cb227fd2428f306c/href</a></iframe><h3>Resources</h3><ul><li><a href="https://golang.org/pkg/context/">context - The Go Programming Language</a></li><li><a href="https://blog.golang.org/context">Go Concurrency Patterns: Context - The Go Blog</a></li><li><a href="https://faiface.github.io/post/context-should-go-away-go2/">Context should go away for Go 2</a></li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FQDDwwePbDtw%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DQDDwwePbDtw&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FQDDwwePbDtw%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/62aade5e585abf8a446ba29c1c6f3152/href">https://medium.com/media/62aade5e585abf8a446ba29c1c6f3152/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c44d681da2e8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/engineeringexception/go-context-c44d681da2e8">Go Context</a> was originally published in <a href="https://medium.com/engineeringexception">Engineering Exception</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[SSH Port Forwarding]]></title>
            <link>https://medium.com/engineeringexception/ssh-port-forwarding-60b9dc67e545?source=rss----cbb0b91da30b---4</link>
            <guid isPermaLink="false">https://medium.com/p/60b9dc67e545</guid>
            <category><![CDATA[port-forwarding]]></category>
            <category><![CDATA[terminal]]></category>
            <category><![CDATA[ssh]]></category>
            <dc:creator><![CDATA[Random Exception]]></dc:creator>
            <pubDate>Thu, 04 Jan 2018 12:16:46 GMT</pubDate>
            <atom:updated>2018-01-04T12:18:38.041Z</atom:updated>
            <content:encoded><![CDATA[<p>It is easy to forward port external service to your local machine. You can do it with ssh protocol.</p><pre>ssh \<br>   -L <strong>local_port:service_host:service_port</strong> \<br>   -l <strong>ssh_server_username</strong> \<br>   -N <strong>ssh_server_host</strong></pre><p><strong>local_port</strong>: this is your local machine port</p><p><strong>service_host:</strong> IP address of the server to expose</p><p><strong>service_port: </strong>port of the external service</p><p><strong>ssh_server_username:</strong> SSH username of external machine</p><p><strong>ssh_server_host:</strong> host of external machine</p><p>If you want to do it with port forwarding <strong>8001</strong>, with server <strong>51.15.202.40</strong>, with user <strong>root</strong> on <strong>local(127.0.0.1)</strong> machine, use the following line</p><pre>ssh -L 8001:127.0.0.1:8001 -N 51.15.202.40 -l root</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=60b9dc67e545" width="1" height="1" alt=""><hr><p><a href="https://medium.com/engineeringexception/ssh-port-forwarding-60b9dc67e545">SSH Port Forwarding</a> was originally published in <a href="https://medium.com/engineeringexception">Engineering Exception</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Docker Container Cluster with Docker Swarm]]></title>
            <link>https://medium.com/engineeringexception/docker-container-cluster-with-docker-swarm-6fce09200765?source=rss----cbb0b91da30b---4</link>
            <guid isPermaLink="false">https://medium.com/p/6fce09200765</guid>
            <category><![CDATA[cluster]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[swarm]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[docker-swarm]]></category>
            <dc:creator><![CDATA[Random Exception]]></dc:creator>
            <pubDate>Fri, 29 Dec 2017 10:18:14 GMT</pubDate>
            <atom:updated>2018-01-05T07:57:48.016Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1JVMJleGOPUcXnAbW5MPsQ.jpeg" /></figure><p>Aim of this story to create cluster of Docker Containers and let containers to talk each other to scale up easily.</p><p>First start by installing Docker. To get the latest Docker version, install Docker from the official Docker repository.</p><pre>$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -</pre><pre>$ sudo add-apt-repository &quot;deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable&quot;</pre><pre>$ sudo apt-get update<br>$ apt-cache policy docker-ce<br>$ sudo apt-get install -y docker-ce</pre><p>That&#39;s all. It should be installed. You can check the status.</p><pre>$ systemctl status docker</pre><p>Let&#39;s create 3 Docker Containers.</p><pre>$ for i in 1 2 3; do <strong>docker-machine create node-$i</strong>; done</pre><p>After this command, you can list all machines that are created.</p><pre>$ docker-machine ls</pre><pre>NAME     ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER        ERRORS<br>node-1   -        virtualbox   Running   tcp://192.168.99.100:2376           v17.12.0-ce<br>node-2   -        virtualbox   Running   tcp://192.168.99.101:2376           v17.12.0-ce<br>node-3   -        virtualbox   Running   tcp://192.168.99.102:2376           v17.12.0-ce</pre><p>Before starting clustering node, to be sure that certain network ports must be opened.</p><p><strong>TCP port 2376:</strong> it is used to orchestrate Docker hosts</p><p><strong>TCP port 2377:</strong> it is used by only manager nodes</p><p><strong>TCP and UDP port 7946:</strong> it is used for container network discovery</p><p><strong>UDP port 4789:</strong> it is used for container ingress networking</p><p>Let&#39;s connect to the first container node (<em>node-1</em>).</p><pre>$ docker-machine ssh node-1 &quot;docker swarm init --advertise-addr 192.168.99.100&quot;</pre><p><strong><em>192.168.99.100</em> </strong>is IP address of <strong>node-1</strong>. It might be different for you.</p><p>With this command, we set <strong>node-1</strong> as manager node on the cluster.</p><pre>Swarm initialized: current node (8yzq3jjzw4v7qe253k1n43osp) is now a manager.</pre><pre>To add a worker to this swarm, run the following command:</pre><pre><strong>docker swarm join --token SWMTKN-1-5th3c7zonmbje7gctguwybhlzfn8est939lfn5hbopn79bwmal-3fgg23njgcsng14nwy2nwlwq0 192.168.99.100:2377</strong></pre><pre>To add a manager to this swarm, run &#39;docker swarm join-token manager&#39; and follow the instructions.</pre><p>You can see the output as above. Let&#39;s set other nodes as worker node. Copy the bold text above and run on other nodes.</p><pre>$ docker-machine ssh node-2 &quot;<strong>docker swarm join --token SWMTKN-1-5th3c7zonmbje7gctguwybhlzfn8est939lfn5hbopn79bwmal-3fgg23njgcsng14nwy2nwlwq0 192.168.99.100:2377</strong>&quot;</pre><pre>This node joined a swarm as a worker.</pre><pre>$ docker-machine ssh node-3 &quot;<strong>docker swarm join --token SWMTKN-1-5th3c7zonmbje7gctguwybhlzfn8est939lfn5hbopn79bwmal-3fgg23njgcsng14nwy2nwlwq0 192.168.99.100:2377</strong>&quot;</pre><pre>This node joined a swarm as a worker.</pre><pre>$ docker-machine ssh node-1 &quot;<strong>docker node ls</strong>&quot;</pre><pre>ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS<br>8yzq3jjzw4v7qe253k1n43osp *   node-1              Ready               Active              Leader<br>vmkmggeg6thzqyihnd01n7m69     node-2              Ready               Active<br>3qcxsuxewsmvty4y856230555     node-3              Ready               Active</pre><p>Our Docker Container Cluster is ready now.</p><p>Connect to <strong>node-1</strong> node</p><pre>$ <strong>docker-machine ssh node-1</strong></pre><pre>node-1 &gt; $ <strong>docker service create --name blog -p 3001:2368 ghost</strong><br>node-1 &gt; $ <strong>docker service scale blog=5</strong><br>node-1 &gt; $ <strong>docker service ps blog</strong></pre><pre>ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE                ERROR               PORTS<br>uycqq2fzto1m        blog.1              ghost:latest        <strong>node-3</strong>              Running             Running 2 minutes ago<br>ayd8hgmgptdq        blog.2              ghost:latest        <strong>node-2</strong>              Running             Running 6 seconds ago<br>k4cd4getxs7b        blog.3              ghost:latest        <strong>node-1</strong>              Running             Running about a minute ago<br>t8dkumwllnjs        blog.4              ghost:latest        <strong>node-1</strong>              Running             Running about a minute ago<br>40nqv5ex7s5l        blog.5              ghost:latest        <strong>node-3</strong>              Running             Running about a minute ago</pre><p>You see that we successfully deployed ghost image to Docker Containers and it is distributed from <strong>node-1</strong> to <strong>node-3</strong>.</p><p>Check your ghost blog is running.</p><p><a href="http://192.168.99.100:3001/">http://192.168.99.100:3001/</a></p><p>On the next blog, we will implement load balancer between those containers.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6fce09200765" width="1" height="1" alt=""><hr><p><a href="https://medium.com/engineeringexception/docker-container-cluster-with-docker-swarm-6fce09200765">Docker Container Cluster with Docker Swarm</a> was originally published in <a href="https://medium.com/engineeringexception">Engineering Exception</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>