<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kamil Piotrowski on Medium]]></title>
        <description><![CDATA[Stories by Kamil Piotrowski on Medium]]></description>
        <link>https://medium.com/@kamil.piotrowski.pol?source=rss-88c3b17f23d3------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*fKrCcQhRI3sv8vD0</url>
            <title>Stories by Kamil Piotrowski on Medium</title>
            <link>https://medium.com/@kamil.piotrowski.pol?source=rss-88c3b17f23d3------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 00:08:37 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kamil.piotrowski.pol/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Create beautiful PDFs with Golang and AWS Lambda]]></title>
            <link>https://medium.com/nordcloud-engineering/create-beautiful-pdfs-with-golang-and-aws-lambda-efb8163dc817?source=rss-88c3b17f23d3------2</link>
            <guid isPermaLink="false">https://medium.com/p/efb8163dc817</guid>
            <category><![CDATA[html]]></category>
            <category><![CDATA[template]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Kamil Piotrowski]]></dc:creator>
            <pubDate>Mon, 04 Mar 2019 08:10:19 GMT</pubDate>
            <atom:updated>2019-03-04T08:10:19.490Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hP3s8faPjoR_APG-Xvcxlw.png" /></figure><p>Golang templates allow you to generate customizable HTML documents based on the schema and input data. They are an advanced formatting language with conditions, loops, pipelines and custom formatting functions. Templates are mostly used for generating HTML documents that can be served by an HTTP server. However, they can be used for many more use cases. In this article, I will show how to generate an HTML document that will be used as an input for PDF-builder Lambda. If you want to know how we have created a Lambda that generates PDF - please follow us. We will release this article soon.</p><h3>Prepare data structure</h3><p>Because Go is a strongly types language, before we create a template schema we need to think about what we want to present and prepare a data structure. For our example, I will use the following, simple structure:</p><pre>type Report struct {<br>    ID string  `json: &quot;id&quot;`<br>    Jobs []Job `json: &quot;jobs&quot;<br>}</pre><pre>Type Job struct {<br>    ID string                         `json: &quot;id&quot;<br>    Start time.Time                   `json: &quot;start&quot;<br>    End time.Time                     `json: &quot;end&quot;<br>    User *string                      `json: &quot;user&quot;<br>    Status string                     `json: &quot;status&quot;<br>    MetaInformation map[string]string `json: &quot;meta_information&quot;<br>}</pre><p>Report contains a list of the jobs that have been processed by the system. Every job is described by execution status and the date time. Our task is to convert this structure to a PDF document that will be understandable for the end user.</p><p>Please note that every field that you want to use in the template has to be exported, this means it needs to start with a capital letter. In other case, template engine will return an error. It’s fine to use pointers for the fields that may be empty. However, accessing nil value will result in an error so you need to make sure that field is safe to display by using if else statement in your template.</p><h3>Create template</h3><p>Once the data structure is ready, it’s time to create the template schema. You can either assign it to the variable or create a separate file containing the template. The difference is the function used for parsing the template. In this section, I presented the basic features of the template engine. If you want to know more please visit this <a href="https://golang.org/pkg/text/template">site</a>.</p><h4>Accessing fields</h4><p>Golang template engine uses moustache{{ }} syntax to represent dynamic actions. You can access the data using the{{ .FieldName }} or the {{ . }} (for the non-struct types) syntax. Please remember that the variable you want to refer to cannot be nil, otherwise, engine return nil pointer dereference error.</p><p>Example:</p><pre>&lt;h1&gt;{{ .ID }}&lt;/h1&gt; //Report.ID</pre><h4>Loop over a list and a map</h4><p>For displaying iterable data types, like lists or maps, you can use the loop syntax in a template. There is a slight difference in how you iterate over list and map items. For the list, you have to use the {{ range .ListFieldName }} … {{ end }} syntax. Inside the loop you are in the iterated item scope, so you can access its data in the standard way.</p><p>Example:</p><pre>{{ range .Jobs }}<br>    &lt;h2&gt;{{ .ID }}&lt;/h2&gt;                      //Report.Jobs[].ID<br>    &lt;h4&gt;{{ .Status }}&lt;/h4&gt;                  //Report.Jobs[].Status<br>    &lt;h4&gt;{{ .StartTime.Format &quot;02.01.2006 15:04&quot; }}&lt;/h4&gt;<br>    &lt;h4&gt;{{ .StartTime.Format &quot;02.01.2006 15:04&quot; }}&lt;/h4&gt;<br>{{ end }}</pre><p>To iterate over the map you have to use the {{ range $key, $value := .MapFieldName }} .. {{ end }} syntax. Inside loop you can access <em>$key</em> and <em>$value</em> variables using the {{ $key }} or {{ $value }} statement.</p><p>Example:</p><pre>{{ range $key, $value := .MetaInformation }}<br>    &lt;h5&gt;&lt;b&gt;{{ $key }}&lt;/b&gt;{{ $value }}&lt;/h5&gt;<br>{{ end }}</pre><h4>If … else statements</h4><p>For the conditional actions, you can use the {{ if condition }} .. {{ else }} .. {{ end }} syntax. What’s more, you can combine multiple conditions using the <em>and</em> or the <em>or</em> operator inside the if function: {{ if and condition1 condition2 }}.</p><p>Example:</p><pre>{{ if .User }} <br>    {{ .User }}<br>{{ else }} <br>    unknown user<br>{{ end }}</pre><pre>{{ if not (eq (len .MetaInformation) 0 ) }}<br>     …<br>{{else }} <br>    Missing meta information <br>{{ end }}</pre><p>The above example protects us from the accessing nil variable (User field is a pointer) and shows how to check if map or list is not empty.</p><h4>Pipelines and custom formatters</h4><p>Another interesting feature of golang templates are the pipelines. A pipeline is a sequence of formatting commands and arguments separated with the <strong>|</strong> character. The real fun begins when you can create your own, custom formatting function. What you need to do is to create a variable of type template.FuncMap containing the functions definitions.</p><p>Example:</p><pre>import (<br>    &quot;html/template&quot;<br>)</pre><pre>var funcMap = template.FuncMap{<br>    &quot;ToLower&quot;: strings.ToLower,<br>    &quot;Remove_&quot;: func(val string) string {<br>        return strings.Replace(val, &quot;_&quot;, &quot;&quot;, -1)<br>    },<br>}</pre><p>Example:</p><pre>&lt;h4&gt;{{ .Status | ToLower | Remove_ }}&lt;/h4&gt; //Report.Jobs[].Status</pre><h3>Execute Template</h3><p>Once the data structure and the template are ready we can combine them and generate the final document:</p><pre>buf := &amp;bytes.Buffer{}<br>t := template.New(&quot;template.html&quot;).Funcs(funcMap)<br>t, err := t.ParseFiles(&quot;/templates/template.html&quot;) //path to the template</pre><pre>if err != nil {<br>    log.Error(&quot;Failed to parse report template&quot;, err.Error())<br>    return err<br>}</pre><pre>err = t.Execute(buf, data) //Execute template, output in buf<br>if err != nil {<br>    log.Error(&quot;Failed to generate report template&quot;, err.Error())<br>    return err<br>}</pre><p>The above example shows how to load the template from the file. Please note that the name of the template has to be the same as one of the parsing files names. Otherwise, template parser will fail. If you defined your own formatting functions you can bind them with the template using the .Funcs() method.</p><h3>Summary</h3><p>Golang templates deliver really powerful formatting language that can be used for generating various documents. Advanced actions like loops, conditions, and pipelines allows you to focus more on how the document should look like rather than how to build it. If you want you can even use your own, custom formatting functions. That feature makes golang templates really customizable.</p><p>In Nordcloud we are using templates to generate reports from our applications, that are converted to PDF’s, and delivered to the end users. If you are interested in how we built our pdf-builder lambda please follow us. We will release this article soon.</p><p>At Nordcloud we are always looking for talented people. If you enjoy reading this post and would like to work with GO projects on a daily basis — check out our open positions <a href="https://nordcloud.com/careers/">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=efb8163dc817" width="1" height="1" alt=""><hr><p><a href="https://medium.com/nordcloud-engineering/create-beautiful-pdfs-with-golang-and-aws-lambda-efb8163dc817">Create beautiful PDFs with Golang and AWS Lambda</a> was originally published in <a href="https://medium.com/nordcloud-engineering">Nordcloud Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tracing serverless application with AWS X-Ray]]></title>
            <link>https://medium.com/nordcloud-engineering/tracing-serverless-application-with-aws-x-ray-2b5e1a9e9447?source=rss-88c3b17f23d3------2</link>
            <guid isPermaLink="false">https://medium.com/p/2b5e1a9e9447</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[x-rays]]></category>
            <category><![CDATA[tracing]]></category>
            <category><![CDATA[lambda]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Kamil Piotrowski]]></dc:creator>
            <pubDate>Wed, 26 Dec 2018 23:06:09 GMT</pubDate>
            <atom:updated>2018-12-26T23:15:04.004Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is AWS X-ray?</h3><p><a href="https://aws.amazon.com/xray/">AWS X-Ray</a> is a service that records and visualize requests made by your application. It can be used to analyze, monitor and debug distributed applications based on the AWS Lambdas or microservices architecture. With AWS X-Ray you can f.ex.:</p><ul><li>Keep track of every request made to other AWS services,</li><li>Discover issues with an application performance</li><li>Visualize the path of a single request through the entire application,</li><li>Detect problems with application permissions.</li></ul><p>AWS X-Ray supports many computing services such as EC2, ECS, Elastic Bean Stack, <strong>AWS Lambda</strong> and runtimes. In this article, I will focus on the serverless applications built with AWS Lambdas based on the Golang 1.X runtime.</p><h4>Segments and subsegments</h4><p>AWS X-Ray receives information about requests made by your application in the objects named <strong>segments</strong>. Every segment may contain multiple <strong>subsegments</strong> which give you a more detailed view of the operations made by the application. Segments with the same <strong><em>Trace-ID</em></strong> are grouped into the single trace and visualized as a service graph. The Trace-ID is generated by the first service in the request and propagated to the other services by adding the X-Amzn-Trace-Id header.</p><h4>Service Graph</h4><p>Every trace is visualized as a graph containing nodes that represent different AWS services or requests. Every node has its status presented with a different color:</p><ul><li>Green - Success</li><li>Orange - Error (4XX Response)</li><li>Red - Fault (5XX Response)</li><li>Purple - Throttled request (429 Response)</li></ul><h3>How to use it?</h3><h4>Configuration</h4><p>Enabling AWS X-Ray for AWS Lambda is really simple. You just need to enable <strong><em>active tracing</em></strong> in the AWS console:</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*LSKRqtpeANipF-Lw" /></figure><p>or if you use cloud-formation to deploy your Lambda, you need to add the following parameter:</p><pre>Properties:<br>  TracingConfig:<br>    Mode: Active</pre><p>You also need to allow your Lambda to send X-Ray segments by adding at least the following permissions to it’s IAM policy:</p><pre>- “xray:PutTraceSegments”<br>- “xray:PutTelemetryRecords”</pre><p>These two changes are enough for your Lambda to start sending X-ray segment containing information that Lambda has been invoked. However, any other request won’t be recorded. To achieve that you need to install xray-sdk library and make few changes in your function code. The proper library for golang runtime is github.com/aws/aws-xray-sdk-go. Please note that version v1.0.0-rc.1 or higher is required as X-Ray support for AWS Lambda has been added in this version.</p><h4>HTTP requests</h4><p>AWS X-Ray allows you to record every outgoing HTTP request made by your application. However, it won’t be done automatically. For every request you want to record, you have to pass the additional <strong><em>Context</em></strong> containing Trace-ID and the segment data. The following example contains the HTTP client configuration adjusted to send the X-Ray segment:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/08f28880cd803145db029fc07e678341/href">https://medium.com/media/08f28880cd803145db029fc07e678341/href</a></iframe><p>First, we wrapped the default net/http client with the X-Ray client. However, it’s not enough. We also need to pass the <strong><em>Context</em></strong> variable to the function that makes HTTP request. To achieve that, we need to use the golang.org/x/net/context/ctxhttp library that allows us to do it. Example code from above should result in the following tracing graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/0*V-tRloa0CU90Kcbd" /></figure><h4>AWS services</h4><p>AWS X-Ray allows you to record every call to the other AWS service, such as dynamoDB, S3, SNS and many many more. Similarly to HTTP requests you need to put some effort to force AWS Lambda to send X-Ray segments containing this information:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/28940fa905da289fd6053329e62257f0/href">https://medium.com/media/28940fa905da289fd6053329e62257f0/href</a></iframe><p>As you can see we have to wrap created dynamoDB Client with the xray.AWS() function that adds the X-Ray tracing configuration to the DynamoDB Client. The next step is to pass the <strong><em>Context</em></strong> to the AWS function that we want to execute. Basic functions do not allow that, however, AWS provided equivalent WithContext functions. The above example should result in the following tracing graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/791/0*mumDqaTOLafgkl07" /></figure><p>In this case, AWS Lambda does not have permissions to call DynamoDB API. This is why you can see orange nodes representing an error.</p><h4>API Gateway support</h4><p>If your Lambda works as an API gateway proxy integration that handles API requests, you can benefit from the X-Ray support for this service. What you need to do, is to enable support for the active tracing in the API gateway stage configuration:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/534/0*cue1qy3nnrpqwGzC" /></figure><pre>ApiGatewayDeployment:<br>  Type: “AWS::ApiGateway::Deployment”<br>  Properties:<br>    StageDescription:<br>       TracingEnabled: true</pre><p>After this change API gateway will generate (if missing) and add X-Amzn-Trace-Id header to every incoming request. Based on this header AWS Lambda creates the first segment. The following function contains example how to handle API gw request in the AWS Lambda:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7a35a13707bbf8cf0d5263b39f00ab3b/href">https://medium.com/media/7a35a13707bbf8cf0d5263b39f00ab3b/href</a></iframe><p>It results with the following tracing graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/925/0*Jm9tsR7HNq1Ew2-C" /></figure><h4>Custom segments</h4><p>We already know how to record HTTP, AWS, and API gateway requests. AWS X-Ray gives you also a possibility to define your own segments or subsegments to highlight a piece of your code. You can use it for measuring the performance, adding metadata or just splitting the whole logic into smaller sections for better visualization.</p><p>The following example shows how to create custom subsegment and pass to it additional metadata:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7351d602cb01f960f0cefb90657278b8/href">https://medium.com/media/7351d602cb01f960f0cefb90657278b8/href</a></iframe><p>It results with the following tracing graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/0*CpZSruFbl6CLirs7" /></figure><p>You may be surprised that custom subsegment is not visible on the graph, but it can be found in the trace details:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5JqUqG7-QuilK7K6" /></figure><h3>Example application</h3><p>Now it’s time to create a whole application and show the real power of the X-Ray service. As an example, I have created a simple distributed application containing three HTTP endpoints, from each one is handled by the different Lambda:</p><ul><li><em>POST </em><strong><em>/</em></strong><em>items - </em>adds a new item to the DynamoDB table and sends a notification,</li><li><em>GET /items/{id} - </em>reads item from the dynamoDB,</li><li><em>GET /id_generator </em>- returns new UUID.</li></ul><p>I tried to use all the X-Ray features described in the previous section. The detailed implementation is not that important, but you can check it on my GitHub <a href="https://github.com/kpiotrowski/aws_lambda_xray_example/tree/master/simple">page</a>. Are you able to understand how the application works only by looking at the following X-Ray graph?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FPKMIf4NYF4k13Ps" /><figcaption>Application service graph — generated by the X-Ray service</figcaption></figure><h4>Find a bug</h4><p>This application is not perfect. I hid there at least 3 bugs/performance issues (it was written only for the test purpose so it may contain many more :) ). Are you able to find them by checking the node graph or reviewing attached trace details? To simulate production environment I executed around 200 POST and 2000 GET item requests.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*buslS54J1TeRhDqTTACvGw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nupgpyUUWDaqjWdyGLnLtA.png" /></figure><p>If you have a problem with finding the bugs here is the solution:</p><ul><li>Look at the SNS node - it’s orange which indicates an error. In the trace details, you can find that the call to this service returned 403 status code. It’s because I forgot to add required sns:Publish permissions to the Lambda IAM role.</li><li>Creating a new item takes too much time. As you can see, the requests took more than 3 seconds. When you analyze the whole trace you will understand that it’s because of the poor performance of the service that generates an ID. I have added 3s sleep to simulate this :)</li><li>Some of the requests made to the DynamoDB service returned status 400. In the trace details, we can find information that it was due to the Provisioned Throughput Exceeded Exception. AWS SDK tried to retry the request to dynamoDB a few times, but because of the Lambda timeout, it eventually returned an error. It seems that read throughput set to 1 was not enough to handle around 2k requests :)</li></ul><h3>Problems and issues</h3><p>While working with the AWS X-Ray I found some issues:</p><ul><li>You cannot trace async operations. For example, my use case is that one of the functions publishes notification to the SNS topic that triggers another function to send a slack notification. AWS X-Ray does not allow me to trace requests like that. However, I think there can be a workaround for this. But this is a topic for another article.</li><li>You should handle errors in the proper way, depends on what do you want to see in the X-Ray graph. For example, if you return an error from the Lambda function you’ll see that the function segment resulted in an error. In other case, if you return a response containing an error message and status 500 you won’t see that Lambda failed, but the result for the end user will be the same.</li><li>Custom subsegments that you create are not visible on the node graph, but it’s intended behavior. Subsegments are made to group some lines of code together and measure it’s performance or add additional metadata. However, I managed to create a custom segment inside my Lambda that was visible on the node graph:</li></ul><pre>//Begin segment generates new Trace ID so we need to override it with the existing one</pre><pre>ctx, seg := xray.BeginSegment(ctx, “MySegment”)<br>seg.ParentID = xray.GetSegment(ctx).ID<br>seg.TraceID = xray.GetSegment(ctx).TraceID<br>time.Sleep(time.Second*2)<br>seg.Close(nil)</pre><h3>Other runtimes</h3><p>When comparing golang Xray SDK with other implementation, we find out they are quite similar - the idea is the same. The main difference is how we use it on our code. Let’s compare it with the <strong>xray-python</strong> SDK. Recording request is much much easier than doing it in the golang app. It requires you to run a single patch() function to catch and record <strong>EVERY</strong> request in your code:</p><pre>from aws_xray_sdk.core import patch</pre><pre>libs_to_patch = (‘boto3’, ‘requests’)<br>patch(libs_to_patch)</pre><p>We can say that it works out of the box, without big changes in your application. Unfortunately, the golang app may require some refactoring to support X-Ray.</p><h3>Summary</h3><p>It was a lot of fun to test AWS X-Ray service. I’m impressed by its features and how it helped to debug my test application. However, using it in the golang application requires a lot of additional work like wrapping HTTP or AWS clients and passing Context to every function. When we build a new application it shouldn’t be a problem. We just need to remember that we write an application that uses X-Ray. But for the existing applications, it may require a lot of refactoring to make it really beneficial.</p><p>I’m not sure if would waste a lot of time adjusting the golang application to support X-Ray. Especially if it has good logging and monitoring. In this case, The X-Ray may not be very beneficial. What’s more, in my opinion, X-Ray cannot replace well-designed monitoring so it should be used as an additional tool to help debug the application, not as the main one.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b5e1a9e9447" width="1" height="1" alt=""><hr><p><a href="https://medium.com/nordcloud-engineering/tracing-serverless-application-with-aws-x-ray-2b5e1a9e9447">Tracing serverless application with AWS X-Ray</a> was originally published in <a href="https://medium.com/nordcloud-engineering">Nordcloud Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to build pluggable Golang application and benefit from AWS Lambda Layers.]]></title>
            <link>https://medium.com/nordcloud-engineering/how-to-build-pluggable-golang-application-and-benefit-from-aws-lambda-layers-154c8117df9b?source=rss-88c3b17f23d3------2</link>
            <guid isPermaLink="false">https://medium.com/p/154c8117df9b</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[lambda]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[plugins]]></category>
            <dc:creator><![CDATA[Kamil Piotrowski]]></dc:creator>
            <pubDate>Sat, 15 Dec 2018 16:30:12 GMT</pubDate>
            <atom:updated>2019-01-17T00:06:38.928Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TArmNwMaoXjR1MegmucwyQ.png" /></figure><h3>Golang — why is it worth your attention?</h3><p>Golang is an open source programming language designed and implemented by Google. It’s very widely used in modern applications especially in the cloud. It’s the most characteristic features are:</p><ul><li>Golang is statically typed - it provides less flexibility, but protects you from making mistakes,</li><li>It is not object-oriented. However, you can create structures and interfaces and that gives you 3 of 4 OOP principles: data abstraction, encapsulation, and polymorphism. Inheritance is the only one missing,</li><li><strong>Goroutines</strong>! — the greatest implementation of the light threads I have been ever using. It allows you to create a new thread in the super easy way using go operator and communicate between different goroutines using channels,</li><li>It compiles to the single binary with all dependencies — no more packages’ conflicts!</li></ul><p>Personally, I consider Golang as the greatest language which I use on the daily basis. However, this article won’t be about creating your first function or printing “Hello World”. I will show You a little bit more advanced stuff. If you are a beginner and want to learn more about Golang please visit its main <a href="https://golang.org/">page</a>.</p><h3>AWS Lambda &amp; Golang</h3><p><a href="https://aws.amazon.com/lambda/"><strong>AWS Lambda</strong></a> is one of the most popular serverless compute services in the public cloud, released in November 2014 by Amazon Web Services. It allows you to run your code in response to the events like DynamoDB, SNS or HTTP triggers without provisioning or managing servers! Do you know what is really great? Since January 2018 it supports Golang runtime. Working with AWS Lambda is really simple - just upload a zipped package with your code and all the dependencies (single binary when using Golang).</p><p>Fast forward,4 years later at 2018 re:Invent AWS releases <a href="https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html"><strong>Lambda Layers</strong></a> that allows you to store and manage data that is shared across different functions in the single or even multiple AWS accounts! For example, While using Python you can put all the dependencies in an additional layer which can be later used by other Lambdas. There is no need to put different dependencies in each zipped packages anymore! In the Golang world situation is different as AWS Lambda requires you to upload compiled binary. How can we benefit from the AWS Lambda Layers? The answer is simple - build a modular application using <strong>Golang Plugins</strong>!</p><h3>Golang Plugins — a way to build a modular application</h3><p>Golang Plugins is the feature released in the Go1.8 that allows you to dynamically load shared libraries (.so files). It gives you an opportunity to export some of your code to the separate library or use the plugin prepared and compiled by someone else. It is promising, however, there are a few limitations:</p><ul><li>Your Plugin has to be single main module,</li><li>You can load only functions and variables that are exported as ELF symbols,</li><li>Due to static typing, you have to cast every loaded symbol to the correct type. In the worst scenario, you need to define the correct interface in your code,</li><li>It works only for the Linux and MacOS. Personally, I do not consider this as a disadvantage :)</li></ul><h3>Building and testing your first plugin</h3><p>Now let’s create our first plugin. As an example, we will create a simple module for the string encryption. Let’s go back to the basics and implement 2 simple encryption algorithms - Ceasar and Verman.</p><ul><li>Caesar cipher is the algorithm firstly used by Julius Ceases. It shifts every letter in the text by the fixed number of positions. For example, if you want to encrypt the word golang with the key 4 you will get ktpek. The decryption works in the same way. You just need to shift the letters in the opposite direction.</li><li>Verman cipher is similar to the Ceaser, based on the same shifting idea, the difference is that you shift every letter by the different number of positions. To decrypt the text you need to have the key containing the positions used to encrypt the text. For example, if you want to encrypt the word golang with the key [-1, 4, 7, 20, 4, -2]you will get future.</li></ul><p>The full implementation of this example is available <a href="https://github.com/kpiotrowski/golang_modular_app">here</a>.</p><h4>Plugin implementation</h4><p>The following snippet contains the implementation of the two algorithms mentioned above. For each one we implement 2 methods of encrypting and decrypting our text:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/647debf1ff1dbea4d3af1cdb2829dca9/href">https://medium.com/media/647debf1ff1dbea4d3af1cdb2829dca9/href</a></iframe><p>As you can see, we exported here 3 different symbols (Golang exports only these identifiers that starts with the upper letter):</p><ul><li><strong>EncryptCeasar</strong> - func (int, string) string that encrypts text using Ceasar algorithm,</li><li><strong>DecryptCeaser</strong> - func (int, string) string that decrypts text using Caeser algorithm,</li><li><strong>VermanCipher</strong> - variable of type vermanCipher implementing 2 methods: <strong>Encrypt</strong>: func (string) string and <strong>Decrypt</strong>: func () (*string, error)</li></ul><p>To compile this plugin you have to run following command:</p><pre>go build -buildmode=plugin -o plugin/cipher.so plugin/cipher.go</pre><p>For now, there is nothing special - few simple functions were created and a module was compiled as a plugin by adding the -buildmode=plugin argument.</p><h4>Load and test plugin</h4><p>The fun begins when we want to use compiled plugin in our app. Let’s create a simple example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9861e44248f0e6a52b6d38e478c505b4/href">https://medium.com/media/9861e44248f0e6a52b6d38e478c505b4/href</a></iframe><p>First, you need to import the golang plugin package. It contains only two functions - the first one is for loading shared library and the second one is for finding an exported symbol. To load your library you have to use Open function which requires providing the path to your shared plugin and returns variable of type Plugin. If loading the library is not possible (ex. incorrect path or corrupted file) this function returns the error that has to be handled.</p><p>The next step is to load every exported symbol using the Lookup method. A slight inconvenience is that you need to load every exported function separately. However, you can combine multiple functions together in the same way as it was done for the VermanCipher symbol. Once you have loaded all the symbols you want to use you have to cast them to the correct type. Golang is statically typed language so there is no other way to use these symbols without casting. Remember, when you export a variable that implements a few methods, you need to cast it to the correct interface type (I had to define encryptionEngine interface to handle this).\newline \newline</p><p>To compile and run the app use the following command:</p><pre>go build app.go<br>./app</pre><p>In the output, you should see the encrypted and decrypted text as a proof that algorithm works correctly.</p><h3>Use plugin in AWS lambda</h3><p>To use our plugin in the AWS Lambda we need to make a few modifications in our application:</p><ul><li>AWS Lambda mounts layers to the /opt directory in the lambda container, so we have to load our plugin from this directory.</li><li>We need to create a handler function that will be used by the Lambda engine to handle our test event.</li></ul><p>The following snippet contains our application adjusted to be used by the Lambda:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7d7f88ba90845eacfa68f4c6954cc3f3/href">https://medium.com/media/7d7f88ba90845eacfa68f4c6954cc3f3/href</a></iframe><p>As you can see the implementation is very similar to the previous one. We have only changed the directory from which we loaded our plugin and added the function response instead of printing values. If you want to learn more about writing Lambdas in golang please check the AWS <a href="https://docs.aws.amazon.com/lambda/latest/dg/go-programming-model-handler-types.html">documentation</a>.</p><h4>AWS Lambda deployment</h4><p>There are two ways of deploying AWS Lambda functions and layers. You can create and upload zipped package manually or use the more advanced framework, which makes it much easier and faster. For most of my projects, I use the <a href="https://serverless.com/">Serverless</a> framework, so I have already prepared the simple serverless.yml configuration file using this tool:</p><pre>service: cipherService<br>frameworkVersion: &quot;&gt;=1.28.0 &lt;2.0.0&quot;<br>provider:<br>  name: aws<br>  runtime: go1.x</pre><pre>layers:<br>  cipherLayer:<br>    path: bin/plugin<br>    compatibleRuntimes:<br>      - go1.x</pre><pre>functions:<br>  engine:<br>    handler: bin/cipherEngine<br>    package:<br>      exclude:<br>        - ./**<br>      include:<br>        - ./bin/cipherEngine<br>    layers:<br>      - {Ref: CipherLayerLambdaLayer }</pre><p>In the layers section we defined a single layer with the path to the already created plugin - it will be deployed together with the lambda function. You can define up to 5 different layers which order is really important. They are mounted to the same /opt directory, so layers with the higher number can override files from the previously mounted layers. For every layer, you need to provide at least 2 parameters: path to the directory containing layer source (path to the plugin binary in your case) and the list of compatible runtimes.</p><p>The next functions section is a place where you define the list of the functions to be deployed. For every function, you need to provide at least the path to the compiled application. In addition, to that, we need to define the layers parameter with the reference to the layer defined above. This will automatically attach the layer to our Lambda function during the deployment. The funny thing is that you have to convert your lambda layer name to be TitleCased and add the LambdaLayer suffix if you want to refer to that resource. It seems that the Serverless team implemented it in this way to solve the conflict with reference to the different type of resources.</p><p>Once our serverless.yml configuration file is ready, the last thing to do is to compile our app, plugin and deploy it. We can use simple Makefile for that:</p><pre>.PHONY: build buildPlugin clean deploy</pre><pre>build:<br> dep ensure -v<br> env GOOS=linux go build -ldflags=&quot;-s -w&quot; -o bin/cipherEngine cipherEngine/main.go</pre><pre>buildPlugin:<br> env GOOS=linux go build -ldflags=&quot;-s -w&quot; -buildmode=plugin -o bin/plugin/cipher.so ../plugin/cipher.go</pre><pre>clean:<br> rm -rf ./bin ./vendor Gopkg.lock</pre><pre>deploy: clean buildPlugin build<br> sls deploy --verbose</pre><p>You can build and deploy your function by running the following command:</p><pre>make deploy</pre><h4>Test AWS Lambda</h4><p>As I mentioned earlier AWS Lambda executes code in the response to the event. However we did not configure any event triggers, so it won’t be invoked without our help. We have to do it manually using the Serverless framework or the awscli tool:</p><pre>sls invoke -f function_name<br>aws lambda invoke — function-name function_name output_file</pre><p>In the response, you should see the same output as before, which proves that our lambda function works correctly and loads the plugin from the additional layer. Now you can create other functions that will use the same layer or even share it with other AWS accounts.</p><h3>Summary</h3><p>It was a lot of fun to use Golang modules and test how to integrate them with the newly released AWS Lambda Layers. Plugin library is really awesome, however because of its limitations and Golang specification it can be used only in some special scenarios. I think that for most of the developers who are working on the standard projects it won’t be needed or even possible to use plugins. Only two reasons come to my mind:</p><ul><li>Implementing complicated algorithms that can be used by the other applications ex. video coding or encryption algorithms.</li><li>Sharing your algorithm with others without publishing its code.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=154c8117df9b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/nordcloud-engineering/how-to-build-pluggable-golang-application-and-benefit-from-aws-lambda-layers-154c8117df9b">How to build pluggable Golang application and benefit from AWS Lambda Layers.</a> was originally published in <a href="https://medium.com/nordcloud-engineering">Nordcloud Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>