<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Serif &amp; Semaphore - Medium]]></title>
        <description><![CDATA[The best-laid plans of unicorns and men… - Medium]]></description>
        <link>https://medium.com/serif-semaphore?source=rss----1f6afab5b862---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Serif &amp;amp; Semaphore - Medium</title>
            <link>https://medium.com/serif-semaphore?source=rss----1f6afab5b862---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 17:01:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/serif-semaphore" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Serverless Framework Landscape]]></title>
            <link>https://medium.com/serif-semaphore/the-serverless-framework-landscape-ef58365b3f17?source=rss----1f6afab5b862---4</link>
            <guid isPermaLink="false">https://medium.com/p/ef58365b3f17</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[framework]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <dc:creator><![CDATA[Tom Maiaroto]]></dc:creator>
            <pubDate>Fri, 13 Jul 2018 17:34:04 GMT</pubDate>
            <atom:updated>2018-07-13T17:34:03.854Z</atom:updated>
            <content:encoded><![CDATA[<h4>Approaches, considerations, and the state of affairs in 2018</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uVnkwpO44sSu6RyYOKFTYw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/oZPwn40zCK4?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Thomas Kvistholt</a> on <a href="https://unsplash.com/?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>We’re just over half way through the year, but I feel it’s a fair time to reflect on the serverless space. I will also make some predictions.</p><p>I’ve been researching and using serverless in production since AWS Lambda was announced. I follow many frameworks out there. My work has also included personal research and development of an application framework called <a href="https://github.com/tmaiaroto/aegis">Aegis</a>.</p><p>So where are we at with the frameworks? I’m an advocate for many projects. I believe everyone is doing something very important —<em> there’s a huge change occurring in the industry right now.</em></p><h4>The Number of Serverless Frameworks</h4><p>It’s been increasing. I think one of the first (April 2015?), and most well known, is <a href="https://serverless.com/">Serverless</a>. Another early one was <a href="http://apex.run/">Apex</a> (December 2015?). If you were into Go then you may have heard of <a href="http://gosparta.io/">Sparta</a> (October 2015?). In fact, Tim Wagner told me about some collaboration with Sparta’s <a href="https://twitter.com/mweagle">Matt Weagle</a> for Go in AWS Lambda. Clearly AWS was working with the community from very early on. Another early framework was <a href="https://github.com/jorgebastida/gordon">Gordon</a>; also started in 2015. I consider these some of the earliest contributors to the movement that also had big impact.</p><p>Sorry if I left anyone out. Of course there are others. I could rattle off a list, or simply link a few “awesome” repositories that have tons of resources. Here’s <a href="https://github.com/anaibol/awesome-serverless">one</a>. Here’s <a href="https://github.com/pmuens/awesome-serverless">another about Serverless framework</a> specifically and <a href="https://github.com/SerifAndSemaphore/go-serverless-list">another specific to Golang</a> that I maintain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/930/1*PX-YYXUmXF3YGZl8-Pk9wQ.png" /><figcaption>Top most starred Serverless Frameworks as of July 12th, 2018 (chart created by <a href="https://livegap.com/charts">https://livegap.com/charts</a>).</figcaption></figure><p>By the following year, 2016, the number of frameworks certainly more than doubled. <a href="https://github.com/claudiajs/claudia">ClaudiaJS</a>, <a href="https://github.com/aws/chalice">Chalice</a>, and <a href="https://github.com/Miserlou/Zappa">Zappa</a> were quick out of the gate that year following on the heels of 2015’s frameworks.</p><p>By this point I had been using Apex, Serverless, and tinkered with Sparta and some others. I think <a href="https://twitter.com/tjholowaychuk">TJ Holowaychuk</a>’s work with Apex spoke to me the most then. However, I wasn’t finding exactly what I was after so I created <a href="https://github.com/tmaiaroto/aegis">Aegis</a> toward the end of 2016. This was before official Golang support so there were inherit bugs and inconsistencies when using shims (all fixed with official Go support).</p><blockquote>Today, I now count <strong><em>at least 20</em></strong> different serverless frameworks.</blockquote><p>For more usage statistics, <a href="https://serverless.com/blog/serverless-by-the-numbers-2018-data-report/">Serverless released an article</a> recently that has some in depth metrics based on their observations.</p><p>So why use a framework? The biggest reason, in my mind, is convention. You want to work with others and follow a certain paradigm. You also want to ship applications faster.</p><h4>The Considerations of Serverless Frameworks</h4><p>There is a lot to consider. It’s no surprise there’s so many frameworks. What exactly are they trying to solve though?</p><p>For many of these frameworks, a big consideration is language support. Everyone wants a serverless framework in the language they feel most comfortable with and this is one way to get just that.</p><p>Cloud providers are doing a fair job of supporting more languages. Amazon is in the lead, Microsoft supports some that others don’t (like F#, C#, and PHP). Google Cloud Functions surprise me. There’s a hack to run Go binaries, but why they aren’t officially supporting the language? Bonkers to me.</p><blockquote>The most universally supported language is Node.js.</blockquote><p>At this point you can write serverless functions in Java, C#, Python, Go, PHP, Rust, Swift, and more. It’s just a matter of finding the right framework and/or cloud provider.</p><h4>Infrastructure Management</h4><p>Perhaps the biggest concern for the majority of frameworks is infrastructure management. Many frameworks could care less about your code. Their goal is remove the complexities of infrastructure. A favorite passage comes to mind:</p><blockquote>No slave in the Roman galleys could have toiled so frightfully and lived, for this man thought himself a free man, and that he was working for his wife and babes. — Hamlin Garland</blockquote><p>Right? We go on about how much easier it is <em>without</em> servers and start coining phrases like <em>NoOps</em>…Yet, ironically, it’s not so simple. We need these frameworks and tools to help configure, deploy, and maintain the stack.</p><p>I think <a href="https://www.terraform.io/">Terraform</a> is perhaps the leader in the infrastructure management. Certainly at least when it comes to supporting multiple cloud providers.</p><p>The <strong><em>infrastructure as code</em></strong> movement has been well adopted by people in the serverless space. In fact, I think the newer framework, <a href="https://pulumi.io/">Pulumi</a> got a lot of things right when it decided to leverage Terraform.</p><p>Of course for AWS, there CloudFormation. Many frameworks, like Serverless, leverage that too. Others have taken to SDKs. Then Amazon spotted this growing concern and gave us <a href="https://github.com/awslabs/serverless-application-model">SAM</a>.</p><blockquote>Clearly, we’re still searching for something here.</blockquote><p>I’m not sure anyone has cracked the code yet. I think it boils down to the approach you’re most comfortable with. That varies a great deal depending on your role.</p><p>A few frameworks, like <a href="https://pulumi.io/">Pulumi</a> and <a href="http://gosparta.io/">Sparta</a>, are taking a different approach to infrastructure. They use application code to provision and manage infrastructure. Sparta is about “self deploying” applications and I think Pulumi may not be far behind in this concept as well.</p><p>I do think self deploying, self updating, serverless apps are important. Great opportunities exist for licensed apps that run in the cloud based off this idea.</p><h4>The Divide Between Application &amp; Infrastructure</h4><p>Or is it a blend? Again, frameworks like Pulumi and Sparta are designed to allow the application provision infrastructure itself.</p><p>So what does your serverless application look like? Should it define the infrastructure it needs? Or do you consider that a separate concern?</p><blockquote>I think, in 2018, we’re beginning to test this marriage of infrastructure and application through code.</blockquote><p>I think that’s where we’re at. People are going to weigh in with their opinions and I think the masses will give us an answer. Perhaps even before the year is through. It could be a divided opinion, or preference, too and that’s perfectly fine and healthy.</p><p><strong><em>Where does this leave the application developer?</em></strong></p><p>For starters, it’s a lot easier for them to manage infrastructure. Whether it’s through their code or through configuration files, there’s an abundance of tooling now.</p><p>However, what I see lacking in the serverless landscape right now is a focus on the application development. It’s all about infrastructure management, monitoring, and debugging.</p><p>That’s part of why I created <a href="https://github.com/tmaiaroto/aegis">Aegis</a>. I wanted to consider the application architecture and make the actual coding of serverless functions faster (well, at least if you’re using Go and AWS).</p><h4>A History of Web Application Frameworks</h4><p>I have a long history with RAD, or <em>rapid application development</em>, frameworks. I started with <a href="https://cakephp.org/">CakePHP</a> in alpha (2005), quickly followed the <a href="http://li3.me/">Li3 (Lithium)</a> gang, then went on to be exposed to and used several other frameworks since. A lot of this history is steeped in the <em>convention over configuration</em> movement and Ruby on Rails.</p><p>Honestly, that’s almost as far back as web application frameworks go (not forgetting about ColdFusion). There were various projects that helped you out before the whole RoR stuff…But that was a big change in the industry. <em>Just like right now.</em></p><p>Simultaneously, Flash/ActionScript was in hyper growth. It slowed JavaScript progress for a few years. That made it all the more easy for these MVC frameworks to handle the “view” aspect of things.</p><p>Every language then jumped on that bandwagon. From Django to Grails (formerly Groovy on Rails) to Sails (much later on). It was ORM, MVC, “monolith” nirvana. You could use practically any database you wanted right out of the box. There was support for <em>everything; </em>if there wasn’t, it was often considered a “bad” framework.</p><p>Though Sails was among one of the only such frameworks for Node.js, something started to change by then. Before serverless, people were fearing the monolith. So package managers became more important; Composer, NPM, Ruby Gems, Bower, and so on. SVN and submodules were out. Git was in. <em>We began to break things apart.</em></p><p>About the time Node.js came around (2009, 2010), many frameworks started to take a more minimalist approach.</p><p>As the years went on and the rise of SPAs (single page applications) became evident, more backend frameworks slimmed down and changed their approach. Everything was mainly API focused. A separation of frontend and backend code. It was a change from rendering HTML and serving JavaScript from the web server. In fact, S3/CloudFront was foreshadowing the serverless movement at this time.</p><blockquote><em>Around 2012, the monolith began to fall out of favor.</em></blockquote><p>JavaScript frameworks were here to stay and sprinkles of jQuery on various pages was out. AngularJS (2010) led the charge, but here we are now with Vue.js and React.js among others. We realized we didn’t need to configure and load balance a web server for the frontend. Then the Heroku era pondered, was it required for the backend?</p><h4>Tying it all Together</h4><p>So how would you tie all these things together in the serverless era? The “frontend” isn’t really just about JavaScript. Think about IoT and mobile. Of course it may not even be a user or device’s interaction with the backend at all. There can be many “actors” in the cloud.</p><p>It boils down to <em>events</em>. Events are the messages that act as the glue between the frontend, backend, our IoT devices, automated processes, and more.</p><p>There’s no shortage of clever names in technology so we have projects like <a href="https://gloo.solo.io/">Gloo</a> that take aim at this consideration.</p><p>Serverless again pushes forward with research here too in their <a href="https://serverless.com/event-gateway/">Event Gateway</a>.</p><p>I think technologies like <a href="https://www.envoyproxy.io/">Envoy</a> are going to become increasingly important here. Though many of us won’t realize it because only few will need to work at that level. The idea of being “cloud native” is taking off and as such we have organizations like the <a href="https://www.cncf.io/">Cloud Native Computing Foundation</a>.</p><p>These kind of committees remind me of <a href="https://www.php-fig.org/">PHP-FIG</a>; it’s a sign of maturity.</p><p>The CNCF is helping to draft some specs and put some definition around <a href="https://github.com/cncf/wg-serverless/tree/master/proposals/cloudevents">cloud events</a> as well.</p><p>More and more I see serverless finding its spot in this cloud ecosystem. I’m aware of the debate between serverless and containers, but I don’t see these technologies as competing at all. I see them as complimentary and I see everything being tied together.</p><h4>The Future</h4><p>So we now have this third type of framework for infrastructure.</p><p>Yes, Salt, Chef, Ansible, Puppet, and others existed well before serverless, but I feel that configuration and infrastructure frameworks are broadening their audience now.</p><p>I was recently asked by both <a href="https://twitter.com/austencollins">Austen Collins</a> and <a href="https://twitter.com/Ganesh__RK">Ganesh RK</a> of <a href="https://twitter.com/goserverless">Serverless</a> where I thought serverless was going. I mostly agreed with them about “events,” but to each of them I also said there’s a lot of maturation, growth, and adoption ahead of us still.</p><p>I promised my prediction and here it is summed up: <strong>Convention over configuration.</strong></p><p>Eventually, the configuration gets so complex and detailed that it stands out as an issue. I’ve seen configuration files that end up being longer than code in some cases. <em>That’s a problem.</em></p><blockquote>Anything is well organized if you’re the only one using it.</blockquote><p>The challenge is in how we communicate and work with one another. Same goes for services. The more services needing to “talk” to each other, the more complex and disorganized things get.</p><p><em>I believe that these experiments in using code to define and provision infrastructure needed will be critical in guiding the future.</em></p><p>Think about those RAD frameworks — they took years of fragmented pieces of code and helped bring convention and speed to the table.</p><p>I believe convention over configuration, along with some other themes, has a bigger role to play here too.</p><p><strong>The things I’m looking for in the future:</strong></p><ul><li>A reduction of lengthy, over engineered, static configuration</li><li>Code to provision infrastructure (perhaps even some clever introspection, generators, scaffolding)</li><li>Self deploying and updating functions</li><li>Self managed/healing functions (automatic rollbacks, circuit breakers)</li><li>A potential move away from all the CLI “deploy” commands and change in toolchains</li><li>Conventions in messaging and “events”</li></ul><p>This is the dawn of a new era. There will always be variations in features and approaches, but I think the future does hold some sort of consensus or pattern that stands out the most. History has a way of repeating itself.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef58365b3f17" width="1" height="1" alt=""><hr><p><a href="https://medium.com/serif-semaphore/the-serverless-framework-landscape-ef58365b3f17">The Serverless Framework Landscape</a> was originally published in <a href="https://medium.com/serif-semaphore">Serif &amp; Semaphore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Cognito Protected Serverless API with Golang in Minutes]]></title>
            <link>https://medium.com/serif-semaphore/a-cognito-protected-serverless-api-with-golang-in-minutes-a054c9f50cf3?source=rss----1f6afab5b862---4</link>
            <guid isPermaLink="false">https://medium.com/p/a054c9f50cf3</guid>
            <category><![CDATA[serverless-architecture]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Tom Maiaroto]]></dc:creator>
            <pubDate>Thu, 12 Apr 2018 17:02:07 GMT</pubDate>
            <atom:updated>2018-04-12T17:02:06.598Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PpLinNInbgNM01IykypLpA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@ja5on">Jason Chen</a></figcaption></figure><h4>The Need for Speed</h4><p>Over a year ago I set out on a fun side project to build myself a solution to all the boilerplate associated with building Lambda functions. Second to that, a solution for using Go with AWS Lambda. The goal was convenience and speed.</p><p>While there are a few other frameworks, none of them seemed to capture what I was after. Don’t get me wrong, they’re great and all, but I just wanted something different. So I started <a href="https://github.com/tmaiaroto/aegis">Aegis</a>.</p><p>Recent native Go support for Lambda rekindled my interest in the project; I was able to quickly ditch the Node.js shim and expand upon the framework. Enough to now call it a “framework” instead of just a tool. As I went to use it in production, I found more helpful features to add.</p><p>The latest feature added was a big one on my list — Cognito support. That is to say, interfaces and helper functions for making life easier when using Cognito.</p><h4>Enough Talk, Show me the <a href="https://github.com/tmaiaroto/aegis/blob/master/examples/cognito/main.go">Code</a>!</h4><p>The deploy tool won’t manage Cognito user pools for you. So you’ll first want to ensure you setup a new user pool (a quick process from AWS web console). You can follow the instructions in the <a href="https://github.com/tmaiaroto/aegis/tree/master/examples/cognito">readme of the example.</a> You can also check out the main.go file in there, and it’s whopping <strong><em>114</em></strong> lines of code! Whopping? Wait, no, that’s measly.</p><p>I’m sure you can make it less than 114, I wasn’t trying to make it the shortest thing ever, just easy to follow and deploy.</p><p>Go get Aegis setup, change to that example directory, plugin your user pool ID, etc. and deploy. The deploy took 1 minute and 32 seconds and most of that is in the upload time. You should be able to have a Cognito protected API up in less time than it takes to read this article.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*SiHPTcWFa95hhwtNXmXQ9Q.png" /><figcaption>Sure, looks legit.</figcaption></figure><p>We’re leveraging AWS Cognito hosted pages for registering users and logging in. Under the hood, we’re exchanging an authorization code for JWTs. Then we’re verifying the access_token. Your typical OAuth 2.0 workflow really.</p><p>Then we’re using some middleware on our event handlers to protect paths in the API. All of this occurs inside one Lambda. So we aren’t restricting access to the invocation of a Lambda, but rather preventing access to certain types of events handled by the Lambda…And we have very granular control over this from an easy to use interface.</p><p>Of course if you wanted to have a separate Lambda handling other events or APIs, you still could use the same JWTs for securing it as well.</p><p>The two meatiest pieces of code for auth here are the callback handler and the token verification function. Here’s the example callback that would be part of your application code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/441883a372762584f17dca8bd2bc4a96/href">https://medium.com/media/441883a372762584f17dca8bd2bc4a96/href</a></iframe><h4>Under the Hood</h4><p>There’s a lot you won’t need to worry about coding too. We take the “code” and exchange it for a few tokens (id_token, access_token, refresh_token). We take the access_token and verify it. If valid, we set a Secure HttpOnly cookie so we can check it in our middleware later on.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/802bbbeb430d6b02ffb76d0fb407dcfa/href">https://medium.com/media/802bbbeb430d6b02ffb76d0fb407dcfa/href</a></iframe><p>CognitoAppClient is an interface that helps us out quite a bit. Two major functions are exchanging authorization codes for JWTs and then verifying JWTs. It also figures out AWS Cognito hosted page URLs for you as well. It just saves you from a lot of boilerplate.</p><p>Two heroes here are the <a href="https://github.com/dgrijalva/jwt-go">https://github.com/dgrijalva/jwt-go</a> and <a href="https://github.com/lestrrat/go-jwx">https://github.com/lestrrat/go-jwx</a> packages. They do most of the heavy lifting when it comes to parsing and verifying.</p><p>I think you can figure out how the middleware provided by Aegis works now too. It just takes the incoming API Gateway HTTP request (event, remember Lambda passes events as map[string]interface{}) and finds the access_token cookie to verify. It returns an error if not valid. Otherwise it lets the flow of execution continue into the handler.</p><p>You can also use your own middleware of course.</p><h4>Even More Helpers!</h4><p>Another thing Aegis provides are helpers to get cookies. So if you did want to write your own middleware, this should help you out.</p><p>While an API Gateway HTTP request doesn’t actually come in as map[string]interface{} because Amazon’s Go Lambda package marshals it to an APIGatewayProxyRequest, it still doesn’t make getting the cookies much easier. If you’re familiar with Go’s http package, you’ll know getting cookies is very simple.</p><blockquote>Part of Aegis’ goal is to be conventional.</blockquote><p>So Aegis provides the same helper function to get cookies. Either all cookies or a single cookie by name. The way it does this is by creating a fake http.Request from that Lambda event and then proxying the very same functions.</p><p>Now you can call req.Cookie(&quot;access_token&quot;) for example.</p><p>While I constantly have to remind myself Lambda works with events, in this context I want to code against those messages as if they were incoming HTTP requests. I want to use Go’s standard library.</p><p>Perhaps the biggest helper for you is the example code. Just copy it and adjust to taste. You should be up and running with Cognito, API Gateway, and Lambda in no time.</p><h4>Speed as in Performance?</h4><p>Perhaps an important question for many. Just how long does it take for a response here? Is this feasible?</p><p>Yes! I believe so. Aegis will look up some information from Cognito using AWS SDK so your initial “cold” Lambda invocation will incur this penalty of waiting on a few requests. However, after that, everything necessary to verify the JWT is there. So subsequent “warm” invocations will be much faster.</p><p>I’ve seen response times vary in the range of 25ms to 2 seconds. Obviously the 2 second responses are those that need to deal with AWS SDK operations. The shorter responses are those that are just simply verifying a token and returning a simple API response.</p><blockquote>Don’t worry, performance is acceptable.</blockquote><p>Obviously, your mileage will vary here depending on what your event handlers are doing.</p><p>I think many people focus on the performance of AWS Lambda and API Gateway too much. I see it come up as the chief complaint all the time. That’s actually really good for Amazon if that’s all people can come up with.</p><p>Over the past year API Gateway has become a lot faster and now with native Go support, I’m seeing Lambda invocation times decreasing as well (for all languages actually). So folks…This is an addressable concern.</p><p>The reality is that yes, there is a small “tax” to pay for the convenience. Your typical Docker based API with Go will likely be faster. However, you’re going to need to load balance it in order to scale and you aren’t coding and deploying your Cognito protected API in a matter of minutes that way either.</p><p>Only you can determine if the development, deployment, and response speed is fast enough for your needs. For most general applications, I think it is.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a054c9f50cf3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/serif-semaphore/a-cognito-protected-serverless-api-with-golang-in-minutes-a054c9f50cf3">A Cognito Protected Serverless API with Golang in Minutes</a> was originally published in <a href="https://medium.com/serif-semaphore">Serif &amp; Semaphore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Quickly Deploy a Golang GeoIP Microservice]]></title>
            <link>https://medium.com/serif-semaphore/aws-lambda-geoip-golang-microservice-with-aegis-91bae736c1b2?source=rss----1f6afab5b862---4</link>
            <guid isPermaLink="false">https://medium.com/p/91bae736c1b2</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Tom Maiaroto]]></dc:creator>
            <pubDate>Thu, 29 Mar 2018 00:19:38 GMT</pubDate>
            <atom:updated>2018-04-13T15:43:58.015Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A7ngTDB0QHVhYeiBuzm_Dw.jpeg" /><figcaption>Wonderful photo by lenabell (<a href="https://unsplash.com/@lenabell">https://unsplash.com/@lenabell</a>)</figcaption></figure><h4>With the help of Aegis for AWS Lambda</h4><p>Recently Amazon announced native support for Go in Lambda. They also cut someone in half on stage. Cool. Honestly, the announcements made at re:Invent were exciting and soon after, Go was available to use with Lambda in early 2018.</p><p>I’d even say <em>deploy a serverless geoip microservice service in seconds</em> if it weren’t for the size of the data set. However, for about 30 minutes of coding and less than 100 lines, we really have a nice service here.</p><h4>The Results &amp; Performance</h4><p>Let’s cut to the chase. You know how a simple geoip lookup works. We’ll go over how the <a href="https://github.com/tmaiaroto/go-lambda-geoip">code</a> works, but most folks are interested in how it actually works. Is putting a geoip lookup microservice in AWS Lambda something you’d want to do? Sure!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/826/1*fo8MuLFC6MDDHikWbsiiEA.png" /></figure><p>The end result is an API that you can call to get a JSON response with geolocation data based on your IP address as well as an RPC to use with other Aegis based functions (or anything that makes AWS SDK calls to invoke a Lambda).</p><p>The API is pretty quick too. On average, it takes about 1 second to return a response.</p><p>There’s a few things you can do to tweak performance as well with Lambda, such as increasing the amount of memory the Lambda uses and keeping the container “warm.”</p><p>The actual time spent loading the geoip data set (using Maxmind’s free GeoLite2 City database…thank you Maxmind!) is about 1–1.5 seconds. However, subsequent Lambda invocations can re-use the same container if executed soon after the initial invocation. So pinging the API every few minutes would be what’s known as keeping it warm. That way, the burden of the initial load of that data set is avoided/minimized.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wMT137y0SS4GfwuvscZkyg.png" /><figcaption>The initial data set load was pulled out as a separate segment and traced with XRay. Incredibly helpful!</figcaption></figure><p>After that data set is loaded, handling the request is extremely fast. Using 1,204MB of RAM with the Lambda, along with Go, produces some great results. Most of the time is really spent in the actual invocation of the Lambda, API Gateway, some measurable time with DNS as well. The Aegis framework will wrap all handlers with XRay tracing by default (this strategy can be changed) and so we can easily see that it took 1ms for the Lambda to handle the request and 0.1ms for the actual function that did the look up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*52swGbfPbADrhhdYfPYGEg.png" /><figcaption>Go on AWS Lambda is quite fast.</figcaption></figure><p>Again, this service not only works with API Gateway, but is a microservice in its own right. That is to say, other Lambdas can invoke it. It’s still acceptably fast (an average of 60ms or so isn’t bad, you’re still billed for 100ms anyway). Here’s an example XRay trace:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hLSjJeu3TR3DdjOa1AwFxg.png" /><figcaption>Lambda calling Lambda using Aegis framework’s RPC helpers.</figcaption></figure><h4>Show me the Code</h4><p>You can take a look at the <a href="https://github.com/tmaiaroto/go-lambda-geoip">go-lambda-geoip repository here</a>. It’s pretty small. Just a main.go file of under 100 lines and a config file for <a href="https://github.com/tmaiaroto/aegis">Aegis</a>.</p><p>You’ll need a few prerequisites in order to deploy it into AWS yourself. The readme has instructions. It should only be a few commands and a few minutes before you’re in business. Given you are a Go developer with AWS credentials all setup on your machine already, it is likely the fastest way to deploy a geoip service for you, period.</p><p>The way Aegis (and Lambda) works with Go is through a blocking process that listens for incoming messages. AWS uses TCP under the covers for this, so it’s fair to think of it like a web server to a degree.</p><blockquote><strong>Side Note:</strong> In the past there have been several shim approaches for getting Lambda to work with Go. Some involved piping data through stdin while others used HTTP servers. Looks like Amazon went with a similar TCP solution. Many of their structs were nearly identical to Aegis, so migrating it from a Node.js shim to native Go was a breeze!</blockquote><p>Aegis adds some convenience in by allowing you to route the events to various handlers along with providing default/fall through handlers. It does this for various types of events; requests from API Gateway, CloudWatch event rules, or Lambda invocations via the SDK. Aegis will continue to add more triggers/routers/handlers over time.</p><p>In this case, we registered a handler for the root path to the API endpoint as well as an RPC handler which will handle any invocation of Lambda with a conventional JSON message format.</p><pre>{<br>  _rpcName: &quot;lookup&quot;,<br>  ipAddress: &quot;127.0.0.1&quot;<br>}</pre><p>Since there is no way to know if it was invoked via the SDK (unlike the API Gateway which carries a very specific signature), the convention is an <em>”_rpcName”</em> key with a value that references the handler. Like all “routers” in Aegis, there could also be a catch all.</p><p>The intent of “RPC” handlers is to allow this Lambda to be called from other Lambdas (or anything using the AWS SDK really).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZwpJaUOPajnav9TTR8EUKw.png" /><figcaption>An example service map with an “rpc_test” Lambda calling this “aegis_geoip” Lambda.</figcaption></figure><p>So that’s basically what you’re seeing in the code. The bulk of it being the loading of the data set and registering the handlers. This all then “starts” with a simple, blocking, call:</p><pre>listeners.Handle()</pre><p>Note that we could load the data set in each handler, but that’s wasteful. That means every single invocation would go through the process of loading that large-ish file each time. It takes about 1–1.5 seconds for this. So you’re going to have a slow function on your hands.</p><p>To mitigate this, the code loads that data set in the<em> main()</em> function <em>before</em> the handlers listen for events. Now on the initial invocation, it loads the data set, but on subsequent invocations the Lambda container is likely to be re-used and so your responses will be much quicker.</p><p>The <em>APIGatewayProxyResponse</em> struct provides us with the client IP address if using the API Gateway. Otherwise, you’ll need to pass it in the RPC event JSON. Then the <em>lookup()</em> function is very simple here. It just uses another Go package that reads from the data set. It returns city level information.</p><p>That’s all there is to it. For illustrative purposes, here’s the code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/10966e5dc30ef78c3c9e6fde1c9e63ac/href">https://medium.com/media/10966e5dc30ef78c3c9e6fde1c9e63ac/href</a></iframe><p>Aegis is a deploy tool as well (actually that was its first goal). It will build, zip, and upload the Lambda code along with setting up the API. It will create an execution role in IAM for you too. You can adjust configuration if you like in the <em>aegis.yaml</em> file.</p><p>The snippet above is likely to become dated, so if you want to actually deploy this service, use the repository. Note that it also does not include the XRay timings around the geoip data set loading that are seen in the screenshots above.</p><p>The best part of all this, it’s <em>serverless</em>. It runs fast enough to stay within the minimum billing per request (100ms) and if you don’t use it, you won’t be charged. In general, geolocation services (more the geocoding aspects) can get rather expensive, but using Lambda really helps reduce that cost.</p><p>In future articles (or perhaps even videos), I’ll go through how to use RPCs with Aegis and cover more architectural decisions and options available to you with Aegis.</p><p><a href="https://github.com/tmaiaroto/go-lambda-geoip/blob/master/main.go">tmaiaroto/go-lambda-geoip</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=91bae736c1b2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/serif-semaphore/aws-lambda-geoip-golang-microservice-with-aegis-91bae736c1b2">Quickly Deploy a Golang GeoIP Microservice</a> was originally published in <a href="https://medium.com/serif-semaphore">Serif &amp; Semaphore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Setup Google Analytics for Medium]]></title>
            <link>https://medium.com/serif-semaphore/how-to-setup-google-analytics-for-medium-e04b14f56f25?source=rss----1f6afab5b862---4</link>
            <guid isPermaLink="false">https://medium.com/p/e04b14f56f25</guid>
            <category><![CDATA[google-analytics]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[seo]]></category>
            <category><![CDATA[medium]]></category>
            <category><![CDATA[marketing]]></category>
            <dc:creator><![CDATA[Tom Maiaroto]]></dc:creator>
            <pubDate>Fri, 06 Jan 2017 19:36:55 GMT</pubDate>
            <atom:updated>2018-05-08T04:52:37.902Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rxuW29UDTKriLKUcemOKWg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@shwetha">Shwetha Shankar</a></figcaption></figure><h4>A guide for not flying blind</h4><p><strong><em>Note: Medium has since changed some things, so this information is dated in some cases.</em></strong></p><p>I still debate whether or not using Medium for my blog was a smart move. The transition has been brutal on my traffic numbers, but it’s getting there. However, Medium affords you some extra visibility. It’s a social network as much as a news platform and simple hassle-free place to write.</p><p>The biggest disadvantage one has with Medium is that there is no way to add custom code. The second thorn in many people’s paw is the lack of insight. Medium’s stats are limited.</p><p>Fortunately, Medium now allows you to use Google Analytics! You can <a href="https://help.medium.com/hc/en-us/articles/215832277-Are-Google-Analytics-available-on-Medium-">submit a request</a> giving them your Analytics ID and they will add it for you (pretty quickly too).</p><p><strong>Note: You will need a publication and </strong><a href="https://help.medium.com/hc/en-us/articles/214110127-About-custom-domains"><strong>custom domain</strong></a><strong> for this.</strong></p><p>Once done, you’ll start seeing session data and such. It’s pretty nice because you now get demographic information (age, gender, location, etc.).</p><h4>1. Making Things More Useful With Segments</h4><p>So this is worlds better than Medium’s stats, but still not wonderful. What else can you do? Well, you can set up a few segments in GA.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BZFtb3EU0vE1NHHrDRfbEQ.png" /><figcaption>A segment of users coming from Medium’s reading list</figcaption></figure><p>If you look at all of the pages visited you can see Medium’s querystrings and how they send referral data. These are not exactly standard UTM params unfortunately, so things aren’t automatically picked up by GA.</p><p><strong><em>Reading Lists<br></em></strong>One of the important keys is source and it can contain a variety of values. The above screenshot shows how you’d go about building a segment of users who came to your publication from Medium’s reading list (reading_list is the value here).</p><p>However, you should note that there are multiple reading lists on Medium and each source value will being with the phrase reading_list, so if you want to look at specific ones you’ll need to craft your regular expression a bit differently.</p><p>For example, here’s the regex match for visitors from the Golang reading list: \?source\=reading\_list.+golang</p><p>How many are there? Good question. Unlimited I imagine, I believe it’s based on tags. So you’ll want to look at your site content filtering URLs to only show those containing ?source=</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Qg75HW3XZfPSCvKYbQ5n2Q.png" /><figcaption>A variety of values for source=</figcaption></figure><p>You’ll see some pretty odd ones that contain a series of dashes with numbers. You may want to ignore these, my best guess is that the numbers represent some sort of position in a grid.</p><p><strong><em>Footer Placement Cards</em></strong><br>You know at the bottom of each article how there’s some related stories in the footer? Ever wonder if you’re appearing in those places and how many people are reading your content due to that placement? Well, here ya go. You can define a segment for those with ?source=placement_card_footer</p><p><strong><em>E-mail &amp; RSS</em></strong><br>Medium sends out e-mails and referrals from those too have this source param; digest.reader is a good way to spot them. In fact, those e-mails are also as specific as the reading lists in that there are specific types of e-mail digests for each tag (or at least the popular ones).</p><p>Did you know Medium has <a href="https://help.medium.com/hc/en-us/articles/214874118-RSS-Feeds-of-publications-and-profiles">RSS for you</a>? One of the listed limitations on the support page for setting up GA says that sessions from RSS won’t be reflected. This is only if they are reading from the RSS itself and not if they came from a link within it. So you can see referrals from RSS feed. How often do people come from the RSS feed instead of just read it and move on? Your mileage will vary, but it’s there under ?source=rss</p><p><strong><em>Tags</em></strong><br>As you inspect your site content more you’ll pick up some URL schemes that you may find useful. For example, you can create a segment of users who viewed the pages for each article tag. This can be useful if you are trying to figure out which tags are working for you.</p><p>Medium’s URL for tag lists is pretty simple; /tagged/golang for example. I’ll use this to see how many people were looking for similar stories on my publication.</p><p>Keep in mind this is not Medium’s page with other people’s content, it’s your own publication’s tagged stories. Their tagged articles list is under medium.com/tag/golang for example.</p><p>Want to know how many visitors you got from Medium’s tagged lists? Instead of building your segment with <strong>Page</strong> or <strong>Landing Page</strong>, instead use <strong>Referral Path</strong> and <em>contains</em> /tag/xxxx</p><h4>2. Google Webmaster Tools (Search Console)</h4><p>Oh yea, you didn’t forget about this handy tool did you? It goes by the name <a href="https://www.google.com/webmasters/tools/home?hl=en">Search Console</a> these days. If you are using Medium with a custom domain, <strong><em>be sure to add your Medium publication to it.</em></strong></p><blockquote>Google Search Console is extremely valuable for your Medium Publication.</blockquote><p>This is extremely valuable as it includes information about your Google indexed pages, search appearance, structured data (yup, Medium makes use of it), sitemaps, and <strong>search analytics </strong>(also visible in GA).</p><p><strong><em>Indexed Pages &amp; Sitemaps</em></strong><br>Medium does indeed have an XML sitemap for your publication. It’s at /sitemap/sitemap.xml and it won’t be given to Google by default (as far as I can tell). By adding it yourself it may help Google discover and index your content faster.</p><p>You can also easily see how many indexed pages you have with the search console and if anything is blocked or removed.</p><p>Granted, you don’t have tons of control to change things should there be a problem, but this tool does provide some insight.</p><p><strong><em>Structured Data</em></strong><br>Medium does make use of schema.org structured data on its pages. It’ll take a little bit to see the data appear in Search Console, but in a few days you’ll see it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qDI4d4kFDad9hzBO-82igA.png" /><figcaption>Medium makes use of structured data for your</figcaption></figure><p>Not happy with the data? Well, if you’re daring you could make use of Search Console’s <strong>Data Highlighter</strong> tool. <em>Here’s where things could get interesting.</em></p><p>First off let me first say that it’s very hard to highlight the date from your articles using this tool because the date on your pages is in a hard to parse format. It will say things like days ago, etc. and it also won’t have a year anywhere to pick up. So relying on Medium’s structured data tags may be a safer bet.</p><p>However. If you want to add your own date to each of your articles, you are free to do so within the body of your story (maybe place it at the end in italics or something). Then you should be able to highlight this value to generate the structured data.</p><p>Ok, not super clean, but ok. What else? Well, perhaps <em>star ratings</em>? Do you use Medium to review things?</p><p>You might be able to put a consistent element for star ratings with some text that can be highlighted. Then pick it out with Data Highlighter to help generate the structured data for it. These star ratings would then appear in Google search results. Neato! Those orange stars in your Google search result can have a dramatic affect.</p><p><strong><em>Search Analytics</em></strong><br>By far, this is the most valuable thing that the Search Console provides and it also feeds that data into Google Analytics for you too.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8wKdJmYNLOd8uqNnbi-ghg.png" /><figcaption>Google search info for your Medium content!</figcaption></figure><blockquote>Search Analytics for Medium is one of the most valuable things you can setup.</blockquote><p>It’ll take a few days to start appearing, but it will provide you with a bunch of handy information. It shows you which search queries you are ranking well for. It also shows how many impressions and clicks you are receiving from them.</p><p>This really can help guide your content by identifying specific keywords to be sure to include in your articles.</p><p><strong><em>Links to Your Content</em></strong><br>Last, but not least the Search Console also shows you other sites/pages linking to your content. It also shows which content is linked the most.</p><p>This is also very valuable because it can help you figure out which articles are more likely to get picked up by other publications and aggregators. It also helps you identify content that could be linked more; so you can actually do something about promoting it.</p><h4>Action Items, Final Thoughts</h4><p>I covered a lot here, but it’s important to note that the two biggest steps here are:</p><ol><li>create a GA property and request Medium to add its GA tracking ID</li><li>add your publication’s domain in Google Search Console</li></ol><p>The sooner you do this, the sooner data starts flowing.</p><p>You will need a custom domain (or subdomain) to use with Medium as well as a Medium Publication. Again, this guide isn’t really intended for users without publications.</p><p>Creating a publication is easy and free on Medium. They provide you a lot of flexibility that you won’t otherwise get (layout options, etc.). So I do highly suggest every writer to set one up.</p><p>However, one thing you still won’t get are any events. I wish Medium would instrument everything. Hey, Medium if you’re listening??? Please send a GA event each time someone <em>clicks any link in an article</em>, clicks follow, share, favorite, highlights, or comments…Please…</p><p>I highly doubt they’ll let us use GTM because that could lead to a mess, but it would fix all of Medium’s disadvantages — it would allow you to add custom code to your pages.</p><p>Regardless,<strong> you don’t have to fly blind on Medium.</strong> It’s a wonderful tool and while you can’t control every detail, it’s still a very viable replacement for your blog.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e04b14f56f25" width="1" height="1" alt=""><hr><p><a href="https://medium.com/serif-semaphore/how-to-setup-google-analytics-for-medium-e04b14f56f25">How to Setup Google Analytics for Medium</a> was originally published in <a href="https://medium.com/serif-semaphore">Serif &amp; Semaphore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build a Serverless API in Seconds with Go]]></title>
            <link>https://medium.com/serif-semaphore/build-a-serverless-api-in-seconds-with-go-c504398d86f6?source=rss----1f6afab5b862---4</link>
            <guid isPermaLink="false">https://medium.com/p/c504398d86f6</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <dc:creator><![CDATA[Tom Maiaroto]]></dc:creator>
            <pubDate>Sun, 18 Dec 2016 17:51:55 GMT</pubDate>
            <atom:updated>2016-12-18T17:55:31.581Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f5PEulQxutJXcq5eYDYdEQ.jpeg" /><figcaption><a href="https://unsplash.com/@igorovsyannykov">Photo by Igor Ovsyannykov</a></figcaption></figure><h4>The future of rapid application development</h4><p>There’s a landscape change on the internet every six years or so. One is happening right now. Maybe you’ve heard buzzwords such as <em>“serverless”</em> or <em>“microservices”</em> or <em>“NoOps.”</em></p><p>During the course of my research, I built a few open-source projects. One of these I nicknamed <a href="https://github.com/tmaiaroto/aegis"><em>Aegis</em></a>. What this project has allowed me to do is build a scalable RESTful API with Go (golang) in an extremely rapid fashion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GTeUe8nUsPBAf_Oo03M_ZQ.jpeg" /><figcaption>Then the`up` command deployed a scalable API in seconds.</figcaption></figure><p>While building <a href="https://github.com/tmaiaroto/aegis">Aegis</a>, I didn’t want to re-invent any wheels per se. I have used several tools for serverless hosting providers and have enjoyed them all. I have great respect for them and their maintainers. Their goals are for speed and scalability as well. I believe this is the future of rapid application development.</p><p>Though some of these frameworks and tools set out to accomplish <em>many goals for many users</em>. They support multiple languages, service providers, and are feature rich. <a href="https://github.com/SerifAndSemaphore/go-serverless-list">Here’s a complete list of serverless resources for Go</a>.</p><blockquote>I was looking for a simpler solution.</blockquote><p>I have worked on open-source frameworks before and have used many others. I enjoy their internals and comparing them to understand the different approaches they take. I like to understand the benefits and tradeoffs. I don’t believe in the word <em>best</em> in this regard.</p><p>I’m a firm believer in keeping frameworks lean yet modular and open. Sometimes this means being a little opinionated when it allows for a common problem to be solved really well. It’s a delicate balance.</p><p>When I was comparing <a href="https://serifandsemaphore.io/azure-cloud-functions-vs-aws-lambda-caf8a90605dd">Azure Functions with AWS Lambda</a> I noticed something extremely convenient about Azure. That was that it provided you with a URL to call your function. It supported any HTTP method. It was essentially an effortless mini API. A <em>hook</em> or <em>trigger</em> is fair, but in it I saw a glimmer of something with a lot more potential.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*eNLleKBATeNvBuHfFqdp9Q.png" /><figcaption>API Gateway previously required each HTTP method to be configured if you wanted them all.</figcaption></figure><p>Now, at that time AWS API Gateway was a bit of a painstaking process to get setup to work the same. I think someone was listening though. Since then, Amazon has added an ANY HTTP method to API Gateway resources which handles any method (POST, GET, PUT, etc.). They also have a <em>Lambda Proxy</em> which can be used with a <em>proxy path</em> to handle any URL path with any number of querystring parameters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RwGxUXxmd36G0Y6D-HpN2w.png" /><figcaption>You can now simplify your API Gateway configuration. This can handle the same requests as the one above.</figcaption></figure><p>The configuration is less involved now, but still not as simple as Azure’s. Regardless, it functionally provides what I was after. Yet, none of the other tools take advantage of it (yet).</p><p>I started Aegis under an opinionated pretense as an experiment. <strong>It uses AWS Lambda and API Gateway with Go</strong>. The scope is small, but it has room to grow. I’m not sure I want to call it a framework, today, but it’s beginning to really define a paradigm. A very familiar one if you’re used to writing APIs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PzKbXMhAZrdXbsG-ue9ARA.png" /><figcaption>Hey, this looks rather familiar...</figcaption></figure><p>Along with being a little opinionated, <em>Aegis</em> also was able to simplify things. That was by design.</p><blockquote><strong>It takes just one command to deploy your highly scalable API.</strong></blockquote><p>I was further motivated to build this tool by <a href="https://medium.com/@tjholowaychuk/blueprints-for-up-1-5f8197179275#.6b24imoqt">TJ Holowaychuk’s article</a> (the author of <a href="http://apex.run/">Apex</a>). Though I also understand why he is not making API Gateway more a part of Apex for now (including noted concerns about API Gateway’s speed).</p><p>There’s certainly more features that I’d like to add to Aegis. This is only the beginning, but already it’s proving to be a valuable time saver for me.</p><p><strong>So what exactly do you get from Aegis?<br>What’s the point?</strong></p><ul><li>Single command deploys/updates for your API</li><li>A single AWS Lambda to handle each API<br><em>(this is a fairly big differentiator, more on this below)</em></li><li>A “router” that will allow you to write your API in a more familiar way</li><li>Route handlers</li><li>Middleware</li><li>POST form-data support</li><li>A variety of helpers for working with incoming event data</li><li>Environment variable support</li><li>API response cache support</li><li>Simple YAML based configuration for your API and Lambda settings</li><li>Automatic logging to CloudWatch (Go’s log functions will go straight to CloudWatch and there’s also Logrus included with Aegis so you can easily add adapters to log anywhere)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wPHdPXl53gdGZmdJlzE6ug.png" /><figcaption>Aegis has a helper function that will parse multipart form data.</figcaption></figure><p><a href="https://github.com/tmaiaroto/aegis">Aegis</a> was written in Go and can be installed as a binary.</p><p>If you’re starting from scratch, in a clean directory, you’ll likely want to run aegis init to create a little starter app and configuration.</p><p>Then simply by running aegis up the program will use your configured AWS credentials (typically in a simple ~/.aws/credentials file) and deploy from the current working directory to Amazon. It will show you the API endpoint URL in your terminal. You need not even log into AWS web console. That’s it, you’re done. Enjoy your API.</p><blockquote>It’s very fast to deploy or update and it uses Amazon’s API under the hood. It does not involve CloudFormation or lead you to using a tool like Terraform.</blockquote><h4>A Note About an Architecture Decision</h4><p>Many serverless frameworks aim to follow a stricter practice of microservices or cloud functions where each function is small. Actually, not small — tiny.</p><p>With Aegis, I decided to make it bigger. By that I mean I wanted each Lambda (or microservice) to comprise of an entire component or concern. So a “blog” would be its own API and perhaps its comment system another Lambda with its own API.</p><blockquote>The beauty of a service oriented architecture is that you can really choose how you define those services and break up concerns.</blockquote><p>Your application architecture should not just be for server resource management, but also for people management. Teams can more easily work in parallel when you think in terms of (micro)services. That’s part of why Uber can support so many engineers for example.</p><p>The only exception here is if the limitations of the cloud function service are such that they support only a few concurrent runs of the <em>same</em> function, while <em>different</em> functions don’t count toward that limit.</p><p>In that case, you can better avoid service limits by having multiple functions. Fortunately, the concurrent limit with AWS Lambda is per region — not function. Though with Azure functions, the concurrent limit <em>is</em> per function (used to be 4, then 10, but now varies based on resources). So with Azure, you might want to have your functions smaller and more spread out. With AWS, I’m not so sure you need things <em>that</em> spread out.</p><p>In fact, critics of microservices maintain that having too many tiny functions creates a burden on maintenance and debugging. The choice is yours to make though.</p><h4>The Future?</h4><p>Serverless is a hot topic. I think with new hosting services, tools and technology, we’ve only seen the beginning. I already see more and more companies adopting it into their stacks.</p><p>Of course Docker is already a very big deal and companies like Uber are really championing in microservices. I feel that serverless is merely a companion to this story. It helps form this new landscape.</p><p><strong><em>Will Aegis support providers other than AWS?</em></strong> Unlikely. If I were to use another provider, it would be under a whole new project.</p><p><strong><em>Will Aegis support more than Go?</em></strong> I’m not sure just yet. You never know where my research may lead. I think building a simple serverless RESTful API in Swift would be awesome for iOS developers (and yes, you can run Swift in AWS Lambda with a wrapper too). So I might look into that next.</p><p>One thing is for certain — it will remain simple.</p><p><a href="https://github.com/tmaiaroto/aegis">Give it a try</a> and feel free to leave feedback or even contribute. It’s still under active development and there’s more to come. While I never intended on building a framework or tool to put out there, I do believe in open-source so most of my research and projects simply end up becoming open-source. Take from it what you will. Maybe the tool can help you too.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c504398d86f6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/serif-semaphore/build-a-serverless-api-in-seconds-with-go-c504398d86f6">Build a Serverless API in Seconds with Go</a> was originally published in <a href="https://medium.com/serif-semaphore">Serif &amp; Semaphore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Conversion Rate Optimization for Startups]]></title>
            <link>https://medium.com/serif-semaphore/conversion-rate-optimization-for-startups-997d685c2ee?source=rss----1f6afab5b862---4</link>
            <guid isPermaLink="false">https://medium.com/p/997d685c2ee</guid>
            <category><![CDATA[google-analytics]]></category>
            <category><![CDATA[startup-lessons]]></category>
            <category><![CDATA[conversion-optimization]]></category>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[startup]]></category>
            <dc:creator><![CDATA[Tom Maiaroto]]></dc:creator>
            <pubDate>Sun, 30 Oct 2016 19:48:50 GMT</pubDate>
            <atom:updated>2016-12-25T22:13:05.312Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1M2QMT_HBRjs6N-KSvbxsg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@szolkin">Sergey Zolkin</a></figcaption></figure><h4>Your First CRO Lesson: Do you even need to optimize?</h4><p>Conversion rate optimization (CRO) is becoming a hotter and hotter topic these days. There’s many tools to do it; <a href="https://www.optimizely.com/">Optimizely</a>, <a href="https://vwo.com/">VWO</a>, and now <a href="https://www.google.com/analytics/optimize/">Google</a> is even launching a new tool.</p><p>It’s a highly specialized function of marketing and UX. Not many people do it well and it’s not always an easy thing to understand. Newcomers often need to adjust their expectations (probably because of all the hype and bad literature out there).</p><p>I used to work at <a href="http://www.funnelenvy.com/">FunnelEnvy</a> and I would <em>highly</em> recommend them if you have CRO needs. I saw the results and was privileged to be able to assist with some amazing work. I learned so much from them. They are experts in this field.</p><p>One of the best learnings is that<strong> it’s important to discuss <em>when</em> you should think about CRO. </strong><em>Especially</em> as a resource limited startup.</p><p>Now, the truth is, most startups don’t need a serious investment in CRO. It’s usually a later stage game (arguably except for e-commerce/shopping sites). For many, it’s often their only option after growth seems to plateau. There comes a time when it’s cheaper to optimize than it is to reach new users.</p><p>That said, there are still <em>many</em> ways a startup can optimize or otherwise “growth hack” their business. I’d look at sites like <a href="https://growthhackers.com/">GrowthHackers.com</a> and <a href="https://inbound.org/">Inbound.org</a> for many ideas, but at the core of everything is <strong><em>measurement</em></strong>.</p><p>You will waste your time (and money) if you are not setup to measure and if you don’t understand the opportunity cost.</p><blockquote>Everyone with a digital presence should be measuring day one. No excuses.</blockquote><p>Your measurements don’t need to be perfect (because we actually deal with an imperfect science here). You don’t even need to measure everything. You’ll often hear about KPIs (key performance indicators) or the “metrics that matter most.” It need not even take a long time to set up measurement. In fact, it’s something you continue to do over time as things change. If you leave it for later, you actually create more work for yourself and you’ll wind up putting it off indefinitely. Come up with a convention and start day one. You need <em>directional</em> metrics.</p><h4>Measuring Conversions &amp; Drop Outs</h4><p>Aside from the basics, the first thing you’re going to want to setup is a measurement for conversions and when people drop out of funnels. This can be done, for free, with Google Analytics.</p><p>I’d also suggest using Google Tag Manager to manage the events being sent to Google Analytics. I’d suggest this over triggering events from your front-end code base, for the following reasons:</p><ol><li>You won’t tie your event tracking to your codebase or deploy process (which can get all sorts of delayed and messed up). Remember: separate concerns.</li><li>You won’t always need a developer to track an event.</li></ol><p>Some back-end events are unavoidable, but 99% of your behavioral events are from the front-end. I would manage those with GTM (at least when you’re dealing with a web site). This is a position reversal for me. As a developer I was always used to just tracking in the codebase, but over the years I have since seen the light so to speak.</p><p>Ensure your HTML has good class tags on each element. <em>When it doubt, add a class. </em>You’ll be able to easily use that element with GTM.</p><blockquote><em>Ensure your developers get into the habit of putting meaningful class names on elements for marketing.</em></blockquote><p>You can even put together a funnel in Google Analytics if your web site has page paths or titles to use. This is one of the most simple, yet effective, measurements you can perform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EOBS5Kti2HRw1toqlr92ow.png" /><figcaption>A hypothetical example of a story publish funnel (measured by regular expression matches for URLs).</figcaption></figure><p>In this example, there are 6 parts to the funnel with a 7th page being the goal. Google will measure this as a conversion once a visitor hits the final destination page.</p><p>With it, you can tell at which point users are dropping out of the funnel so you can optimize those stages. The example above is a rather long funnel…But does breaking up the form help? Or hurt? Now you can measure.</p><p>Over time, you’ll want to get more detailed about your measurements. For example, you’ll want to send events to Google about which particular fields of a form a visitor completed. It’s possible to track precisely where a visitor abandoned a form.</p><p>Google even lets you visualize these basic funnels, so you can see where visitors are dropping out and how many convert.</p><p><em>Note: Google doesn’t care the order in which these pages were seen.</em> Visitors can also come back later to complete the steps of the funnel. So more advanced measurement may be required in the future.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/1*6X7niYMpeQacdChnWz6D4A.png" /></figure><h4>Understanding the Opportunity Cost</h4><p>What’s <em>opportunity cost</em>? It’s an economics term, but I like to apply it to CRO and various UX operations. <em>It’s the loss of potential gain from other alternatives when one alternative is chosen.</em></p><p>In other words, it’s what you are missing out on by not optimizing <em>(and it may not always be money)</em>.</p><p>By now you very quickly see visitors dropping out of your funnels. People aren’t signing up or publishing content or buying or whatever your desired goals are. <strong><em>Keep in mind that you won’t ever get a perfect 100% conversion rate.</em></strong></p><p>Why? Well, first off, it doesn’t mean you had a <em>qualified lead</em> just because someone hits the top of your funnel. People and robots click all sorts of things on your web site and ads.</p><blockquote>Let me riddle you this side note batman: Did you know that there’s a certain number of people who will click your ad over your organic search result listing? It’s true. If you aren’t ranking toward the top, but you run an ad that is appearing…Users will click that. Now, if that ad happens to immediately go to the top of your funnel, guess what? Not everyone hitting that page wanted or needed to go through your funnel. They just wanted to get to your site.</blockquote><p>If you <em>do</em> have 100% conversion, I’d double checking your configuration and measurement process.</p><p>The second thing to consider is that startups typically have low traffic. They are new businesses after all. No one should ever expect to get it right at first.</p><blockquote>Often times pushing more traffic into the leaky funnel is going to yield better results.</blockquote><p>For a long while, it’ll be better to just push traffic than it will be to spend time optimizing. You likely aren’t talking about thousands, you’re talking about hundreds or even dozens of visitors.</p><p>You’re going to have to think about statistical relevance here at some point. For that, you need a healthy sample size. 1 of 2 people converting does not mean 50 of 100 will. I’ll leave probability out of this, but I think you get it.</p><h4><strong><em>Let’s calculate an opportunity cost with the following scenario:</em></strong></h4><blockquote>Let’s say you have 100 people hit your funnel with an 80% conversion rate for the month. So assuming it’s signups/purchases, you have 20 people who didn’t purchase.</blockquote><p>Do you know the life time value (LTV) for each user yet? It’s something you should measure or at least estimate.</p><p><em>Let’s say our LTV is $100.</em> The opportunity cost of not optimizing this funnel is, at most (and hang on a minute), $2,000.</p><p>I say <em>at most </em>because, remember, you won’t have a perfect 100% conversion rate. So really, if you get half of those who dropped out you’re talking about <strong>$1,000.</strong></p><p>That’s what you’re chasing for the month — which pays out over the lifetime of the user. This may not be immediate revenue depending on your pricing model. Remember it’s LTV. So the actual revenue for the month could be even less.</p><p>Now figure out how much it will cost you to have a developer (and anyone else who is involved in the planning and building) make the changes to optimize this funnel. Let’s say it takes two people two weeks to build and that’s roughly $2,000 a person per week. That’s $8,000 to address. Sorry…<em>Attempt</em> to address. Remember CRO is a process that involves many tests. This will not be a once and done situation.</p><p>This means it could actually cost your business <em>more money </em>to go after those 20 people for the month than you will ever earn from them. Of course, this starts to shift the other way over a long enough period of time. <strong><em>Or…</em></strong> <em>with more traffic.</em></p><p><em>So what’s the LTV of your users? How many more will you convert? How much will it cost to do so?</em></p><p>Armed with this information, you can start to determine the priority of optimizing. Without this information, you have absolutely no clue how important it is to address. Do not make the mistake of pretending otherwise. You can have a gut feeling, you can suspect, but you do not know (and knowing is half the battle).</p><p>Again, your mileage will vary because the numbers will be different. You might not have an already great conversion rate. The conversion might not even be part of something that generates revenue. Your LTV may be lower.</p><p>There’s many variables at play, but you shouldn’t even think about optimizing until you know the basic ones at least. These are <em>always</em> the questions you need to ask when determining to optimize or not.</p><p>Hey, I know. This sounds simple. Believe me though, you’d be surprised how many people don’t ask the right questions. We’re often very anxious to improve things. It’s in our nature. Try to remember to take just a few moments to ask why.</p><blockquote>Speed is a byproduct of moving smart.</blockquote><h4>Action Items</h4><p>I’m going to leave you with just 3 action items to help you jump start things. It’s ok if you don’t have measurements in place already — <em>I can’t stress enough how many people don’t bother to measure.</em> You are not alone and you very likely have more than enough to think about. So let’s make this easy:</p><ol><li>Setup goals in Google Analytics (or whichever analytics tool you are using). This should be fairly straight forward (if not, Google is your friend). Having trouble? I would check out <a href="http://www.lunametrics.com/blog/2008/06/25/funnel-problems-google-analytics/">this article</a> for common problems with your GA funnel setup.</li><li>Figure out the LTV or revenue associated with each conversion in this case. You likely already have an idea and even an estimate is ok…But a measurement is even better.</li><li>After you get some data flowing, calculate your opportunity cost. It could very likely be as easy as number of people not converting multiplied by LTV. You should be able to say, <em>“We are potentially missing out on $X.” </em>Not number of people, number of dollars.</li></ol><p>Once you are done with all of that, you can confidently continue on with your CRO journey (or you might wait). Would you like to read more? If so, please be sure to like and share this. Thanks!</p><p><a href="https://clarity.fm/shift8">Tom Maiaroto - Web Applications &amp; PHP Expert - Clarity</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=997d685c2ee" width="1" height="1" alt=""><hr><p><a href="https://medium.com/serif-semaphore/conversion-rate-optimization-for-startups-997d685c2ee">Conversion Rate Optimization for Startups</a> was originally published in <a href="https://medium.com/serif-semaphore">Serif &amp; Semaphore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing discfg, a Configuration Service for Your Applications]]></title>
            <link>https://medium.com/serif-semaphore/discfg-serverless-config-service-972fdbb355a?source=rss----1f6afab5b862---4</link>
            <guid isPermaLink="false">https://medium.com/p/972fdbb355a</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Tom Maiaroto]]></dc:creator>
            <pubDate>Wed, 05 Oct 2016 13:11:02 GMT</pubDate>
            <atom:updated>2016-10-05T18:02:40.762Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vAzpoCxz1eAdeNdXHAKwXg.jpeg" /></figure><h4>Distributed and Serverless</h4><p>An application configuration service is used to maintain a configuration in a consistent or conventional fashion. It helps separate concerns in software architecture. The service allows for configuration changes to be updated from outside the application, though the service often requires hosting itself.</p><p>Sometimes this service is built right into the application where configuration data is stored alongside other application data (look at applications such as WordPress for example).</p><p>It’s a common need among many applications, but in particular it’s a larger concern for distributed systems or microservices. When you move away from the monolithic application, you begin to need configuration information in many places so a service makes sense for maintainability reasons alone.</p><p>Though there are actually many reasons you’d want a dedicated configuration service. Here are just a few reasons:</p><ul><li>You don’t want to store sensitive credentials in your application code where it might be accidentally committed into revision control.</li><li>You don’t want to store config data in the application database.</li><li>You want to easily share configurations with other trusted developers.</li><li>You want to manage customer’s application configurations without them being able to.</li><li>Any other configuration access control scenario.</li><li>You want a consistent way to manage configuration and update values on the fly.</li><li>You’re building stateless microservices where you can’t easily keep a configuration around (ie. AWS Lambda).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/594/1*8U5OElUMBWXVqJ6KxFiRrw.png" /><figcaption><a href="https://github.com/tmaiaroto/discfg"><em>Check it out on GitHub</em></a></figcaption></figure><h4>Why Did I Build it?</h4><p>This simple tool came about from a need of mine when building microservices with AWS Lambda. One of the downsides to Lambda is that you can’t easily set environment variables. With ECS, which uses Docker, you sure can and I found myself gravitating more and more toward ECS despite the better fit with Lambda.</p><p>Why was that? Well, it was because configuration and state was a real pain with AWS Lambda. Nothing persists there (<a href="https://serifandsemaphore.io/azure-cloud-functions-vs-aws-lambda-caf8a90605dd">unlike Azure Functions</a>) so you have to bake everything into the function code. That’s not maintainable. A configuration service is clearly needed (and I’m surprised AWS has not offered one yet).</p><p>I wasn’t so interested in <em>service discovery</em> at the time, just configuration. Specifically a simple, yet conventional, key/value store.</p><p>So I looked into etcd and some other solutions…Before I knew it, I was back to worrying about the health of clusters — for simple configuration needs. Setting aside the fact that etcd was costing more to host than the actual serverless application itself, it was simply a maintenance I didn’t want.</p><p>Now don’t get me wrong. I <strong><em>love</em></strong> etcd. I drew so much inspiration from it and discfg is simply <strong>not</strong> a replacement for etcd (nor was that ever the intent). It is, however, the configuration tool for many of my needs (maybe yours too). Even still, I’ll usually use it as a point of reference.</p><p>Since the whole <em>serverless</em> concept is fairly new, there really aren’t any solutions around like this. It really just required a bit of boilerplate code in each of your functions every time you turned around.</p><p>Discfg is intended to be hosted in AWS and be serverless. It will have no problem scaling and there is no major concern with monitoring either. It just sits there and does its job.</p><h4>How Does it Work?</h4><p>Simple. It’s just a key/value store, but it’s the conveniences around that key/value store that make it what it is. It can expose a RESTful API to make accessing it easy from nearly any web application.</p><blockquote>discfg is essentially a convenient wrapper around DynamoDB (with more storage engines to come)</blockquote><p>It’s written in Go so, if your web application is written in Go, you could even just import the project to use a consistent API for config values and never need to run the API server. You would need to configure AWS credentials of course, but that should be fairly straightforward.</p><p>Most folks will be using AWS Lambda + API Gateway to host the API. <a href="https://www.terraform.io/">Terraform</a> and <a href="http://apex.run/">Apex</a> were used to build and deploy this, so you’d need those tools to get it setup. It should just be a few simple commands after that (assuming you have an AWS account and set credentials on your system, most likely because you used the aws-cli tool before).</p><pre>&gt; apex infra apply -var &#39;aws_account_id=XXXXX&#39;<br>&gt; apex deploy</pre><p>After that, you can simply access the RESTful API to create configurations, set, read, update, and delete keys.</p><p>The discfg binary is a CLI for your local machine. So you can work with configurations from your command line. Need to update something? It’s as quick as running:</p><pre>&gt; discfg set mycfg mykey &quot;some value&quot;</pre><p>You can set the current configuration that you’re using so you only need to specify the keys and values when working with the CLI. It’s just a simple shortcut for convenience.</p><pre>&gt; discfg use mycfg<br>&gt; discfg set mykey &quot;some value&quot;</pre><p>Retrieving the value? Easy:</p><pre>&gt; discfg get mycfg mykey</pre><p>You can even set data from a file:</p><pre>&gt; discfg use mycfg<br>&gt; discfg set mykey -d file.json</pre><p>Depending on how your application refreshes configuration data, you could very easily update its configuration straight from your command line in seconds.</p><h4>Wait a Minute…What About Security?</h4><p>Like other configuration tools, it does not take into consideration security itself.</p><p>However, keep in mind that through Amazon you have access control with IAM. No one is updating this configuration except for those you allow. Remember that API Gateway allows the use of authentication tokens as well. So you’re really covered in terms of security.</p><h4>How Can my Apps Use it?</h4><p>They can use the API which can be restricted to certain IP addresses or protected with an access token. Depending on your application, you may even want to cache the configuration data so you aren’t retrieving it more than necessary.</p><p>You can get a bit more sophisticated as well. For example, you could adjust the API so that only certain operations are available — for instance just read operations. This way your front-end applications can get the settings they need without allowing any curious people to inspect your endpoint to then add their own data.</p><p>You could then update data from the CLI or a separate API endpoint that has more permission. AWS API Gateway offers you a <em>lot</em> of flexibility and options here (a lot is an understatement).</p><p>Alternatively, if your application is built with Go, you could import discfg into your project and call its commands from Go.</p><h4>Trade-Offs? Other Tools?</h4><p>Well, it’s not really about competing with other tools for performance.</p><p>DynamoDB is pretty darn fast, but it’s no Raft. DynamoDB is technically an <em>eventually consistent</em> storage engine. But it does have conditional writes and discfg can leverage this feature. So you can work with some guarantees when necessary.</p><p>There’s most definitely a huge difference between something like the locking etcd system, the Raft consensus and then DynamoDB. <em>Huge</em>.</p><p>What is your application doing and what are you trying to build? If you’re looking to build systems level services then I would probably take a look at etcd or something else. If you’re trying to configure a web application, then I really see no reason not to use discfg…But that’s going to depend on the needs of your application.</p><p>Service discovery, for instance, may be better handled by other tools.</p><p>One big trade-off is watching for updates. With etcd you can have your application watch for a key value to update. Since discfg was designed to run using AWS Lambda, there can’t be long polling. So this feature doesn’t exist today (there’s some ideas around using AWS Kinesis to work in similar functionality in the future).</p><p>This is part of why I don’t really compare discfg to things like etcd, Zookeeper or Consul. It’s just a different use case.</p><p>Any other comparisons? Yes, just one comes to mind. <a href="http://webd.is/">Webdis</a> is an HTTP interface for Redis. It’s fairly similar but again you’re on the hook for hosting. Though Webdis does make it a point to address access control itself unlike many other tools.</p><p>Redis is yet again an altogether different storage engine too. It comes with slightly different use cases, but nothing says you can’t use discfg for a small fast cache too. Who knows, maybe discfg will even use Redis as storage engine one day.</p><blockquote>I suspect the simplicity of discfg will lead it to being used in a variety of ways. I think it makes DynamoDB a bit more approachable for starters.</blockquote><h4>It’s Free as in Beer and it’s Also in Beta</h4><p>It’s a free tool with an Apache 2.0 license. Keep in mind this tool is in beta. Definitely, please, bring up bugs on GitHub if you spot any.</p><p>In the future, I plan to add support for S3 as a storage engine too. There might be reasons why one would choose S3 over DynamoDB. It all depends.</p><p>Feedback is very much welcome too. There’s some issues and a loose roadmap for the tool on the GitHub project.</p><p><a href="https://github.com/tmaiaroto/discfg"><em>Check out discfg on GitHub</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=972fdbb355a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/serif-semaphore/discfg-serverless-config-service-972fdbb355a">Introducing discfg, a Configuration Service for Your Applications</a> was originally published in <a href="https://medium.com/serif-semaphore">Serif &amp; Semaphore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Host WordPress Like a Boss]]></title>
            <link>https://medium.com/serif-semaphore/how-to-host-wordpress-like-a-boss-b5993fcfbd8e?source=rss----1f6afab5b862---4</link>
            <guid isPermaLink="false">https://medium.com/p/b5993fcfbd8e</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[wordpress]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloud-hosting]]></category>
            <dc:creator><![CDATA[Tom Maiaroto]]></dc:creator>
            <pubDate>Wed, 24 Aug 2016 15:23:34 GMT</pubDate>
            <atom:updated>2016-12-06T20:41:08.554Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*0edgxpmELHY66NjS7hjISA.jpeg" /></figure><h4>Scale WordPress on AWS with Docker</h4><p>If your primary business is not to build a CMS, then <a href="https://wordpress.com/">WordPress</a> becomes a very attractive option. I also have to thank it because it was one of the first open-source projects that helped me learn PHP.</p><p>Yes, I’ve moved on to PHP frameworks and understand why WordPress doesn’t scale well…But there’s also a reason why companies like <a href="https://pantheon.io/">Pantheon</a> exist and why WordPress is still popular and still around to this day.</p><p>All fairness aside, I’ve long maintained that WordPress doesn’t scale. Trust me, having built sites like UsMagazine.com (which at first did use WordPress) I’ve seen WordPress and Drupal fail — hard. When you work on sites with that kind of traffic, you’ll quickly realize this.</p><p>So it’s been 8? 10? I’ve lost count. Let’s just say it’s been many years since I’ve used WordPress but I recently came back to it.</p><p>Though this time around I wanted to see if I could find a scalable solution using the latest web hosting technology. And through my previous research and experimentation with AWS and microservices, I have.</p><p>Want host WordPress like a boss? Ok, the easy answer is <a href="https://pantheon.io/">Pantheon</a>. However, if that’s not an option or you want more control over the situation, I’ve come up with a pretty awesome solution. Ready for the winning formula?</p><h3>Docker + Nginx + PHP-FPM + Amazon ECS + EC2 (Auto Scaling) + ELB + EFS + RDS (MariaDB) + CloudWatch = Winning</h3><p>If that makes sense, you can stop reading. It’s ok if you keep reading though because I just threw a metric shit ton of acronyms at you. No one should be expected to understand that, so I’ll gladly break it down.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_d2ywolCMUd61Zl353PR3g.jpeg" /><figcaption>The scalable and resilient architecture in a nutshell.</figcaption></figure><p>Let’s start with the easy one — Docker. Docker is taking the hosting world by storm. It’s rapidly transforming the landscape at a pace I’ve never seen before.</p><p>Docker let’s us <em>containerize</em> our server and, optionally, application together (some used to use Amazon AMIs to do this in the past — this is cooler). It then can be configured to work with other containers and scale very effectively. There’s many more benefits, but I won’t go on about them here.</p><p><a href="https://aws.amazon.com/ecs/">Amazon ECS</a> is a service that makes Docker management and deployment a breeze. It lets you configure the containers and keep everything under version control and it also schedules where those containers run.</p><p>In this case I’m using Docker purely as a service (Amazon ECS Tasks/Service) that is the “web server.” It runs Nginx and PHP-FPM. That’s it. In this setup, it does <em>not</em> host the codebase. This is atypical from many architectures you’ll see using Docker where the codebase is baked in as to use it as a method of versioning/tagging the application deployments.</p><p>Put another way, only an Nginx or PHP config change would prompt a new Docker image and a deploy of a new version. This keeps your Docker image very stable. You’ll almost forget about it.</p><p>Why didn’t I put WordPress in the container? Because user uploads. The assets in a WordPress application don’t go off to a shared storage solution like S3 or a MongoDB GridFS. They go straight to where the codebase lives and that’s problem if you want to think of your Docker containers as immutable so they can be easily thrown away and restarted.</p><h4>The user upload solution? Amazon EFS</h4><p><a href="https://aws.amazon.com/efs/">Amazon EFS</a> is a relatively new service, this is basically a modern NAS. It uses the NFS protocol, but it’s performant (at least for reads).</p><p>I put the entire codebase on EFS and not just the <em>uploads</em> directory. This also makes cloning environments, backups, and deployments much easier.</p><p>Docker containers in ECS can mount EFS. So it was trivial to have Nginx configure its webroot to use a directory on EFS. Now each of your Docker web servers can read from the same version of the codebase and access the same file assets.</p><h4>Scaling the web server</h4><p>In this case, our web server runs Nginx with PHP-FPM (with PHP7). However, there are other solutions for WordPress you could use here. You could use Apache for example, but I prefer Nginx.</p><p>For those unfamiliar, Amazon ECS runs Docker containers on EC2 instances. Those EC2 instances can be part of an auto-scaling group. The ECS Tasks can also run in their own auto-scaling group defined as an ECS Service.</p><p>Confused? Here’s what happens: Amazon ECS uses EC2 instances like a resource pool. Your ECS Tasks are defined to require a certain amount of CPU and RAM. ECS’ scheduler runs those containers on the pool of EC2 instances where they fit. You must have enough EC2 instance resources to run a Task/Service.</p><blockquote>When using ECS, just think of EC2 as total RAM and CPU available across all subscribed instances and never think about individual servers again. It’s all about resource units — not how many servers you have.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SmTz61-6_KG4r0z0qBvZNQ.png" /><figcaption>You can assign how much CPU and RAM each container uses as well as a host of other things.</figcaption></figure><p>So the ECS Service also has rules for scaling. If so many containers are unhealthy, turn on another one. There’s rules you can set. That’s great, but then you also need to ensure you have enough EC2 instances for that. So those too need to auto-scale.</p><p>With a bit of setup, you could end up with a situation where you’ll always have Amazon turn on more servers and run more containers to keep up with demand. <em>Set it and forget it.</em><strong> Save money.</strong></p><h4>Load balancing</h4><p>Of course you need to use <a href="https://aws.amazon.com/elasticloadbalancing/">Amazon Elastic Load Balancer</a> with ECS and EC2. Otherwise you’d be running your site on just one EC2 server despite having a bunch up and running with Docker containers.</p><p>This is relatively straight forward to setup and comes with a whole other set of useful options.</p><h4>The Database</h4><p>The last piece here is the database. I chose MariaDB instead of MySQL, but in either case you’ll want to leverage <a href="https://aws.amazon.com/rds/">Amazon RDS</a>.</p><p>Amazon RDS allows you to easily scale and back things up. There do exist plugins for WordPress that let you leverage database replication.</p><h4>Bonus! CloudWatch</h4><p>What about centralized logging? Well, ECS Tasks can log out to <a href="https://aws.amazon.com/cloudwatch/">Amazon CloudWatch</a> conveniently enough. You create a log group and configure it in your ECS Task definition (<em>just an important note: you must have your AWS log group created </em><strong><em>first</em></strong><em> before running your Task and they don’t explain that in the error message if you try to run a Task that points to a non-existent log group</em>).</p><p>Log groups are also convenient when it comes to multiple environments too. While I have to admit looking through logs in AWS web console is not the prettiest, they do work.</p><blockquote>Protip: You can “tail” AWS logs using this tool <a href="https://github.com/jorgebastida/awslogs">https://github.com/jorgebastida/awslogs</a></blockquote><p>Just ensure your Nginx (and anything else you want) logs out to <em>stdout</em> and <em>stderr</em>. Amazon will pick that up and throw it into CloudWatch. Pretty cool, huh?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PhGU5thLzaw1ko2rfK3Ygw.png" /><figcaption>The error_log setting that will allow Nginx logs to be picked up by CloudWatch.</figcaption></figure><h4>Wiring it all up</h4><p>I cover the major components of the architecture, but there’s actually a lot that goes on in terms of configuration and wiring it all up.</p><p>I’d like to adjust the Dockerfile I’m using to be a bit more generic and then also setup some Terraform files or maybe CloudFormation. Until then, I’ll skip on diving too deep on the Dockerfile or the configuration scripts. If you’re familiar with Docker and deploying code, I’m sure you’ll be able to come up with something similar and very likely better for your needs.</p><p>One of the most important things to note is to leverage environment variables because you can then address them in your ECS Task as well as override them when you launch each Task. Things like your webroot path for example or your database connection information.</p><p>By using environment variables and a setup script, you can really ensure that you’re only building one Docker image for your staging and production environments.</p><h4>Deployment strategy &amp; other concerns</h4><p>Once again I turn to ECS and Docker for deployment. I created a separate Docker image that is responsible for grabbing the latest codebase and getting it onto EFS.</p><p>This deployment Task can be as simple or as complex as you need. It could create entire backups simply by copying directories so you can rollback if you need to. Again, keep in mind that WordPress will put uploads within your codebase.</p><p>I found deployments pretty manageable with a simple shell script. Literally all you need to do is get the latest files on EFS.</p><blockquote>The great thing about separating the codebase from the Docker image is that Nginx and PHP config updates can be deployed separately from the codebase. Even if you like to keep everything in the same GitHub repo.</blockquote><p>Aside from getting your site up to date, you’ll occasionally need to update your web server. This is as simple as pushing a new image to the ECS registry and updating your ECS Task (unless you like to live on the edge and use the latest tag — then you just kill your ECS Tasks and the next ones to automatically online are updated). You could also be using something like <a href="https://www.terraform.io/">Terraform.io</a> to be managing this from your console or an automated process somewhere.</p><p>You could create additional ECS Tasks (and Docker images) for other needs. For example, you could have an ECS Task that clones the current production filesystem and database and sets up a whole new environment for development. Keep in mind you can also use the AWS SDK to run ECS Tasks from other ECS Tasks. Yes, you could basically replicate the features you see in a service like Pantheon.</p><p>That’s all for now, hope this has given you some ideas to go on for your next WordPress project.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b5993fcfbd8e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/serif-semaphore/how-to-host-wordpress-like-a-boss-b5993fcfbd8e">How to Host WordPress Like a Boss</a> was originally published in <a href="https://medium.com/serif-semaphore">Serif &amp; Semaphore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tools to Help you Go]]></title>
            <link>https://medium.com/serif-semaphore/tools-to-help-you-go-d6f782055ce7?source=rss----1f6afab5b862---4</link>
            <guid isPermaLink="false">https://medium.com/p/d6f782055ce7</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Tom Maiaroto]]></dc:creator>
            <pubDate>Sat, 09 Apr 2016 19:10:56 GMT</pubDate>
            <atom:updated>2016-04-11T18:22:11.785Z</atom:updated>
            <content:encoded><![CDATA[<h4>Write Maintainable Golang Code</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RzE9hBgmsl6gaxCfO4n5Wg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/Q47eNv_UvfM">Dmitrii Vaccinium</a>.</figcaption></figure><p>We’re all guilty of it — we sometimes don’t follow style guidelines, we pull in a dependency and then fail to really “depend” on it, or we end up with a bit more complexity in our code than we’d like.</p><p>I’ll be the first to admit that I do this <em>all</em> the time. I see writing software like sculpting. There’s two primary ways to go about it; 1. you can either start with a block and remove material (subtractive) or, 2. you can add more material to create the work (additive).</p><p>Perhaps not a perfect analogy, but my point is that coding is as much a creative process as it is a technical one. Creative processes are subject to change and a little mess and that’s ok.</p><p>Though this process leaves us with technical debt and maintenance challenges. Code readability is as important as test cases; especially when working with teams or in an open-source context.</p><p>I’ve always found Node.js to have good tools to help with this. Perhaps it’s due to static typing and <em>gofmt</em> that Go has less tools to help with code quality than Node.js. Or maybe it’s just because Node.js currently has a larger community.</p><p>I’m happy that there’s been more tooling for Go over the past year or so. Maybe this is a sign of adoption, maturity, and a growing community.</p><p>I’ve come up with a new practice for all of my Go code (and I’ll try to do a good job practicing what I preach). I use the following tools in a Git hook.</p><h4>Linting &amp; Style Guidelines</h4><p><a href="https://github.com/golang/lint"><strong>Golint</strong></a><br>Go already comes with <em>gofmt</em> and <em>govet</em> which are wonderful tools, but they simply format your code and make sure it runs. Honestly, that’s all they should do because we may want to adopt our own style guidelines.</p><p>Many editors and IDEs will automatically run <em>gofmt</em> for you upon file save anyway. So it makes little sense to run on a Git hook.</p><p>If you want to follow Golang’s community style guidelines, you’ll want to use Golint.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V3WOaq7zPqh8jgt7mhsBFQ.png" /><figcaption>My very dirty code.</figcaption></figure><h4>Cyclomatic Complexity</h4><p><a href="https://github.com/fzipp/gocyclo"><strong>Gocyclo</strong></a><br>This one always escapes me! I realize later on that I could have reduced complexity or broke things into separate functions, but it’s always been something I looked at later on.</p><p><a href="https://codeclimate.com/engines">Code Climate</a> is a wonderful service and it does support Golang! However, it doesn’t show you cyclomatic complexity (which it does for other languages like JavaScript). It only runs <em>gofmt</em>, <em>govet</em>, and <em>golint</em> (above).</p><p>There are actually several packages that give you the tools for this. I prefer gocyclo. Your tolerance for this will likely be to taste.</p><h4>Dependencies &amp; Redundancy</h4><p><a href="https://github.com/divan/depscheck"><strong>Depscheck</strong></a><strong><br></strong><a href="https://github.com/mvdan/interfacer"><strong>Interfacer</strong></a><strong><br></strong>A very important component of writing maintainable code if your project’s dependencies.</p><p>You can “vendor” packages and avoid <a href="http://www.haneycodes.net/npm-left-pad-have-we-forgotten-how-to-program/">leftpad syndrome</a> — it’s a common thing for Gophers to do. However, maybe you could’ve simplified things and simply put what you needed from the dependency into your own code (license depending).</p><p>It’s very easy to find a package and want to use it because it’s awesome. The problem is when you then only end up using a very small portion of it. Sometimes for a single function. Maybe it’s still awesome, but just more awesome for someone else’s code.</p><p>Depscheck is a great tool to help you determine when you might not be so dependent upon a dependency.</p><p>How about unnecessary interfaces? How many times have you wrote an interface with good intentions, but then later realized it wasn’t necessary? There’s a tool for this too, it’s called interfacer.</p><h4>Unused Code</h4><p><a href="https://github.com/opennota/check/"><strong>check</strong></a><strong><br></strong><a href="https://github.com/kisielk/errcheck"><strong>errcheck</strong></a></p><p>Aside from pointless interfaces, I think a bigger culprit of maintenance woes is unused code. Don’t allow variables, struct fields, and constants to become orphans! It can really confuse people (and you, a few months later).</p><p>Check and errcheck both help tremendously with this. Check can even find inefficiently packed structs.</p><h4>SQL Injection</h4><p><a href="https://github.com/stripe/safesql"><strong>safesql</strong></a></p><p>This one is a bit of a bonus, but certainly noteworthy if you use SQL. It’s not so much a concern of maintenance, but security is related to code quality.</p><h4>Trust Yourself</h4><p>With all of these tools, it’s important to note that there can be some false positives. Treat all of these as suggestions, but I feel that they do a good job and will help lead you to making good decisions. Don’t forget to trust your own best judgement.</p><p>I’m sure I’m missing a few handy tools; however, I feel that these are among the most common tools needed to ensure code quality and help reduce technical debt.</p><h4>Gitting Hooked</h4><p>What good are all these tools if they are a pain in the neck to use? You’ll forget them all, really.</p><p>To help me form a good habit, I’ve added all of the above tools to <a href="https://gist.github.com/tmaiaroto/247f3735b58f736450dd5c6c7780a3f8">a Git hook that you can find <strong>here</strong></a><strong> </strong>(also seen below). You’re more than welcome to use it yourself of course. You may wish to edit it a bit first if you don’t want or need all of the tools listed above. You might also want to add a few.</p><p>A few coloring decisions have been made. You can remove those by easily commenting and uncommenting specific lines. It should be fairly straight forward.</p><p>Remember, these tools take a while to run. You may want to put this on a <em>pre-push</em> hook so you aren’t slowed down by every commit. That is, of course, if you like to make frequent commits.</p><p>You should even be able to run this script through your favorite CI tool.</p><p>I hope this helps you write better Go code, like it helped me!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/98b17007a0648078e1406d36f52b69bd/href">https://medium.com/media/98b17007a0648078e1406d36f52b69bd/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d6f782055ce7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/serif-semaphore/tools-to-help-you-go-d6f782055ce7">Tools to Help you Go</a> was originally published in <a href="https://medium.com/serif-semaphore">Serif &amp; Semaphore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Azure Cloud Functions vs. AWS Lambda]]></title>
            <link>https://medium.com/serif-semaphore/azure-cloud-functions-vs-aws-lambda-caf8a90605dd?source=rss----1f6afab5b862---4</link>
            <guid isPermaLink="false">https://medium.com/p/caf8a90605dd</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[microsoft-azure]]></category>
            <dc:creator><![CDATA[Tom Maiaroto]]></dc:creator>
            <pubDate>Mon, 04 Apr 2016 00:47:26 GMT</pubDate>
            <atom:updated>2016-11-22T21:29:05.858Z</atom:updated>
            <content:encoded><![CDATA[<h4>What you Need to Know</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VuyH9Yybcf9QEcSCUrL3Hw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/iIg4F2IWbTM">Olivier Miche</a>.</figcaption></figure><p>Microsoft announced their answer to AWS Lambda at their <em>Build 2016</em> conference. Brace yourself for the next cloud hosting change; because, it now seems like everyone is going “serverless.” <a href="https://aws.amazon.com/lambda/">Amazon Web Service’s Lambda</a>, <a href="https://cloud.google.com/functions/docs">Google’s Cloud Functions</a>, <a href="https://developer.ibm.com/openwhisk/">IBM’s OpenWhisk</a>, and now <a href="https://azure.microsoft.com/en-us/services/functions/">Azure’s Functions</a>.</p><p>There’s too much to compare in one article, so I’ll write about the differences in a series of articles. I’ve had the opportunity to play with some. Google’s Cloud Functions are in private alpha, so I can’t share much about them, but Microsoft’s are in public preview.</p><p>I’ve been using Lambda a lot lately and it’s been a <strong>game changer</strong>.</p><p>While Azure’s cloud function usage is billed the same way, the service is <em>radically</em> different than Lambda under the hood.</p><p>Before you make any major decisions, keep in mind that these services are evolving.</p><h4>Supported Languages</h4><p>This will change over time for every cloud function service.</p><p>Azure Functions supports Node.js, C#, F#, Python and PHP. They also list Java, bash, and batch but it’s not clear how to use these (again, it’s in preview).</p><p>AWS Lambda supports Node.js, Python, and Java for now.</p><p>I think PHP support is a very strategic choice given its popularity, but I wish one of these services would better support Go.</p><p>Why Go? Because it runs fast and its code is easy to read. It is a language of brevity and therefore fits well for the cloud and (micro)services. Add Rust and Swift to the list too.</p><p>It’s important to note that you can execute binaries in both services through one of these languages. This is the method in which <a href="https://serifandsemaphore.io/go-amazon-lambda-7e95a147cec8">Go works in Lambda</a>. Both <a href="http://apex.run/">Apex</a> and also <a href="http://gosparta.io/">Sparta</a> are serverless frameworks that let you use Go in Lambda as well.</p><blockquote>It would be great for a cloud function service to support Go natively. Swift for mobile developers who often need simple back-end services would also be a smart move.</blockquote><h4>Web Dashboard</h4><p>No amount of fancy devops tricks or tooling will save you from having to use the web dashboard at least every now and then.</p><p>Azure’s Function usage charts are still being developed, but they will be there and at least on par with Lambda’s.</p><blockquote><em>Azure’s new dashboard is beautiful.</em></blockquote><p>Once you understand the organizational structure of Azure Function it becomes a little easier. The same can be said for AWS Lambda too.</p><p>One important difference is the editor. Both are on par when it comes to the function’s settings screen. However, Azure has the more robust Visual Studio Online which can also be used.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bnf5ZE6WQ8XGnDtCpt42Fg.png" /><figcaption>Azure Functions can be authored with Visual Studio Online.</figcaption></figure><p>Visual Studio Online is found under the Function’s App Service’s tools section. It’s worth noting that there are some other cool tools here too, including a console.</p><p>Most of your development time should be outside of dashboards anyway. I don’t consider them a huge factor in my decision making process but they can offer convenience.</p><h4>System Structure</h4><p>Let’s dive into the architecture of Azure Functions compared to Lambda.</p><p>An “App Service” is a container, or environment, for a set of Azure Functions. This is much different than Lambda. In fact, the two services couldn’t be more different.</p><p>Lambdas are organizationally independent, where Azure Functions are grouped logically into an “application.”</p><p>This App Service can either by “dynamic” or “classic.” The dynamic option is where you pay only for the time and memory your functions use. This is the biggest similarity between Lambda and Azure Functions.</p><p>It’s important to note that the memory you allocate is per app service (with potentially many functions), not per function like Lambda.</p><p>I think of the “classic” App Service more like Amazon ECS where you have EC2 instances running to handle the tasks or functions. The pricing model for this is then like EC2.</p><p>Azure is actually more like a blend between ECS Tasks and Lambda. For example, you can set environment variables on App Services which are then available for your Azure Functions. AWS Lambda cannot persist environment variables, but ECS Tasks can (correction/update: Lambda <a href="http://docs.aws.amazon.com/lambda/latest/dg/env_variables.html">now can</a> as pointed out by <a href="https://medium.com/u/feac242fa8b0">Jeremy Axmacher</a>, thank you).</p><p>The entire container architecture is different. Lambdas provision a brand new one and deploys your code (from a zip file) on a cold request. Subsequent requests <em>can</em> be subject to container re-use and be handled much faster. However, you need to understand there is no persistence and with Node.js Lambdas you need to watch your variable scope because the container <em>can</em> be re-used.</p><p>However, Azure Functions are less subject to the cold/warm request effects. Azure still provisions resources as needed, but your files aren’t “frozen” somewhere in a zip file. They run on top of <a href="https://azure.microsoft.com/en-us/documentation/articles/web-sites-create-web-jobs/">Azure’s WebJobs</a>.</p><p>Azure Functions are also supposed to be accessible via FTP by connecting to the App Service’s FTP. Though I couldn’t get it to work so far during my review (no matter how many times I reset my credentials).</p><p>Azure Functions of course is a Windows system (32bit for dynamic services) while <a href="http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html">Lambda is a Linux system</a>. Though I do hope that changes since <a href="http://www.pcworld.com/article/3050473/windows/heres-how-windows-10s-ubuntu-based-bash-shell-will-actually-work.html">Ubuntu can now run on Windows</a>. In fact, that would put Azure in a good position with cloud hosting.</p><h4>Continuous Deployment</h4><p>Speaking of working with the files, you can deploy your Azure Functions in a variety of ways:</p><ul><li>Git (you can connect GitHub, Bitbucket, etc.)</li><li>Dropbox</li><li>OneDrive</li><li>Visual Studio Team Services</li><li>Visual Studio Online editor</li><li>Cloud function’s own editor under it’s settings area (limited to just the code and the function.json)</li></ul><p>You have many more inherent options than AWS Lambda, but I didn’t explore any sort of CLI deployment. Lambda has some great 3rd party tools for deploying Lambdas and both cloud services have SDKs to make just about anything possible.</p><p>I do have to hand it to Microsoft here though. You can hook up GitHub, even with your favorite CI tool, and easily deploy your cloud functions to Azure. Release managers rejoice.</p><p>Though it’s important to note that Lambdas can be versioned whereas Azure Functions can not (at least not yet). So if you wanted to rollback, you’ll need to rely on Git or some other version control system.</p><h4>Run Your Functions Anywhere</h4><p>One challenge with Lambda is development workflow. People have created <a href="https://github.com/motdotla/node-lambda">tools</a> to test your Lambda locally so you can write code, run, and then even deploy if all looks good.</p><p>There are also several “serverless frameworks” that help with this process too.</p><p>Regardless, Lambda is closed source. However, <a href="https://github.com/azure/azure-webjobs-sdk-script">Azure Function’s runtime is open-source</a>. I would expect some clever CI integrations in the future.</p><h4>Function Triggers &amp; Integrations</h4><p>I also have to concede that Azure Functions make things a bit easier in this department too. Every function automatically maps to an HTTP endpoint if enabled. Whereas with Lambda, you must configure API Gateway separately.</p><p>API Gateway is fine, but complex and time consuming. Again, some serverless frameworks ease this pain point by automatically setting up an API for Lambdas.</p><p>Microsoft gets points for UX because you have a <em>lot less</em> to configure. I’m not sure you can run Azure Functions from multiple triggers though. It doesn’t appear that way, but maybe by editing the function.json directly you can.</p><p>I would say that AWS Lambda has a slight edge in flexibility here, but both services are constantly changing.</p><p>The Azure app and function names are reflected in the URLs to trigger the cloud functions. This helps avoid confusion, but might also be limiting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5YySwHeBE8pK03d4ex6SiA.png" /></figure><p>Apparently all HTTP methods are supported (GET, POST, etc.) through the endpoint assigned to each cloud function. With API Gateway, you need to configure each method manually.</p><p>One thing I don’t see is path variables. It seems that everything must be passed as a querystring or in the request body.</p><p>To be fair, it’s called an “HTTP trigger” so when thinking about it in those terms, it makes sense. If you need more robust settings, then you’re talking about “API configuration and management.”</p><p>Azure Functions also have super easy authentication. You can protect the cloud function endpoints with 3rd party authentication; this includes Facebook, Twitter, Google, and of course Microsoft’s auth.</p><blockquote>I’ve never setup Facebook auth for an HTTP endpoint so quickly and easily before in my life.</blockquote><p>You can configure CORS and you can also manage your own domain name for the endpoints. Though I couldn’t see if there was a way to limit the HTTP request methods. I imagine with Azure’s API management service more will be possible, but then you’re going through as much work as API Gateway.</p><p>I don’t think there’s a clear “winner” here. Both Microsoft and Amazon have powerful features for triggering cloud functions with HTTP requests. Though I definitely think Amazon could learn a thing or two about UX from Microsoft.</p><h4>Azure Solves a Few Pain Points</h4><p>I was thoroughly impressed by Azure and I think it’s in part due to its new dashboard. The new dashboard and the process of setting up cloud functions in a logical group called an “app” makes complete sense.</p><p>To illustrate how obvious this pain point is, look no further than <a href="https://github.com/serverless/serverless">Serverless</a> (formerly JAWS) and <a href="http://apex.run/">Apex</a>. These are two serverless frameworks that address this specific organizational concern. They help a developer logically group functions.</p><p>Another pain point with Lambda is the maximum execution time limit. It wasn’t always clear and many people never thought their functions would run that long (at least in my experience).</p><p>The time limit used to be 1 minute for Lambda, but increased to 5 to help with “ETL” operations according to Amazon. I’m happy for the time limit increase, but even 5 minutes may not be sufficient for all operations.</p><p><a href="https://www.iron.io/">Iron.io</a> pointed out that their workers had no time limits in a comparison with Lambda. <a href="https://www.iron.io/platform/ironworker/">Iron workers</a> are another option, though I stayed the course with AWS Lambda. I even found a way to <a href="https://serifandsemaphore.io/aws-lambda-going-beyond-5-minutes-34e381e71231">re-purpose your Lambda code as ECS Tasks to cope with the 5 minute limit</a>.</p><p>While that solved my own problems with the time limit, I understand that it’s not a solution for everyone. It also requires EC2 instances which are billed in a different fashion. So it’s not a perfect solution if “pay as you go” is a top priority.</p><p>Azure’s Functions really are the perfect blend of Lambdas and ECS Tasks. You can write and execute the same exact code regardless of the environment and billing model. There are apparently no time limits either(so long as your function is not idle).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pkyj9uSRHW4-I5gx2M_VDg.png" /><figcaption>“Always On” will change the billing model.</figcaption></figure><p>On the other hand, I imagine there could be some surprises in billing too. Lambda’s 5 minute execution time limit does serve to protect you from expense. Will Microsoft keep billing you for some accidental loop in a cloud function? Assuming the same amount of resources were used, will the cost work out to be about the same regardless of billing models? It’s perhaps too early to say since it’s still in preview and there is no pricing information available yet.</p><p>There is some chatter about this concern though if you follow the <a href="https://github.com/Azure/azure-webjobs-sdk-script/issues/18">GitHub issue</a>.</p><h4>Other Observations</h4><p>As I explored Azure Functions and it’s underlying architecture (as best I could), I realized a few other things of note.</p><p>First, since there is persistence and no execution time limits, working with large files and ETL is easier. <em>Just watch out for how cost effective this ends up being or not being.</em></p><p>With Lambda + API Gateway, the content-type of the response is set via API Gateway. With Azure Functions, the content-type is set by the code. This can be very useful, but could also lead to some bloated functions.</p><p>Remember, your goal is not to build a complete application within a cloud function. It’s not really possible to do with <a href="http://docs.aws.amazon.com/lambda/latest/dg/limits.html">Lambda’s limitations</a>, but it is with Azure. So be careful not to fall into any bad practices.</p><blockquote>Watch out for Azure Function’s limitation — only 10 concurrent executions per function.</blockquote><p>Again, Azure Functions run on a server with a persistent filesystem and webroot. It’s just not accessible to the outside world. However, it is accessible to you while logged in.</p><p>All of your code and assets can be seen via a URL like this: <em>https://yourAppName.scm.azurewebsites.net/dev/api/files/wwwroot/HttpTriggerNodeJS1/index.js</em></p><p>This maps this location on disk: <em>/home/site/wwwroot/HttpTriggerNodeJS1</em></p><p>Is there a cost to request these files this way? Can you somehow open this to the public? I don’t know. It definitely might raise an eye-brow or two though.</p><p>You have more insight, access to and control over the environment running your cloud function than you do with AWS Lambda…For better or worse.</p><p>Speaking of insight, when listing the directory of <em>D:\home\site\wwwroot </em>where your files are, it indicates that there is over <strong>5 terabytes</strong> of disk space available!</p><p>This is much more than Lambda’s <em>default ephemeral</em> disk space of <strong>500MB</strong>. There isn’t as much info about <a href="https://azure.microsoft.com/en-us/documentation/articles/functions-reference/">Azure’s Cloud Functions limitations</a> just yet, but it is important to note that the instance limit is <strong>10</strong> <em>while using “dynamic” service apps </em>(up from 4). The concurrent execution limit per instance is a bit unknown. <a href="http://docs.aws.amazon.com/lambda/latest/dg/limits.html">Lambda’s limit</a> is <strong>100</strong> concurrent executions total (which is a soft limit).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N0wuzlWSEw24pcLybmGLKA.png" /><figcaption>Your Azure Functions are but a drop in a very large bucket.</figcaption></figure><p>Is everyone’s code on the same storage volume? Or does each Azure account gets its own, very large, storage volume? I’m not familiar enough with what’s going on to know, but exploring is fun.</p><h4>So Which Path?</h4><p>I would encourage everyone to explore and try each of these services. Understand why and when they they can help you. The end result can be a game changer for you too.</p><p>The cloud has powerful magic, but not all clouds are the same. Get to know the environment your code lives in as well as the capabilities of the services available to you. Read the fine print. Understand the limitations and tradeoffs.</p><p>Only with this information can you make an informed decision. Remember, your choice will be the path less taken here.</p><p>My choice? The jury is still out. I use a variety of services because I believe there is strength and opportunity in diversity.</p><p>I primarily use Lambda and ECS because it’s well established. I love AWS and will continue to use it happily, but I definitely think these companies can learn a thing or two from each other.</p><p>There’s a lot of opportunity in this fast changing landscape. There aren’t many books. Few best practices. Barely any frameworks. Less than ideal monitoring and debugging. There will be trial and error. You are subject to the possibility of having to re-build code and move your operations. If it gives you any solace, you will be in good company. Some major companies and smart developers are on board with cloud services and the (micro)service movement. This is the new frontier on the web. Buy the ticket, take the ride.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=caf8a90605dd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/serif-semaphore/azure-cloud-functions-vs-aws-lambda-caf8a90605dd">Azure Cloud Functions vs. AWS Lambda</a> was originally published in <a href="https://medium.com/serif-semaphore">Serif &amp; Semaphore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>