<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Paul Swail on Medium]]></title>
        <description><![CDATA[Stories by Paul Swail on Medium]]></description>
        <link>https://medium.com/@paulswail?source=rss-3a509aed10a8------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*mf5ooNqIh1Lm_ppg4r4GgA.jpeg</url>
            <title>Stories by Paul Swail on Medium</title>
            <link>https://medium.com/@paulswail?source=rss-3a509aed10a8------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 05:39:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@paulswail/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Back-end Primer for Front-end Web Developers]]></title>
            <link>https://medium.com/@paulswail/back-end-primer-for-front-end-web-developers-80339b5b5c3a?source=rss-3a509aed10a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/80339b5b5c3a</guid>
            <category><![CDATA[cloud-services]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[front-end-development]]></category>
            <category><![CDATA[full-stack]]></category>
            <dc:creator><![CDATA[Paul Swail]]></dc:creator>
            <pubDate>Thu, 01 Nov 2018 07:06:01 GMT</pubDate>
            <atom:updated>2018-11-01T07:06:01.749Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sTlT9rF33xsDhuYF81LEhg.png" /></figure><p><em>“I do feel envious of back-end developers and am sorta clueless concerning what they do.”</em></p><p><em>“Full-stack devs, you amaze me with your awesome voodoo stuff!”</em></p><p>If you’re a front-end developer wanting to become more familiar with the back-end, the number of different concepts to learn can be overwhelming.</p><p>Maybe you want to feel the power of shipping an end-to-end production app by yourself without needing to rely on the skills of others.</p><p>Or maybe you simply want to be able to chat with your back-end teammates without feeling dumb.</p><p>Whatever your motivation, there are so many pieces to the puzzle that is a modern day web app. How do all these servers, storage and services fit together? And when (or why) would I want to use a thing?</p><p>In this article, I will give you a high-level overview of the main components used in today’s web apps. You will learn:</p><ul><li>each component’s core purpose and benefits</li><li>the other components it interacts with</li><li>examples of the commonly used technologies which implement it</li></ul><p><em>💌 </em><strong><em>Want to become a full-stack web developer and ship your own end-to-end apps?</em></strong><em> </em><a href="https://winterwindsoftware.com/newsletter/"><em>My weekly newsletter</em></a><em> provides tips on how you can use cloud services to build scalable web apps without needing to be a back-end master.</em></p><h3>Web Application Server</h3><p>A web application server is where your custom code runs that processes a client’s request (either a request for a HTML page or an API call). Your custom code will call out to other components to send or fetch data (e.g. those listed below) and then send a response back to the client.</p><p>You write this custom code in a server-side language such as Node.js, Python, PHP, Java, C# or Ruby. Each language has its own “web framework” (e.g. Ruby on Rails, ASP.NET MVC for C# or Express for Node.js). These frameworks enable the developer to write less boilerplate code to handle the request.</p><p>In addition to hosting custom application code, some web app architectures also employ a “web server process” such as <a href="https://httpd.apache.org/">Apache HTTP Server</a> or <a href="https://nginx.org/en/">nginx</a>. These server processes intercept the client request before it reaches your custom code. They are used for a few reasons:</p><ul><li>To quickly redirect certain requests rather than having to do it via custom code.</li><li>To serve static content (e.g. images, CSS, JS) stored on the web server’s file system faster than going through your custom code.</li><li>Certain server-side languages (e.g. PHP) don’t have a production-grade web server built into them and so need to be launched by a dedicated web server process.</li></ul><p>The web server machine itself where these tools and your custom code are installed to can take a few forms:</p><ul><li>a physical machine</li><li>a virtual private server (e.g. Rackspace, Linode)</li><li>a managed virtual machine instance (e.g. AWS EC2, Google Compute Engine)</li><li>a Platform-as-a-Service (PaaS) host (e.g. Heroku, AWS Elastic Beanstalk)</li></ul><p>You may hear these different locations where your application code runs collectively referred to as “compute”.</p><h3>Load Balancer</h3><p>If you are expecting traffic spikes or if high uptime is important, you may decide to install your custom code on 2 or more application servers. In this case, you will then need to use a load balancer.</p><p>A load balancer is a server which receives the request from the client and then forwards it on to a web application server.</p><p>You need to tell it the locations (IP addresses or domain names) of the web application servers to which it will be forwarding client requests. It will then send regular ping messages called “health checks” to each application server (e.g. every 5 seconds) to ensure that the server is responsive. When the load balancer receives a client request, it uses an algorithm (e.g. round robin) to forward the request to the next <em>healthy</em> application server.</p><p>There are 2 key benefits of using a load balancer in a web app:</p><ol><li>It helps maintain <strong>consistent response times</strong> by ensuring that a single web server isn’t swamped with all the requests and thus would be slower to process each one.</li><li>It maintains <strong>high availability</strong>. If a server crashes, all subsequent client requests will still succeed as they will be routed to a healthy server and your end users won’t notice any issues.</li></ol><p>Some examples of load balancer implementations are:</p><ul><li><a href="https://www.haproxy.org/">HAProxy</a> — open-sourced software which you would install yourself on your own virtual machine</li><li><a href="https://aws.amazon.com/elasticloadbalancing/">AWS Elastic Load Balancer</a> — this is a suite of managed load balancers meaning that you don’t need to provision virtual servers and install the load balancing software on them. They can be effectively treated as a cloud service.</li></ul><h3>Domain Name System (DNS)</h3><p>When a user enters a URL into their address bar, the browser takes the domain part of the URL (e.g. www.google.com) and makes a call to a DNS <a href="https://en.wikipedia.org/wiki/Name_server">nameserver</a>. The nameserver sends back an IP Address location for that website’s server (e.g. 172.217.23.4). Once it has an IP address it can then send the actual request for the web page.</p><p>If your web application is using a load balancer you would configure the domain name to point to your load balancer’s domain name or IP address. If you’re not using a load balancer, then you would point the domain name directly at your application server’s domain name/IP address.</p><p>Most internet domain name registration services (e.g. GoDaddy, NameCheap) provide a DNS management console. These allow you to configure your domain names (and subdomains) to point to the location of your application.</p><p>If you wish you can also transfer your nameservers over to a cloud provider such as <a href="https://aws.amazon.com/route53/">AWS Route 53</a> and manage it from there. A benefit of doing this is that you keep all your application environment configuration in the one place as well as making it easier to automate.</p><h3>HTTPS / SSL Certificates</h3><p>If you’re building a web app (or static website), you need to serve it over HTTPS to ensure secure communication between your users and your servers. There are now also <a href="https://moz.com/blog/seo-tips-https-ssl">SEO benefits</a> to using HTTPS, so there’s no excuse not to use it.</p><p>This means that you need SSL certificates installed on your back-end. Specifically, you need to install them on any server which is the first point of contact for a client request. That usually means the load balancer and the CDN server, but could also be the application servers if you’re not using a load balancer.</p><p>You can use <a href="https://letsencrypt.org/">LetsEncrypt</a> to generate a certificate for free. Alternatively, if you’re using cloud infrastructure you can use a managed service such as <a href="https://aws.amazon.com/certificate-manager/">AWS Certificate Manager</a>. This allows you to create and automatically renew and distribute the SSL certificates to your application servers, load balancers and CDN servers.</p><h3>Database</h3><p>Almost all web applications need to persist data somewhere. In most cases, that somewhere is a form of database. The database’s main job is to persist data reliably to permanent storage and to allow that data to be retrieved back out via queries. It may also enforce some rules around the structure of the data it stores.</p><p>There are many different database implementations which we don’t have time to go into here. There are, however, 2 high-level categories that most fall under: relational databases (SQL-based) and “NoSQL” databases. Relational databases (e.g. MySql, Postgres, MS SQL Server, Oracle, SQLite) have been around for over 40 years and have been the mainstay of most web apps. During the past decade or so, NoSQL databases (e.g. MongoDB, Cassandra, CouchDB, DynamoDB) have become much more common in web apps, largely because of their scalability benefits and data structure flexibility.</p><p>You can host a database on one server but more common in production scenarios is to host it on 2 or more servers in some form of a cluster. This ensures that the database is highly available and reduces the risk of data loss, e.g. if one server’s storage gets corrupted.</p><p>In recent years, a small number of cloud-hosted “serverless databases” have become available. These are databases which you can call via an API but that you don’t need to worry about setting up servers to host them. The cloud vendor invisibly does that for you, in addition to handling things such as automated backups. Examples of these are <a href="https://aws.amazon.com/dynamodb/">DynamoDB</a> (NoSQL), <a href="https://firebase.google.com/docs/database/">Firebase Realtime Database</a> (NoSQL) and <a href="https://aws.amazon.com/rds/aurora/serverless/">Aurora Serverless</a> (relational).</p><h3>Blob/File Storage</h3><p>Whilst databases are typically used to store dynamic data (e.g. generated by end users or API clients), there are certain categories of data which are not changeable by the user or which are file-based that aren’t a great fit for database storage. Examples of this are:</p><ul><li>Front-end website assets such as images, Javascript, CSS, fonts, audio, video files</li><li>Files uploaded by a user via a form</li></ul><p>Rather than store these in a database, cloud vendors provide dedicated services to store these such as <a href="https://aws.amazon.com/s3/">AWS Simple Storage Service (S3)</a>, <a href="https://cloud.google.com/storage/">Google Cloud Storage</a> and <a href="https://docs.microsoft.com/en-us/azure/storage/">Azure Storage</a>.</p><p>The benefits of this are that the cloud vendor stores the files securely and can make redundant copies of it to minimise the risk of data loss.</p><p>A “bucket” is the term used to denote a top-level storage folder which a web app would use to store its static content.</p><h3>Content Delivery Network (CDN)</h3><p>Blob/file storage services allow clients to access the files over a HTTP endpoint. For example, your web app’s HTML markup could simply link to the URLs of the images and CSS files stored in AWS S3. However, let’s say my user is based in Paris and my S3 storage is hosted in West U.S. — that is several thousand miles for the data to travel and so my user will observe a delay.</p><p>A CDN is a service provided by cloud vendors where they have “edge servers” distributed all across the globe. These edge servers take a copy of a file from an “origin” (e.g. a blob/file storage location). Rather than pointing at the blob storage URL for the static assets, your front-end web app would point to their CDN URL. Now rather than a round-trip of a few thousand miles, the distance between the client and the “edge” is a lot less and so the file is fetched much quicker.</p><p>Examples of commonly used CDNs are <a href="https://www.cloudflare.com/">CloudFlare</a> and <a href="https://aws.amazon.com/cloudfront/">AWS CloudFront</a>.</p><h3>Caching Service</h3><p>Whilst a CDN acts a form of cache for static files, web applications may have a need to temporarily cache <em>dynamic</em> data.</p><p>For example, say there is a database query which performs a calculation over yesterday’s data whose result is frequently accessed by thousands of users every day. It doesn’t make sense to contact the database every time a user requests this data.</p><p>A solution for this is to use a caching service to store the result for a certain time period after the first user has requested it. Subsequent requests for that data will be served much faster via the cache.</p><p>A caching service is essentially a specialised type of database. The server-based clustering comments in the Database section above are all applicable here. A cache takes the form of a key-value store, where the key is a string which your application code uses to query for the data (e.g. DailySiteStats_2018-10-17) and the value is the actual data being cached. A cache’s data is often held fully in memory which makes retrieval of the data from the cache extremely fast.</p><p>Common caching services are <a href="https://redis.io/">Redis</a> and <a href="https://www.memcached.org/">Memcached</a>. AWS provides a managed version of both these via its <a href="https://aws.amazon.com/elasticache/">Elasticache</a> service.</p><h3>Message Queue</h3><p>Sometimes there are tasks you need to perform that aren’t directly related to responding to a user’s request.</p><p>For example, say a user has uploaded a video which you now need to encode and watermark. But this is a long-running task so it doesn’t make sense to make the user wait while it completes. A better approach is to do this asynchronously. Your web app code creates a job message in a queue and informs your user that they will receive an email when the watermarked video is ready.</p><p>You would then have a worker task which would do the following:</p><ol><li>Read the message from the queue</li><li>Start processing the video</li><li>Once finished, save encoded copy of the video</li><li>Send the user a notification email</li><li>Delete message from the queue</li></ol><p>There are 2 architectural components at work here:</p><p>You can implement the worker task in a few ways:</p><ul><li>Scheduling a <a href="https://en.wikipedia.org/wiki/Cron">CRON job</a> to trigger some custom code installed on your application server to read from the queue on a certain schedule</li><li>Using a Functions-as-a-Service platform to invoke your worker code whenever a message is added to the queue (see section below)</li></ul><h3>Functions-as-a-Service (FaaS)</h3><p><a href="https://en.wikipedia.org/wiki/Function_as_a_service">Functions-as-a-Service</a> is a way that you can run your own custom code (“compute”) on cloud infrastructure without needing to worry about provisioning and maintaining the servers upon which that code will be running. You may also hear this referred to as “serverless” computing (although strictly speaking FaaS is a subset of serverless).</p><p>There are many use cases for FaaS :</p><ul><li>Data processing jobs</li><li>Back-end service integrations</li><li>Automated server maintenance and orchestration tasks</li><li>Building a web API</li><li>Server-side HTML rendering</li></ul><p>Examples of FaaS providers are <a href="https://aws.amazon.com/lambda/">AWS Lambda</a>, <a href="https://cloud.google.com/functions/">Google Cloud Functions</a> and <a href="https://azure.microsoft.com/en-us/services/functions/">Azure Functions</a>.</p><h3>A “service-ful” future for full-stack development</h3><p>As serverless technologies mature and their limitations reduce, I believe FaaS and other fully managed cloud services will become much more commonplace in web app architectures over the next few years.</p><p>The good news for aspiring full-stack developers is this means that many of the above architectural concepts (e.g. application servers, load balancers) <a href="https://winterwindsoftware.com/concerns-that-serverless-takes-away/">will become redundant</a>. Their complexities are being absorbed into the cloud services you will be using. <strong>The bar to becoming an accomplished full-stack engineer is being lowered.</strong></p><p>That leaves you to focus much more on where the real business value is — <strong>designing and writing application code for your users.</strong></p><p><strong><em>💌 If you enjoyed this article, you can sign up </em></strong><a href="https://winterwindsoftware.com/newsletter/"><strong><em>to my weekly newsletter</em></strong></a><strong><em> where I provide tips on how you can use cloud services to build scalable web apps without needing to be a back-end master.</em></strong></p><p><em>Originally published at </em><a href="https://winterwindsoftware.com/back-end-primer/"><em>winterwindsoftware.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=80339b5b5c3a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Concerns that go away in a serverless world]]></title>
            <link>https://medium.com/@paulswail/concerns-that-go-away-in-a-serverless-world-3f250e28a15d?source=rss-3a509aed10a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f250e28a15d</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Paul Swail]]></dc:creator>
            <pubDate>Thu, 27 Sep 2018 06:46:02 GMT</pubDate>
            <atom:updated>2018-09-27T06:46:02.662Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0qjE_0C_vqpI2TaUcRNwqQ.jpeg" /></figure><p>A common claim from serverless advocates is that it can <a href="https://medium.com/@PaulDJohnston/feature-velocity-and-serverless-588a2ff2d59a">increase a team’s feature velocity</a> and reduce the ops overhead of running an application in the cloud. But if you’re an engineering manager, developer or ops engineer who has never run a serverless workload in production, you may find this claim hard to visualise and quantify.</p><p><strong><em>How</em></strong><em> will serverless increase our team’s productivity?</em></p><p><em>In what ways will it affect the skillsets we need in our engineering team?</em></p><p>To help answer these questions, I’ve compiled a list of specific concerns and tasks encountered when building and operating server-based applications <strong>that no longer apply when using a serverless architecture</strong>.</p><p>By the end of the article, you should have a clearer picture of how selecting a serverless architecture can potentially reduce the total cost of ownership of a new application within your organisation.</p><p>When reading through the list, try to think of a server-based app you are currently or have previously worked on, and ask yourself the following questions about each list item:</p><ul><li>Do I have the <strong>knowledge to complete this task myself</strong> or is this something a specialist or more experienced member of my team takes care of? If the latter, how much time would it take me to understand what’s involved in doing this?</li><li>How much <strong>time do I (or another member of my team) currently spend doing this</strong> and how frequently does it need repeated?</li><li>Even though I can do this task myself, am I <strong>confident that I’ve implemented it correctly</strong> using best practices and that I haven’t introduced any security holes or performance issues?</li><li>Is this something that we don’t currently do on our team (due to time or skill constraints), but that <strong>I know that we <em>should</em> be doing</strong>, and thus we’ve taken on extra risk by omitting it?</li></ul><p><em>A note on scope: I focus on the AWS ecosystem so the items listed below primarily relate to the EC2, VPC, RDS, ECS/EKS and ELB services Amazon provides. Nevertheless, most points are generally applicable to other major cloud providers. I’ve assumed the server-based system is run either in containers or directly on the virtual machine instances.</em></p><p>Ok, let’s dive into the list of concerns which which go away in a serverless world…</p><h3>Server Provisioning and Scaling</h3><ul><li>Configure AMIs for your VM instances with specific OS version and any required application software</li><li>Set up a VPC and subnets using best-practice security settings</li><li>Configure security groups and identify what ports need to be open on each instance</li><li>Create launch configurations and auto scaling groups for each EC2 instance type</li><li>Configure load balancers and associated health checks</li><li>Set up internet gateways</li><li>Configure route tables</li><li>Configure VPC peering</li><li>Configure RDS cluster with appropriate storage and instance size</li><li>Regularly observe load-related metrics and modify scaling limits or instance resource allocation accordingly</li><li>Repeat most of above steps for each environment (dev, test, staging, production)</li></ul><h3>Application Development &amp; Maintenance</h3><ul><li>Define your container environment (Dockerfile)</li><li>Configure your container orchestration cluster (ECS, Kubernetes, etc)</li><li>Configure the pods/services/task definitions within your cluster</li><li>Debug container inter-connectivity/service discovery issues</li><li>Write script to deploy build artifact (Docker image, zip file) to EC2 instance</li><li>Regularly update base Docker image with latest patches (e.g. to Node.js/Python/Java or whatever language your app uses)</li></ul><h3>Server Maintenance</h3><ul><li>Set up a secure VPN/SSH bastion instance (and keep it patched)</li><li>Manage VPN/SSH access to different servers for authorised engineers</li><li>Manage regular patching of all VM instances (either manually via SSH or automated via script/Systems Manager)</li><li>Be available to promptly deploy emergency patches (e.g. heartbleed)</li><li>Set up alerts to be notified about emergency patches</li><li>Set up monitoring to watch for low disk space</li><li>Manually expand a volume when it’s out of space</li><li>Handle SSL certificate renewal and deployment (if installing keys directly to instances and not just to load balancers where it’s managed by AWS)</li><li>Repeat most of above steps for each environment (dev, test, staging, production)</li></ul><h3>Cost Control</h3><ul><li>Pay for an EC2/RDS/ElastiCache instance when it’s not in use</li><li>Over-provision instances to handle occasional sudden traffic spikes</li><li>Write cron jobs to spin down dev/test environment instances at evenings and weekends</li></ul><p><strong><em>Related: </em></strong><a href="https://winterwindsoftware.com/ec2-to-serverless-migration-calculator/"><strong><em>How to calculate the billing savings of moving an EC2 app to Lambda</em></strong></a><strong><em>.</em></strong></p><p>By now, you’re probably shouting <em>“That’s great Paul, but what about all the new concerns that serverless brings?”</em> And you would be correct to do so, my healthily skeptical friend!</p><p>I do intend to write about this soon, but for now I will point you at this article on <a href="https://dev.to/adnanrahic/containers-vs-serverless-from-a-devops-standpoint-e4n">Containers vs Serverless from a devops standpoint</a> and also Martin Fowler’s section on <a href="https://martinfowler.com/articles/serverless.html#drawbacks">the drawbacks of operating serverless systems</a>.</p><p>That said, and while this is very difficult to objectively measure, I personally believe that for most development teams building a greenfield production system in the cloud today <strong>the total cost of ownership of a serverless app will be lower than that of a server-based app</strong>.</p><p>This is especially true if your organisation doesn’t have a skilled ops team already in place with availability to help your dev team build out the infrastructure and automation required to provision and maintain your application.</p><p>Got any more items that I should add to the list? I’m sure there are plenty of things I’ve forgotten so please leave me a comment below and I’ll add it in.</p><p>💌 <strong><em>If you enjoyed this article, you can sign up </em></strong><a href="https://winterwindsoftware.com/newsletter/"><strong><em>to my weekly newsletter on building serverless apps in AWS</em></strong></a><strong><em>.</em></strong></p><p><em>Originally published at </em><a href="https://winterwindsoftware.com/concerns-that-serverless-takes-away/"><em>winterwindsoftware.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f250e28a15d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to connect to the AWS Console as multiple IAM users at the same time]]></title>
            <link>https://medium.com/@paulswail/how-to-connect-to-the-aws-console-as-multiple-iam-users-at-the-same-time-f5dae029e21c?source=rss-3a509aed10a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/f5dae029e21c</guid>
            <dc:creator><![CDATA[Paul Swail]]></dc:creator>
            <pubDate>Thu, 20 Sep 2018 07:31:03 GMT</pubDate>
            <atom:updated>2018-09-20T07:31:03.581Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1t69ZQLu-7nf8iyT6m5oVg.png" /></figure><p>If you’ve ever had to manage multiple AWS accounts concurrently, whether it’s for <a href="https://winterwindsoftware.com/managing-separate-projects-in-aws/">different client projects</a> or different environments of the same app, you’ve probably encountered the pain of having to log out of the AWS Console as one user and log in as another. If you’ve multi-factor authentication enabled (<a href="https://aws.amazon.com/iam/details/mfa/">which you should</a>), this just adds to the friction.</p><p>You tell yourself this is a UX trade-off in favour of security.</p><p>But we can have our cake and eat it! 🍰</p><p>By using the somewhat obscure <a href="https://support.google.com/chrome/answer/2364824">People Profiles</a> feature in Chrome, you will be able to:</p><ul><li>Open an AWS Console window for a new user profile with 2 clicks</li><li>Avoid needing to switch to the separate login screen if your profile is for a root user rather than an IAM one</li></ul><p>People profiles effectively act like a sandbox whereby each person gets their own cookies, cache, local storage, extensions, etc.</p><p>Just follow these quick steps to create your profile, repeating for each AWS user you frequently use:</p><ol><li>Open a Chrome window and click on the circular user image in the top right corner to open a menu</li><li>Click on the “Manage People” button and then the “Add Person” button.</li><li>Name the user the same as their AWS username (optionally prefixed with “AWS” or “IAM”).</li></ol><p>That’s it!</p><p>Now go back to your original user’s window and open the Manage People popup again. You should see all the users that you’ve created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/811/0*48DvEHHYrWdk5ZEd.png" /></figure><p>Just click on the user you wish to connect with and a new sandboxed browser window will be opened.</p><p>You will still have to log in as each user once their session expires (<a href="https://aws.amazon.com/console/faqs/#session_expire">currently set to 12 hours</a>), but at least now you can go through your login process once in the morning and keep multiple windows open for your whole day’s work.</p><p>Some bonus tips:</p><ul><li>Within your person profile, set your homepage to <a href="https://console.aws.amazon.com/console/home">https://console.aws.amazon.com/console/home</a> so that you’ll be immediately taken to the AWS console home page.</li><li>Make sure to install your favourite password manager extension (1Password/LastPass) to each user profile if you want quick password auto-filling when logging in.</li><li>Use CMD+ ` to quickly switch between open browser windows in OSX.</li><li>Customise the <a href="https://support.google.com/chrome_webstore/answer/148695?hl=en-GB">theme</a> of your browser window at the user-level (e.g. you might want a <a href="https://chrome.google.com/webstore/detail/fire-flame-close-up-lsp/apahdohilpbfhfbnjepjilngffpoepif?hl=en-GB">fire 🔥 theme</a> for your production environment user).</li></ul><p>Go forth and be productive!</p><p>💌 <strong><em>If you enjoyed this article, you can sign up </em></strong><a href="https://winterwindsoftware.com/newsletter/"><strong><em>to my weekly newsletter on building serverless apps in AWS</em></strong></a><strong><em>.</em></strong></p><p><em>Originally published at </em><a href="https://winterwindsoftware.com/aws-console-multiple-users/"><em>winterwindsoftware.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f5dae029e21c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The differing definitions of “serverless”]]></title>
            <link>https://medium.com/@paulswail/the-differing-definitions-of-serverless-47cf33e5751f?source=rss-3a509aed10a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/47cf33e5751f</guid>
            <dc:creator><![CDATA[Paul Swail]]></dc:creator>
            <pubDate>Thu, 13 Sep 2018 07:26:02 GMT</pubDate>
            <atom:updated>2018-09-13T07:26:02.220Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_etWAzBBQ257FIn3vbzS-A.png" /></figure><p>Like many industry buzzwords before it — “cloud computing”, “DevOps”, “big data” — debates over the definition of the term “serverless” have become somewhat of a honeypot for pedants.</p><blockquote>“But there are still servers involved!” <br> — Dave from Reddit.</blockquote><p>Thanks for that, Dave. 😏</p><p>If you’re reading an article about anything serverless-related today, there’s a fair chance that the author will have stated their definition of the term in the introduction (probably to avoid incurring the wrath of the Daves of the internet).</p><p>Experienced practitioners will just skip to the next paragraph, but if you’re new to the space, adding some depth to your understanding of the concept is worthwhile.</p><p>In this article, we’ll walk through the definitions provided by several prominent advocates in the serverless space and extract some common themes along the way, finishing off with me attempting my own inelegant definition.</p><p><em>Note: All </em><strong><em>emphasis</em></strong><em> is mine in the quotes below.</em></p><h3>What the experts said</h3><p>We’ll start with AWS themselves:</p><blockquote>“Serverless computing allows you to build and run applications and services without thinking about servers. Serverless applications don’t require you to provision, scale, and manage any servers. You can build them for nearly any type of application or backend service, and everything required to run and scale your application with high availability is handled for you.” (<a href="https://aws.amazon.com/serverless/">source</a>)</blockquote><p>The authors of the <a href="https://serverless.com/">Serverless Framework</a> share a similar message:</p><blockquote>“Just like wireless internet has wires somewhere, serverless architectures still have servers somewhere. What ‘serverless’ really means is that, as a developer <strong>you don’t have to think about those servers. You just focus on code.</strong></blockquote><blockquote>… You don’t have to actively manage scaling for your applications. You don’t have to provision servers, or pay for resources that go unused.” (<a href="https://serverless.com/learn/">source</a>)</blockquote><p>In his definition article, <a href="https://twitter.com/tjanczuk">Tomasz Janczuk</a>, VP at Auth0, provides a high-level summation:</p><blockquote>“The essence of the serverless trend is the absence of the server concept during software development.” (<a href="https://auth0.com/blog/what-is-serverless/">source</a>)</blockquote><p><a href="https://twitter.com/ajaynairthinks">Ajay Nair</a>, Product Manager of AWS Lambda, focuses on the specifics of the tasks which go away:</p><blockquote>“Owning servers comes with responsibilities:</blockquote><blockquote>- You own how your primitives (functions in the case of applications, or objects when it comes to storage) map to server primitives (CPU, memory, disk etc.)</blockquote><blockquote>- You own how your capacity scales to handle your application scale</blockquote><blockquote>- You own provisioning (and therefore paying) for the capacity to handle your application’s projected traffic, independent of whether there’s actual traffic or not</blockquote><blockquote>- You own managing reliability and availability constructs like redundancy, failover, retries etc.</blockquote><blockquote>For me, serverless means <strong>activities/responsibilities related to servers are no longer on your radar</strong>.” (<a href="https://www.quora.com/What-is-Serverless-Computing/answer/Ajay-Nair">source</a>)</blockquote><p><a href="https://twitter.com/PaulDJohnston">Paul Johnston</a>, ex AWS Serverless Snr DA, gives a pithy, financially-focused explanation:</p><blockquote>“A Serverless solution is one that <strong>costs you nothing to run if nobody is using it</strong> (excluding data storage).” (<a href="https://medium.com/@PaulDJohnston/a-simple-definition-of-serverless-8492adfb175a">source</a>)</blockquote><p>Paul also points out a common misconception that I often encounter:</p><blockquote>“Often in people’s minds, Serverless means that you’re using Functions as a Service (FaaS) which is only one part of a possible Serverless solution.”</blockquote><p><a href="https://twitter.com/sbarski">Peter Sbarski</a>, VP Engineering at A Cloud Guru, gives his take:</p><blockquote>“Serverless is about <strong>abstracting users away from servers, infrastructure, and having to deal with low-level configuration or the core operating system</strong>. Instead, developers make use of single purpose services (such as S3 for storage or Auth0 for identity management) and elastic compute platforms (such as AWS Lambda) to execute code.” (<a href="https://www.quora.com/What-is-Serverless-Computing/answer/Peter-Sbarski">source</a>)</blockquote><h3>Common themes</h3><p>Extracting the meat from the quotes above, there are re-occurring ideas around what serverless includes and what it excludes:</p><h3>What’s in</h3><ul><li>Resource costs must be tightly aligned with usage levels</li><li>Greater focus on application code</li></ul><h3>What’s out</h3><ul><li>Server provisioning and maintenance</li><li>Active management of service availability and scaling</li></ul><h3>My definition</h3><p>I’m going to stand on the shoulders of the giants mentioned above and organise these points into prose to give my own definition:</p><blockquote><strong><em>A serverless solution is one where developers are freed up to focus more on application development without needing to worry about provisioning, scaling or maintaining servers (or containers), and whose running costs are proportional to the system’s usage levels.</em></strong></blockquote><p>And finally, if you’re that person writing an article about serverless and feel the need to define the term for your readers, just copy and paste the following into your post and get back to writing! 😎</p><blockquote><em>👉 Not sure what serverless is? Check out this </em><a href="https://winterwindsoftware.com/serverless-definitions/"><em>curation of serverless definitions from the experts</em></a><em>.</em></blockquote><p>💌 <strong><em>If you enjoyed this article, you can sign up </em></strong><a href="https://winterwindsoftware.com/newsletter/"><strong><em>to my weekly newsletter on building serverless apps in AWS</em></strong></a><strong><em>.</em></strong></p><p><em>Originally published at </em><a href="https://winterwindsoftware.com/serverless-definitions/"><em>winterwindsoftware.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=47cf33e5751f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Should you be concerned about vendor lock-in when writing FaaS functions?]]></title>
            <link>https://medium.com/@paulswail/should-you-be-concerned-about-vendor-lock-in-when-writing-faas-functions-8780870923f8?source=rss-3a509aed10a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/8780870923f8</guid>
            <category><![CDATA[functions-as-a-service]]></category>
            <category><![CDATA[google-cloud-functions]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[azure-functions]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <dc:creator><![CDATA[Paul Swail]]></dc:creator>
            <pubDate>Thu, 06 Sep 2018 11:06:01 GMT</pubDate>
            <atom:updated>2018-09-06T11:06:01.389Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pj9QdxW3bqObJmRGExDgXw.jpeg" /></figure><p>Vendor lock-in is a topic which has been discussed at length since the early days of cloud computing and is particularly prevalent today in the context of serverless applications.</p><blockquote>“Lambda and serverless is one of the worst forms of proprietary lock-in that we’ve ever seen” — CEO of CoreOS (source: <a href="https://www.theregister.co.uk/2017/11/06/coreos_kubernetes_v_world/">The Register</a>)</blockquote><p>But is it a real issue that you need to be concerned about when picking a Functions-as-a-Service (FaaS) provider?</p><p>Or is it just fear-mongering put out by competing vendors and “traditional” software architects averse to learning a new paradigm?</p><p>In this article, we will walk through the FaaS offerings of the 3 major cloud providers (AWS Lambda, Google Cloud Functions and Microsoft Azure Functions) and examine what would be required to switch providers down the line.</p><h3>Why might you want to switch?</h3><p>Firstly, let’s look at some potential reasons why you may want to switch FaaS provider:</p><ul><li>You want to integrate your function with a new back-end service which is only offered by another provider</li><li>Runtime issues or limitations of current provider being hit</li><li>Function execution is becoming too costly and would be much cheaper elsewhere</li><li>Provider drops support for your language runtime</li><li>Organisational strategy or support reasons</li></ul><h3>What would be involved in switching?</h3><p>In terms of gauging the effort to switch FaaS providers, we will look at:</p><ul><li>changes required to the function codebase.</li><li>changes to execution configuration, deployment and triggers.</li></ul><h3>Code changes</h3><p>Each cloud provider specifies a function signature which your code must adhere to in order to be executable in their cloud. Each provider’s signature is slightly different. The following Node.js code examples show how, for each provider, you can read the input from the triggering event and send a response back. Each example assumes it’s been triggered by a HTTP request.</p><h4>AWS Lambda (<a href="https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html">docs</a>)</h4><pre>exports.myHandler = async (event, context) =&gt; {<br>  const input = JSON.parse(event.body);<br>  return {<br>    statusCode: 200, body: { message: `Hello ${input.myName}` } <br>  };<br>};</pre><h4>Azure Functions (<a href="https://docs.microsoft.com/en-us/azure/azure-functions/functions-reference-node">docs</a>)</h4><pre>module.exports = (context, req) =&gt; { <br>  const input = JSON.parse(req.body);<br>  context.res = {<br>    status: 200,<br>    body: { message: `Hello   ${input.myName}` } <br>  };<br>  context.done();<br>};</pre><h4>Google Cloud Functions (<a href="https://cloud.google.com/functions/docs/quickstart">docs</a>)</h4><pre>exports.myHandler = (req, res) =&gt; {<br>  res.status(200).send({ message: `Hello ${req.body.myName}` });<br>};</pre><p>As you can see from the above 3 examples, the differences between each provider are small so it should not be a big job to refactor them. Initiatives such as <a href="https://cloudevents.io/">Cloud Events</a> (aimed at standardising the schema for events which trigger functions), should help reduce these differences even more in the future.</p><p>Another point to consider is the runtime of your chosen language that is supported by the cloud provider, in particular if you are using new language features. For example, at the time of writing AWS Lambda supports Node.js v8.10 which has the new async/await JavaScript syntax. However, Google Cloud functions only supports up to v6.14 which doesn’t support this feature.</p><h3>Changes to deployment, trigger and execution configuration</h3><p>Each provider gives different settings to allow you to configure how your functions will execute, in particular around how they’re triggered and what compute resources (memory, CPU) should be allocated to them. For deployment, each provider has their own CLI for packaging and publishing functions. So your deployment scripts would need to be updated if you choose to use this CLI. Regarding triggers, the main difference to be aware of at the time of writing is that to invoke AWS Lambda functions over HTTPS, you need to use the <a href="https://aws.amazon.com/api-gateway/">API Gateway service</a>. This will involve setting up its own configuration and it’s also billed separately. Azure functions and Google Cloud Functions, on the other hand, support HTTP triggers natively with no extra charge.</p><h3>How can you mitigate the risks of needing to switch in the future?</h3><p>The following recommendations should help you design your FaaS solution in such a way as to minimise any future effort required to migrate your functions to a different FaaS provider:</p><ul><li>Check that your organisation’s preferred language runtime is supported by multiple providers.</li><li>Don’t pass provider-specific parameters (e.g. context or res) through multiple helper functions. Instead parse the fields you need out of the parameter object in the main handler function and pass those through.</li><li>Extract any calls to provider-specific services (e.g. AWS S3) into their own helper function/module so you can easily swap these out in the future.</li><li>Maintain locally runnable unit tests for each handler function to act as a safety net for your code refactoring.</li><li>Check the triggers supported required for your app (e.g. HTTP request, queue message, CRON schedule) have equivalents in the other cloud providers.</li><li>Check that the limits imposed by your provider (e.g. around memory allocation and maximum request timeout) are within the requirements of your app and compare these to the other cloud providers. For example, at the time of writing, AWS Lambda supports up to 3GB memory allocation and max timeout of 5 minutes, whereas Azure supports a maximum of 1.5GB of memory but a max timeout of 10 minutes.</li><li>Use the <a href="https://serverless.com/">Serverless framework</a> to package and deploy your app. It supports all the main cloud providers and means you can master your function and trigger configuration in a single file (serverless.yml) and use the same CLI command to deploy to each provider. This CLI can be invoked from a developer machine or from within a CI pipeline.</li></ul><h3>Vendor lock-in beyond FaaS</h3><p>In this article, we’ve dealt with the FaaS component of a serverless architecture. In reality, I don’t believe vendor lock-in should be a major concern for the vast majority of organisations in terms of changes required to the code and configuration of your functions and your build and deployment process, in particular if the above recommendations are heeded.</p><p>However, a bigger concern in terms of vendor lock-in a wider serverless architecture is the services which your functions integrate with. Your app will inevitably need to store data, either in relational or non-relational databases, file storage, caches or queues. The big 3 vendors provide a myriad of services in these areas, all with differing levels of lock-in. There are many trade-offs to consider in making these architectural decisions which I hope to dive into in more detail in a future article.</p><p>💌 <strong><em>If you enjoyed this article, you can sign up </em></strong><a href="https://winterwindsoftware.com/newsletter/"><strong><em>to my weekly newsletter on building serverless apps in AWS</em></strong></a><strong><em>.</em></strong></p><p><em>Originally published at </em><a href="https://winterwindsoftware.com/faas-vendor-lockin/"><em>winterwindsoftware.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8780870923f8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Scaling Lambdas inside a VPC]]></title>
            <link>https://medium.com/@paulswail/scaling-lambdas-inside-a-vpc-6320ce18e02?source=rss-3a509aed10a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/6320ce18e02</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <dc:creator><![CDATA[Paul Swail]]></dc:creator>
            <pubDate>Thu, 30 Aug 2018 09:55:14 GMT</pubDate>
            <atom:updated>2018-08-30T09:55:14.492Z</atom:updated>
            <content:encoded><![CDATA[<p>So you’re building a Lambda function and you need it to talk to resources inside your VPC, maybe an RDS database or a Redis Elasticache.</p><p>You’ve heard VPC Lambda functions are more complicated than standard functions and are finding it difficult to get the full picture from the AWS documentation. It’s hard to know what you don’t know when the relevant docs are spread over so many pages.</p><p>However, you <em>do</em> know the questions you need answers to in order to be confident that your function will perform at scale:</p><ul><li>Will it be able to handle my maximum projected throughput?</li><li>Is there a ramp-up limit?</li><li>How will my app be affected if the actual usage surpasses these limits?</li><li>How can cold starts affect my Lambda function while under load?</li></ul><p>In this article, we’ll try to answer these questions and help <strong>ensure that your VPC-enabled function is optimally configured for scalability</strong>.</p><p><em>Don’t care about understanding the “why” and just want to know the “how”? Jump straight to the </em><strong><em>Action Plan</em></strong><em> section at the bottom of the article.</em>👇</p><h3>Overview of Lambda scaling</h3><p>Let’s first look at the <a href="https://docs.aws.amazon.com/lambda/latest/dg/scaling.html">scaling behaviour of Lambda functions</a>:</p><blockquote>AWS Lambda will dynamically scale capacity in response to increased traffic, subject to your account’s Account Level Concurrent Execution Limit. To handle any burst in traffic, Lambda will immediately increase your concurrently executing functions by a predetermined amount, dependent on which region it’s executed.</blockquote><p>To elaborate on what is meant by “scale capacity” here, under the hood each Lambda function is run inside a container (think Docker). A single container can handle sequential requests for several hours before Lambda eventually disposes of it. However, if another request comes in while one is in progress, a new container needs to be spun up to handle it.</p><p>The <a href="https://docs.aws.amazon.com/lambda/latest/dg/concurrent-executions.html#concurrent-execution-safety-limit">Account Level Concurrent Execution Limit</a> is set to to 1,000 per region (although it can be increased by contacting AWS Support). This means that the upper limit of throughput for all Lambda functions in your account can be calculated as follows:</p><p>max_requests_per_second = 1000 / function_duration_seconds</p><p>So for a function which takes an average of 0.5 seconds to complete, the absolute maximum throughput that can you can achieve is 1000 / 0.5 = 2000 requests per second. (Although if you have other functions in your account also executing during this period, this limit will be lower).</p><h3>Adding a VPC into the mix</h3><p>When you <a href="https://docs.aws.amazon.com/lambda/latest/dg/vpc.html">connect your Lambda function to a VPC</a>, scalability gets more complicated:</p><blockquote>AWS Lambda runs your function code securely within a VPC by default. However, to enable your Lambda function to access resources inside your private VPC, you must provide additional VPC-specific configuration information that includes VPC subnet IDs and security group IDs. <strong>AWS Lambda uses this information to set up elastic network interfaces (ENIs) that enable your function to connect securely to other resources within your private VPC.</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*Oxsa_7zAPdAgkST4.png" /><figcaption>Lambda functions connecting to a VPC via an ENI</figcaption></figure><p>This requirement to set up an ENI between each function container and the VPC constrains the scalability of your Lambda function in 3 ways:</p><ol><li>There’s an upper limit on the number of ENIs that can be created (ENI capacity), thus capping the max achievable requests per second</li><li>Cold start overhead is greater</li><li>Ramp-up rate is capped</li></ol><h3>Constraint #1: ENI Capacity</h3><p>The docs advise:</p><blockquote>If your Lambda function accesses a VPC, you must make sure that your VPC has sufficient ENI capacity to support the scale requirements of your Lambda function.</blockquote><p>… and they provide the following formula to tell you how many ENIs that your function will <em>need</em>:</p><p>ENI Capacity = Projected peak concurrent executions * (Memory in GB / 3GB)</p><p>So if your projected peak load is 1,000,000 requests/hr, your average function duration is 0.5 seconds and has 1GB of memory assigned to it, then the ENI capacity can be calculated as follows:</p><p>Peak concurrent executions = 1000000/(60*60) * 0.5 = 139</p><p>ENI Capacity = 139 * (1GB / 3GB) = 47</p><h4>Calculating your actual ENI Capacity</h4><p>But this formula doesn’t tell you how much ENI capacity <em>you currently have available</em> within your VPC, only how much your function <em>needs</em> to reach your projected peak load. Each ENI is assigned a private IP address from the IP address range within the subnets that are connected to the function. Therefore, in order to calculate the ENI capacity, we need to get a total of available IP addresses in all subnets.</p><p>To do this, you can use the <a href="https://console.aws.amazon.com/vpc/home#subnets:sort=SubnetId">VPC Console</a> to list all the subnets in your VPC. Then get the value of the IPv4 CIDR column and plug it into the <a href="https://mxtoolbox.com/subnetcalculator.aspx">IP CIDR calculator</a> to get the number of available IP addresses in that range. Do this for every subnet connected to your function and sum the values to get your total Actual ENI Capacity.</p><h3>Constraint #2: Cold start overhead</h3><p>All Lambda functions incur a “cold start”, which is the time taken for AWS to provision a container to service a function request. However, this impact <a href="https://docs.aws.amazon.com/lambda/latest/dg/vpc.html#vpc-configuring">is greater for VPC-enabled Lambda functions</a>:</p><blockquote>When a Lambda function is configured to run within a VPC, it incurs an additional ENI start-up penalty. This means address resolution may be delayed when trying to connect to network resources.</blockquote><p>In addition to affecting the individual requests which incur the cold start, it will also affect your ramp-up rate as it effectively adds to your average function duration during a ramp-up period.</p><h3>Constraint #3: Ramp-up rate</h3><p>Ramp-up rate is the rate at which Lambda can provision new function containers to service an increase in requests to your function. <a href="https://docs.aws.amazon.com/lambda/latest/dg/scaling.html#scaling-behavior">The docs</a> describe the ramp-up behaviour as follows:</p><blockquote>To handle any burst in traffic, Lambda will immediately increase your concurrently executing functions by a predetermined amount, dependent on which region it’s executed. If the default Immediate Concurrency Increase value is not sufficient to accommodate the traffic surge, AWS Lambda will continue to increase the number of concurrent function executions by 500 per minute until your account safety limit has been reached or the number of concurrently executing functions is sufficient to successfully process the increased load.</blockquote><p>The “Immediate Concurrency Increase” value is <a href="https://docs.aws.amazon.com/lambda/latest/dg/limits.html">different for each region</a>. For US East (N.Virginia), it is 3,000.</p><p>However, for VPC-enabled functions:</p><blockquote>Because Lambda depends on Amazon EC2 to provide Elastic Network Interfaces for VPC-enabled Lambda functions, these functions are also subject to Amazon EC2’s rate limits as they scale.</blockquote><p>You will need to use the EC2 Console to <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html">check the EC2 limits of your account</a>, in particular the “Network interfaces per region” limit, which defaults to 350. If this value is less than the “Peak concurrent executions” figure for your function, then you will need to request that this limit is increased.</p><h3>What happens when the concurrency limit is hit?</h3><p>If your function reaches its concurrency limit, any further invocations to it are throttled. The <a href="https://docs.aws.amazon.com/lambda/latest/dg/concurrent-executions.html#throttling-behavior">throttling behaviour</a> that Lambda applies is different depending on whether the event source is stream-based or not and whether it was invoked synchronously or asynchronously. For synchronously invoked, non-stream based event sources, the behaviour is:</p><blockquote>Lambda returns a 429 error and the invoking service is responsible for retries. The ThrottledReason error code explains whether you ran into a function level throttle (if specified) or an account level throttle. Each service may have its own retry policy. Each throttled invocation increases the Amazon CloudWatch Throttles metric for the function.</blockquote><p>So if you have a Lambda function behind an API Gateway, your web API clients will receive a 429 error.</p><p>Another <a href="https://docs.aws.amazon.com/lambda/latest/dg/vpc.html#vpc-setup-guidelines">item to be aware of</a> is:</p><blockquote>If your VPC does not have sufficient ENIs or subnet IPs, your Lambda function will not scale as requests increase, and you will see an increase in function failures. <strong>AWS Lambda currently does not log errors to CloudWatch Logs that are caused by insufficient ENIs or IP addresses.</strong></blockquote><p>Not having a log entry describing why this error is happening would be a pain to diagnose. The only indicator you will have that this is happening is the <a href="https://docs.aws.amazon.com/lambda/latest/dg/monitoring-functions-metrics.html">Invocation Errors</a> CloudWatch metric increasing.</p><h3>Your action plan</h3><p>So now you know how Lambda scales your VPC-enabled Lambda function, here’s a step-by-step guide to ensure your function is optimally configured for scalability:</p><ol><li>Calculate your Peak Concurrent Executions with this formula: <br> Peak Concurrent Executions = Peak Requests per Second * Average Function Duration (in seconds)</li><li>Now calculate your Required ENI Capacity:<br> Required ENI Capacity = Projected peak concurrent executions * (Function Memory Allocation in GB / 3GB)</li><li>If Peak Concurrent Executions &gt; Account Level Concurrent Execution Limit (default=1,000), then you will need to <a href="https://docs.aws.amazon.com/lambda/latest/dg/concurrent-executions.html#concurrent-execution-safety-limit">ask AWS to increase this limit</a>.</li><li>Configure your function to use all the subnets available inside the VPC that have access to the resource that your function needs to connect to. This both maximises Actual ENI Capacity and provides higher availability (assuming subnets are spread across 2+ availability zones).</li><li>Calculate your Actual ENI Capacity using <a href="https://winterwindsoftware.com/scaling-lambdas-inside-vpc/#actual-eni-capacity">these steps</a>.</li><li>If Required ENI Capacity &gt; Actual ENI Capacity, then you will need to do one or more of the following: (a) Decrease your function’s memory allocation to decrease your Required ENI Capacity ;(b) Refactor any time-consuming code which doesn’t require VPC access into a separate Lambda function; (c) Implement throttle-handling logic in your app (e.g. by building retries into client).</li><li>If Required ENI Capacity &gt; your EC2 <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html">Network Interfaces per region</a> account limit then you will need to <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html#request-increase">request that AWS increase this limit</a>.</li><li>Consider <a href="https://docs.aws.amazon.com/lambda/latest/dg/concurrent-executions.html#per-function-concurrency">configuring a function-level concurrency limit</a> to ensure your function doesn’t hit the ENI Capacity limit and also if you wish to force throttling at a certain limit due to downstream architectural limitations.</li><li><a href="https://docs.aws.amazon.com/lambda/latest/dg/concurrent-executions.html#monitoring-concurrent-usage">Monitor the concurrency levels</a> of your functions in production using CloudWatch metrics so you know if invocations are being throttled or erroring out due to insufficient ENI capacity.</li></ol><p>🔢 If you have high-volume Lambda functions running in your AWS account, you should try out the <a href="https://winterwindsoftware.com/lambda-scaling-calculator/"><strong>Lambda Scaling Calculator</strong></a> to get an interactive view on how your different function profiles will scale under load and when you will reach capacity.</p><p>💌 <strong><em>If you enjoyed this article, you can sign up </em></strong><a href="https://winterwindsoftware.com/newsletter/"><strong><em>to my weekly newsletter on building serverless apps in AWS</em></strong></a><strong><em>.</em></strong></p><p><em>Originally published at </em><a href="https://winterwindsoftware.com/scaling-lambdas-inside-vpc/"><em>winterwindsoftware.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6320ce18e02" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[5 common mistakes deploying Lambda functions]]></title>
            <link>https://medium.com/@paulswail/5-common-mistakes-deploying-lambda-functions-87bf30b0a01a?source=rss-3a509aed10a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/87bf30b0a01a</guid>
            <category><![CDATA[functions-as-a-service]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <dc:creator><![CDATA[Paul Swail]]></dc:creator>
            <pubDate>Wed, 29 Aug 2018 15:12:36 GMT</pubDate>
            <atom:updated>2018-08-29T15:12:36.658Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’re new to building Functions-as-a-Service applications and to AWS Lambda in particular, there are several snags you will likely hit when you’re deploying your first few functions. The list below will hopefully ensure you don’t make the same mistakes as I did.</p><h3>1. Not using a deployment toolkit</h3><p>Manually uploading code packages then configuring them in the AWS Console is both slow and error-prone. If you use a deployment toolkit such as the <a href="https://serverless.com/">Serverless framework</a>, this automates all the packaging and deployment as well as allowing your Lambda config to be version controlled and related functions to be grouped into the same deployment package.</p><h3>2. Setting and forgetting memory allocation</h3><p>You’re billed on how much memory is allocated to each function invocation so you don’t want to allocate significantly more memory than is needed to complete the task. However, this is not the whole story. As memory allocation increases <a href="https://docs.aws.amazon.com/lambda/latest/dg/resource-model.html">so does the amount of CPU power that AWS allocates to your function</a>. This means that by allocating more memory your function could complete quicker (especially if it is CPU bound), thus cutting your execution time charges.</p><p>This will involve some trial and error to get the sweet spot and you should edge on the side of over-allocating to start off with (say around 512MB). Once your function is deployed, your Lambda invocations are automatically logged to CloudWatch where you can see a line similar to the following which you can use to compare the Max Memory Used to the Memory Size (allocated):</p><pre>REPORT RequestId: be2fc81e-8345-11e8-b334-cb4ce1d5d651	Duration: 30914.88 ms Billed Duration: 31000 ms Memory Size: 256 MB	Max Memory Used: 111 MB</pre><p>This particular example shows a function which uses less than half its allocated memory. If you know that this function is IO bound (e.g. it hands off an event to another API), then this would be a good candidate for decreasing the allocation as it probably doesn’t need all that’s allocated to it. The <a href="https://serverless.com/framework/docs/providers/aws/guide/serverless.yml/">Serverless framework</a> defaults to allocating 1024MB to a function which may be too much if your function is only doing some simple pass-through network calls.</p><p>However, if your function is doing something more computationally expensive, then you should try out increasing the memory allocation to see if it brings down the total overall cost. See <a href="https://www.jeremydaly.com/15-key-takeaways-from-the-serverless-talk-at-aws-startup-day/">point 1 in this article</a> for an example of a Lambda performance test showing the benefits of a larger memory allocation.</p><p>AWS does allow you to pass secrets (passwords, API keys, database connection strings, etc) as environment variables which are <a href="https://docs.aws.amazon.com/lambda/latest/dg/env_variables.html">stored encrypted at rest using AWS KMS</a> when you are deploying your Lambda function. However, at deployment time you still need to tell AWS where to fetch the values for these environment variables from. You obviously do not want to hardcode them into your deployment scripts.</p><p>A recommended approach is to use the <a href="https://aws.amazon.com/blogs/compute/sharing-secrets-with-aws-lambda-using-aws-systems-manager-parameter-store/">Systems Manager Parameter Store</a> to store your secrets. Deployment toolkits such as the Serverless framework <a href="https://serverless.com/blog/serverless-secrets-api-keys#handling-secrets-for-small-teams--projects">allows you to map a Parameter Store key to an environment variable key</a> which it will read from at deploy-time and store encrypted in the Lambda’s configuration.</p><h3>4. Timeouts in Node.js functions</h3><p>If you’re implementing your Lambdas using Node.js, there is the possibility that your functions may timeout, even after seeming to have completed their processing.</p><p>The reason for this is the JavaScript event loop may not have cleared (e.g. if there is an unhandled callback in a downstream function). When this happens you will end up paying for the full period of time until the function times out (the default timeout is 3 seconds) rather than just the time doing the actual processing.</p><p>To fix this, you should always set the following as the first line of code in your handler function:</p><pre>context.callbackWaitsForEmptyEventLoop = false;</pre><h3>5. Not setting up budget alerts</h3><p>This mistake isn’t specific to Lambda, but if you’re new to developing on AWS, the last thing you want to do is to rack up an unexpected bill. Since Lambda is so easy to get up-and-running with (especially when using a deployment toolkit), you could accidentally trigger your function to run on a regular schedule and be unaware of it until the end of the month bill comes in.</p><p>To avoid this, <a href="https://aws.amazon.com/premiumsupport/knowledge-center/create-monthly-budget/">create a budget alert</a> so you will be emailed whenever a threshold is hit.</p><p>💌 <strong><em>If you enjoyed this article, you can sign up </em></strong><a href="https://winterwindsoftware.com/newsletter/"><strong><em>to my weekly newsletter on building serverless apps in AWS</em></strong></a><strong><em>.</em></strong></p><p><em>Originally published at </em><a href="https://winterwindsoftware.com/common-mistakes-deploying-lambda/"><em>winterwindsoftware.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=87bf30b0a01a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Deploy a High Availability Web App to AWS ECS]]></title>
            <link>https://medium.com/@paulswail/how-to-deploy-a-high-availability-web-app-to-aws-ecs-702a3520dc69?source=rss-3a509aed10a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/702a3520dc69</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-ecs]]></category>
            <category><![CDATA[cloudformation]]></category>
            <dc:creator><![CDATA[Paul Swail]]></dc:creator>
            <pubDate>Wed, 25 Apr 2018 12:43:53 GMT</pubDate>
            <atom:updated>2018-04-25T13:12:10.274Z</atom:updated>
            <content:encoded><![CDATA[<p>Running your containerised web app locally is easy. Just docker run and you’re done.</p><p>But running it in a production-style environment using AWS infrastructure is much harder. If you’re not an ops expert, the decisions can be overwhelming.</p><p>You need to consider:</p><ul><li>What do I need to do to ensure that the <strong>servers are securely locked down</strong>?</li><li>How do I make my app <strong>always available</strong> in the event of a container or entire server instance going down?</li><li>How will developers on my team be able to <strong>deploy new versions</strong> of the app without breaking everything or causing downtime? Will I need to get them all set up with SSH keys?</li><li>How will my team <strong>access the logs</strong> to debug app issues?</li><li>What AWS services should I be using to help me with all the above?</li></ul><p>To answer these questions, you’re worried you’ll need to spend a week or more buried in the depths of the vast AWS documentation vortex researching how to use their different services, with lots of trial and error before finally arriving at a working environment which your test users can access.</p><p>And after completing this, you realise you now need to spend another day documenting all the steps you took so that your colleagues can rebuild the environment in the future (because most AWS documentation tells you where to point and click in the AWS Console — pretty useless for creating a repeatable or automatable build process).</p><p>This is a significant upfront time investment — probably time you don’t have if you’re under pressure to get the app in front of users.</p><p>So you make compromises.</p><p>You stick to what you know and just manually create a couple of EC2 instances using the AWS Console. You SSH and SCP over your built Docker image and invoke docker run on each one. Most steps don’t get documented, and live on only in your head. Your team can always work backwards from the configuration in the Console if they need to rebuild the environment. And they can always SSH in to the instances to do deployments and check logs. Not ideal, but it will do for now.</p><p>But you don’t need to take on all this advance technical debt.</p><p>You can instead get a production-ready app up and working in less than an hour using the AWS Elastic Container Service (ECS). By the end of this guide, you will have:</p><ul><li>Highly available, load-balanced requests to your container instances (via Application Load Balancer)</li><li>A durable and secure set of server instances (via an Auto Scaling Group) running an OS optimised for hosting containers</li><li>Centralised, searchable app logs (via CloudWatch)</li><li>Versioned Docker images of your app hosted in AWS Elastic Container Repository (ECR)</li><li>A fast, reliable and SSH-free application release process</li><li>Repeatable and version controlled environment configuration process (via a CloudFormation stack template)</li></ul><p>This process and architecture is based on the tried and tested approach I have been using to host the API and management portal for <a href="https://autochart.io/">my SaaS business</a> on AWS for the past 3 years.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-up_J37kmFHoxAOZ.png" /><figcaption>ECS Web App Architecture</figcaption></figure><h3>Core ECS Concepts</h3><p>Before getting started with building our environment, let’s quickly cover a few basic concepts and terms which we’ll be referencing.</p><p>Firstly, ECS is the service which AWS provides to orchestrate containers running on top of EC2 instances. It works by having an agent process run on each host instance which the central ECS controller service communicates with, instructing the agent to spawn or kill specific containers as well as performing health checks on running containers.</p><p>ECS provides the following resources which we will be using:</p><ul><li>A <strong>Cluster</strong> is a logical grouping of EC2 instances that are made available for application containers to be run upon. ECS will run its agent process on each instance in the cluster.</li><li>A <strong>Task Definition</strong> is where you configure how a container will run. In particular, you will specify the Docker image which will be used, port mappings, its memory/CPU requirements and any environment variables which should be set in the container at runtime.</li><li>A <strong>Task</strong> is what ECS calls a running Docker container that was created using the configuration provided in a Task Definition.</li><li>A <strong>Service</strong> is where you define the orchestration requirements for a particular task definition. The main item here is defining the desired number of tasks for a task definition which should be running at any time. It is possible to configure different task definitions within a single service, but for our web app we will just have a 1:1 mapping from service to task definition. Within the service, you will also tell ECS what load balancer to use for routing requests to it.</li></ul><p>Check out <a href="https://medium.com/@hossein.kazemi/aws-ecs-in-simple-words-9e083e3ec2ee">this article</a> for a more in-depth look at ECS definitions.</p><h3>Building your Environment</h3><p>Ok, time to start creating resources…</p><h3>Step 0: Pre-requisites</h3><p>The rest of the guide assumes that:</p><ul><li>you already have an AWS account</li><li>it has a VPC and at least 2 public subnets configured (preferably in different availability zones)</li><li>you have created an <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html">EC2 Key-Pair</a> which would be used to SSH into EC2 instances from your computer</li><li>you have the <a href="https://aws.amazon.com/cli/">AWS CLI</a> installed on your computer</li></ul><h3>Step 1: Prep your app source code</h3><p>If you would prefer to use a ready-made basic Node.js web app to work through the remaining steps, you can pull <a href="https://github.com/WinterWindSoftware/simple-express-app">this repo</a> to your machine and go to Step 2.</p><p>If you have already Dockerised your app and are now ready to deploy it, then just copy <a href="https://github.com/WinterWindSoftware/simple-express-app/blob/master/docker-task.sh">this shell file</a> into the root folder of your application source code. It contains a few helper scripts we will be using to build and deploy the Docker image.</p><h3>Step 2: Configure Docker Variables</h3><p>Open the docker-task.sh file and update the following variables:</p><ul><li>IMAGE_NAME: this must be set to the name of the image you wish to create (e.g. winterwindsoftware/simple-express-app)</li><li>AWS_REGION: name of the region where you’re deploying to (e.g. us-east-1)</li><li>CONTAINER_PORT: port number that is exposed from your container image (if using sample app, this will be 8080)</li></ul><p>Save the file.</p><h3>Step 3: Create ECR Repository</h3><pre>./docker-task.sh createrepo</pre><p>This will create a new repository named the same as the IMAGE_NAME variable.</p><p>Note down the value of the repositoryUri JSON field printed in the response from AWS in your terminal. Open the docker-task.sh file and update the REPOSITORY_PATH with the first part of this value (right up to the “amazonaws.com” bit). It should look something like &lt;aws_account_id&gt;.dkr.ecr.&lt;aws_region&gt;.amazonaws.com.</p><p>You can then open the <a href="https://console.aws.amazon.com/ecs/home#/repositories">ECR Console</a> to verify that the repo was created ok.</p><h3>Step 4: Build Docker image locally</h3><pre>./docker-task.sh build</pre><h3>Step 5: Deploy image to ECR</h3><p>To upload the image to ECR, run:</p><pre>./docker-task.sh push</pre><p>If you refresh the AWS console once this completes, you should see the new image listed there.</p><h3>Step 6: Create CloudFormation Stack</h3><p>We will use a CloudFormation stack to launch all the requisite AWS resources. Copy over <a href="https://github.com/WinterWindSoftware/simple-express-app/blob/master/cloudformation/ecs-webapp-stack.yml">this stack file</a> into a new /cloudformation folder in your app’s codebase.</p><p>Then open your terminal and run the CLI command below to create your stack. Before running it, you will need to update all values surrounded by angle brackets, e.g. (&lt;vpc_id&gt;).</p><pre>aws cloudformation deploy \ <br>  --stack-name SimpleExpressApp \ <br>  --template-file ./cloudformation/ecs-webapp-stack.yml \<br>  --capabilities CAPABILITY_IAM \<br>  --parameter-overrides KeyName=&#39;&lt;keypair_id&gt;&#39; \ <br>    VpcId=&#39;&lt;vpc_id&gt;&#39; \<br>    SubnetId=&#39;&lt;subnet_id_1&gt;,&lt;subnet_id_2&gt;&#39; \<br>    ContainerPort=8080 \<br>    DesiredCapacity=2 \<br>    EcsImageUri=&#39;&lt;ecr_image_uri&gt;&#39; \<br>    EcsImageVersion=&#39;&lt;app_version&gt;&#39; \<br>    InstanceType=t2.micro \<br>    MaxSize=3</pre><p>⚠️ <em>Be aware that this will create some paid-for hourly resources, including 2 micro EC2 instances and an Application Load Balancer. If you’re just creating this stack as a test, you can delete your stack using the </em><a href="https://console.aws.amazon.com/cloudformation/home"><em>CloudFormation console</em></a><em> once you’re finished to remove all resources.</em></p><h3>Step 7: Test in browser</h3><p>The stack will take around 5 minutes to create. Once it completes, go to the <a href="https://console.aws.amazon.com/ec2/home#LoadBalancers">Load Balancers console</a> and select the ALB which was created. Copy the value of the DNS name setting into your browser address bar. It should look something like ecsalb-12345678.us-east-1.elb.amazonaws.com. This should open the app home page.</p><p>You can also view the logs from your app by going to the <a href="https://console.aws.amazon.com/cloudwatch/home#logs:">CloudWatch Logs Console</a>.</p><h3>How do I deploy new versions of my app?</h3><p><strong>Push new image version</strong></p><p>To deploy an updated version of the app, first make a small change to your codebase. If you’re using the sample Node.js app, you can simply run npm version minor to bump the minor version number as it’s dynamically read and rendered on the home page.</p><p>Then run the following to build and push the new version of the image to ECR:</p><pre>./docker-task.sh build &amp;&amp; ./docker-task.sh push</pre><p><strong>Update running containers to use new version of the image</strong></p><p>Note down the new version number. Then run the same CLI command as in Step 6, except with the EciImageVersion parameter value updated (this example has a new version number of 1.1.0):</p><pre>aws cloudformation deploy \ <br>  --stack-name SimpleExpressApp \ <br>  --template-file ./cloudformation/ecs-webapp-stack.yml \ <br>  --capabilities CAPABILITY_IAM \ <br>  --parameter-overrides KeyName=&#39;&lt;keypair_id&gt;&#39; \ <br>    VpcId=&#39;&lt;vpc_id&gt;&#39; \ <br>    SubnetId=&#39;&lt;subnet_id_1&gt;,&lt;subnet_id_2&gt;&#39; \<br>    ContainerPort=8080 \<br>    DesiredCapacity=2 \<br>    EcsImageUri=&#39;&lt;ecr_image_uri&gt;&#39; \<br>    EcsImageVersion=&#39;1.1.0&#39; \<br>    InstanceType=t2.micro \<br>    MaxSize=3</pre><p>Once the stack update completes, refresh your browser to see the new version of the app. That’s it! No SSH-ing into servers or copying files across the network.</p><p><strong>Rolling back</strong></p><p>You’ve done your post-release smoke test and spot a newly introduced high priority bug. 😰 It will take a while to investigate/fix so you decide to rollback to the previous version in the meantime. No problem! Simply re-run the CloudFormation deploy command specifying the previous version number. Sweat over 😅.</p><h3>Extending your stack</h3><p>So you now have a running, high availability containerised web app.</p><p>If you want to take this further and extend the stack for your own requirements, you can do so by editing the <a href="https://github.com/WinterWindSoftware/simple-express-app/blob/master/cloudformation/ecs-webapp-stack.yml">ecs-webapp-stack.yml</a> CloudFormation template.</p><p>Here are a few suggestions for things to try out:</p><ul><li>Add an SSL certificate to your Application Load Balancer to secure requests to your web app</li><li>Add a Route53 DNS entry to give your app a friendly domain name</li><li>Create a separate staging environment using this same stack (hint: just rename the stack in your CloudFormation CLI deploy command)</li></ul><p><em>Originally published at </em><a href="https://winterwindsoftware.com/deploy-high-availability-web-app-to-aws-ecs/"><em>winterwindsoftware.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=702a3520dc69" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Books I read in 2017]]></title>
            <link>https://medium.com/@paulswail/books-i-read-in-2017-d0793f42a4a?source=rss-3a509aed10a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/d0793f42a4a</guid>
            <category><![CDATA[books]]></category>
            <category><![CDATA[reading]]></category>
            <dc:creator><![CDATA[Paul Swail]]></dc:creator>
            <pubDate>Tue, 23 Jan 2018 08:11:02 GMT</pubDate>
            <atom:updated>2018-01-23T08:11:02.359Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3BFG8PiDB1DiL3wB62N22g.jpeg" /></figure><ul><li>🎧 <a href="https://www.audible.co.uk/pd/The-E-Myth-Revisited-Why-Most-Small-Businesses-Dont-Work-and-What-to-Do-About-It-Audiobook/B00FZURU8A">The E-Myth Revisited: Why Most Small Businesses Don’t Work and What to Do About It</a> by Michael E Gerber. Great book for learning how to systematise your small business. My main takeaway was to spend more time working <em>on</em> my business rather than <em>in</em> it, in particular to start planning on how it’s overall success can be less dependent on my time. [<strong>8/10</strong>]</li><li>🎧 <a href="https://www.audible.co.uk/pd/The-Power-of-Habit-Part-1-Why-We-Do-What-We-Do-and-How-to-Change-Audiobook/B007AK4V62">The Power of Habit</a> by Charles Duhigg. A book which had been recommended by several folks I respect, I read the Kindle version a few years ago and listened to the audiobook in 2017. It describes the Cue-Routine-Reward cycle that habits follow and helped me to get a more objective outlook on my own habits as well as generating ideas on how to implement new habits I would like to adopt. [<strong>9/10</strong>]</li><li>📖 <a href="https://www.amazon.co.uk/dp/B00I0C46BO/">The Life Changing Magic of Tidying Up</a> by Marie Kondo. Short book about decluttering your home. A departure from my usual genres but I particularly enjoyed this. It helped me be more aware about items around my house, particularly useful after having just moved into a new home. My main takeaway was to make throwing out an item my default action unless I could really justify keeping it. [<strong>8/10</strong>]</li><li>🎧 <a href="https://www.audible.co.uk/pd/Deep-Work-Rules-for-Focused-Success-in-a-Distracted-World-Audiobook/B01D0E32T8">Deep Work</a> by Cal Newport. I listened to this book twice last year. A highly practical guide on how to cut out all the distractions that the modern world brings in order to carve out time and circumstance to do your best work and to switch off effectively when the work day is over. [<strong>9/10</strong>]</li><li>🎧 <a href="https://www.audible.co.uk/pd/Sapiens-Part-1-Audiobook/B00VY24CWU">Sapiens: A Brief History of Humankind</a> by Yuval Noah Hariri. Walks through the history of humans, from caveman through ancient civilisations to the modern day, touching on all the main philosophical concepts of war, money, politics, nationality, religion and technology. Hariri has a particular style of making his opinion sound like fact (e.g. on existence or not of deities) but once I got used to it, it challenged me to think, even if I didn’t always agree with him. [<strong>9/10</strong>]</li><li>🎧 <a href="https://www.audible.co.uk/pd/Homo-Deus-Part-1-A-Brief-History-of-Tomorrow-Audiobook/B01HGY2730">Homo Deus: A Brief History of Tomorrow</a> by Yuval Noah Hariri. A follow up to Sapiens, this book touches on the same philosophical concepts as the first but rather looking into the future and making prediction on how things might look. As you might expect, technology is a major theme throughout. [<strong>9/10</strong>]</li><li>🎧 <a href="https://www.audible.co.uk/pd/A-Short-History-of-Nearly-Everything-Part-1-Audiobook/B004FTIFOK">A Short History of Nearly Everything</a> by Bill Bryson. A review of the history of people and the universe in language an everyday person can understand. Somewhat similar to Sapiens in subject matter, the main differences are that it’s not just focussed on humans and also covers origins of the universe and physics concepts. It’s also more factual and less opinionated in tone. I mainly enjoyed it, though I found some of the historical episodes slightly boring. [<strong>7.5/10</strong>]</li><li>🎧 <a href="https://www.audible.co.uk/pd/Astrophysics-for-People-in-a-Hurry-Audiobook/B06X9GZ62F">Astrophysics for People in a Hurry</a> by Neil deGrasse Tyson. “A Short History of Nearly Everything” rekindled my childhood interest in astrophysics and influenced this purchase. This book is quite short but is very fast-paced (hence “people in a hurry”, duh!). I found it quite hard going and it needed my full attention in order to not get lost. [<strong>6/10</strong>]</li><li>🎧 <a href="https://www.audible.co.uk/pd/Money-Part-1-Know-More-Make-More-Give-More-Audiobook/B071X9ZPGR">Money: Know More, Make More, Give More</a> by Rob Moore. The first few chapters contain some useful info around understanding and making money. He cuts through taboos around talking about money and wealth which I liked. However, I found the author (who also narrates it) hard to empathise with. He’s upfront about being somewhat of a rich stereotype (only buying designer clothes and a super car addict) which is fine, but the bravado did get a bit grating. Covers a lot on property investment (which I wasn’t particularly interested in) and got very repetitive in later chapters. [<strong>6/10</strong>]</li><li>🎧 <a href="https://www.audible.co.uk/pd/The-4-Hour-Work-Week-Part-1-Audiobook/B005QQWGPA">The 4 Hour Work Week</a> by Tim Ferriss. A book I’ve heard a lot about and referred to as somewhat of a bible for entrepreneurs. I enjoyed the content around systematising and automating your business (e.g. through using virtual assistants) but found the content around travelling the world while running a business less applicable to my situation. Wish I’d read this book 10 years ago. Later chapters are somewhat dated as they go into detail on specific online services you can use to accomplish the 4hr week goal, many of which are obsolete now. [<strong>7/10</strong>]</li><li>🎧 <a href="https://www.audible.co.uk/pd/Black-Box-Thinking-Part-1-The-Surprising-Truth-About-Success-Audiobook/B00WKKAGZ2">Black Box Thinking</a> by Matthew Syed. A book about how people and organisations should change their attitude to failure, and embrace it rather than punish it. I found the title a bit confusing as the computer scientist in me seen “Black Box” as a reference to putting something in and getting something out without knowing what happens in between. It is in fact a reference to an aircraft’s black box and how it’s used to diagnose why a failure happened. It’s well known that failure is a great learning tool but yet still so few people or organisations embrace this concept today. [<strong>8/10</strong>]</li><li>📖 <a href="https://www.amazon.co.uk/Online-Business-Lifestyle-Successful-Internet-ebook/dp/B0761WBVSN">The Lifestyle Startup</a> by Kyle Gawley. Written by a young Belfast entrepreneur, this book provides a lot of good advice on how to start your own 1-person business. Most of the suggestions are things I’ve read before but I would definitely recommend this to any software developers or designers considering the move to self-employment, in particular if you want to take the bootstrapped route. [<strong>8/10</strong>]</li><li>🎧 <a href="https://www.audible.co.uk/pd/Business/Principles-Audiobook/B074B83BMZ">Principles (Life &amp; Work)</a> by Ray Dalio. Sets out the principles of a billionaire Wall Street investor. I really liked the concept of documenting a set of principles to live and lead by. The first half of the book is about Dalio’s personal principles which I found very interesting, although sometimes too specific to financial investing to be applicable to my own situation. The second half of the book focusses on big organisational principles and large company culture which I wasn’t particularly interested in but if you’re interested in growing your own company to a large enterprise, you would like this. [<strong>7/10</strong>]</li><li>📖 <a href="https://www.amazon.co.uk/gp/product/B002RI9KAY/ref=kinw_myk_ro_title">The Automatic Millionaire: A Powerful One-step Plan to Live and Finish Rich</a> by David Bach. Short book on personal finance. I went on a bit of an economy drive last year to cut out wasted spending and general inefficient use of money and found some useful principles and tactics in this book. My single most memorable piece of advice was to pay yourself first, i.e. as soon as you get paid then immediately put away a fixed amount into your pension and savings and don’t just save what’s rest at the end of the month. Simple but effective. [<strong>8/10</strong>]</li><li>🎧 <a href="https://www.audible.co.uk/pd/Sum-Tales-from-the-Afterlives-Audiobook/B004EW24Q8">Sum: Tales from the Afterlives</a> by David Eagleman. A very thought-provoking and uplifting series of short stories around the concept of an afterlife. Each story is about 5 minutes long and is narrated by a different actor. They don’t align with any particular religious viewpoint on the afterlife so if you’re Christian, Buddhist or Atheist you will still get something from this. Rather than making me wonder about an afterlife, I found each story made me reflect on my life in the here and now. [<strong>9.5/10</strong>]</li><li>📖 <a href="https://www.goodreads.com/book/show/31578285-extracted">Extracted: Extracted Trilogy №1</a> by R R Haywood. [<strong>7/10</strong>]</li><li>📖 <a href="https://www.goodreads.com/book/show/22915453-from-the-cradle">From the Cradle</a> by Louise Voss and Mark Edwards. [<strong>8/10</strong>]</li><li>📖 <a href="https://www.goodreads.com/book/show/34219007-the-sixth-window">The Sixth Window</a> by Rachel Abbott. [<strong>9/10</strong>]</li><li>📖 <a href="https://www.goodreads.com/book/show/29481890-the-devil-s-work">The Devil’s Work</a> by Mark Edwards. [<strong>8/10</strong>]</li><li>📖 <a href="https://www.goodreads.com/book/show/18913806-the-magpies">The Magpies</a> by Mark Edwards. [<strong>8/10</strong>]</li><li>📖 <a href="https://www.goodreads.com/book/show/25153134-follow-you-home">Follow You Home</a> by Mark Edwards. [<strong>9/10</strong>]</li><li>📖 <a href="https://www.goodreads.com/book/show/33833390-the-lucky-ones">The Lucky Ones</a> by Mark Edwards. [<strong>9/10</strong>]</li><li>📖 <a href="https://www.goodreads.com/book/show/30518988-dark-matter">Dark Matter</a> by Crouch Blake. [<strong>9.5/10</strong>]</li><li>📖 <a href="https://www.amazon.co.uk/gp/product/B01NAHCIRB/ref=kinw_myk_ro_title">For Your Own Protection</a> by Paul Pilkington. [<strong>8/10</strong>]</li></ul><p>I generally consume non-fiction in the form of audiobooks (🎧) on <a href="https://www.audible.co.uk/">Audible</a>. This allows me to listen to it when travelling or doing housework. Another benefit is that I find Audible’s subscription of 1 credit/month a good frequency. A minor drawback is that when listening I sometimes lose focus and don’t take in as much as I would when reading it.</p><p>All my fiction reading tends to be on my Kindle Paperwhite (📖). These are almost always read in bed as I avoid most non-fiction books late at night due to their tendency to initiate reflective thought processes which don’t make for a good night’s sleep.</p><p>General themes from last year’s reading material:</p><ul><li>Human history</li><li>Habits and focus</li><li>Personal finance</li><li>Crime mystery</li></ul><p>I noticed that I didn’t finish any technology books last year (though I did start a few and not finish them). This wasn’t a deliberate decision but I guess is symptomatic of my interests right now. Whilst I do still love tech and it’s an integral part of my business, I don’t see it as a major differentiator for me or my business going forward. That said, there are a few hyped tech topics (e.g. AI and Blockchain) which I do plan to read up more on this coming year.</p><p>Another observation is that the vast majority of the authors (particularly in non-fiction) are male.</p><h3>Goals for Reading in 2018</h3><ul><li>Machine Learning</li><li>Cryptocurrency &amp; Blockchain (hop aboard the bandwagons!)</li><li>More female non-fiction authors</li><li>Make time for more focussed reading and note taking on certain non-fiction books</li><li>Publish a few in-depth book reviews on this blog</li></ul><p>Any good recommendations on the above topics would be much appreciated in the comments below.</p><p><em>Originally published at </em><a href="https://winterwindsoftware.com/articles/books-read-in-2017/"><em>winterwindsoftware.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d0793f42a4a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Services and tools for running a small software business]]></title>
            <link>https://medium.com/@paulswail/services-and-tools-for-running-a-small-software-business-34b0f96b9430?source=rss-3a509aed10a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/34b0f96b9430</guid>
            <category><![CDATA[saas]]></category>
            <category><![CDATA[apps]]></category>
            <category><![CDATA[solopreneur]]></category>
            <category><![CDATA[productivity]]></category>
            <dc:creator><![CDATA[Paul Swail]]></dc:creator>
            <pubDate>Fri, 12 Jan 2018 14:19:59 GMT</pubDate>
            <atom:updated>2018-01-12T14:19:59.157Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rqyhAkYMGWExSDUlDKQChQ.jpeg" /></figure><p>I’ve been running my own business building a <a href="https://autochart.io/">SaaS product</a> and doing consulting/contracting projects for over 4 years now. During this time I’ve used hundreds of tools and services to help get things done across the different areas of my business.</p><p>The list below is everything I use right now (as of January 2018). I will make an effort to update this over time. All services which I pay for are marked with a 💰.</p><h3>General Business Operations</h3><ul><li><a href="https://1password.com/">1Password</a> 💰 — Securely stores all passwords and secrets for business services. Previously used <a href="https://www.lastpass.com/">LastPass</a>.</li><li><a href="https://www.boomeranggmail.com/">Boomerang for Gmail</a> — For scheduling emails to be sent in the future. Main use case is writing the occasional late night email and scheduling to send first thing in morning so that you don’t set the expectation with a consulting client that you check email outside of work hours.</li><li><a href="https://www.dropbox.com/">Dropbox</a> 💰 — File storage. I’ve mostly migrated business files to Google Drive but maintain this more for personal use (e.g. photo backup).</li><li><a href="https://evernote.com/">Evernote</a> — For capturing interesting articles (using webclip) and quickly scanning paper documents which I don’t want to have to file.</li><li><a href="https://letsfreckle.com/">Freckle</a> 💰 — Tracks my time on a per-project basis. Mainly used for consulting engagements to ensure fixed price projects are profitable with respect to my time. I have also used it for tracking time spent on recurring Autochart product maintenance activities to identify tasks which should be more automated.</li><li><a href="https://gocardless.com/">GoCardless</a> 💰 — Default payment processor for new Autochart subscriptions. Automatically takes monthly subscription fees as a direct debit with very low fees compared to credit card processors. I’m currently trying to get more customers moved over to Direct Debit from invoice-based payments to alleviate the pain that is monthly invoice chasing. This is proving tough for larger enterprise customers with dedicated credit control departments.</li><li><a href="https://gsuite.google.com/">GSuite</a> 💰 — Gmail (2 custom domains), Calendar, Google Docs/Sheets, Google Drive (bulk of business file storage), Hangouts for video demos.</li><li><a href="https://www.office.com/">Microsoft Office</a> 💰 — Mainly just for Excel. I’ve managed to avoid needing to use Word and Powerpoint for over a year now.</li><li><a href="https://www.nuclino.com/">Nuclino</a> — Internal wiki/knowledge base for documenting standard business processes and procedures.</li><li><a href="http://gingerlabs.com/">Notability</a> 💰 — For regular note-taking and casual diagramming. Great for using Apple Pencil on iPad and syncs to Mac. Sadly no Android version but also syncs to Google Drive in PDF format, so I can still read notes on my phone.</li><li><a href="https://slack.com/">Slack</a> — For team communication mainly when I’m working on a custom consulting engagement and also for chat with fellow co-working space residents.</li><li><a href="https://stripe.com/">Stripe</a> 💰 — Payments processor. Integrated into Xero for customers who prefer to pay invoice by credit card.</li><li><a href="https://sweep.cards/">Sweep</a> 💰 — My new primary day-to-day task management tool after moving from Trello. Still in beta but the recurring task support is where it shines as SaaS products require a lot of recurring love. Great support team behind it.</li><li><a href="https://trello.com/">Trello</a> — Previously used for task management, now primarily used for marketing content ideas and engineering kanban boards for custom dev engagements.</li><li><a href="https://xero.com/">Xero</a> 💰 — For book-keeping, client invoicing and HMRC returns. Automatic bank feed for current account combined with reconciliation rules is great for automated book-keeping, though the feed isn’t yet available for credit card accounts where most of my spend is. The automatic HMRC integration for monthly salary payroll and quarterly VAT returns means I only need my accountant for annual corporation and self-assessment tax returns.</li><li><a href="https://www.youneedabudget.com/">YNAB</a> 💰 — I’ve used this for personal budgeting for almost a year and have recently started using it for business cashflow predictions since Xero’s budgeting feature is pretty poor.</li><li><a href="https://zapier.com/">Zapier</a> 💰 — For automating workflow between online services I use. I should probably use this a lot more, currently my main use is for automating transfer of receipts from Gmail and Google Drive into Xero.</li></ul><h3>Sales, Marketing and Customer Support</h3><ul><li><a href="https://buffer.com/">Buffer</a> — For scheduling tweets and LinkedIn posts. Social media isn’t a major channel for Autochart sales, so I use this very infrequently.</li><li><a href="https://www.drip.com/">Drip</a> 💰 — For email marketing and workflow automation. Current main use is an onboarding email sequence for new customers though I’m hoping to use it more for pre-customer educational marketing. Previously used Mailchimp, but Drip’s model of subscribers over lists makes more sense, and the workflow automation is very powerful.</li><li><a href="https://analytics.google.com/">Google Analytics</a> — For tracking traffic and high-level usage of the Autochart marketing website.</li><li><a href="https://www.helpscout.net/">HelpScout</a> — Great service which I use for 2 purposes: support ticket management (for all emails to Autochart Support) and as a CMS for the <a href="http://support.autochart.io/">Autochart Support knowledge base</a>.</li><li><a href="https://mixpanel.com/">Mixpanel</a> — Tracking usage of the Autochart portal, specifically for identifying feature usage within the app.</li><li><a href="https://www.pipedrive.com/">Pipedrive</a> 💰 — Recently started using this to help manage sales pipeline and be more structured in how sales are progressed. It helps me follow-up with prospects who’ve shown interest in Autochart or a custom consulting engagement, as well as keeping in touch with existing or past clients. Its automatic email synchronisation keeps all conversations for a specific opportunity in one place (although this sync is slightly buggy).</li><li><a href="https://www.screencastify.com/">Screencastify</a> 💰 — Chrome extension for recording screencasts. I bought paid version so can record longer sessions and convert to MP4.</li></ul><h3>Engineering and Design</h3><ul><li><a href="https://aws.amazon.com/">Amazon Web Services</a> 💰 — Main infrastructure for the Autochart application, marketing site and other company web properties. Individual AWS services are listed in the next section.</li><li><a href="https://www.draw.io/">Draw.io</a> — Drawing structured diagrams.</li><li><a href="https://github.com/">GitHub</a> 💰 — Used for private source code hosting, pull request workflows and product issue tracking.</li><li><a href="https://www.iterm2.com/">iTerm 2</a> — Better terminal for MacOSX (although somewhat replaced by integrated terminal within VS Code).</li><li><a href="https://jekyllrb.com/">Jekyll</a> — Simple CMS for generating static websites. It’s used for all company marketing websites.</li><li><a href="https://keen.io/">Keen.io</a> 💰 — An “Analytics as a Service” provider, it’s used to produce metrics and graphs on how Autochart customer websites are performing.</li><li><a href="https://mlab.com/">mLab</a> 💰 — Managed hosting for MongoDB (Autochart’s main datastore), running inside an AWS data center. They take care of backups, replica sets, patching servers, etc. Provide good monitoring tools in their web portal for assessing performance issues.</li><li><a href="https://namecheap.com/">Namecheap</a> 💰 — Purchasing domain names.</li><li><a href="https://www.parallels.com/">Parallels</a> 💰 — For running Windows VM on Mac. Mainly used for browser testing in IE.</li><li><a href="https://postmarkapp.com/">Postmark</a> 💰 — Transactional email provider. A key feature of Autochart is sending out returning lead emails to customer sales reps (1000s sent out daily). Previously used Mandrill and then Sendgrid but recently switched to Postmark due to its better .deliverability and debugging options.</li><li><a href="https://www.purevpn.com/">PureVPN</a> 💰 — For secure internet access when connected to public wifi.</li><li><a href="https://robomongo.org/">Robomongo</a> — Desktop client for accessing MongoDB.</li><li><a href="https://serverless.com/">Serverless</a> — Used to simplify configuration and deployment of Lambda/API Gateway and other related resources. Uses Cloudformation under the hood.</li><li><a href="https://uptimerobot.com/">Uptime Robot</a> — Get notified if any key websites go down. Used as a secondary backup to Cloudwatch alarms.</li><li><a href="https://code.visualstudio.com/">Visual Studio Code</a> — Main code editor since about 6 months ago. Great for JavaScript development (Autochart backend is NodeJS). Switched from <a href="https://www.sublimetext.com/">SublimeText 3</a> which is also great, but lost out due to Code’s intellisense, integrated terminal and debugging support.</li></ul><h4>AWS Services</h4><ul><li>EC2 + ECS + ELB — Hosting docker containers where Autochart portal web app runs.</li><li>S3 + Cloudfront — hosting of static assets for portal app, client-side tracking library and static marketing websites.</li><li>Cloudwatch — Infrastructure monitoring dashboards, alarms and logs.</li><li>Cloudformation — Infrastructure configuration management. Have now moved most resources to Cloudformation config as many were initially created using the AWS web console (for shame!).</li><li>Elasticache (Redis) — Used for caching some expensively computed data used by the portal app.</li><li>Route 53 — DNS settings for all company domains.</li><li>API Gateway — HTTP endpoint where the Autochart tracking library running on customer websites sends its event data to.</li><li>Lambda — Handles API Gateway events. Also used to run background CRON jobs which previously lived inside their own containers on ECS.</li><li>Kinesis — For persisting the incoming stream of website event data sent to API Gateway.</li><li>SNS — Notifications for Cloudwatch alarms and also as a dead letter queue for failed Lambda functions.</li><li>RDS Aurora — SQL database used as a secondary datastore, mainly for internal analytics/data warehouse reporting functions.</li></ul><h3>Other Resources</h3><ul><li>Inspiration for this post: <a href="https://baremetrics.com/blog/tools-and-services-we-use-to-run-our-startup">Tools used by Baremetrics</a></li></ul><p>If you run a small business, I’d be interested to hear what tools you use. Leave me a comment below.</p><p><em>Originally published at </em><a href="https://winterwindsoftware.com/articles/services-and-tools-used-to-run-business/"><em>winterwindsoftware.com</em></a><em>.</em></p><p><a href="https://winterwindsoftware.com/newsletter/"><em>Subscribe to my mailing list</em></a><em> to be notified of future articles.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=34b0f96b9430" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>