<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mayank Lahiri on Medium]]></title>
        <description><![CDATA[Stories by Mayank Lahiri on Medium]]></description>
        <link>https://medium.com/@_mayank_?source=rss-e1b8f5b99cf6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Okgpy0lT40vOIk2CdM-MVg.jpeg</url>
            <title>Stories by Mayank Lahiri on Medium</title>
            <link>https://medium.com/@_mayank_?source=rss-e1b8f5b99cf6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 00:20:06 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@_mayank_/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Understanding and Controlling AWS Lambda Costs]]></title>
            <link>https://medium.com/@_mayank_/note-this-article-was-originally-published-on-the-serverless-blog-aa88b75cbab0?source=rss-e1b8f5b99cf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/aa88b75cbab0</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[faa]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[pricing]]></category>
            <category><![CDATA[lambda]]></category>
            <dc:creator><![CDATA[Mayank Lahiri]]></dc:creator>
            <pubDate>Mon, 18 Sep 2017 17:08:57 GMT</pubDate>
            <atom:updated>2017-09-18T17:09:27.584Z</atom:updated>
            <content:encoded><![CDATA[<p><strong><em>Note: this article was originally published on the </em></strong><a href="https://serverless.com/blog/understanding-and-controlling-aws-lambda-costs/"><strong><em>Serverless Blog</em></strong></a><strong><em>.</em></strong></p><p><a href="https://aws.amazon.com/lambda">AWS Lambda</a> is the archetype for a class of cloud computing products called serverless functions-as-a-service, or FaaS.</p><p>Others in this product class include <a href="https://cloud.google.com/functions/">Google Cloud Functions</a> and <a href="https://azure.microsoft.com/en-us/services/functions/">Azure Functions</a>, which share the same billing model as Lambda, but with different rates and service limits.</p><p>This post looks at the serverless FaaS billing model in general, but focuses on AWS Lambda’s current pricing (as of September 2017). Its goal is to point out some observations that may help in reducing or preventing unexpectedly large AWS Lambda bills. Many of these may also be relevant to other products that share a similar billing model.</p><h3>Lambda Functions</h3><p>A Lambda function is a piece of application software that runs in a short-lived container to service a single request or event. Although the use of the term “function” can suggest that the code must consist of a single function, Lambda functions are regular processes that can also, for example, spawn child processes. They must conform to a specified interface, but can otherwise contain arbitrary code.</p><p>Each Lambda function can be “sized” by setting the <strong>maximum memory size (GB)</strong> parameter in the Lambda Console or using the API. This value also affects the CPU shares allocated to the function when it runs, but in a manner that is not disclosed by AWS. Lambda also allows limiting the <strong>maximum function execution time (seconds)</strong> for a function, to prevent runaway or hanging functions from driving up cost.</p><p>Since Lambda functions run only when a request must be serviced, they only incur charges when used. The general pricing model adopted by serverless FaaS providers is based on two numbers per function invocation:</p><ul><li><strong>Maximum memory size (GB): </strong>note that this is <strong>not </strong>the actual memory used by the function, but rather the maximum memory size parameter in the Lambda function’s configuration. If you reduce your function’s memory usage, but do not adjust this configuration parameter accordingly, you will not see a cost reduction from the reduced memory usage.</li><li><strong>Function execution time (seconds): </strong>the actual amount of clock (wall) time that the function invocation took to run. Note that if a Lambda function makes an outgoing network call and sits idle waiting for the result, the time spent idle is still counted in the function’s execution time, i.e., this is not a measure of CPU usage.</li></ul><p>For each function invocation, these two values are <strong>multiplied</strong> together to produce a number with the unit <strong>GB-sec</strong>. After allowing for a monthly allowance of free GB-sec from the free tier, the billable compute cost is the total GB-sec across all function invocations, multiplied by a fixed <strong>GB-sec rate</strong>. This fixed rate is currently the following for three popular FaaS services.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/933/1*CUxf_2hkTNRqlbm-f9rHcQ.png" /><figcaption>Cost summary of various serverless FaaS services in US Dollars, as of September 2017. Free tier GB-sec is per month.</figcaption></figure><p>Since GB-sec is a composite unit, it can be unintuitive to reason about. Focusing on AWS Lambda’s current pricing, the following chart shows the cost of executing 100,000 invocations of a Lambda function that executes for a varying amount of time, broken down by three different maximum memory sizes. Note that each invocation may have to start a language runtime and load third-party libraries before getting to user code, all of which add to the function’s billable runtime <strong>for each invocation</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kxZTolFAtiBCWr3C." /></figure><p>It can be helpful to remember that a GB-sec represents neither a gigabyte nor a second, and is the first composite unit to be used for commercial cloud product pricing in recent memory.</p><p>Spreadsheet-inclined users may want to look at the data in this <a href="https://docs.google.com/spreadsheets/d/1xh_rNQwGVLos7Fshq2eM4gbIz546LwvrY1fMcLGqPmw/edit?zx=p7c6vgh0j97k#gid=541900098">public, read-only Google Sheet</a>.</p><h3>Observations</h3><ul><li><strong>Costs are multiplicative in function memory size and execution time. </strong>Suppose that a Lambda function uses 512 MB of memory and executes in slightly less than 200 milliseconds. After a code change, the function now needs 400 milliseconds to run (double), and 1024 MB of memory (double). The total compute cost increases <strong>4 times</strong>. If the memory requirement is instead tripled to 1536 MB, the total cost increases <strong>6 times</strong>. If both the memory requirement and execution time are tripled, the total cost increases <strong>9 times</strong> (<a href="https://docs.google.com/spreadsheets/d/1xh_rNQwGVLos7Fshq2eM4gbIz546LwvrY1fMcLGqPmw/edit?zx=p7c6vgh0j97k#gid=541900098">spreadsheet</a>). The impact of multiplicative costs may not be intuitive in that small changes to <em>either</em> function memory size or execution time can cause large changes in the total billable cost.</li><li><strong>Processing delays can be expensive. </strong>Suppose that a 512 MB Lambda function uses executes in slightly less than <strong>200 milliseconds</strong>, with a hard limit of 5 seconds. As part of its processing, the function calls an external service over HTTPS and waits for a response before ending. Suppose that network congestion or an external service degradation adds a spike of <strong>2 seconds </strong>to each network call. For the duration of the latency spike, the extra 2 seconds of Lambda running time will increase the cost of each invocation <strong>11 times</strong>, from $0.17 per 100k requests to $1.83 (<a href="https://docs.google.com/spreadsheets/d/1xh_rNQwGVLos7Fshq2eM4gbIz546LwvrY1fMcLGqPmw/edit?zx=p7c6vgh0j97k#gid=541900098">spreadsheet</a>).</li><li><strong>The free tier can run out quickly.</strong> Suppose that a service must support a sustained request rate of 100,000 requests per hour (sustained 27 requests per second). At this rate, the free tier of 400,000 GB-sec per month will run out in approximately <strong>1 hour</strong> for the following function sizes (<a href="https://docs.google.com/spreadsheets/d/1xh_rNQwGVLos7Fshq2eM4gbIz546LwvrY1fMcLGqPmw/edit?zx=p7c6vgh0j97k#gid=240985719">spreadsheet</a>):<br> • 512 MB, executing for 8 seconds per invocation<br> • 1024 MB, executing for 4 seconds per invocation<br> • 1536 MB, executing for 2 seconds per invocation</li><li><strong>Spot pricing may sometimes be cheaper than Lambda pricing. </strong>For some types of sustained workloads, using a reliable, distributed work queue in conjunction with preemptible, spot priced instances can allow fine tuning of the price/performance ratio, while still allowing a function-based, event-driven application architecture.</li></ul><p>As a concrete example of the observation on spot pricing, consider the following numbers, drawn from the <a href="https://iceroad.github.io/baresoil-benchmark-image-resizer/">Baresoil Image Resizing Benchmark</a>:</p><ul><li>A fixed-sized cluster of 20 c4.2xlarge spot instances can resize at least 120,449 images per hour for a total cost of <strong>$1.82 per hour</strong>, using a median of 1 second of processing time per image (<a href="https://iceroad.github.io/baresoil-benchmark-image-resizer/">benchmark</a>).</li><li>The exact same code and workload on Lambda costs <strong>$2.28 </strong>to run using a 320 MB function memory size (the smallest possible for this task), which is <strong>33% higher</strong> than the spot fleet. With the largest 1536 MB function size, the Lambda cost is <strong>59% higher </strong>than the spot fleet (<a href="https://docs.google.com/spreadsheets/d/1xh_rNQwGVLos7Fshq2eM4gbIz546LwvrY1fMcLGqPmw/edit?zx=p7c6vgh0j97k#gid=0">spreadsheet</a>). Processing times ranged from 800 ms to 3700 ms depending on the Lambda function size (<a href="https://docs.google.com/spreadsheets/d/1xh_rNQwGVLos7Fshq2eM4gbIz546LwvrY1fMcLGqPmw/">raw data</a>).</li></ul><p>On the other hand, if you are currently processing 120,449 images per hour using spot priced queue workers, you may be able to process the same actual workload in <strong>minutes rather than an hour </strong>for just a 33% increase in costs, using Lambda’s default concurrency limit of 1000.</p><h3>Summary</h3><ul><li><strong>Monitor and adjust maximum memory size and execution time parameters. </strong>Since the cost incurred by a Lambda function invocation is multiplicative in its execution time and memory size, increasing or reducing both by even a small amount can have an unexpectedly large impact on billable cost. Monitoring actual function execution time and memory usage allows these parameters to be set closer to their required value, which also limits the cost impact of runaway or hanging functions. Note that this does (and should) cut into the “no operations” myth surrounding serverless services.</li><li><strong>Avoid high maximum execution time. </strong>A common engineering instinct is to build in a safety buffer beyond minimum requirements. If a Lambda function normally runs in 200ms, it may be tempting to set the maximum execution time parameter to something large, like 10 seconds. However, consider the additional cost incurred by Lambda functions that are waiting on flaky external services, versus having the function terminate earlier and return an error to its caller.</li><li><strong>Consider EC2 spot pricing for queue-driven workloads. </strong>If traffic is predictable and sustained, an application architecture based around a reliable queue and spot priced queue worker instances may be cheaper than Lambda functions. This may change if Lambda starts supporting spot priced Lambda functions in the future, such as is the case with <a href="https://spotinst.com/products/spotinst-functions/">SpotInst</a>.</li><li><strong>Expect a price war soon</strong>. The serverless FaaS model is already heading towards a degree of organic standardization when it comes to features and interfaces, helped by frameworks like the <a href="https://serverless.com/">Serverless Framework</a>. Once serverless FaaS becomes a commodity cloud service, providers may only have room to compete on pricing. We can already see that Google has chosen different rates than Amazon and Microsoft, opting for a lower GB-sec rate but a higher per-request rate. In the future, some things we could conceivably expect are native spot (preemptible) pricing for serverless FaaS, and a lower slope to the cost curves shown in the earlier chart.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aa88b75cbab0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Servers to Serverless (long-form)]]></title>
            <link>https://medium.com/@_mayank_/from-servers-to-serverless-long-form-70d5f40ebfb5?source=rss-e1b8f5b99cf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/70d5f40ebfb5</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[paas]]></category>
            <category><![CDATA[virtualization]]></category>
            <dc:creator><![CDATA[Mayank Lahiri]]></dc:creator>
            <pubDate>Mon, 11 Sep 2017 18:22:34 GMT</pubDate>
            <atom:updated>2017-09-11T19:34:26.046Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Note: a condensed version of this article appears on </em><a href="https://read.acloud.guru/the-evolution-from-servers-to-functions-21833b576744"><em>acloud.guru</em></a><em>.</em></p><p><strong>Serverless computing, </strong>or just <strong>serverless, </strong>is a new cloud computing paradigm that promises to deliver lower overall development and operational costs, due to a combination of new technologies and pricing methods. Like other pieces of organically coined technical jargon, it does not have a precise, standardized definition, and is used interchangeably to refer to commercial products, open-source projects, as well as aspects of the underlying technology itself. In addition, the term can misleadingly suggest that no actual servers are involved, which is confusing and false.</p><p>Instead of trying to formulate <a href="https://xkcd.com/927/">one true definition</a> of serverless, this article lays out a possible history of its evolution, seen through the lens of monetary cost savings. In this scenario, there is the loosely-defined <strong>developer </strong>(a person or company), and a commercial entity called the <strong>provider </strong>(generally a company). The developer wants to use a certain number of Internet-connected servers to build a new application, and the provider wants to make the developer’s task easier by taking over varying degrees of mundane, lower-level tasks for them, in exchange for cold, hard currency.</p><p>This trade-off of labor for cost between developer and provider is the focus of this article, as a means of explaining how “serverless” came to be. The precise trade-off chosen by any given developer is affected by many factors, such as their budget, reliability needs, and general degree of do-it-yourself spirit. Serverless computing is a new point in this spectrum of trading labor for cost, and in order to understand how we got there, we first start at servers.</p><h3>Servers</h3><p>In the beginning, there were servers.</p><p>These were large, flat, physical boxes with expensive, industrial-grade computers inside. Servers were procured by buying or leasing, or building them from parts. They were generally placed inside data centers with redundant power supplies and fast Internet connections. The developer would generally have to perform (or pay someone to perform) manual tasks like physically installing the box inside a metal frame called a rack, checking for and replacing malfunctioning parts, physically pressing a power reset button when necessary, and putting out <a href="http://greenmonk.net/2011/11/14/data-center-war-stories-maxim-samo-from-ubs/">literal fires</a> caused by overheating equipment.</p><p>Only once the server was physically placed in a server rack, powered up, and connected to the Internet could the developer start using it by installing an operating system. After the operating system came installing and configuring various software like web servers, databases, and caches. The actual application’s code would then have to be copied to each server and started, and finally, the servers would have to be monitored, operating system security patches applied, and new servers added when application usage increased or servers failed.</p><p>The time and effort involved in administering a server in this manner can far exceed the time required to develop an application, and represents an ongoing cost for as long as the application is running. The benefit of this model, however, is the absolute control over the physical hardware, similar to buying a server and connecting it to a home Internet connection. Since the developer retains complete control of the hardware and all software running on it, a $10,000 server will deliver a constant $10,000 worth of performance to the developer, unlike some of the value-added models discussed later.</p><p>Today, this model of running servers has been streamlined into commercial forms such as <a href="https://en.wikipedia.org/wiki/Colocation_centre">server colocation</a>, <a href="https://en.wikipedia.org/wiki/Bare-metal_server">bare metal hosting</a>, and <a href="https://en.wikipedia.org/wiki/Dedicated_hosting_service">dedicated servers</a>, each varying in the degree of manual labor required, method of hardware procurement, and pricing model. It is still best suited for those who need absolute control over their own hardware (for maximum performance, custom hardware, or legal compliance reasons, for example), and are willing and able to put in the effort required to maintain it. Generally speaking, this group comprises large organizations with dedicated I.T. staff and specialized needs.</p><p><strong>Commercial examples (August 2017)</strong>: <a href="https://www.rackspace.com/en-us/managed-hosting/colocation">Rackspace Managed Colocation</a>, <a href="https://joesdatacenter.com/colocation/">Joe’s Data Center Colocation</a>, <a href="https://www.rackspace.com/en-us/cloud/servers/onmetal">Rackspace OnMetal</a>, <a href="https://www.scaleway.com/">Scaleway</a></p><h3>Virtualization</h3><p>With the advent of <a href="https://en.wikipedia.org/wiki/Full_virtualization">virtualization</a> technology, and particularly hardware support for it <a href="https://en.wikipedia.org/wiki/Hardware-assisted_virtualization">baked into CPUs</a> around 2005, a physical server could be efficiently split into multiple, smaller <strong>virtual servers</strong> (or <strong>virtual machines</strong>). Each virtual server would run its own separate copy of a standard operating system like Linux, Windows, or FreeBSD, completed isolated from any other virtual servers that may be running on the same physical machine. The expensive $10,000 server could now be split into five smaller $2,000 servers, each running a different operating system, with no modifications required to any existing software. If two such servers had been procured by an over-zealous developer, and were idle 50% of the time, they could be consolidated into a single server running two virtual machines.</p><p>Virtualization works by having an additional hardware-assisted layer of software called a <strong>hypervisor</strong> that sits below a traditional operating system. The purpose of the hypervisor is to effectively emulate a physical server, with or without the cooperation of the operating system. Virtualization had an interesting side effect: since the hypervisor acts an intermediary between a virtual machine and the actual hardware, a running virtual machine could be “frozen” at any time into a system snapshot, essentially a large file. Later, this file could be copied to a different server, and an exact clone of the virtual machine could be restored. The snapshot can be <a href="https://www.vmware.com/support/ws5/doc/ws_learning_files_in_a_vm.html">comprehensive</a> and include everything from the running state of programs and the contents of allocated memory, to just an image of the “hard disk” that the virtual machine sees.</p><p>Commercial hosting providers took advantage of this by offering <a href="https://en.wikipedia.org/wiki/Virtual_private_server">virtual private servers</a>, or VPSs, as an alternative to running a dedicated server. By installing a standard operating system like Linux or Windows into a virtual machines, and then freezing the snapshot post-installation (and configuration), they were able to obtain master snapshots of the base state of all popular operating systems. These snapshots could then be copied to any number of physical servers, and restored to a running virtual machine in a matter of minutes or seconds. Since this process could be repeated any number of times, large fleets of virtual servers running identical software could be created and destroyed quickly, rather than having to set up physical servers individually.</p><p>Economically, the providers were also able to make better use of their server fleet. Previously, if they had wanted to offer their customers a small $500 server in addition to large $10,000 ones, they would have had to maintain an inventory of $500 servers, which are generally neither cost effective nor performant. With virtual machines, a single $10,000 server could be efficiently split into a mix of five $1,000 servers and ten $500 servers (for example), each possibly leased out to a different developer. Furthermore, the mix of virtual servers could be altered at any time, allowing not just a wider range of server sizes to offer to developers, but also more efficient utilization of their inventory of physical servers.</p><p>The use of VPSs gave developers a few new abilities:</p><ul><li><strong>Scale</strong>. The server operating system only had to be configured once, and the frozen state could then be re-used to quickly create many, identical virtual servers. This meant that individual servers no longer had to be manually set up, and large fleets of identical servers could be created very quickly.</li><li><strong>Reliability. </strong>If the physical server failed for any reason (e.g., due to a failing hard disk), the hosting provider could automatically re-create the VPS on a different physical server and automatically switch the old IP address over to the new VPS. Users would see service disruption on the order of minutes, rather than hours for physical server repairs, and developers would no longer have to wake up in the middle of the night to fix routine hardware failures.</li><li><strong>Cost Control. </strong>Not only were more types of smaller (and cheaper) instances available than with physical servers, they could be mixed arbitrarily with larger instances to optimize a developer’s cost vs. performance needs. In addition, because a VPS could be created and destroyed relatively cheaply, providers generally billed for their use by the hour or month, rather than year. Large fleets could be created for a few hours, used, and then destroyed without incurring the cost of an annual server lease (for example).</li></ul><p>Virtualization technology had a large impact on the economics of running fleets of servers. It lead to the popularity of cheap, managed installations of various software frameworks like <a href="https://en.wikipedia.org/wiki/WordPress">Wordpress</a>, as well as the now-ubiquitous $5 per month virtual server. Virtual private servers are also frequently cited in conversations in terms like “but why can’t I just run it on a $5 VPS?”, which is often a valid question, but also often has valid answers as to why not.</p><p><strong>Commercial hosting examples (August 2017)</strong>: <a href="https://cloud.google.com/compute/">Google Compute Engine</a>, <a href="https://aws.amazon.com/ec2">Amazon EC2</a>, <a href="https://www.linode.com/">Linode</a>, <a href="https://www.digitalocean.com/">DigitalOcean</a></p><h3>Containers</h3><p>In an independent development, a feature called <a href="https://en.wikipedia.org/wiki/Cgroups">cgroups</a> was added to the Linux kernel in 2007, offering a way to <em>effectively</em> replicate a limited form of virtualization within the Linux operating system itself. Contributed by Google, its primary purpose was to safely bundle a Linux program and all its dependencies into a <strong>container image</strong>, which could then be cloned and run on other Linux machines, with a degree of isolation from other containers running on the same machine. Although this sounds very similar to virtualization, there are a number of <a href="https://blog.docker.com/2016/03/containers-are-not-vms/">important technical differences</a>between virtual machines and containers.</p><p>One key difference between containers and virtual machines for the purpose of this discussion is the following: <strong>virtual machines run an operating system on top of a hypervisor, but containers run programs on top of an operating system.</strong></p><p>Starting a virtual machine can involve either the entire operating system boot process, or restoring running system state from a frozen snapshot. Both of these procedures can take on the order of minutes to complete. After starting, running a virtual machine involves running an entire operating system, with all the associated CPU and memory overhead. Containers, on the other hand, run within a single host operating system, and only incur the CPU and memory overhead of the programs being run within the container. When a container is started, the application program inside it starts like a regular program, but is restricted by the operating system to run in isolation on only a slice of overall system resources.</p><p>This feature leads to two interesting properties of containers relative to virtual machines:</p><ul><li><strong>Lower resource usage. </strong>Since containers do not have an entire operating system’s worth of background processes, device drivers, and other paraphernalia running along with the developer’s programs, the memory and CPU overhead of a container is far less than that of a virtual machine.</li><li><strong>Faster startup. </strong>Virtual machines must be either booted through a standard operating system boot procedure, or restored from a suspended state. A container, on the other hand, starts with latency that is comparable to double-clicking a program on your desktop, while offering many of the benefits of virtual machines.</li></ul><p>Providers now had a new tradeoff available for developers: if the developer did not need to control or customize various low-level hardware facets of a virtual machine or physical server, and could express their application and all its dependencies as a file system (similar to a ZIP file) that ran on a handful of operating systems that supported containers (notably, Linux and FreeBSD, but now also Windows), the provider could run the application inside a container rather than a virtual machine. Since containers consume less memory and fewer CPU cycles than entire virtual machines, providers could utilize their fleet of physical servers even more efficiently. So instead of packing ten $10 virtual machines to a single server, they could now perhaps pack fifty $5 running containers into the same machine, due to the lower memory and CPU usage requirements of the containers.</p><p>With containers, developers had many of the benefits of virtual machines, such as installing custom system libraries and modifying many operating system settings, but with a faster deployment process than virtual machines. Since container images were generally small and easily testable, they could be quickly created by scripts and tested locally. The containers themselves would start much faster, and due to the use of <a href="https://en.wikipedia.org/wiki/Aufs">layered filesystems</a> by container runtimes, images could be built, updated, and uploaded much faster than virtual machine images.</p><p>However, one thing that had not <em>yet </em>changed from renting VPSs was that providers ran containers continuously and charged for the number of hours that each container ran for. This billing would occur even if the container consisted of an idle program that did no work, matching the experience of renting a VPS by the hour.</p><p><strong>Open-source container runtimes (August 2017)</strong>: <a href="https://docs.docker.com/engine/">Docker Engine</a>, <a href="https://linuxcontainers.org/lxd/">LXD</a><br><strong>Open-source distributed container runtimes (August 2017): </strong><a href="http://mesos.apache.org/">Mesos</a>, <a href="https://kubernetes.io/">Kubernetes</a><br><strong>Commercial managed container runtimes (August 2017)</strong>: <a href="https://aws.amazon.com/elasticbeanstalk/">AWS ElasticBeanstalk</a></p><h3>Platform as a Service (PaaS)</h3><p>At some point, providers may have realized that many developers were using similarly configured containers to run their applications. For example, a typical web-facing setup might contain a web server and a web application runtime and framework, like Ruby on Rails or PHP, along with the developer’s application code. These would typically be served as an HTTP-based API, allowing easy access from web frontends, native applications, and desktop applications.</p><p>Of all the components and auxiliary software that went into the “stack” of software, the only one that most developers truly cared about was their own application code, or their <strong>business logic. </strong>If the remaining components (e.g., web server, language runtime, system libraries, etc.) could be managed by the provider, then the developer would be free to focus on writing just their application code, reducing development time as well as ongoing maintenance costs. The provider would be in charge of setting up a largely standard OS environment for the developer, which the developer could count on for consistency.</p><p>In order to achieve this “business logic only” goal, the serverless methodology is to factor out all the common parts that are not pure business logic. The developer can choose to relinquish control of everything else to the provider, counting on the provider to create a standard environment in which to run the developer’s code. The provider therefore takes on all the tasks associated with maintaining and running a scalable web service, from provisioning virtual or physical servers, to configuring the operating system and server software used, to determining the exact version of a language’s runtime to use (in most cases). Since the developer no longer has to concern themselves with these tasks, the size of the code required to bring their application live is drastically reduced.</p><p>While this approach lead to the evolution of the Platform-as-a-Service, where <strong>continuously running</strong> containers could be dynamically created and destroyed by developers, there was a further cost saving to be had.</p><p><strong>Commercial PaaS (August 2017)</strong>: <a href="https://www.heroku.com/">Heroku</a>, <a href="https://www.meteor.com/pricing">Meteor Galaxy</a></p><h3>Serverless</h3><p>Due to the relatively small size and startup time offered by containers, providers realized that they could run the developer’s application only when an actual request was received, and then immediately shut it down to save resources. Instead of keeping a copy of the developer’s code running continuously at all times (and incurring billing charges), providers could wait for a user request to come in, and only then create a container with the developer’s code to service the request. After the developer’s code responds to the request, the container would be destroyed, freeing up system resources for other requests (and other developers’ code).</p><p>This leads to one of the defining characteristics of the serverless paradigm: <strong>short-lived, container environments</strong> that are created to service individual requests and other events. These events can be HTTP requests, WebSocket connections, work queue items, and database notifications, among many other possible triggers.</p><p>From the provider’s point of view, requests for multiple developers’ applications can be efficiently distributed across a large fleet of web servers. Since each container is short-lived, and only consumes resources for the duration of servicing a request, the provider’s fleet can be utilized even more efficiently than having continuously-running containers (which consume memory and CPU resources even when idle). Furthermore, since individual requests can be routed to any physical machine in the fleet, providers were able to offer developers the holy grail of scaling: <strong>instant, massive parallelism</strong>, without any service degradation for large, sudden spikes in traffic (say, after a Superbowl advertisement).</p><p>For developers, the serverless experience offers many other perks as well, in addition to the instant parallelism.</p><ul><li><strong>Simpler code</strong>: instead of writing long-lived web serving programs, the serverless paradigm encourages small, stateless programs (such as “functions”) that can be created and destroyed on demand. Instead of keeping track of <em>every</em> client connected to the server, the developer’s code can now assume that it is created to communicate with exactly <em>one</em> client.</li><li><strong>No provisioning: </strong>instead of having to determine the capacity needs for an application before it is in use, developers can now deploy a serverless application without worrying about how many servers or instances to reserve.</li><li><strong>No paying for idle time</strong>: since serverless providers no longer run virtual machines or containers continuously, they can no longer bill for server instances by the hour. With a serverless application, developer code is only executed in response to trigger events, on an “invisible” fleet of actual servers hidden from the developer’s view. Thus, the developer has no control over how many servers or instances they want to reserve, and the only sensible billable unit becomes the total amount of time actually spent by the provider servicing all the incoming events.</li></ul><p>A major requirement for writing serverless code, however, is to express business logic as programs that are created to serve a single trigger event, and only that event. After the program finishes handling the trigger event, it is destroyed. This is in contrast to traditional client/server programming, where a single program with multiple threads may accept many simultaneous HTTP connections and respond to them (for example), using conveniences like shared memory between requests. There is, thus, a new onus on developers to write or rewrite their applications in a different paradigm than what they were used to. In many cases, however, this rewrite can actually <em>reduce</em> the amount of code required for a particular application.</p><p><strong>Commercial Serverless Services (August 2017)</strong>: <a href="https://aws.amazon.com/lambda">AWS Lambda</a>, <a href="https://cloud.google.com/functions/">Google Cloud Functions</a>, <a href="https://azure.microsoft.com/en-us/services/functions/">Azure Functions</a>.</p><h3>Summary</h3><p>The serverless paradigm has applications beyond web serving, although that is currently the largest use case. There are already <a href="https://cloud.google.com/bigquery/">serverless data warehouses</a>, <a href="https://cloud.google.com/datastore/docs/concepts/overview">serverless databases</a>, and <a href="https://databricks.com/blog/2017/06/07/databricks-serverless-next-generation-resource-management-for-apache-spark.html">serverless stream processing systems</a>. In the future, we can probably expect this paradigm to extend to other classes of products, since the benefits for developers are so glaringly obvious.</p><p>An easy way to tell if a product or service is “serverless” is often to look at the pricing page. In particular, a serverless solution will have the following properties:</p><ul><li><strong>No provisioning</strong>: if a product or service bills requires the developer to reserve an explicit number of instances/servers/containers, it is likely not a serverless platform, since it requires explicit capacity provisioning. A serverless platform would add capacity as and when needed, keeping the details transparent from the user.</li><li><strong>Bill by usage: </strong>if a product or service bills by the time than an application is kept available for use, rather than for the actual usage of the application, then it is likely not a serverless platform. A quick way to check is to ensure that an idle application (i.e., one without any user activity), when deployed to the platform, will result in a negligible bill at the end of the month.</li></ul><p>The following figure attempts to break down the product categories described here based on how much of the stack is managed by the developer rather than the provider, using the loose definitions of each laid out in the introduction. Each row represents a slice of the stack from the first line of application code at the top, to the physical data center that hosts the server ultimately running the application. Note that the categorization, and example commercial products, is neither comprehensive nor static. Product categories and features change over time, blurring the lines between categories.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hseDNhpswAuD25E4." /></figure><p><em>One possible categorization of different product classes based on the labor tradeoff between developer and provider.</em></p><p>This has been a somewhat subjective view of the evolution of serverless computing. Future posts will look at more defining features of serverless platforms and architectures, as well as some of their caveats.</p><p><em>Many thanks to Oliver Bowen, Drew Firment, Merhawi Redda, Kapil Thadani, and Ishaan Joshi for comments and edits on versions of this post. Please send corrections, suggestions, and comments to </em><strong><em>mlahiri@gmail.com</em></strong><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70d5f40ebfb5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The evolution from servers to functions]]></title>
            <link>https://medium.com/a-cloud-guru/the-evolution-from-servers-to-functions-21833b576744?source=rss-e1b8f5b99cf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/21833b576744</guid>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Mayank Lahiri]]></dc:creator>
            <pubDate>Mon, 11 Sep 2017 02:51:23 GMT</pubDate>
            <atom:updated>2017-10-02T21:38:00.467Z</atom:updated>
            <content:encoded><![CDATA[<h4>Peer through the lens of a developer’s most common decision criteria — time and money</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*naNRufZO4QP-Gi2ld0kGWg.png" /></figure><p>Serverless architecture<strong> </strong>is the newest cloud computing paradigm that promises to lower overall development and operational costs — with a combination of new technologies and pricing structures.</p><p>While the architecture is quickly evolving, s<em>erverless </em>lacks a standard definition and general consensus within the technical community.</p><p>The term <em>serverless</em> is used liberally to label everything from commercial products, open-source projects, to aspects of the underlying architecture. It doesn’t help that the term falsely implies the absence of servers.</p><h3>Werner Vogels on Twitter</h3><p>I am the only one who doesn&#39;t like Serverless as an architectural description? It says what it is *not*, so what would describe it better?</p><p>Before trying to define serverless, it’s important to understand how we even arrived at this inflection point of modern architecture. To trace the evolution of <em>servers to serverless</em>, let’s peer through the lens of a developer’s most common decision criteria— time and money.</p><h4>Rack ‘em and Stack ‘em</h4><p>In the beginning, there were servers.</p><p>Remember servers? Those large, flat, physical boxes with expensive, industrial-grade components inside? They were generally placed inside data centers with redundant power supplies and high-speed networks.</p><p>The server was installed in a server rack, powered up, and connected to the network. After the initial install of the operating system and patches, the server needed to configured with web servers, databases, and caches.</p><p>Once properly configured, the application code would finally be copied to each server. The end of the setup was just the beginning — servers would have to be monitored, patches continuously applied, and new servers added when application usage increased or servers failed.</p><h3>saddest server on Twitter</h3><p>the nice thing about our datacenter is you cannot hear the techs crying over the fan noise</p><p>The effort involved in administering a server far exceeds the time required to develop an application — and represents an ongoing cost for the duration of the application’s lifespan. The only rationale driver and advantage for installing physical servers is retaining absolute control over the hardware.</p><p>Today, this model of running servers has been streamlined into commercial forms such as server colocation, bare metal hosting, and dedicated servers. Each of the models vary in the degree of manual labor required, method of hardware procurement, and pricing model — but they are all still best suited for those who need absolute control over their own hardware.</p><p>Generally speaking, this approach comprises large organizations with dedicated I.T. staff and specialized needs.</p><blockquote><strong>Examples</strong>: <a href="https://www.rackspace.com/en-us/managed-hosting/colocation">Rackspace Managed Colocation</a>, <a href="https://joesdatacenter.com/colocation/">Joe’s Data Center Colocation</a>, <a href="https://www.rackspace.com/en-us/cloud/servers/onmetal">Rackspace OnMetal</a>, <a href="https://www.scaleway.com/">Scaleway</a></blockquote><h4>It’s virtually a server</h4><p>With the advent of virtualization technology, and particularly hardware support for it baked into CPUs around 2005, a physical server could be efficiently split into multiple, smaller virtual servers — or virtual machines.</p><p>Each virtual server could run its own separate copy of a standard operating system like Linux, Windows, or FreeBSD. Each instance is completed isolated from the other virtual servers running on the same physical machine.</p><p>An expensive $10,000 server that was over-provisioned could now be split into five smaller $2,o00 servers — each running a different operating system, with no modifications required to any existing software.</p><p>Virtualization works by having an additional hardware-assisted layer of software called a <em>hypervisor</em> that sits below a traditional operating system. The purpose of the hypervisor is to effectively emulate a physical server, with or without the cooperation of the operating system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*LgKUkyFikAC866YNV8o8uw.png" /></figure><p>Virtualization had an interesting side effect: since the hypervisor acts an intermediary between a virtual machine and the actual hardware, a running virtual machine could be “frozen” at any time into a system snapshot — essentially creating a large snapshot file.</p><p>The snapshot could be copied to a different server, and an exact clone of the virtual machine could be restored. The snapshot could comprehensively include everything from the running state of programs and the contents of allocated memory, or simply capture an image of the “hard disk”.</p><h4>A fleet of virtual servers</h4><p>Commercial hosting providers quickly began to offer virtual private servers, or VPSs, as an alternative to running a dedicated server. The vendors would install and configure popular operating system like Linux or Windows onto a virtual machines, and then take a snapshot to obtain a master copy.</p><p>These snapshots could then be copied to any number of physical servers, and restored to a fully-functioning virtual machine in a matter of minutes or seconds. Since this process could be repeated any number of times, large fleets of virtual servers running identical software could be created and destroyed quickly — rather than setting up physical servers individually.</p><p>By leveraging virtual machines and economies of scale, the providers were also able to make better use of their fleet of physical servers.</p><p>With virtual machines, a single server could be efficiently split into a mix of various sized servers — with each possibly leased out to a different developer. The mix of virtual servers could be altered at any time allowing a wider range of server sizes to be offered to developers, while enabling more efficient utilization of their inventory of physical servers.</p><p>The use of VPSs providers developers with a few new advantages:</p><ul><li><strong>Scale</strong>. The server operating system only had to be configured once, and the frozen state could then be re-used to quickly create many, identical virtual servers.</li><li><strong>Reliability. </strong>If the physical server failed for any reason, the hosting provider could automatically re-create the VPS on a different physical server and automatically switch the old IP address over to the new VPS.</li><li><strong>Flexbility. </strong>The availability of smaller and cheaper instances increased, as well as the flexibility to mix with larger instances to optimize cost vs. performance needs.</li><li><strong>Cost Control</strong>. Since a VPS could be easily created and destroyed , providers generally billed by the hour or month instead of year. Large fleets could be created for a few hours, used, and then destroyed without incurring the cost of an annual server lease.</li></ul><p>Virtualization had a massive impact on the economics of running fleets of servers. The technology lead to the popularity of cheap, managed installations of various software frameworks like Wordpress, as well as the now-ubiquitous $5 per month “pay as you go” virtual server.</p><blockquote><strong>Examples</strong>: <a href="https://cloud.google.com/compute/">Google Compute Engine</a>, <a href="https://aws.amazon.com/ec2">Amazon EC2</a>, <a href="https://www.linode.com/">Linode</a>, <a href="https://www.digitalocean.com/">DigitalOcean</a></blockquote><h4>Contain yourself</h4><p>In 2007, a new feature was contributed by Google to the Linux kernel that offered a way to <em>effectively</em> replicate a limited form of virtualization within the Linux operating system itself.</p><p>The primary purpose of the feature was to safely bundle a Linux program and all its dependencies into a <em>container image. </em>The image<em> </em>could then be cloned and run on other Linux machines — with a degree of isolation from other containers running on the same machine.</p><p>Although this sounds very similar to virtualization, there are a number of important technical differences between virtual machines and containers.</p><p>One key difference between containers and virtual machines is that <em>virtual machines run an operating system on top of a hypervisor,</em> but <em>containers run programs on top of an operating system</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*1t0asuPdSQm9slYsdp2AMw.png" /><figcaption>Containers vs Virtual Machines, courtesy of Docker Inc. and RightScale Inc.</figcaption></figure><p>One of the drawbacks of a starting a virtual machine is that the process involves an entire operating system boot process, or requires the restoration of running system state from a frozen snapshot.</p><p>Both of these procedures can take on the order of minutes to complete. After the startup sequence completes — running a virtual machine consumes an entire operating system with all the associated CPU and memory overhead.</p><p>Containers, on the other hand, run within a single host operating system. The program being executed only consumes the CPU and memory overhead within the container. When a container is started, the application program inside it starts like a regular program — but it’s restricted by the operating system to run in isolation on only a slice of overall system resources.</p><p>This feature leads to two interesting properties of containers relative to virtual machines:</p><ul><li><strong>Lower resource usage. T</strong>he memory and CPU overhead of a container is far less than that of a virtual machine. Containers do not have an entire operating system worth of background processes, device drivers, and other paraphernalia running alongside the developer’s programs.</li><li><strong>Faster startup. </strong>Virtual machines must be either booted through a standard operating system boot procedure, or restored from a suspended state. A container, on the other hand, starts with latency that is comparable to double-clicking a program on your desktop, while offering many of the benefits of virtual machines.</li></ul><p>Since containers consume less memory and fewer CPU cycles than entire virtual machines, providers could utilize their fleet of physical servers even more efficiently. So instead of packing ten virtual machines to a single server, they could now pack fifty containers onto the same machine.</p><h3>Honest Status Page on Twitter</h3><p>Between containers, cluster managers and virtual machines we&#39;ve lost track of where our code even is. #inception</p><p>Providers could now offer a new tradeoff available for developers — allowing them to run applications instead of a container instead of a virtual machine. Containers provided a great option assuming the developer didn’t need to interact with system-level components, could express their application and all its dependencies as a file system, and it ran on the handful of operating systems that supported containers.</p><p>With containers, developers still had many of the benefits of virtual machines, such as installing custom system libraries and modifying many operating system settings — but with a faster deployment process than virtual machines.</p><p>Container images were generally small, faster to start, easy to test, and could be quickly created by scripts and tested locally. Due to the use of layered filesystems by container runtimes, images could be built, updated, and uploaded much faster than virtual machine images.</p><p>However, one thing that didn’t change was the billing method — providers ran containers continuously and charged for the number of hours that each container executed. The billing would occur even if the container consisted of an idle program that did no work, matching the experience of renting a VPS by the hour.</p><blockquote><strong>Examples</strong>: <a href="https://docs.docker.com/engine/">Docker Engine</a>, <a href="https://linuxcontainers.org/lxd/">LXD</a>, <a href="http://mesos.apache.org/">Mesos</a>, <a href="https://kubernetes.io/">Kubernetes</a>, <a href="https://aws.amazon.com/elasticbeanstalk/">AWS ElasticBeanstalk</a></blockquote><h4>Platform as a Service (PaaS)</h4><p>At some point, providers realized that many developers were using similarly configured containers to run their applications. A typical web-facing setup might contain a web server and a web application runtime and framework, like Ruby on Rails or PHP, along with the developer’s application code.</p><p>These containers would typically be served as an HTTP-based API, allowing easy access from web frontends, native applications, and desktop applications. Of all the components and auxiliary software that went into the “stack” of software, the only one that most developers truly cared about was their own application code — or their <em>business logic</em><strong>.</strong></p><p>If the remaining components could be managed by the provider— such as the web server, language runtime, system libraries — then the developer could be free to focus on writing just their application code. The provider would be responsible for setting up a largely standard OS environment for the developer. In return, the developer would get a consistent environment that reduced development time as well as maintenance costs.</p><p>In order to achieve the goal of focusing on only the business logic that differentiates your product, then everything else should be abstracted into a commoditized service. The developer must learn to relinquish control of everything else to the provider, counting on the provider to create a standard environment in which to run the developer’s code.</p><p>The provider takes on all the tasks associated with maintaining and running a scalable web service — from provisioning virtual or physical servers, to configuring the operating system and server software used, to determining the exact version of a language’s runtime. Since the developer no longer has to concern themselves with these tasks, the size of the code required to bring their application live is drastically reduced.</p><p>While this approach lead to the evolution of the Platform-as-a-Service, where<em> continuously running</em> containers could be dynamically created and destroyed by developers, there were still further cost saving to be gained.</p><blockquote><strong>Examples</strong>: <a href="https://www.heroku.com/">Heroku</a>, <a href="https://www.meteor.com/pricing">Meteor Galaxy</a></blockquote><h4>Functions as a Service (FaaS)</h4><p>Due to the small size and execution times of containers, providers soon realized they could run the developer’s application when a request was received — and then immediately shut it down to save resources.</p><p>Instead of keeping a copy of the developer’s code running continuously at all times and incurring billing charges, providers could now wait for a user request — and only then create a container with the developer’s code to service the request. After the developer’s code responds to the request, the container would be destroyed freeing up system resources for other requests.</p><p>This leads to one of the defining characteristics of the serverless paradigm: <em>short-lived, container environments</em> that are created to service individual requests and other events. These events can be HTTP requests, WebSocket connections, work queue items, and database notifications, among many other possible triggers.</p><p>From the provider’s point of view, requests for multiple developers’ applications can be efficiently distributed across a large fleet of web servers. Each container is short-lived — -and only consumes resources for the duration of servicing a request. The provider’s fleet can be utilized even more efficiently than having continuously-running containers, which consume memory and CPU resources even when idle.</p><p>Since individual requests can be routed to any physical machine in the fleet, providers were able to offer developers the holy grail of scaling — <em>instant, massive parallelism — </em>without any service degradation for large, sudden spikes in traffic (e.g., after a Super Bowl advertisement).</p><p>For developers, the serverless experience offers many other perks as well, in addition to the instant parallelism.</p><ul><li><strong>Simpler code</strong>: instead of writing long-lived web serving programs, the serverless paradigm encourages small, stateless programs — or functions — that can be created and destroyed on demand. Instead of keeping track of <em>every</em> client connected to the server, the developer’s code can now assume that it is created to communicate with exactly <em>one</em> client.</li><li><strong>No provisioning: </strong>instead of having to determine the performance capacity needs for an application before it’s launched into production, developers can now deploy a serverless application without worrying about the number of servers or instances to reserve.</li><li><strong>No paying for idle time</strong>: With a serverless application, developer code is only executed in response to trigger events, on an “invisible” fleet of actual servers hidden from the developer’s view. The developer has zero control over how many servers or instances they can reserve — so the only sensible billable unit is the total amount of time actually spent by the provider servicing all the incoming events.</li></ul><p>A major tenet for writing serverless code is to express business logic as programs that are created to serve a single trigger event — and only that event. After the program finishes handling the trigger event, it is destroyed.</p><p>This approach to serverless architecture is in stark contrast to traditional client/server programming, where a single program with multiple threads may accept and respond to many simultaneous HTTP connections using conveniences like shared memory between requests.</p><h3>Simon Wardley on Twitter</h3><p>Things I care about in a serverless environment - 1) Does my code run? 2) Does it scale? 3) Do you bill me for what my code consumes?</p><h3>Simon Wardley on Twitter</h3><p>Things I don&#39;t care about in a serverless environment - 1) The servers you use 2) Containers 3) Your electricity provider 4) Nuts and bolts</p><p>As a result, there is a new onus on developers to write or rewrite their applications using a different paradigm. In many cases, however, this rewrite can actually <em>reduce</em> the amount of code required for a particular application.</p><blockquote><strong>Examples</strong>: <a href="https://aws.amazon.com/lambda">AWS Lambda</a>, <a href="https://cloud.google.com/functions/">Google Cloud Functions</a>, <a href="https://azure.microsoft.com/en-us/services/functions/">Azure Functions</a>.</blockquote><h4>Serverless services</h4><p>While serving websites is currently the largest use case, the serverless paradigm extends well beyond this common pattern. Providers are already offering serverless data warehouses, serverless databases, and serverless stream processing systems.</p><p>In the future, we can expect this paradigm to quickly extend to other classes of products — since the benefits for developers are so glaringly obvious. To help determine if a product or service is “<em>serverless</em>” — you can often just look look at the pricing page for the following properties:</p><ul><li><strong>No provisioning</strong>: If it requires explicit capacity provisioning — it is likely not a serverless platform. The product or service should not require the developer to reserve an explicit number of instances, servers, or containers. A serverless platform increases capacity as and when needed, keeping the details transparent from the user.</li><li><strong>Bill by usage: </strong>If a product or service bills by the time an application is kept available for use, rather than for the actual usage of the application — then it is likely not a serverless platform. A quick litmus test is to ensure that an idle application will result in a negligible bill at the end of the month.</li></ul><h4>TL; DR — check out the summary table</h4><p>The following table breaks down the product categories based on how much of the stack is managed by the developer. Each row represents a slice of the stack — from the first line of application code at the top, to the physical data center that hosts the server ultimately running the application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hseDNhpswAuD25E4." /></figure><p><em>Thanks for reading!</em></p><p><em>Many thanks to Oliver Bowen, Drew Firment, Merhawi Redda Tamrat, Kapil Thadani, and Ishaan Joshi for their contributions to this post.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=21833b576744" width="1" height="1" alt=""><hr><p><a href="https://medium.com/a-cloud-guru/the-evolution-from-servers-to-functions-21833b576744">The evolution from servers to functions</a> was originally published in <a href="https://medium.com/a-cloud-guru">A Cloud Guru</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How many 53-bit integers fit in your language’s map data structure?]]></title>
            <link>https://medium.com/hackernoon/how-many-53-bit-integers-fit-in-your-languages-map-data-structure-41605402b7de?source=rss-e1b8f5b99cf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/41605402b7de</guid>
            <category><![CDATA[benchmark]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[cplusplus]]></category>
            <dc:creator><![CDATA[Mayank Lahiri]]></dc:creator>
            <pubDate>Sat, 09 Sep 2017 00:07:53 GMT</pubDate>
            <atom:updated>2017-09-09T17:11:13.969Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<p>In many applications, generating a unique user identifier is a common task that is frequently taken for granted. A simple, effective way to generate user identifiers is to generate a large random number, and let the properties of small probabilities all but guarantee that each generated number will be different from any previously generated identifiers (see: <a href="https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_.28random.29">UUIDv4</a>, or <a href="https://en.wikipedia.org/wiki/Cryptographic_nonce">cryptographic nonces</a>).</p><p>Suppose that you don’t feeling like breaking out the pencil, paper, and brain to work out the collision probabilities as the size of your user base increases. Instead, you reach for a nearby, relatively crumb-free keyboard and decide to amuse yourself by writing a simulation.</p><p>The program is simple. It executes the following:</p><ol><li>Create a map / dict / object data structure to serve as an associative array.</li><li>Set the global collision counter to 0.</li><li>Generate a random 53-bit random integer (in order to allow parity with Javascript, which only allows 53-bit integers).</li><li>Check if this number exists in the associative array. If it does not, add it to the array. If it does, increment the global collision counter.</li><li>Jump to step 3 until the program crashes due to hitting a 2 GB memory limit.</li></ol><p>This is clearly a simple program that can be written in multiple ways (for example, by using each language’s set data structure). However, the point of this micro-benchmark is to roughly answer the following questions for a number of languages:</p><ol><li><strong>How many unique integers can we stuff into an associative array before the program crashes?</strong></li><li><strong>How much time does each language take to get to its crash point?</strong></li></ol><p>And of course, to see how many collisions are actually encountered.</p><p>The contenders are as follows. Before looking at the results, try to formulate your guesses for the answers to the questions above for each language.</p><ul><li><strong>C++ </strong>compiled with gcc 5.4.0 and C++11 support</li><li><strong>Python 2</strong> running on CPython 2.7.12</li><li><strong>Python 2</strong> running on PyPy 5.1.2</li><li><strong>Python 3</strong> running on CPython 3.5.2</li><li><strong>node.js 6</strong>.11.3 (LTS as of September 2017)</li><li><strong>node.js 8</strong>.4.0 (latest as of September 2017)</li></ul><h3>Results</h3><p>The graph below shows the number of unique integers put into the associative array before the program either hit the 2 GB memory limit, or crashed for other reasons.</p><p>The x-axis is time since the program was started, and the y-axis is the number of integers in the associative map in millions. In this graph, lines terminate when the program crashes. Steeper lines represent faster associative array performance, since a lookup and insertion is performed at each iteration of the loop.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*N_4Bp4yfLiIXQJKbwCJTlw.png" /><figcaption>Size of the associative array over time for a number of languages. Each series ends when the program crashed. Lines that reach higher y-values represent more memory-efficient languages, and the slope of each line represents the performance of data structures in each language.</figcaption></figure><ul><li>Naturally, <strong>C++ achieves the best memory performance</strong>, storing more than <strong>32.5 million long integers</strong> in a std::map before crashing. However, Python exceeds its performance in terms of speed, as seen by the steeper lines for all three implementations. This may be somewhat unexpected to people accustomed to C/C++ being the gold standard for performance, but can be explained by the fact that Python uses a <a href="http://www.laurentluce.com/posts/python-dictionary-implementation/">plain hash map</a> as the implementation for its dict data structure, whereas C++’s map uses a <a href="https://stackoverflow.com/questions/5288320/why-is-stdmap-implemented-as-a-red-black-tree">red-black tree</a>.</li><li>Running CPython in Python 2 and 3 modes show very similar performance profiles, with Python 2 being a tad faster than Python 3 at storing the same number of items — about <strong>22.3 million integers</strong>.</li><li>Pypy’s performance exceeds that of CPython in both Python 2 and 3 flavors, but its memory management causes it to crash after adding just <strong>10 million integers</strong> to the map.</li><li>node.js shows the worst memory performance characteristics. Of these, the current LTS release of node (v6) is slightly better than the latest release (v8). However, both are not only the slowest profiled here, but also show among the worst memory performance. Node 6 stores just <strong>11.1 million integers</strong> in an object, and node 8 fares the worst with <strong>8.3 million integers</strong>.</li></ul><p>As for the original intent of this expedition, the total number of collisions across all runs for all languages was precisely <strong>0.</strong></p><h3>Code and raw results</h3><p><a href="https://github.com/mayanklahiri/53bit-ids">Github</a></p><h3>Notes</h3><ul><li><strong>g++ </strong>was run with <strong>-O3</strong> and with the C++11 standard.</li><li><strong>node 8.4.0 hung without crashing</strong> after consuming approximately 1.1 GB of memory. This is in spite of running with the <strong>— max-old-space-size=2048 </strong>command line flag. Without limiting process memory using ulimit, memory consumption appeared to increase without bound, in spite of producing no new output. Disabling TurboFan using <strong>— noturbo </strong>had no impact. Disabling garbage collection using<strong> — noconcurrent_sweeping</strong> also had no impact.</li><li><strong>node 6.11.3</strong>’s garbage collector prints its crash stack trace to stdout instead of stderr, which is non-standard behavior.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=41605402b7de" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/how-many-53-bit-integers-fit-in-your-languages-map-data-structure-41605402b7de">How many 53-bit integers fit in your language’s map data structure?</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>