<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Christophe Bougère on Medium]]></title>
        <description><![CDATA[Stories by Christophe Bougère on Medium]]></description>
        <link>https://medium.com/@christophe.bougere?source=rss-7934ff29dd7c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*T8ri22YfGN9xQ8Z4BV7xQA.jpeg</url>
            <title>Stories by Christophe Bougère on Medium</title>
            <link>https://medium.com/@christophe.bougere?source=rss-7934ff29dd7c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 18:44:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@christophe.bougere/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AWS Amplify beyond the quickstart]]></title>
            <link>https://medium.com/@christophe.bougere/aws-amplify-beyond-the-quickstart-c389f8e44c92?source=rss-7934ff29dd7c------2</link>
            <guid isPermaLink="false">https://medium.com/p/c389f8e44c92</guid>
            <category><![CDATA[amplify]]></category>
            <category><![CDATA[cloud-architecture]]></category>
            <category><![CDATA[serverless-framework]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws-amplify]]></category>
            <dc:creator><![CDATA[Christophe Bougère]]></dc:creator>
            <pubDate>Tue, 06 Aug 2019 11:43:55 GMT</pubDate>
            <atom:updated>2019-08-06T11:44:31.970Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-nc-sa/4.0/</cc:license>
            <content:encoded><![CDATA[<h4>Tips &amp; tricks to integrate amplify in a real-world project</h4><p>If you know a little bit the React ecosystem, the arrival of Create React App (CRA) has probably been a relief for you. We all know how complicated it can be to set up modern tooling (including webpack, eslint, babel, hot reload server, …). CRA offers all this in a simple command line: Simply run 2 or 3 commands and voilà! Of course, it won&#39;t fit for every project. But when you just want to prototype something, you will save many hours. And you&#39;re free to eject your app at any time if you need more configuration capabilities.</p><p>Well, I am not going to speak about CRA today, but I find the analogy with AWS Amplify quite relevant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*KOFkHUYkw8sXtP8U.jpg" /><figcaption>Ready to go?</figcaption></figure><p>I will start with the basics and go deeper progressively, so feel free to skip to the section that interests you the best:</p><ol><li><a href="#8ca5">It does everything, even the coffee, right?</a></li><li><a href="#c607">So what’s the deal, should I use it?</a></li><li><a href="#45d8">Integrating into a real-world project</a></li><li><a href="#715a">Final thoughts</a></li></ol><p>This article is the logical continuation of previous articles I wrote about serverless architectures and the Serverless framework. If you feel a bit lost at some point, you should have a quick look at these two articles:</p><ul><li><a href="https://medium.com/@christophe.bougere/the-10-commandments-of-serverless-e5525e2c4bb6">The 10 commandments of serverless</a></li><li><a href="https://medium.com/@christophe.bougere/serverless-framework-tips-and-tricks-7ee88a01c4ab">Serverless framework tips and tricks</a></li></ul><h3>1. It does everything, even the coffee, right?</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FTfItZ1HqJCTza%2Ftwitter%2Fiframe&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FTfItZ1HqJCTza%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FTfItZ1HqJCTza%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/d0aab15b42f27141a0415b84d80abe23/href">https://medium.com/media/d0aab15b42f27141a0415b84d80abe23/href</a></iframe><p>Sorry to disappoint you, but the coffee category does not exist yet (I&#39;m sure it&#39;s in the roadmap though). However, it&#39;s doing plenty of other cool things! After experiencing with it on a side project for 5 months, if I had to sum up Amplify in a sentence this is what I would say:</p><blockquote>Amplify is a full-stack serverless framework</blockquote><p>Before Amplify, AWS was already providing a great <strong>Backend as a Service</strong> (BaaS) toolset, meaning you were able to create a complete serverless backend without managing anything (including API, storage, database, authentication, authorization, …). But when it comes to integrating with your frontends (web or mobile), you were left alone with no pre-designed architectures recommendations. This is, I think, the main innovation brought by Amplify.</p><p>So, what is amplify made of?</p><ul><li><strong>A CLI</strong>, this is where it all starts. Run npm install -g @aws-amplify/cli and <a href="https://aws-amplify.github.io/docs/cli-toolchain/quickstart#quickstart">start your project</a>. The CLI allows you to create a project and add resources from various categories in it (api, auth, hosting, storage, …). Under the hood, it creates CloudFormation stacks and manages them.</li><li><strong>An API as a service</strong>: simply configure your data model using GraphQL Schema Definition Language (SDL), and Amplify will take care of creating and managing all the resources for you (DynamoDB tables, AppSync resolvers, Lambda functions, …). It also offers subscription using WebSockets to add some real-time features to your app.</li><li><strong>A user management system</strong>, on top of Cognito User Pool and Cognito Federated Identities. Handle MFA, social providers and all the boring stuff in a snap.</li><li><strong>A static website hosting service.</strong> Running amplify push in the configured environments will build your frontend app, deploy it on S3 and configure a CloudFront distribution for you. This is very similar to <a href="https://www.netlify.com/">Netlify</a>.</li><li><strong>A UI library.</strong> Amplify provides <a href="https://aws-amplify.github.io/docs/js/ui">UI components</a> for React, React Native, Angular, Ionic, and Vue. Don’t expect too much on this, but I find the Authenticator component quite useful when building a prototype.</li><li><strong>A CI/CD platform.</strong> Yep, with <a href="https://aws.amazon.com/amplify/console/">Amplify Console</a> you can setup a proper CI/CD pipeline that follows the git flow, create feature environments on-the-fly, configure domains and restrict access, well all your DevOps!</li><li>Analytics (user tracking), push notifications, AR/VR, machine learning, chatbots, and probably more to come. I feel that AWS is using Amplify to showcase lots of new features.</li></ul><h3>2. So what&#39;s the deal, should I use it?</h3><p>Well, it depends on your use case.</p><h4>Pros</h4><ul><li><strong>Focus on your business, do not reinvent the wheel.</strong> What is the point of building your own authentication system while you could use an existing one that already handles MFA, social providers, and all the enterprise features?</li><li><strong>Prototype cool things with Amplify and eject features one by one when you have to.</strong> Amplify allows you to do high-quality proof of concepts at no cost, so why not take advantage of it? As an example, you could build your API with generated AppSync resolvers and replace them one by one by custom resolvers when required (similarly to Create React App).</li><li><strong>Good multi-environment design for good DevOps.</strong> You can bootstrap new environments on the fly. Also, your amplify environments can follow your git workflow.</li><li><strong>Serverless and cheap as hell.</strong> In most cases, the free-tier should be enough. My 4 environments (production, staging, and 2 users) only cost me a 2–3$ a month in total, and most of it is the CI/CD, which won&#39;t scale in the same proportion that resources (like compute or storage).</li></ul><h4>Cons</h4><ul><li><strong>A steep learning curve.</strong> Integrating all these modern technologies implies understanding their inner mechanism if you want to go beyond the examples. However, once you get the experience, it&#39;s as easy as pie!</li><li><strong>Limitations of the framework.</strong> While it is heavily maintained by AWS, some small issues can be really blocking and prevent you from using it.</li><li><strong>Strong AWS lock-in.</strong> Don&#39;t go for Amplify if you&#39;re thinking about migrating to another cloud provider, it’s obvious.</li></ul><p>My recommendation would be to try it on a side project to understand it better and gain experience, and then decide by yourself if it fits your application.</p><h3>3. Integrating into a real-world project</h3><p>So you decided to give Amplify a try? OK, let&#39;s move on to the serious stuff now! In this section, I will share how to integrate Amplify with Serverless framework, how to manage environments and a bunch of other tips. This is not an introduction to Amplify, so have a look at the <a href="https://aws-amplify.github.io/docs/cli-toolchain/quickstart">quick start</a> before reading this part.</p><p>To illustrate my words, let&#39;s say we want to create a blogging app with Amplify. Also, let&#39;s name it <strong>Podium</strong> (yeah, I like bad puns). Authentication will be managed through Cognito, and we will use AppSync and DynamoDB to store the blog posts and comments. The frontend will be a React SPA, but we plan to add mobile apps later. Lastly, we will store post images on S3.</p><p>Also, please note that this isn&#39;t a step-by-step tutorial. The goal is to see on a high-level how we can integrate the different pieces of our project, not to get very specific on details.</p><h4>Code organization</h4><p>Well, I am a monorepo advocate. So this will be our repo structure:</p><ul><li><strong>cli/</strong> We will create a small CLI to help us automate tasks.</li><li><strong>commons/</strong> Shared Node.js library that will be used in the CLI and when packaging our Serverless services.</li><li><strong>fixtures/</strong> Contains all our testing data (JSON files to seed our DynamoDB tables, and images to seed our S3 bucket).</li><li><strong>mobile/</strong> could contain our mobile apps in the future (and reference the <strong>web/amplify/ </strong>folder).</li><li><strong>services/</strong> Serverless services, for everything we can&#39;t do with Amplify.</li><li><strong>web/</strong> Our React application. Will contain a <strong>amplify/</strong> subfolder. Every amplify command should be run inside the web/ folder (which will be our main frontend).</li></ul><h4>GraphQL Schema</h4><p>We will keep it super simple. Two models: Post and Comment.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/16257dc534aeb51d4f569b5a64f5763c/href">https://medium.com/media/16257dc534aeb51d4f569b5a64f5763c/href</a></iframe><h4>Podium CLI</h4><p>To ease development, we can create a custom CLI using tools like <a href="https://oclif.io/">oclif</a>. Some commands that could be useful:</p><ul><li>podium setup-env A command to run after amplify push to customize our environment with features that aren&#39;t handled by Amplify natively. For instance, create a user group &quot;admin&quot;, change the API authentication type to AWS_IAM and customize auth and unauth roles policies, … This command will be used in ou CI/CD.</li><li>podium seed:{all|cognito|dynamodb|s3} Seeds our environment with data from the fixtures/ folder. seed:cognito will create some standard users and add them to the admin group. seed:dynamodb will fill DynamoDB tables with JSON files, and seed:s3 will upload local files to the Amplify S3 bucket.</li><li>podium flush:{all|cognito|dynamodb|s3} Flush our environment (delete Cognito users, flush DynamoDB, empty S3 bucket). This is nice to do before a seed to ensure you have a clean environment (in particular when running tests).</li></ul><p>Nothing complicated here, you can use the AWS SDK to create/update/delete these resources. The only tricky thing is that you need to resolve Amplify resource names, like bucket or table names, and they often contain some ID that change depending on the environment. To overcome this, we will add the following file in the commons/ folder (it will be used by serverless as well later).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/04ad29788699f4e7bb13e08c7b089907/href">https://medium.com/media/04ad29788699f4e7bb13e08c7b089907/href</a></iframe><p>If this looks like an ugly trick, it&#39;s because it is one. We are retrieving the resource names from various amplify files, sometimes by parsing them with regular expressions 🤢. But if you are ready to accept this, I promise the rest will be super smooth 👌. For instance, our Post table name can be generated like this: Post-${getApiConfig().id}-${getCurrentEnv()} so you can easily code to seed your tables.</p><h4>Serverless integration</h4><p>Now, you want to let the user upload images so they can add them in their post. Nothing easier with the <a href="https://aws-amplify.github.io/docs/js/storage">Storage category</a>. But you might also want to run some backend code to ensure the image conforms to your ToS (no sexual content). If you&#39;re used to serverless design, this is what you might want to do:</p><ol><li>Upload the image on S3 using Amplify: Storage.put(`${postId}_filename.jpg`, blob). Note that we are including the post ID so we can use it later.</li><li>Automatically trigger a lambda that will download the image and check its compliance. If it is not, we want to unpublish the post.</li></ol><p>Amplify does offer a <a href="https://github.com/aws-amplify/amplify-cli/tree/master/packages/amplify-category-function">function category</a>, but we&#39;re going to be much more limited than with a framework that focuses on event-driven functions, like <a href="https://serverless.com/framework/docs/providers/aws/">Serverless</a>. Let&#39;s create a podium-images service:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c292e32cfa0316814aab2300bd447b46/href">https://medium.com/media/c292e32cfa0316814aab2300bd447b46/href</a></iframe><ul><li>As you can see, we reused our functions from commons/amplify.js to resolve resource names within the Serverless context, in the serverless-helpers.js file.</li></ul><h4>The web app</h4><p>I don&#39;t have much advice concerning the web app, because here it is a standard React + Apollo application. But still, a few things I learned:</p><ul><li>Keep the default <a href="https://www.apollographql.com/docs/react/api/react-apollo/#optionsfetchpolicy">Apollo fetch policy</a> to cache-first. Sometimes it seems easier to change that parameter to fix update issues, but in general, it is just a workaround. Instead, learn how to use <a href="https://www.apollographql.com/docs/react/api/react-apollo/#datafetchmoreoptions">fetchMore</a> and <a href="https://www.apollographql.com/docs/react/api/react-apollo/#datarefetchvariables">refetch</a>.</li><li>Do not directly use auto-generated queries and mutations. To fit all your needs, you will probably have to set a high depth (like 8), meaning you will always query a huge quantity of useless information. Instead, you can just copy/paste generated queries/mutations and keep only the information you need.</li><li>Amplify UI components aren&#39;t amazing. The <a href="https://aws-amplify.github.io/docs/js/authentication#using-components-in-react--react-native">Authenticator</a> can be useful because it covers a lot of different actions (login, register, confirm identity, reset password, …). But if you feel you are doing too much customization, take some time to rebuild your own components from scratch.</li><li>Last but not least, you might wonder how to merge paginated items when you run fetchMore. For instance, if you run a getPost query, the comments will be paginated and you might want to get them all progressively. Here is a small helper that will help you to achieve that:</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a37b92eeacfc5aefe829ac82cdb1f60a/href">https://medium.com/media/a37b92eeacfc5aefe829ac82cdb1f60a/href</a></iframe><h4>CI/CD</h4><p>Now that we have a full-stack app working like gangbusters, let&#39;s have a look to the DevOps side.</p><p>First, I highly recommend following the <a href="https://aws-amplify.github.io/docs/cli-toolchain/quickstart#environments--teams">multi-environment workflow</a> provided by Amplify. In particular, it allows you to bootstrap a new environment on the fly to test a feature (super useful in your CI/CD). You can also develop with isolated environments for every developer.</p><p>Once you created at least 2 main environments (like staging on develop and production on master), let&#39;s automate deployment with <a href="https://aws.amazon.com/amplify/">Amplify Console</a>. Here again, no step-by-step tutorial but it&#39;s super easy to configure your repository, branches, and domains. You can define environment variables and override them in some environment, you can restrict access to some environment, …</p><p>However, you might wonder how to deploy your Serverless services. Here is my amplify.yml file (must be at the root of your repo):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7b6933590b9dad69620c715a66b5f0d9/href">https://medium.com/media/7b6933590b9dad69620c715a66b5f0d9/href</a></iframe><p>As you can see, we install the common lib and the CLI first. Then we run amplify push (using the <a href="https://gist.github.com/swaminator/7408de774e24ecf031d0d9928f1fbae5">amplifyPush script</a> available by default). Finally, we deploy our podium-images Serverless service.</p><p>Also, note that in the frontend postBuild step, we run podium setup-env —-yes. This applies all the configuration that we can&#39;t do natively with amplify (like creating an Admin group).</p><p>Indeed, we could run tests before deployment, and plenty of other cool things. But this seems like a good start.</p><h3>4. Final thoughts</h3><p>You might like Amplify or not, I believe it is showing the path of tomorrow&#39;s app development. Backend development will become less and less necessary with the adoption of Backend as a Service. I am not saying it will disappear, but it will focus only on your business.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FEPAVvaSqqcvsvMiwAm%2Ftwitter%2Fiframe&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FEPAVvaSqqcvsvMiwAm%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FEPAVvaSqqcvsvMiwAm%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="184" frameborder="0" scrolling="no"><a href="https://medium.com/media/5304df1840a573fb1faf49229d848649/href">https://medium.com/media/5304df1840a573fb1faf49229d848649/href</a></iframe><h4>My wishlist to the team</h4><ul><li>Make amplify console work with multiple AWS accounts, because we don&#39;t want to develop on our production account. The ideal would be to configure the CI/CD in an isolated account, and it would create resources in different accounts.</li><li>Allow to use docker in the CI, so we can build docker images. I know docker is not in the scope of Amplify, but this could prevent from using Amplify Console if you need to run your own image. For instance, our image validation algorithm could run in a Fargate container deployed with Serverless, and we would want to build and deploy in the CI/CD the image from the Dockerfile in the repo.</li><li>Improve the multi-frontend workflow. Ideally, the <strong>amplify/</strong> folder should be at the root of our repo, so we can use the amplify CLI everywhere. This is already tracked <a href="https://github.com/aws-amplify/amplify-cli/issues/582">here</a>.</li><li>Stop using unique IDs in CloudFormation stacks. We should have custom parameters only in parameters files. To go further, maybe the CloudFormation stacks should be generated on-the-fly when running a amplify push. But this will be achievable only when the framework will have reached a good stability level.</li><li>Improve documentation with real-world examples. I bet many people let their API unsecured because the documentation is unclear on many points. Creating custom resolvers is not a piece of cake either.</li></ul><p>That&#39;s all for today. I know many details are omitted, so if something isn&#39;t crystal clear feel free to ask in the comments. And please share your tricks as well.</p><p><strong>Happy coding 🤓</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c389f8e44c92" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Custom business insight using built-in AWS features]]></title>
            <link>https://medium.com/hackernoon/custom-business-insight-using-built-in-aws-features-c399272b44d?source=rss-7934ff29dd7c------2</link>
            <guid isPermaLink="false">https://medium.com/p/c399272b44d</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[dashboard]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <category><![CDATA[insights]]></category>
            <dc:creator><![CDATA[Christophe Bougère]]></dc:creator>
            <pubDate>Tue, 25 Jun 2019 05:44:13 GMT</pubDate>
            <atom:updated>2019-06-25T05:44:13.012Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<h4>Create dashboards easily using a Serverless framework</h4><p>For the purpose of this article, we will get into the shoes of the owner of a newly created online casino. Our first game is roulette, and its backend is a serverless API on AWS. We will discover how we can get some metrics about our game.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bkBtlzwZQZYdVrpLYk6ewQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@kaysha?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Kay</a> on <a href="https://unsplash.com/search/photos/gambling?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h3>The roulette API</h3><p>Our API offers an endpoint that process players bets and returns a random integer between 0 and 36. You can achieve this really easily with a <a href="https://serverless.com/">Serverless framework</a>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c8ef1fddaefdd8a2a860a8b7108211c4/href">https://medium.com/media/c8ef1fddaefdd8a2a860a8b7108211c4/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/eb1d1a5b1c7495eb5463ca7c88207d15/href">https://medium.com/media/eb1d1a5b1c7495eb5463ca7c88207d15/href</a></iframe><p>Our code is ready, and you can deploy it by running:</p><p>serverless deploy —-stage staging --region eu-west-3</p><p>Find the API URL in the deployment logs, and start playing by querying the API:</p><pre>export CASINO_URL=<a href="https://ls7sjcayle.execute-api.eu-west-3.amazonaws.com/staging/roulette">https://xxxxxxxxxx.execute-api.eu-west-3.amazonaws.com/staging/roulette</a></pre><pre>curl -X POST $CASINO_URL -d &#39;{&quot;bet&quot;:50,&quot;number&quot;: 13}&#39; -H &#39;Content-Type: application/json&#39;</pre><pre>{&quot;win&quot;:false,&quot;result&quot;:12}</pre><p>Great, we have a working casino and thanks to our advertising campaign, players are starting to use it. In order for us to drive our next decisions, we need some business insight. In particular, we would like to know how the bet value evolves with time in the day. Our engineering team is already busy building a poker game, so we need something really quick to set up.</p><p>Good news, everyone! You can create that kind of custom metric/dashboard on CloudWatch, and there is even a serverless plugin to automate it. You will see that we don&#39;t even need to update our JavaScript code.</p><h3>CloudWatch Metrics</h3><blockquote>Metrics are the fundamental concept in CloudWatch. <strong>A metric represents a time-ordered set of data points that are published to CloudWatch.</strong> Think of a metric as <strong>a variable to monitor</strong>, and the data points as representing <strong>the values of that variable over time</strong>. For example, the CPU usage of a particular EC2 instance is one metric provided by Amazon EC2. The data points themselves <strong>can come from any application or business activity</strong> from which you collect data. —<a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Metric"> documentation</a></blockquote><p>In our case, we will create a custom metric: the bet value over time. We are already logging the request body, so here are our custom metric properties:</p><ul><li><strong>Filter pattern:</strong> { $.bet = * }<br>The syntax uses JSON path to filter on logs that we want to use for our metric. In our case, we are interested in logs printing the request body (including the bet property).</li><li><strong>Metric value:</strong> $.bet<br>Still using a JSON path, defines what property you want to keep as the metric value: in our case the bet value.</li><li><strong>Log group name:</strong> /aws/lambda/online-casino-staging-roulette<br>This represents the source of data. We want our metric to show insight about our roulette game, but not about our new poker game. Also, don&#39;t forget to update the stage in the name.</li></ul><p>You can get more information about filter and pattern syntax <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html">here</a>.</p><h3>Serverless Plugin Metric</h3><p>Like for <a href="https://github.com/serverless/plugins">almost everything</a>, there is a serverless plugin to automate custom metrics creation. First, install it and we will append a few lines to our serverless.yml to create the metric:</p><p>npm install --save-dev serverless-plugin-metric</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/15c7dcaa3bd377bd9576860c1e99619b/href">https://medium.com/media/15c7dcaa3bd377bd9576860c1e99619b/href</a></iframe><h3>CloudWatch Dashboards</h3><p>Now that we configured our data source, let&#39;s create the dashboard. Append these lines into your serverless.yml:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d2bf492923d6c75b478ce497e4198a75/href">https://medium.com/media/d2bf492923d6c75b478ce497e4198a75/href</a></iframe><p>A few notes:</p><ul><li>You can add many widgets in a dashboard. Each widget is composed of a configuration and a list of metrics. The same configuration will be used for every metrics in the same widget.</li><li>The period property defines how often a value should be computed. Here we set 300 seconds (every 5 minutes).</li><li>The stat property let you choose how values should be aggregated. For our first graph, we want the average bet value, so we used Average. For the second, we want the number of bets and we used SampleCount. Lastly, we will display the total bet amount in the period using Sum. Other possibilities include Minimum, Maximum, and various percentiles.</li><li>Three different views are available:<br>- &quot;view&quot;: &quot;singleValue&quot; displays a single value<br>- &quot;view&quot;: &quot;timeSeries&quot; displays a line chart<br>- &quot;view&quot;: &quot;timeSeries&quot;, &quot;stacked&quot;: true displays a stacked line chart</li><li>You can create your dashboard in CloudWatch console, and export widgets JSON definitions in the source tab when editing.</li></ul><h3>Result</h3><p>After one hour of sending random values through the API, here is the result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lHjOsKRrM3-sBfIFR_XPkA.png" /><figcaption>Our wonderful dashboard</figcaption></figure><p>Pretty good for the small amount of work required, don&#39;t you think? Oh, and here is the pricing:</p><ul><li>$3/dashboard/month</li><li>$0.30/metric/month</li></ul><p>Which makes $3.30 per month for this dashboard.</p><h3>Bonus: Using widgets outside the console</h3><p>One last thing: we would like to share these graphs with the executive board of the casino. But giving them access to the CloudWatch console doesn&#39;t seem like the thing to do. Well, you are lucky because there is an API for that: <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricWidgetImage.html">GetMetricWidgetImage</a>. Instead of creating the dashboard in your serverless.yml, you can retrieve widget images from within a web app that the board has access to and you can keep the AWS console for engineers.</p><p>And voilà! Thank you for reading. Feel free to share your thoughts in the comments and <strong>happy dashboarding</strong>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c399272b44d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/custom-business-insight-using-built-in-aws-features-c399272b44d">Custom business insight using built-in AWS features</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A state of the art of decentralized web — Part 4]]></title>
            <link>https://medium.com/hackernoon/a-state-of-the-art-of-decentralized-web-part-4-212732f74894?source=rss-7934ff29dd7c------2</link>
            <guid isPermaLink="false">https://medium.com/p/212732f74894</guid>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[distributed-database]]></category>
            <category><![CDATA[decentralized-database]]></category>
            <category><![CDATA[decentralized-web]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Christophe Bougère]]></dc:creator>
            <pubDate>Thu, 28 Feb 2019 22:16:00 GMT</pubDate>
            <atom:updated>2019-02-28T22:16:00.733Z</atom:updated>
            <content:encoded><![CDATA[<h3>A state of the art of decentralized web — Part 4</h3><h4>4. Databases</h4><p>This is the fourth and last chapter of a series about web decentralization. As we have seen in the <a href="https://medium.com/@christophe.bougere/a-state-of-the-art-of-decentralized-web-part-3-7d901e09d06f">previous chapter</a>, the blockchain and smart contracts are great to store records for some specific use cases (like financial transactions), but they are not going to replace your whole database. Here we are going to focus on <strong>distributed and decentralized databases</strong>: What are the current candidates and how do they compare each other. Then we will conclude this series and step back on what we have learned.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/0*P34gqgaGo7aTwsmh.png" /><figcaption><a href="https://xkcd.com/327/">SQL, that good old friend!</a></figcaption></figure><h3>Series articles</h3><ol><li><a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-1-54f70fdb7355"><strong>Introduction</strong></a></li><li><a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-2-ea630917332a"><strong>File storage</strong></a></li><li><a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-3-7d901e09d06f"><strong>Blockchain and smart contracts</strong></a></li><li><strong><em>Databases</em></strong></li></ol><h3>Introduction</h3><p>For once, let&#39;s start with a theorem — the <strong>CAP theorem</strong>:</p><blockquote>The CAP theorem states that it is impossible for a distributed data store to simultaneously provide more than two out of the following three guarantees:<br>- <strong>Consistency</strong>: Every read receives the most recent write or an error<br>- <strong>Availability</strong>: Every request receives a (non-error) response — without the guarantee that it contains the most recent write<br>- <strong>Partition tolerance</strong>: The system continues to operate despite an arbitrary number of messages being dropped (or delayed) by the network between nodes<br> — <a href="https://en.wikipedia.org/wiki/CAP_theorem">Source</a></blockquote><p>This theorem is not perfect, but it is simple and gives you an idea of the design trade-off in a quick glance. It will help us to classify the different candidates.</p><p>Also, if you have no idea of what we are talking about, here is a good introduction:</p><p><a href="https://cointelegraph.com/explained/decentralized-and-distributed-databases-explained">Decentralized and Distributed Databases, Explained</a></p><p>All good? Now let&#39;s have a look at these 5 solutions:</p><ol><li><a href="#1ee5">GUN</a></li><li><a href="#4c1f">Fluence</a></li><li><a href="#eed1">Bluzelle</a></li><li><a href="#b57e">BigchainDB</a></li><li><a href="#d3aa">Ties.DB</a></li></ol><h3>1. GUN</h3><p><a href="https://gun.eco/">https://gun.eco</a></p><p>GUN is:</p><ul><li>A <strong>real-time</strong>, <strong>decentralized</strong>, <strong>offline first</strong>, <strong>graph</strong> database.</li><li><strong>AP</strong>: it guarantees <strong>availability</strong>, <strong>partition tolerance</strong>, but only <strong>eventual consistency</strong>. It makes it particularly interesting for real-time (chat, online game, live sensors visualization on IOT, …), but you should probably avoid using it when consistency is critical, like in banking. Read more <a href="https://gun.eco/docs/CAP-Theorem">here</a>.</li><li><strong>Storage-agnostic</strong>. Current adapters include File Storage, Local Storage (in the browser), IPFS and S3.</li><li>Coded in <strong>JavaScript</strong> and super easy to use (see <a href="https://gun.eco/docs/API">API docs</a>).</li><li><strong>Highly scalable</strong>: capable of doing 20M+ API ops/sec.</li><li>VC-backed and <a href="https://techcrunch.com/2018/05/23/gun-raises-more-than-1-5m-for-its-decentralized-database-system/">raised $1.5M in 2018</a>.</li></ul><p>The roadmap is available <a href="https://gun.eco/docs/Roadmap">here</a>.</p><p>Note that GUN can be used without any server (except for peers discovery), but the data will only persist in the browser local storage. So if you need data persistence, you should always keep a few nodes running.</p><blockquote>The Universe is not Strongly Consistent</blockquote><p>GUN made an interesting trade-off by focusing on real-time and offline capability. If you want to understand why <a href="https://gun.eco/distributed/matters.html">this presentation</a> explains well the pros and the cons in a simple and fun way. GUN may not suit for every application, but it definitely worth having a look to it!</p><h3>2. Fluence</h3><p><a href="https://fluence.network/">Fluence Cloudless | Decentralized computing, serverless dePIN platform</a></p><p>Fluence started as a decentralized database network and <a href="https://medium.com/fluence-network/fluence-new-direction-and-the-first-proof-of-concept-cced6e4656e9">recently shifted</a> to become a <strong>trust-less query layer over the Ethereum blockchain</strong>.</p><blockquote>Fluence creates an independent network with its own decentralized consensus for each query, security model and crypto-economics design. This network is optimized for retrieving and processing data from decentralized sources without sacrificing the security and performance. Databases, query languages and arbitrary algorithms can be created and deployed to the Fluence network and can run completely decentralized on the Fluence virtual machine.</blockquote><p>With Fluence, you will be able to query data from the blockchain and from cold storage (Ethereum Swarm, IPFS) using SQL, GraphQL or a custom query language.</p><p>Fluence is CP (<a href="https://www.reddit.com/r/Fluence/comments/99cvwb/fluence_first_ama/e53bm1k">see here</a>). The main network should be available soon (<a href="https://medium.com/fluence-network/fluence-new-direction-and-the-first-proof-of-concept-cced6e4656e9#041e">roadmap</a>).</p><p>In the same vein, you might be interested in <a href="https://github.com/ConsenSys/ethql"><strong>EthQL</strong></a>, a GraphQL interface to Ethereum (which I&#39;m not going to detail here because it relies on a centralized server).</p><h3>3. Bluzelle</h3><p><a href="http://bluzelle.com/">Bluzelle | The Database for Serverless and Decentralized Applications</a></p><p>Bluzelle provides a <strong>NoSQL key-value store decentralized database</strong> with a <strong>CRUD API</strong>. In a similar way than for <a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-2-ea630917332a">many file storage solutions</a>, nodes can be rewarded with <strong>BLZ tokens</strong> for hosting data. If you are a consumer, you use the same cryptocurrency to rent data storage in the swarm. Bluzelle is also <a href="https://github.com/bluzelle/swarmDB">open source</a>.</p><p>You can either <a href="https://devel-docs.bluzelle.com/client-development-guide/">create your own client</a> which communicates with the database through WebSockets or use the <a href="https://devel-docs.bluzelle.com/bluzelle-js/quickstart">existing Node.js client</a>. The <a href="https://bluzelle.github.io/api/#js-api">available operations</a> are quite similar to other NoSQL databases.</p><p>Bluzelle is still in beta and you can try it for free on the TestNet.</p><h3>4. BigchainDB</h3><p><a href="https://www.bigchaindb.com/">BigchainDB * * The blockchain database.</a></p><blockquote>With high throughput, low latency, powerful query functionality, decentralized control, immutable data storage and built-in asset support, BigchainDB is like a database with blockchain characteristics.</blockquote><p>BigchainDB introduces a new paradigm in databases. It is asset-focused (instead of table or document focused) and works with 2 kinds of transactions: CREATE and TRANSFER:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*jZCRB3wbBprby-ZlvoH2aQ.png" /><figcaption><a href="https://www.bigchaindb.com/developers/guide/key-concepts-of-bigchaindb/">Key concepts of BigchainDB</a></figcaption></figure><p>BigchainDB <a href="https://blog.bigchaindb.com/the-next-evolution-for-bigchaindb-software-a104185a8763">is now managed</a> by the <a href="https://ipdb.io/">IPDB foundation</a> which provides a TestNet and maintains the open source code base. A <a href="http://docs.bigchaindb.com/projects/server/en/latest/drivers-clients/">bunch of drivers</a> is available in different languages, as well as a <a href="https://github.com/bigchaindb/js-bigchaindb-graphql">GraphQL interface</a>.</p><p>It is now available in <a href="https://github.com/bigchaindb/bigchaindb/releases/tag/v2.0.0b9">v2.0.0b9</a>. Have a look at <a href="https://www.bigchaindb.com/developers/guide/">this introduction</a> if you want to go further.</p><h3>5. Ties.DB</h3><p><a href="https://tiesdb.com/database">First Decentralized Public Database for dApps</a></p><blockquote>A public, distributed and decentralized database with one common thread: trust. Enforced by integrated fault tolerance, incentive schemes and smart contracts.</blockquote><p>Like with Bluzelle, Ties.DB incentives nodes to process queries and host data by paying them in <strong>TIE tokens</strong>. For now, the network is integrated with Ethereum but they are planning to integrate with other blockchains in the future. Ties.DB is <a href="https://github.com/TiesNetwork/ties-docs/wiki/Where-do-decentralized-applications-store-their-data%3F#distributed-databases">AP</a>. You can give a try to the alpha version <a href="https://github.com/TiesNetwork/ties.db/releases/tag/0.1.2-ALPHA">here</a>.</p><p>You can either use the <a href="https://github.com/TiesNetwork/ties-docs/wiki/Usage-examples-%28NodeJS%29">Node.js client</a> (with some functions like putField) or fetch your data using <a href="https://github.com/TiesNetwork/ties-docs/wiki/TiQL">TiQL</a> (a SQL-like query language).</p><p>Bonus point for offering a <a href="https://github.com/TiesNetwork/ties.db-explorer/releases/tag/v0.1-alpha">user-friendly GUI</a> (something similar to phpMyAdmin for MySQL):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hLldtP1w5_E7cuaU.png" /><figcaption><a href="https://github.com/TiesNetwork/ties.db-explorer">Ties.DB explorer</a></figcaption></figure><h3>Conclusion</h3><p>We have had a look at the 3 most important bricks of the decentralized web:</p><ul><li>How to store files</li><li>How to get a consensus</li><li>How to store relational data</li></ul><p>So what&#39;s the current state of these technologies? Should you start your new project in a decentralized way?</p><p>Well in my opinion… it depends on your project. We are still at a very early stage of web decentralization, and it is a totally new paradigm compared to the current web.</p><p>If you are working on something experimental, then go for it and choose any of those technologies. You will learn A LOT. Even though some tools are still unstable for the moment, it should get better with time and you will be part of it!</p><p>If your project does not aim to generate incomes but you need to store a large amount of data, I would recommend decentralizing the storage layer on IPFS. You will save a lot of money and still have something extremely scalable. You could ask your community to host some IPFS nodes. In addition, if your project dies for some reason, users could still have access to their data on IPFS.</p><p>If your project needs real-time data but does not require strong consistency (like for a chat), you should consider using GUN. It is really nice to use, looks quite stable and is already being used by large dApps.</p><p>Concerning the blockchain, <a href="https://www.technologyreview.com/s/612507/ethereum-thinks-it-can-change-the-world-its-running-out-of-time-to-prove-it/">2018 has seen lots of really interesting projects die</a>. Most of the time the reason was not &quot;Is blockchain a good fit for that purpose?&quot; but &quot;Does it worth affording all the complexity the blockchain brings for that purpose?&quot;. I strongly think that blockchain will keep evolving and getting better in the next few years. However, it was probably a few years too early to start using it for every purpose. There are still some really good reasons to start a project using the blockchain, it&#39;s up to everyone to balance. But if you are willing to use blockchain just traceability purpose, something like BigchainDB can probably provide what you need in a simpler way.</p><p>Also, I would recommend you to keep an eye on a few projects which are developing an abstraction layer making it much easier for developers to integrate with decentralized technologies. They are still young but if they can deliver what they promise, building a dApp will be as easy as developing a normal app! Here are my 3 favorites:</p><ul><li><a href="https://dadi.cloud/">DADI</a>: The decentralized cloud provider. Their roadmap includes storage, compute layer, CDN, API, queues, well everything you need.</li><li><a href="https://blockstack.org/">Blockstack</a>: Like for DADI but limited to identity and storage.</li><li><a href="https://github.com/textileio/">Textile</a>: While building their <a href="https://www.textile.photos/">Photos dApp</a> on top of IPFS, they faced many issues like identity management, pinning files on IPFS, notifications, how to get it working on mobile, … All those things are available in their <a href="https://github.com/textileio/go-textile">go-textile framework</a>. They also provide a <a href="https://github.com/textileio/react-native-sdk">React Native SDK</a>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/0*RmZYqFoLMit_B3Uz.gif" /></figure><p>That&#39;s all for me, thanks for reading! Feel free to share your thoughts and experiments about web decentralization in the comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=212732f74894" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/a-state-of-the-art-of-decentralized-web-part-4-212732f74894">A state of the art of decentralized web — Part 4</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A state of the art of decentralized web — Part 3]]></title>
            <link>https://medium.com/hackernoon/a-state-of-the-art-of-decentralized-web-part-3-7d901e09d06f?source=rss-7934ff29dd7c------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d901e09d06f</guid>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[bitcoin]]></category>
            <category><![CDATA[decentralization]]></category>
            <category><![CDATA[eos]]></category>
            <category><![CDATA[ethereum]]></category>
            <dc:creator><![CDATA[Christophe Bougère]]></dc:creator>
            <pubDate>Wed, 02 Jan 2019 12:45:02 GMT</pubDate>
            <atom:updated>2019-03-01T08:07:07.745Z</atom:updated>
            <content:encoded><![CDATA[<h3>A State of the Art Decentralized Web — Part 3</h3><h4>3. <strong>Blockchain and smart contracts</strong></h4><p>This article is the third chapter of a series about web decentralization. Here we will focus on one of the most trending buzzwords of the year: <strong>the blockchain</strong>. While it has real and unquestionable potential, it also has significant limits. We will have a look at the main public blockchains (meaning available to everybody) and see when it can be a clever choice in a project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/0*ehuZeDiSybCagw_N.jpg" /><figcaption><a href="http://www.commitstrip.com/en/2016/06/20/coder-bingo/">Coder bingo, CommitStrip</a></figcaption></figure><h3>Series articles</h3><ol><li><a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-1-54f70fdb7355"><strong>Introduction</strong></a></li><li><a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-2-ea630917332a"><strong>File storage</strong></a></li><li><strong><em>Blockchain and smart contracts</em></strong></li><li><a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-4-212732f74894"><strong>Databases</strong></a></li></ol><p>Before going further, if you have no idea what blockchain is, watch this 2 minutes video to get the main principles:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FWiRFuHXHBhk%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DWiRFuHXHBhk&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FWiRFuHXHBhk%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b8d1fe4c57fae57dbd5d395b7cfe9ccc/href">https://medium.com/media/b8d1fe4c57fae57dbd5d395b7cfe9ccc/href</a></iframe><h3>A brief history of the blockchain</h3><h4>1995 — The first signs</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/0*sTzmq3XzGpwzzIy-.jpg" /></figure><p>For the anecdote, the first known blockchain has been created in 1995 and its ledger was… a newspaper. Stuart Haber and Scott Stornetta, two researchers in cryptography, decided to use the NOTICES &amp; LOST AND FOUND section of the <em>New York Times</em> to timestamp and certify data. What is the link with the blockchain everybody is speaking about nowadays? Well, it has the same characteristics: Information is encrypted and stored in a timestamped block (here an edition of the newspaper and will be distributed to a network (newspaper readers). This process makes it almost impossible to falsify data. Indeed, one could print a backdated version of the newspaper and change its content. But it would be extremely easy to verify if a version is fake or not by looking at many archives.</p><p>To be precise, the information stored in the newspaper is just a hash that allows verifying the integrity of the data that is stored &quot;off-chain&quot;. Otherwise, they would need much more space in the newspaper! You can read more about the process <a href="http://surety.com/digital-copyright-protection/prove-ownership">here</a>.</p><h4>2009 — And God created Bitcoin</h4><p>Bitcoin is a <strong>decentralized digital currency</strong> created in 2009 by someone under the name Satoshi Nakamoto. Its real identity is still unknown and subject to as many debates than the street artist Banksy. <a href="https://www.google.com/search?q=Satoshi+Nakamoto+identity">Search &quot;Satoshi Nakamoto identity&quot; on Google</a> to read about the best conspiracy theories. But for sure, he is probably extremely rich now as he mined about a million blocks at the very beginning (the exchange rate fluctuates a lot, but makes him multi-billionaire).</p><p>What makes Bitcoin revolutionary is the fact it is decentralized: it isn&#39;t owned by any bank or organization, and anyone can be part of the network. Transactions are verified by network nodes through cryptography and recorded in a public distributed ledger called a blockchain. You might wonder how we can get a consensus without a centralized trusted entity? To achieve that, miners (a.k.a. full nodes, they are rewarded to write data) have to provide a <strong>Proof of Work </strong>(PoW) in order to append new data into the blockchain. A PoW is the result of a complex calculation (time and energy consuming) that can be easily verified. To sum up, a single node will spend resources to append data to the blockchain, which integrity can be verified by any node. Bitcoin uses the <a href="https://en.wikipedia.org/wiki/Hashcash">HashCash</a> PoW and constantly adapt the calculation difficulty.</p><p>The two main drawbacks of the Proof of Work consensus algorithm are the following:</p><ul><li>Mining consumes a lot of energy due to the calculation complexity. Even though the currency is purely virtual, it consumes more energy than 159 countries (not combined though). It isn&#39;t sustainable nor desirable and is even more regrettable since the computations are totally useless (they aren&#39;t helping science for instance).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*yipypn2fCBHU6lB_.png" /></figure><ul><li>In reality, Bitcoin full nodes aren&#39;t that much decentralized because big mining pools have more chances to get their work rewarded over individual miners. Here is the last year pools distribution:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*VuXn0SNkkTIJ9NTXDwCXVw.png" /><figcaption><a href="https://btc.com/stats/pool?pool_mode=year">Mining pools distribution</a> — Last year 58% of the blocks were mined by only 4 mining pools</figcaption></figure><ul><li>It is vulnerable to 51% attacks. To make it simple, if a group was able to control the majority of the mining nodes, they would be able to compromise the whole system integrity. It is, of course, a very expensive attack, but given the current distribution, hackers would &quot;simply&quot; have to take control of a few organizations to make it possible.</li></ul><p>I won’t spend too much time on Bitcoin which is mostly limited to financial transactions (even though <a href="https://en.bitcoin.it/wiki/Script">it has a scripting language</a> and <a href="https://bitcoinmagazine.com/articles/yes-bitcoin-can-do-smart-contracts-and-particl-demonstrates-how/">it is technically possible to write smart contracts</a>).</p><h4>2015 — Ethereum extends the potential of the blockchain</h4><p>Vitalik Buterin, a programmer interested in Bitcoin, saw a much broader potential for the technology behind this crypto-currency. He and his team introduced a new blockchain aiming to create <strong>Decentralized Applications</strong>. Instead of sticking to a basic set of operations, like in Bitcoin&#39;s scripting language, Ethereum offers a Turing-complete language. To make it simple, that means you can theoretically solve any computational problem in a <strong>smart contract, </strong>assuming you have enough computation resources. Let&#39;s be clear, you will face computational limits much faster than in most languages because executing code in Ethereum has a cost (called <strong>gas</strong>) and will take more time. So it is up to the developer to keep a low complexity into the program (avoid nested loops or loops with a high number of iterations).</p><p>The smart contracts can be written in various languages, the most popular one being <a href="https://solidity.readthedocs.io/">Solidity</a>. They are then compiled in an Ethereum-specific bytecode and executed in the <strong>Ethereum Virtual Machine</strong> (EVM), similarly to Java. The concept of smart contract might still be abstract in your head. If you are curious about what does a smart contract can look like, <a href="https://github.com/pbrudny/learning-solidity-2018">here are a bunch of examples</a>.</p><p>When coding smart contracts, security is a primary question because you are dealing with money and a small leak could have huge impacts. One good practice is to reuse libraries that have been audited instead of writing everything by yourself. Take a look to <strong>OpenZeppelin</strong>, probably the most popular one:</p><p><a href="https://openzeppelin.org/">OpenZeppelin</a></p><p>Also, when dealing with Ethereum, the development pipeline can be cumbersome. <strong>Truffle Suite</strong> offers some tools that should make your life much easier: an IDE, a testing pipeline, a simple way to upgrade your contracts (with migrations) and a tool to create a new blockchain on-the-fly for your tests. It is the Swiss army knife of the Ethereum developer.</p><p><a href="https://truffleframework.com/">Truffle Suite | Sweet Tools for Smart Contracts</a></p><p>So, Ethereum looks amazing! You can code everything on it which makes it theoretically possible to decentralize any application! Let&#39;s now have a look at the <strong>main limitations</strong>:</p><ul><li><strong>Environmental disaster</strong>. As for Bitcoin, Ethereum uses a <strong>Proof of Work consensus algorithm</strong> and has the same environmental impact issue due to its electricity consumption. The good news is they are planning to move to a <strong>Proof of Stake</strong> (PoS) in 2019 (project name: Casper). What&#39;s this? Well, instead of resolving time and energy consuming calculations, the miners (here called validators) will have to deposit an amount of tokens that will be held as long as they are mining. They will then be given some blocks to validate based on two factors: <strong>the amount of the stake</strong> (the more you have to lose, the more you are considered trustworthy) and <strong>a degree of chance</strong> (otherwise only the richest would be able to validate blocks).</li><li><strong>Slowness</strong>. A transaction takes about <a href="https://etherscan.io/chart/blocktime">15 seconds to process</a>. It may be acceptable for some specific use cases, but not to decentralize existing applications which are already immediate. But PoS should solve this issue by reducing the time to 1 second.</li><li><strong>Scalability</strong>. For now, Ethereum can support up to 15 transactions per second (1.3 million transactions per day). That&#39;s really low compared its potential of use. In comparison, the VISA system can handle 24,000 transactions per seconds (theoretically). The good news here is that Ethereum is aiming to increase this limit to 1 million TPS by combining 2 technological solutions: <strong>sharding</strong> and a project named <strong>Plasma</strong>. The first one consists of splitting the network into fragments that would only process certain transactions (instead of processing them all), and the second one aims to conduct some transactions on a side chain which should free up the main chain. Casper is planned to arrive during 2019, while we will have to wait until 2020 or 2021 for sharding to show up.</li><li><strong>Cost</strong>. Every transaction has a price, called <strong>gas</strong>, which amount depends on the complexity of the contract. Users are used to a free and unlimited Internet, so having to pay for every transaction in your day-to-day life application may be a serious obstacle. The <a href="https://ethgasstation.info/">average gas price is almost $0.01</a>. In a chat application where every sent message would be a transaction, it would be quite expensive to use Ethereum!</li></ul><p>While Ethereum has a lot to address to allow their vision, they already have a plan for almost everything. But having to wait 1 or 2 years to see these problems solved, if everything goes well, put them in a risky situation.</p><h4>2018 — EOSIO, Ethereum on steroids?</h4><p>EOSIO (or simply EOS) has very similar goals to Ethereum. So instead of a full description, I am going to focus on what makes it different:</p><ul><li>Smart contracts are being executed within <a href="https://webassembly.org/"><strong>WebAssembly</strong></a> (abbreviated Wasm) virtual machine. In short, it is a new web standard allowing websites to run native code in the browser within a sandbox. Wasm could become a concurrent to Javascript for frontend development. In theory, you should be able to write EOS smart contracts in any of <a href="https://github.com/appcypher/awesome-wasm-langs/blob/master/README.md">those languages</a> (but for now C, C++, and Rust seem to be the most supported).</li><li>EOS uses <strong>Delegated Proof of Stake</strong> (DPoS) as the consensus algorithm. It&#39;s very similar to PoS, except that there is a limited number of block producers (21). Even though letting all the power to 21 actors seems very risky, it relies on the community that can vote to fire untrusted producers. It is a new paradigm because the consensus algorithm is a hybrid between democracy where people elect delegates to represent them and a deterministic algorithm between the 21 delegates (which real identities are known). Interesting fact: EOS has a <a href="https://github.com/EOSIO/eos/blob/37ce45c0b60d2710569c2d1a9229945cc0e855a9/governance/constitution.md">community-written constitution</a> (any change can be applied with 15/21 voters).</li><li>The last point allows EOS to get <strong>much better performances</strong>: up to 6000 TPS, and a transaction only takes about 1 second.</li><li>EOS proposes a <strong>fee-less economic model</strong> (no gas to pay to run a transaction), the block producers will be rewarded with 1% of the yearly inflation.</li><li>EOS also brings some <strong>higher level features</strong> like account/password recovery, human-readable usernames (instead of some long hexadecimal addresses) and a file storage solution. These features were usually offered by third-party services, having them integrated is an interesting experiment.</li></ul><p>One of the biggest criticisms to EOS is that it is <strong>too centralized</strong>. They are definitely doing an interesting trade-off here and it&#39;s too soon to know if this choice is viable or not. On the other hand, it will probably make <strong>EOS evolution much simpler</strong>. In many blockchains, some important evolutions require breaking changes which resulted in splitting the community into 2 parts: those who wanted to continue with the old system and those who accept the evolution. Having some delegates should ideally offer more agility. You can read more about the different kinds of forks <a href="https://vitalik.ca/general/2017/03/14/forks_and_markets.html">here</a>.</p><p><a href="https://block.one/">Block.one</a>, the company behind EOS has closed a $4 billion ICO to develop the ecosystem. The team behind the project is also renowned (CTO Daniel Larimer previously founded BitShares and Steemit). Their project is very ambitious as their main concurrent is already the de-facto solution for lots of blockchain projects. 2019 will definitely be an interesting year and both platforms will probably have to incentives users and developers to join them.</p><p>There are many other competitors like <a href="https://www.hyperledger.org/">HyperLedger</a> or <a href="https://lisk.io/">Lisk</a> and I may have forgotten some major ones. Feel free to share your thoughts in the comments.</p><h3>When should you use the blockchain?</h3><p>First of all, even if we are focusing on <strong>public (or permissionless) blockchains</strong> because they aim to decentralize the logic of an application, you should know that there is another kind: <strong>permissioned (or private) blockchains</strong>. They are useful in a project where not anyone should be able to contribute. The limit between these two kinds is not sharp, and there are also semi-pemissioned blockchains.</p><p>To sum up, here are the main benefits of a blockchain:</p><ul><li><strong>No central authority</strong>: When using a system, you don&#39;t need to trust a third party and everything is verifiable and transparent.</li><li><strong>Data immutability</strong>: Once it is written on the chain, it is theoretically impossible to change it (except in case of an attack).</li><li><strong>Single source of truth</strong>: Due to the structure of the blockchain, you can know in a reliable way whether a token belongs to a user or another.</li></ul><p>These features make blockchain particularly interesting to build traceability systems (in a supply chain for instance). But you can also use it to build the logic of a decentralized app: It helps you to get rid of the code that usually runs on your servers.</p><p>Here is a landscape of blockchain applications:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*WGkU27Rs8D5aO9uP" /><figcaption><a href="https://www.quora.com/What-are-the-coolest-blockchain-projects-as-of-January-2018/answer/Diana-Maltseva-1">Source</a></figcaption></figure><p>So when should you use it? You will find lots of decision trees helping you to decide whether you should or not use a blockchain in a project. I find this one particularly relevant:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TgMQCvCfg0e4Efz0.png" /><figcaption><a href="http://www3.weforum.org/docs/48423_Whether_Blockchain_WP.pdf">More details</a></figcaption></figure><p>However, I kinda disagree with the 5th question:</p><blockquote>Do you intend to store large amounts of non-transactional data as part of your solutions?</blockquote><p>If you read the <a href="https://medium.com/@christophe.bougere/a-state-of-the-art-of-decentralized-web-part-2-ea630917332a">second chapter about file storage</a>, you would know that solutions exist to store large amounts of data in a decentralized way. It is then possible to store the transactions on the blockchain and some encrypted files off-chain.</p><h3>Blockchain also brings a lot of complexity</h3><p>While blockchain has a huge potential, don&#39;t forget that it will complexify your application:</p><h4>Evolution will be complicated</h4><p>Upgrading your smart contracts won&#39;t be very easy. Having a good architecture and thinking about all the features you will implement from day one is important.</p><h4>Security as a priority</h4><p>As we have seen previously, a simple mistake can cost a lot and you won&#39;t be able to fix it as easily as with a centralized server. That&#39;s why security must be a number one priority, and you should audit your smart contracts before deploying them!</p><h4>More complex architecture</h4><p>For performance and cost issues, you won&#39;t be able to handle all the logic of your application within your smart contracts. You will also often have to gather data from 3rd party API. To achieve that kind of things with the blockchain, you will have to use <strong>Oracles</strong>:</p><blockquote>An oracle, in blockchain world, is an one-way digital agent that finds &amp; verifies real-world data &amp; cryptographically submits this information to the querying smart contract. An oracle is not the datasource itself <em>but the layer that interfaces with data-sources &amp; the blockchain</em>; it’s a translator for information provided by a 3rd-party API that’s to be added to a blockchain. With oracles, smart contracts have a pathway to interacting with data outside of the immediate blockchain environment.<br> — <a href="https://medium.com/@jesus_notchrist/blockchain-oracles-af3b216bed6b">Blockchain Oracles: What They Are &amp; Why They’re Necessary</a></blockquote><h3>A bunch of cool blockchain projects</h3><h4>Fizzy</h4><p><a href="https://fizzy.axa/">Fizzy</a> is a <strong>flight delay insurance</strong>. If you subscribe to it, it will automatically gather flight data from airlines and immediately compensate you in case your plane was delayed. I like this one because it brings transparency in insurance contracts which are often extremely complicated to decode. If you are interested in the technical implementation, check out <a href="https://medium.com/@humanGamepad/fizzy-axa-smart-contract-explaind-740df52894fd">this article</a>.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FxJZulZ_-CMI%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DxJZulZ_-CMI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FxJZulZ_-CMI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/5bb36bf87c1938f6204c6b9a1b3c693a/href">https://medium.com/media/5bb36bf87c1938f6204c6b9a1b3c693a/href</a></iframe><h4>FOAM</h4><p><a href="http://foam.space/">FOAM</a> is introducing a <strong>crypto-mapping</strong> solution based on a <strong>Proof of Location</strong>. We already have GPS, what&#39;s the point? Well, GPS is a way for a device to know it&#39;s own location, but there is no way for another actor to verify if the coordinates provided by a device are real or not (it could easily falsify them). FOAM relies on a network of beacons that will earn tokens by validating devices location. This could be particularly interesting when used with IOT.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FEJeMVh4tm1w%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DEJeMVh4tm1w&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FEJeMVh4tm1w%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/49f58365f0e7cda7282e2f8da22c5009/href">https://medium.com/media/49f58365f0e7cda7282e2f8da22c5009/href</a></iframe><h4>EXERGY</h4><p><a href="https://exergy.energy/">EXERGY</a> is a <strong>peer-to-peer energy network</strong>. It allows you to buy and sell energy in a secure and transparent way from your neighbors inside a microgrid. What&#39;s interesting is that decentralization can also apply to other networks than the internet.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FLxlMmFKOSUY%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DLxlMmFKOSUY&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FLxlMmFKOSUY%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/44e3ce997e6d4aa4b8282c945b6a0f26/href">https://medium.com/media/44e3ce997e6d4aa4b8282c945b6a0f26/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d901e09d06f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/a-state-of-the-art-of-decentralized-web-part-3-7d901e09d06f">A state of the art of decentralized web — Part 3</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A state of the art of decentralized web — Part 2]]></title>
            <link>https://medium.com/hackernoon/a-state-of-the-art-of-decentralized-web-part-2-ea630917332a?source=rss-7934ff29dd7c------2</link>
            <guid isPermaLink="false">https://medium.com/p/ea630917332a</guid>
            <category><![CDATA[peer-to-peer]]></category>
            <category><![CDATA[decentralization]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[ethereum]]></category>
            <category><![CDATA[blockchain]]></category>
            <dc:creator><![CDATA[Christophe Bougère]]></dc:creator>
            <pubDate>Mon, 03 Dec 2018 09:16:26 GMT</pubDate>
            <atom:updated>2019-03-01T08:07:32.535Z</atom:updated>
            <content:encoded><![CDATA[<h3>A state of the art of decentralized web — Part 2</h3><h4>2. File storage</h4><p>This article is the second chapter of a series about web decentralization. Here we are going to focus on how to store files. I believe storage is the most advanced field when speaking about decentralization because it relies on mature and robust technologies: peer-to-peer and cryptography. If you are an application developer (web or mobile), you can already replace your storage layer by a decentralized one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*l11VtUM3MJlc09H71J-wUQ.png" /><figcaption>And wherever you store your data, always anticipate worst-case scenarios 😉</figcaption></figure><h3>Series articles</h3><ol><li><a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-1-54f70fdb7355"><strong>Introduction</strong></a></li><li><strong><em>File storage</em></strong></li><li><a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-3-7d901e09d06f"><strong>Blockchain and smart contracts</strong></a></li><li><a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-4-212732f74894"><strong>Databases</strong></a></li></ol><h3>Why decentralizing file storage?</h3><p>In typical cloud architectures, we would use cloud services like <a href="https://aws.amazon.com/s3/">Amazon S3</a>, <a href="https://azure.microsoft.com/en-us/services/storage/files/">Azure Files</a> or <a href="https://cloud.google.com/storage/">Google Cloud Storage</a>. On a security and reliability level, no worry this is a really good choice: files can be server-side encrypted (so the cloud provider don’t have access to your files), you can configure backups and cross-region replication. You also have an infinite scalability and a good set of APIs to access your file from any program.</p><p>However, I see two issues with these services:</p><ul><li>They are centralized, which means that in case of an outage, the whole system is impacted. Sometimes, <a href="https://www.theverge.com/2017/3/2/14792442/amazon-s3-outage-cause-typo-internet-server">a typo can turn the web down for four hours</a>.</li><li>They have a significant price (generally in $/GB/Month), which makes it complicated for a non-profit organization to propose an application using a cloud storage service.</li></ul><p>On the other hand, your brand new computer might have a 2TB hard drive while you are only using a tenth of it. Then, why not use this free space to support a collaborative file storage system?</p><p>Let&#39;s have a look at the main candidates offering such a solution:</p><ol><li><a href="#5101">InterPlanetary File System</a></li><li><a href="#7732">Swarm</a></li><li><a href="#0175">Storj</a></li><li><a href="#a45d">DADI</a></li><li><a href="#bbce">Dat Project</a></li><li><a href="#ecb9">Sia</a></li><li><a href="#25bf">Blockstack</a></li></ol><p>That&#39;s a lot of options and they sometimes offer similar features so I will focus on what makes them unique. Ready?</p><h3>1. InterPlanetary File System</h3><p><a href="http://ipfs.io/">IPFS Powers the Distributed Web</a></p><p>Behind this <a href="https://discuss.ipfs.io/t/why-the-name-ipfs/307">enigmatic name</a> stands the most mature system as of now in my opinion.</p><p><strong>Features:</strong></p><ul><li>Both a peer-to-peer protocol and a network. It is a combination of Kademlia, BitTorrent, and Git.</li><li>IPFS can be used standalone, so you don&#39;t need to use a specific stack in order to use IPFS.</li><li>IPFS provides a naming system (IPNS), a CDN and can directly serve websites.</li><li>Provides no guarantees about data availability or redundancy. <a href="https://filecoin.io/">FileCoin</a>, one of IPFS sister-projects, proposes to solve that by introducing an incentive system based on blockchain (users that store data will earn FileCoin tokens). We can see this as an “Enterprise edition” of IPFS with an on-chain SLA.</li><li><a href="https://github.com/ipfs/ipfs">Open source</a>. Some modules of IPFS already became independent projects (<a href="https://libp2p.io/">libp2p</a>, <a href="https://ipld.io/">IPLD</a>).</li></ul><p><strong>Pricing:</strong> IPFS is free to use. Users contribute to the network by serving files to their peers. FileCoin price will depend on the market (the more people propose disk storage, the cheaper it will be).</p><p><strong>Status:</strong> Even though it is still in alpha version since 2015, the protocol seems pretty stable and fast. Skeptical? Then watch this <a href="https://d.tube/#!/v/megamovie/a82s7pkc">funny cats compilation</a> on <a href="https://d.tube/">DTube</a>, a YouTube-like based on IPFS for storage and <a href="https://steemit.com/">Steemit</a> to manage content and reward authors. FileCoin, however, is much younger and not ready to use yet (see the <a href="https://filecoin.io/blog/update-2018-q1-q2/">last update of the roadmap</a>).</p><p><strong>Going further:</strong> For a good theoretical and practical introduction to IPFS, I recommend you this article:</p><p><a href="https://medium.com/coinmonks/a-hands-on-introduction-to-ipfs-ee65b594937">A Hands-on Introduction to IPFS</a></p><p>And if you are interested to understand how IPFS works under the hood (with its name service and various protocols), then watch Juan Benet (founder of IPFS and Filecoin) speaking about the different mechanisms they implemented:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FBqs_LzBjQyk%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DBqs_LzBjQyk&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FBqs_LzBjQyk%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ec7d091f3c08b98f5841d52870c6852c/href">https://medium.com/media/ec7d091f3c08b98f5841d52870c6852c/href</a></iframe><h3>2. Swarm</h3><p><a href="https://ethersphere.github.io/swarm-home/">Swarm</a></p><p><strong>Features:</strong></p><p>On a functional and technical point of view, Swarm is very similar to IPFS with two major differences:</p><ul><li>Swarm is deeply integrated with the Ethereum blockchain. This can be seen as an advantage if you are already using Ethereum in your project, but can also complexify your stack in the other case.</li><li>Swarm has a built-in incentive system to incite people hosting data. Functionally, using Swarm is equivalent to using IPFS along with FileCoin.</li></ul><p>To understand more the similarities and differences between the two projects, <a href="https://github.com/ethersphere/go-ethereum/wiki/IPFS-&amp;-SWARM">a good summary is available here</a>. Ethereum choice of starting its own project has been the subject of many debates, but this kind of concurrence can also make them produce better results (I hope so). Swarm is <a href="https://github.com/ethereum/go-ethereum/tree/master/swarm">open source</a> (part of Ethereum code base).</p><p><strong>Pricing:</strong> Once again, the price will depend on the market</p><p><strong>Status:</strong> Swarm is still experimental. <a href="https://blog.ethereum.org/2018/06/21/announcing-swarm-proof-of-concept-release-3/">The Proof-of-Concept 3 has been released</a>, and a testnet is available. The roadmap is available <a href="https://github.com/orgs/ethersphere/projects/5">here</a>. Keep a close eye to this project, but don’t use it in production yet.</p><p><strong>Going further:</strong> To get started with Swarm, <a href="https://swarm-guide.readthedocs.io/en/latest/introduction.html">read the official documentation</a>.</p><h3>3. Storj</h3><p><a href="https://storj.io/">Globally Distributed Cloud Object Storage</a></p><p><strong>Features:</strong></p><ul><li>S3 compatible. This means it will have the same concept of Buckets and Objects, but no folders like in a standard file system. This will make it super easy to decentralize the storage layer of an existing application using Amazon S3!</li><li>Built-in end-to-end encryption.</li><li><a href="https://github.com/storj/storj">Open source</a>.</li></ul><p><strong>Pricing:</strong></p><ul><li>Storage: $0.015 per GB per month</li><li>Bandwidth: $0.05 per GB downloaded</li></ul><p>Storj proposes a <a href="https://storj.io/partners">partnership program for open source</a>. In short, every open source software that uses Storj will be rewarded by 10% of the benefices.</p><p><strong>Status:</strong> The Storj network will be launched in early 2019. See the roadmap <a href="https://storjlabs.aha.io/published/01ee405b4bd8d14208c5256d70d73a38?page=1">here</a>.</p><p><strong>Going further:</strong> <a href="https://storj.io/white-paper">Whitepaper</a></p><h3>4. DADI</h3><p><a href="https://dadi.cloud/">The local global network</a></p><p>A good description of DADI would be &quot;<strong>A decentralized cloud provider</strong>&quot;. DADI already propose a CDN and a cloud storage service is coming really soon. Let&#39;s focus on the CDN here.</p><p><strong>Features:</strong></p><ul><li>Serves static files (assets, websites, …).</li><li>Data are cached at the edge.</li><li>Full support for caching, header control, image manipulation, image compression, and image format conversion.</li><li><a href="https://github.com/dadi/cdn">Open source</a>.</li></ul><p><strong>Pricing:</strong> <a href="https://docs.google.com/document/d/1lEIfZZtI2QITG0AHxZsAdWsKMwmZaLBhOEVq6wZ4B2w/edit#heading=h.wjvafd9hyvex">This document</a> details the pricing which depends on the volume of requests. DADI is based on proof of stake, so to become a DADI host you must stake 5,000 $DADI tokens (equivalent to 216$ as of now).</p><p><strong>Status:</strong> CDN is already available, and the cloud storage should be available beginning of 2019. See their extremely ambitious roadmap <a href="https://dadi.cloud/en/marketplace/">here</a>. They recently raised $29M with an ICO.</p><p><strong>Going further:</strong> A bunch of tutorials is available <a href="https://dadi.cloud/en/tutorials/">here</a> (look for the CDN tag).</p><h3>5. Dat Project</h3><p><a href="https://datproject.org/">Dat Project - A Community-Driven Web Protocol</a></p><p><strong>Features:</strong> What’s Dat? Dat is a peer-to-peer protocol. It offers a CLI, a Node.js API, and a desktop application. Hosted files can be accessed through a web browser at this address: https://datproject.org/{dat-key}. Nothing new? Well, the particularity here is that you share a folder instead of individual files, like a repository with git (dat clone, dat create, dat share), and you can also update your files. With this in mind, we can understand that it is not the best fit to build a decentralized web application (compared to previous solutions). However, it can have a real added value for researchers and scientists willing to publish data (experiment results, training database, …) to the community and apply change easily. Sources are available <a href="https://github.com/datproject/dat">here</a>.</p><p><strong>Pricing:</strong> Free to use. However, you are responsible for keeping the data alive (peers only share the data they have cloned, like in a classic BitTorrent system).</p><p><strong>Status:</strong> Dat is ready to use. See <a href="https://dat.land/">dat.land</a> for examples of usage.</p><p><strong>Going further:</strong> Dat documentation and tutorials are available <a href="https://docs.datproject.org/">here</a>.</p><h3>6. Sia</h3><p><a href="https://sia.tech/">Sia - Decentralized data storage</a></p><p><strong>Features:</strong> Once again, Sia is blockchain based and remunerates hosts in SiaCoin. When uploading a file, it is guaranteed to stay alive for a period of 3 months, and the contract can be renewed automatically. It is open source too (<a href="https://gitlab.com/NebulousLabs/Sia">recently moved to GitLab</a>). Even though Sia offers an API, I believe its main use case is to store users personal data and get rid of services like Dropbox or Google Drive.</p><p><strong>Pricing:</strong> About $0.002 per GB per month.</p><p><strong>Status:</strong> Sia is ready to use to store non-critical files. In the next year, they plan to improve speed, support warm storage and add a CDN feature.</p><p><strong>Going further:</strong> Get started using the <a href="https://blog.sia.tech/api-quickstart-guide-f1d160c05235">API (CLI or JS)</a> or the <a href="https://blog.sia.tech/getting-started-with-private-decentralized-cloud-storage-c9565dc8c854">GUI</a>.</p><h3>7. Blockstack</h3><p><a href="https://blockstack.org/">Blockstack</a></p><p>Blokstack is more than storage. It is a toolkit that aims to provide everything to create a new decentralized internet. For now, it is composed of a naming system(<a href="https://docs.blockstack.org/core/naming/introduction.html">BNS</a>), a file storage system (<a href="https://github.com/blockstack/gaia">Gaia</a>), an authentication system (you can use the same Blockstack id on every app built on Blokstack), and a <a href="https://github.com/blockstack/blockstack-browser">web browser</a>. Let&#39;s focus on the storage layer.</p><p><strong>Features:</strong></p><ul><li><a href="https://github.com/blockstack/gaia/tree/master/hub/src/server/drivers">Flexible drivers system</a> (compatible with current cloud storage providers).</li><li>When used with the Blockstack browser, users can choose where to store its data by running a Gaia hub and using a driver (either on a local disk or on a cloud provider). What&#39;s interesting is that the storage solution you choose will be used by all Blockstack applications that need to store data. So you are in control of all your data, you can backup them, delete them, host them or use a third party. <strong>You just own your data!</strong></li><li><a href="https://github.com/blockstack/gaia">Open source</a>.</li></ul><p><strong>Pricing:</strong> The default storage provider is free. If you want to store all your data on S3, then you will pay S3 fees.<br>If you are a developer, Blockstack just launched its App Mining program: get paid to develop apps on Blockstack. Starting in December, $100,000 in aggregate will be distributed every month (20% for the 1st ranked app, 20% of the remaining 80% for the 2nd, …). That&#39;s a lot, and you can use this money to incentive users to use your app too (redistribute part of the money). More information on <a href="https://app.co/mining">https://app.co/mining</a>.</p><p><strong>Status:</strong> Ready to use. At the beginning of 2019, Gaia will be released as an independent project so we will be able to use the storage layer in any project.</p><p><strong>Going further:</strong> Have a look at <a href="https://github.com/blockstack/gaia/blob/master/README.md">Gaia README</a> and <a href="http://blockstack.github.io/blockstack.js/">blockstack.js documentation</a>.</p><h3>What’s next?</h3><p>Well, you now have an amazing choice of technologies to store your files depending on your application needs. <a href="https://medium.com/@christophe.bougere/a-state-of-the-art-of-decentralized-web-part-3-7d901e09d06f">Next chapter</a> is about decentralizing the logic of your application using the blockchain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/853/0*_x3PmIQenLw--m3Y.gif" /><figcaption>I hope you are ready because it’s gonna be tricky 🤓</figcaption></figure><p>Give some 👏👏👏 if you appreciated this one and are impatient for the continuation. These articles reflect my comprehension of the subject, and discussions are much welcomed in the comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea630917332a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/a-state-of-the-art-of-decentralized-web-part-2-ea630917332a">A state of the art of decentralized web — Part 2</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A state of the art of decentralized web — Part 1]]></title>
            <link>https://medium.com/hackernoon/a-state-of-the-art-of-decentralized-web-part-1-54f70fdb7355?source=rss-7934ff29dd7c------2</link>
            <guid isPermaLink="false">https://medium.com/p/54f70fdb7355</guid>
            <category><![CDATA[decentralization]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[peer-to-peer]]></category>
            <category><![CDATA[ethereum]]></category>
            <dc:creator><![CDATA[Christophe Bougère]]></dc:creator>
            <pubDate>Mon, 03 Dec 2018 09:14:35 GMT</pubDate>
            <atom:updated>2019-03-01T08:07:56.754Z</atom:updated>
            <content:encoded><![CDATA[<h3>A state of the art of decentralized web — Part 1</h3><h4>1. Introduction</h4><p>I am a full-stack web developer with a good expertise in JavaScript and serverless architectures on AWS (take a look to <a href="https://medium.com/@christophe.bougere">my other articles</a> if you are interested in this topic). I have the feeling that decentralization might be the next big thing in web development. By the way, one very popular ÐApp library is called <a href="https://github.com/ethereum/web3.js/">web3.js</a>, as being the successor of what we currently know as <a href="https://en.wikipedia.org/wiki/Web_2.0">Web 2.0</a>. In a way, decentralization could enable a true web without servers where every client would contribute to the web (like a peer-to-peer network).</p><blockquote>Decentralized is the new Serverless</blockquote><p>In this series, I will try to demystify this whole new ecosystem. I will share with you my findings on a pretty high level. Don&#39;t see this as a complete course or an exhaustive list of technologies, but instead as a good introduction which will help you to understand why and how this architecture could revolutionize the web we currently know.</p><h3>Series articles</h3><ol><li><strong><em>Introduction</em><br></strong>Main concepts and stakes of a decentralized architecture. What are the biggest challenges that we will face?</li><li><a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-2-ea630917332a"><strong>File storage</strong></a><br>How to store files (source code, pictures, …) the decentralized way. Data storage seems to be the most mature area as of now and we will have a look at various solutions.</li><li><a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-3-7d901e09d06f"><strong>Blockchain and smart contracts</strong></a><br>The most trending buzzwords right now. While their potential is unquestionable for some use cases, let&#39;s learn when you should or shouldn&#39;t use blockchain depending on your needs. We will focus on Ethereum and EOS.</li><li><a href="https://hackernoon.com/a-state-of-the-art-of-decentralized-web-part-4-212732f74894"><strong>Databases</strong></a><br>Once you have file storage and processing, the next step for many web apps is to set up a database. What are the distributed and decentralized options?</li></ol><h3>What decentralized means?</h3><p>First things first, let’s make sure we are using the same vocabulary. If you already know the difference, feel free to skip to the next section. Otherwise, here are 3 network topologies and how each of them would react in case of a specific node face an attack:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/852/0*Mxv6E7jAXi84ZQee.gif" /><figcaption>Centralized networks are doing pretty bad, don&#39;t you think?</figcaption></figure><h4>Centralized / Decentralized</h4><p>If the system is <strong>controlled by a single entity</strong>, then it is <strong>centralized</strong>. In the case of a web app, the clients (web browser, mobile app) communicate with a server to render the app. If the server shuts down, then all clients lose their access to the app. This is what we call a <strong>Single Point Of Failure</strong> (SPOF). The <strong>scalability of the system is also very limited</strong>: you can increase the resources of this node by adding more RAM or hard drives (vertical scaling), but you can&#39;t add more nodes (horizontal scaling). On the other hand, this kind of system is <strong>easy to set up, maintain and evolve</strong> because you only need to deploy your system on a single node.</p><p>If the system is <strong>controlled by many entities</strong>, then it is considered as <strong>decentralized</strong>. It is <strong>more resistant</strong>: if one master node is turned off, only the client nodes connected to this master node will stop working. The system is <strong>still vulnerable to failures and attacks</strong>, but the impact is proportional to the number of master nodes impacted. It is also <strong>more scalable</strong> because you can scale horizontally (add more nodes) and vertically.</p><h4>Distributed</h4><p>When it comes to <strong>distributed</strong> systems, <strong>all nodes are equals and have the same control over the system</strong>. One famous example is BitTorrent, a peer-to-peer file sharing protocol. Instead of connecting to a server to retrieve data, you retrieve it from your peers, and you also provide them the data they want if you have it. A distributed system is <strong>highly scalable and robust</strong>. On the other hand, an evolution of the system can be complicated because it involves many entities.</p><p>I am keeping it simple for this article, but the subject can be detailed much more. For instance, Vitalik Buterin, co-founder of Ethereum, distinguish 3 kinds of (de)centralization: architectural, political and logical:</p><p><a href="https://medium.com/@VitalikButerin/the-meaning-of-decentralization-a0c92b76a274">The Meaning of Decentralization</a></p><h3>Why does it matter?</h3><h4>Transparency and decentralized governance</h4><p>This is, in my opinion, the true reason why decentralized systems are trendy right now. With big data, organizations own an incredible quantity of data about you and your personal life. Even if in the case of Google, Amazon or Facebook it is mostly used for advertisement purposes (which can already raise ethical questions), this could quickly go worse if they find an interest into politics, or if some third parties get access to this information (see what happened with Cambridge Analytica). We also know that these companies could be forced to install backdoors in their apps. Data is probably the most valuable thing right now and users are losing trust in these companies and looking for transparency.</p><p>The Internet is also already censored in many countries, and decentralization could be a way to bypass censorship by moving the control of the Internet and of users data in the hands of the users themselves and bypass governments and companies.</p><p>At last, organization structure can be decentralized. Decentralized Autonomous Organization (DAO) offers a transparent solution for governance. Of course, it still requires humans in the loop for some use cases. To go further, the following article explains what kind of organizations could/should be decentralized:</p><p><a href="https://hackernoon.com/blockchain-governance-how-decentralized-do-we-need-to-be-799a7988ce84">Blockchain Governance: How decentralized do we need to be?</a></p><h4>Security</h4><p>Even though the current internet infrastructures are quite robust, it is made of Single Point Of Failure (SPOF) by design. DNS provider is one, and when Dyn went through a DDOS attack in 2016, it was a big part of the web that went down:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FnX_FdkrAnRHUCNGHe2kjw.png" /><figcaption><a href="https://en.wikipedia.org/wiki/2016_Dyn_cyberattack#Affected_services">Dyn attack affected services</a></figcaption></figure><p>Decentralized and distributed systems still need to take care of security (and probably even more), but if the system is built in a strong and reliable way, it will resist better to an attack than a centralized system because it is not enough to take control of one node, in many cases you would need to take control of the majority of the nodes (<a href="https://en.bitcoin.it/wiki/Majority_attack">learn more about 51% attacks</a>).</p><h4>Scalability</h4><p>Last but not least, scalability is the key now. Companies migrated to the cloud to be able to handle the charge in any circumstance, and are starting to migrate to serverless architecture to only pay for what they are really using. Decentralized and distributed architectures are already able to do that by design. The more users a system has, the more compute resources and storage capacity it will dispose of. The scaling capacity of such systems is natural.</p><h3>A few promising examples</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/100/1*UwEacpQmLFQzY4oWsurVBQ.png" /><figcaption><a href="http://textile.photos/"><strong>Textile Photos</strong></a></figcaption></figure><p>Textile Photos proposes a decentralized replacement for cloud photo storage and sharing services like Google Photo and Apple Photos. Images are stored on IPFS which will be the topic of the next chapter. Textile Photos is still in beta but the company is already planning to expand to other use cases (we can imagine a Dropbox/Drive replacement working the same way).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/100/1*9FPq_5QsE8gx13dK5ZSZkA.jpeg" /><figcaption><a href="https://eva.coop/"><strong>Eva</strong></a></figcaption></figure><p>Eva aims to disrupt Uber and other ridesharing applications (that themselves disrupted the Taxi business a few years ago) by directly connecting drivers and passengers through the blockchain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/100/1*QWiYx80io3yyc-palrwEPA.png" /><figcaption><a href="https://openbazaar.org/"><strong>OpenBazaar</strong></a></figcaption></figure><blockquote>A free online marketplace. No platform fees. No restrictions. Earn cryptocurrency</blockquote><p>You get it, OpenBazaar is a decentralized e-commerce platform where anyone can sell and buy. What is interesting though is how they secure the money during the transaction and how they resolve conflicts:</p><blockquote>Scammers may try to take advantage of both buyers and sellers online, but OpenBazaar offers a unique feature of Bitcoin that helps prevent fraud: Multisignature escrow. In this e-commerce marketplace buyers and sellers can choose to agree to a mutually trusted third party OpenBazaar user before they start a trade, and then the buyer sends their bitcoin to an escrow account. Those Bitcoin can only be released when two out of the three parties agree where it will be sent. Normally the buyer and seller are the two parties in agreement, but if there is a dispute the third party will come settle the dispute. These third parties offering dispute resolution are selected in an open marketplace.</blockquote><blockquote>— <a href="https://openbazaar.org/features/">OpenBazaar website</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/100/1*_wjdvZq2SdSJtrP3KohYjQ.png" /><figcaption><a href="https://everipedia.org/"><strong>Everipedia</strong></a></figcaption></figure><blockquote>Everipedia is a wiki-based online encyclopedia. […] The company uses EOS blockchain technology and a cryptocurrency token called IQ to encourage generating information. One of the goals of the company is to stop certain countries from blocking its content by the integration of the blockchain model. Everipedia launched on the EOS blockchain on August 9, 2018.</blockquote><blockquote>— <a href="https://en.wikipedia.org/wiki/Everipedia">Read Everipedia article on Wikipedia</a> (I’m surely going to Hell for this 😈)</blockquote><p>Note that only the governance is stored on-chain, while articles are stored on IPFS. The EOS blockchain is a serious concurrent to Ethereum that claims to solve the challenges the latter faces (it is faster, already based on a Proof Of Stake and does not require gas to proceed transactions). It is still much younger, but very well funded. Only time will tell what cryptocurrency wins the smart-contract war.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/100/1*Z03LhbPyE9CMIso6tF4HtQ.jpeg" /><figcaption><a href="https://www.stateofthedapps.com/"><strong>State of the ÐApps</strong></a></figcaption></figure><p>Finally, is you want to discover new ÐApps (stands for <em>Decentralized Application</em> but with a weird D) or just check if your idea already exists, this website reference about 1800 of them.</p><h3>Challenges</h3><h4>Make it simple for the end user</h4><p>As we will see, using the blockchain requires to have a wallet filled with various cryptocurrencies, and often to pay transaction fees (also called gas). Depending on your project and its users, it can be seen as a pointless setup while some centralized solutions already exist and work well. This is, in my opinion, the biggest challenge because for people to embrace this new paradigm, their user experience should be as easy as before. <a href="https://medium.com/textileio/wip-textile-threads-whitepaper-just-kidding-6ce3a6624338#1705">It has to &quot;just work&quot;</a>.</p><h4>Package it for mobile</h4><p>This young ecosystem is quickly evolving, but as of now many technologies are designed to work on a desktop or on a browser (sometimes with some extensions), and integrating them in mobile applications can be cumbersome. Mobile already represents a huge part of the web use and is vital to make decentralization happen.</p><h4>Make it fast and reliable</h4><p>People have already been using peer-to-peer for a while because it was answering a specific need (freely sharing large files). However, browsing websites using Tor is much slower than directly browsing the web through your favorite browser. This issue has to be solved in the design of the technologies.</p><h4><strong>No more backend</strong></h4><p>This point might be my personal utopia. Even though I love lots of backend technologies, I wish they could run a decentralized way, on users devices. This might not be possible for everything in a near future, but I believe this is the path to take whenever it is possible. In January, <a href="http://dadi.cloud/">DADI cloud</a> raised $29 million to build a decentralized cloud system. They have <a href="https://dadi.cloud/en/marketplace/">a very ambitious roadmap</a> that would allow getting rid of cloud providers for many use cases.</p><h4><strong>Reduce the blockchain environmental footprint</strong></h4><p><a href="https://www.forbes.com/sites/shermanlee/2018/04/19/bitcoins-energy-consumption-can-power-an-entire-country-but-eos-is-trying-to-fix-that/#22fc86201bc8">Alone, the BitCoin Proof of Work (PoW) system is consuming as much energy as Switzerland</a>. This is obviously not conceivable neither desirable to replace the existing web by such solution. While very critical, this challenge is already being addressed and we will see how in Chapter 3.</p><h4><strong>Companies must embrace decentralization</strong></h4><p>Going decentralized is a loss of control over the data and the logic of an application. While open source software is a perfect fit for that, making businesses adopt this new paradigm might be complicated. In some cases, a decentralized concurrent project could force companies to take the same path, but this is going to be a long way.</p><h4><strong>Find the right &quot;business model&quot;</strong></h4><p>Today, most of the web tools (cloud providers, third-party APIs, …) are making money to pay its employees and servers with subscriptions or a “pay as you use” model. With decentralization, I see 2 options:</p><ul><li>Everything could be free, and people would contribute to the web by storing data and sharing their processing capability on their devices. Foundations of the web would be open source and companies would contribute to it. Once again, this might be a utopia and people might try to curb their contribution to save resources and bandwidth so it may not be viable on a large scale.</li><li>Usage of resources could be paid via a cryptocurrency. Users would gain tokens when they share their resources and would spend them to use the web. That method would incentive people to contribute and make sure to keep a good reliability of the system.</li></ul><h3>Ready to start?</h3><p>Well, I hope you are ready and as excited as me to discover this new world!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/0*QU288ho_auSeSFmP.gif" /><figcaption><a href="https://www.wired.com/2017/06/pied-pipers-new-internet-isnt-just-possible-almost/">Haven’t watched last seasons of Silicon Valley yet? It’s all about decentralization!</a></figcaption></figure><p>Let&#39;s go then. <a href="https://medium.com/@christophe.bougere/a-state-of-the-art-of-decentralized-web-part-2-ea630917332a"><strong>Chapter 2 is available here</strong></a> and is about file storage.</p><p>This article reflects my comprehension of the topic which can be subjective so discussions are much welcomed in the comments. And give some 👏👏👏 if you appreciated it 😉.</p><p><strong>P.S.</strong> I know I have been using the words &quot;decentralized&quot; and &quot;decentralization&quot; way too many times in this article (I let you count them), I hope you will forgive me 😬.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=54f70fdb7355" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/a-state-of-the-art-of-decentralized-web-part-1-54f70fdb7355">A state of the art of decentralized web — Part 1</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Serverless framework tips and tricks]]></title>
            <link>https://medium.com/@christophe.bougere/serverless-framework-tips-and-tricks-7ee88a01c4ab?source=rss-7934ff29dd7c------2</link>
            <guid isPermaLink="false">https://medium.com/p/7ee88a01c4ab</guid>
            <category><![CDATA[serverless-framework]]></category>
            <category><![CDATA[tips-and-tricks]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Christophe Bougère]]></dc:creator>
            <pubDate>Thu, 04 Oct 2018 12:50:24 GMT</pubDate>
            <atom:updated>2018-10-04T14:05:32.795Z</atom:updated>
            <content:encoded><![CDATA[<h4>Gain time and improve code quality</h4><p>I already talked about Serverless architectures good practices in a previous article:</p><p><a href="https://medium.com/@christophe.bougere/the-10-commandments-of-serverless-e5525e2c4bb6">The 10 commandments of serverless</a></p><p>In this post I want to focus on the <a href="https://serverless.com/framework">Serverless framework</a> and share many tips &amp; tricks that make my life much easier. Sometimes even too easy…</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*pRUbR7Uql84JL-voK5p8Gg.jpeg" /><figcaption><a href="https://read.acloud.guru/acg-faas-and-furious-b9574b6675c5">https://read.acloud.guru/acg-faas-and-furious-b9574b6675c5</a></figcaption></figure><h3>Prerequisites</h3><p>Before reading, here are a few points to consider:</p><ul><li>The examples and snippets are assuming your services are on a <strong>monorepo</strong>. If you are not, feel free to adapt them to your package management system.</li><li>Some of the advices may be only for <strong>AWS</strong> (usage with SSM, IAM policies, …).</li><li>Knowledge of the <a href="https://serverless.com/framework/docs/providers/aws/guide/variables/">serverless framework variables syntax</a> will help.</li></ul><h3>Create templates</h3><p>Even though you can organise your code very differently between your services, you will probably naturally start coding them in a similar way. The serverless <a href="https://serverless.com/framework/docs/providers/aws/cli-reference/">CLI</a> provides a way to create a new service following a template. This template is generally hosted on a Github repo, but you can also store it on your monorepo and use a local path when creating your service:</p><p>serverless create —-template-path $REPO_DIR/lib/serverless/template-node —-path $REPO_DIR/path-to-the-service —-name service-name</p><p>That way you can have a ready-to-use service backbone already adapted to all your specificities.</p><p>In my company, we have a node template which includes:</p><ol><li>Our babel.config.js and webpack.config.js configuration files.</li><li>A default package.json including most common dependencies (Babel, Webpack, Serverless core and plugins) as well as our standards scripts (npm run local, npm run test, npm run lint).</li><li>A README.md template including a TODO list for new services.</li><li>A serverless.yml template with all our specificities:</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f2c0e1f21e021502cf08cc7fd8e5bd7c/href">https://medium.com/media/f2c0e1f21e021502cf08cc7fd8e5bd7c/href</a></iframe><h3>Share a deployment bucket</h3><p>This feature is kind of hidden in Serverless documentation, still you can use an existing S3 bucket to store Serverless artefacts and stacks. That way, you will not have one bucket created per service.</p><p>To do so, simply define the name in the provider.deploymentBucket.name property (see the previous snippet line 9).</p><h3>Having a shared folder for your services</h3><p>If you have a monorepo, I really recommend you to avoid code duplication by sharing a folder to store some common resources. In this post it will be in $REPO_HOME/lib/serverless/common. You can store a relative path to this folder in a custom.relativePathToCommon property of your serverless.yml (see usage in the previous snippet).</p><p>It will help you to define some global resources like:</p><ul><li>Global constants</li><li>Prefixes</li><li>Common policies</li></ul><p>Let’s dive into these 3 categories.</p><h4>Global constants</h4><p>In thelib/serverless/common/env.yml file, we can store some parameters values that can either be hardcoded or retrieved from SSM:</p><ul><li>${ssm:MY_ENCRYPTED_PARAMETER~true} if the parameter is encrypted.</li><li>${ssm:MY_PARAMETER} otherwise.</li></ul><p>We usually store parameters like AWS account ID, some tokens, or anything that can be used everywhere. Nothing complicated, but it helps to keep a code readable.</p><h4>Prefixes</h4><p>Once again, this is really basic but it can reduce a lot the number of characters of your YAML files.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2be713ab52b1f31ecf3498c375b0c9ab/href">https://medium.com/media/2be713ab52b1f31ecf3498c375b0c9ab/href</a></iframe><p>That way, instead of writing the full ARN every time you need to reference a resource, you can use this prefix and append only the name. For instance:</p><p>${self:custom.prefixes.arn.function}:theLambdaName</p><p>instead of:</p><p>arn:aws:lambda:${self:provider.region}:${self:custom.env.accountId}:function:theLambdaName</p><p>When you have a lot of resources to reference, you will see the difference.</p><h4>Common policies</h4><p>It is very likely that you are going to use common resources across your services, let’s take the example of a RDS database. We will assume you are storing the parameters (database name, host, password, username and port) on SSM. To access it, you need to allow your service to get and decrypt these parameters. A basic IAM policy, nothing complicated, but you will copy it in all your lambdas that use RDS, and probably mix these statements with other authorisations. Here is how your serverless.yml could look like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7a46ef700c811ae676850a89c974f700/href">https://medium.com/media/7a46ef700c811ae676850a89c974f700/href</a></iframe><p>To fix this, you could create <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_managed-vs-inline.html#customer-managed-policies">Customer Managed Policies</a> which are done for this purpose. However, you will have to create these policies either manually or in a stack outside your service (because this policy will be shared). Also, if you want to rename a parameter, it may be more complicated because the code needs to be redeployed at the same time that the policy is changed.</p><p>Another solution is to continue deploying the policy with your service but defining it in your shared folder. That is in my opinion the most microservices oriented solution, and as well the one we have chosen. Following the same example, here are the common policy file and the new serverless.yml:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/224497d6f9d50ca2cf1e1446ba0c05ab/href">https://medium.com/media/224497d6f9d50ca2cf1e1446ba0c05ab/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/98b1d8a0b1d252c7b742cecb8e334f3c/href">https://medium.com/media/98b1d8a0b1d252c7b742cecb8e334f3c/href</a></iframe><h3>Serverless helpers</h3><p>Static YAML syntax is enabling lots of possibilities, but you may be limited if you need some dynamic values. A few examples we had are:</p><ul><li>Get the latest docker image URI from a ECR repository. This could be solved by committing the images values as constants, but I prefer to query them dynamically.</li><li>Generate a string containing the current date (to name an AMI for instance).</li><li>Dump the content of a file, for example a UserData bash script, or the code of a lambda when using CloudFormation custom resources. The content could possibly be written in the serverless.yml file, it’s much cleaner to write a JavaScript or a Bash code in a .js or a .sh file.</li></ul><p>To deal with this, we created a serverless-helpers.js file in the service folders containing that kind of functions:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/94008bbab1ad8ad4600bc4e6042a17b5/href">https://medium.com/media/94008bbab1ad8ad4600bc4e6042a17b5/href</a></iframe><p>Note that the getLastImageUri function is generic and could be written in a shared library. Now, here is how we can use it in our serverless.yml:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0e56f3930c3f2283d9644fbece3b372a/href">https://medium.com/media/0e56f3930c3f2283d9644fbece3b372a/href</a></iframe><p>Thus you can reuse this values wherever in your configuration.</p><h3>Conclusion</h3><p>I do hope you enjoyed the reading, let’s keep in touch in the comment section, my <a href="https://github.com/ChristopheBougere">Github</a> or <a href="https://www.linkedin.com/in/christophebougere/">LinkedIn</a>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7ee88a01c4ab" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sharing a tmp folder within a Step Functions execution]]></title>
            <link>https://medium.com/@christophe.bougere/sharing-a-tmp-folder-within-a-step-functions-execution-5bc2b321c402?source=rss-7934ff29dd7c------2</link>
            <guid isPermaLink="false">https://medium.com/p/5bc2b321c402</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[s3]]></category>
            <category><![CDATA[step-functions]]></category>
            <dc:creator><![CDATA[Christophe Bougère]]></dc:creator>
            <pubDate>Mon, 01 Oct 2018 14:49:21 GMT</pubDate>
            <atom:updated>2018-10-01T14:49:21.399Z</atom:updated>
            <content:encoded><![CDATA[<p>If you don’t know what <a href="https://aws.amazon.com/step-functions/">AWS Step Functions</a> is about, you can have an overview here:</p><p><a href="https://christophebougere.github.io/awesome-stepfunctions">AWeSome Step Functions</a></p><h4>The problem</h4><p>If we look at the <a href="https://docs.aws.amazon.com/step-functions/latest/dg/limits.html#service-limits-task-executions">AWS Step Functions Limits</a>, there is one particularly constraining:</p><blockquote><strong>Maximum input or result data size for a task, state, or execution</strong></blockquote><blockquote>32,768 characters. This limit affects tasks (activity or Lambda function), state or execution result data, and input data when scheduling a task, entering a state, or starting an execution.</blockquote><p>I can totally understand that task Input/Output is not aimed to contain massive amount of data, but this prevent Step Functions use in some cases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/125/1*O5ZPHxEyjYC_0NC40SqZCQ.png" /><figcaption>Simple state machine</figcaption></figure><p>For example, let say your state machine fetch some data at the beginning of the process, pass it in the result of the lambda to a second step which apply some business logic, which pass it to a third step that will persist the processed data.</p><p>This separation is ideal because you can separate the business logic of your process from the basic operations (fetch from various sources, save to a database, …). However this will only work if the amount of data to pass between your tasks is inferior to 32,768 characters.</p><p>Another issue is that you may want to pass binary content in between your tasks, like images to process, sounds, … Even though you could store binary data in base 64 in the event, it is kind of ugly and anyway you would face the size limit even sooner.</p><h4>The dream</h4><p>As I consider this as a service limit, I would love to see this solved by the AWS Step Functions team. The way that intuitively comes to my mind is that the <strong>Lambda </strong><strong>/tmp folder should be shared for all tasks of an execution</strong>. However this seems technically complicated because the /tmp folder belongs to a Lambda container instance. This wouldn’t work well in parallel, this wouldn’t scale, in a nutshell I wouldn’t bet on this option.</p><h4>The workaround</h4><p>In a few projects, I have already been using S3 as a temporary file system for Step Functions executions. But having to manage it by myself in the code seems like an anti-pattern, this is why I wanted to abstract it as much as possible:</p><ul><li>I don’t want to know in my code that I am storing files on S3. I would rather write and read in the tmp folder.</li><li>I want this to work with activities and custom workers too.</li><li>I don’t want to keep track of any execution ID in my code.</li></ul><p>This is where s3fs package becomes really useful:</p><p><a href="https://www.npmjs.com/package/s3fs">s3fs</a></p><p>This library provides a way to read/write on S3 with the standard nodefs API.</p><p>You can then create a bucket that will be exclusively used as a temporary volume for all your Step Functions executions. Note that you will need to allow your services to read and write on this bucket (<a href="https://www.npmjs.com/package/s3fs#minimum-iam-policy">you can use this policy</a>).</p><p>Then, you need to know where the files will be stored on this bucket depending on the execution. Sadly, there is no way to retrieve the execution ID from within a Lambda. In my opinion this is another lack of Step Functions, and this could be easily added in the Lambda context. The trick here is to generate a UUID in the first step of the execution and store it in the event which will be passed as the result. In the next steps, as the UUID already exist, we will use it to find the path on S3 where the data is stored.</p><p>This is a trick, and it will not work in every situation:</p><ul><li>If the state machine starts with a parallel state, we would have many UUIDs generated.</li><li>If you are using InputPath, OutputPath or ResultPath this may be an issue too because the UUID stored in the Input/Output may be lost.</li></ul><h4>The code</h4><p>Let’s bring all the pieces and create a simple class extending S3FS to make it as transparent as possible from within the tasks:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e4d8c687b81d581390a146ef4069af6a/href">https://medium.com/media/e4d8c687b81d581390a146ef4069af6a/href</a></iframe><p>Then simply instantiate this class to get a fs implementation:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/30b18577e8860b6afe8aa7ef6756a1d5/href">https://medium.com/media/30b18577e8860b6afe8aa7ef6756a1d5/href</a></iframe><p>In the last task of your state machine, you can call the clean method to delete the temporary resources that are useless now. You can also use <a href="https://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html">S3 Lifecycle Policy</a> to make it automatically.</p><p>I hope this will help, and that Step Functions team will continue to add new features to it service. Don’t hesitate to share your personal tricks in the comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5bc2b321c402" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The 10 commandments of serverless]]></title>
            <link>https://medium.com/@christophe.bougere/the-10-commandments-of-serverless-e5525e2c4bb6?source=rss-7934ff29dd7c------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5525e2c4bb6</guid>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloud]]></category>
            <dc:creator><![CDATA[Christophe Bougère]]></dc:creator>
            <pubDate>Wed, 11 Jul 2018 16:46:52 GMT</pubDate>
            <atom:updated>2018-07-11T16:46:52.716Z</atom:updated>
            <content:encoded><![CDATA[<h4>Feedbacks on Serverless with AWS</h4><p><strong>Disclaimer:</strong> Despite the title, this is not a holy text. I discovered the serverless world in early 2016, when things were much more handicraft than today. Most of these lessons have been acquired by experiencing and failing. Read it, think about it and question it. Things are evolving quickly and some assumptions may be obsolete soon.</p><h4>#1 You shall think microservices</h4><p>I have always been a big fan of Lego bricks. This may be why I find myself so much in microservices. This analogy may help to understand all the stakes of this architecture.</p><p>Lego pieces are doing one simple thing and you know they are doing it well. Some are really low level and used to build the foundations (bricks), some are really specific to a use case (wheels, doors) and some others are made to help connecting all the parts (cross axes, gears). By combining all those simple Lego elements, you can build lots of specific things like a car or a house.</p><blockquote>Consider your microservices as Lego bricks</blockquote><p>By building small but reliable components and assembling them to build complex things, you will gain on many angles:</p><ul><li>Easier to test. As the scope of your services is very limited (user management, authorizations, emails, storage, …), you can focus your tests on what the service is really doing and not what it is integrating with. Note that integration and end to end tests will still be necessary.</li><li>Easier to maintain. As long as the interface between your services is well defined, you should be able to update a service without thinking too much about how it could impact other services.</li><li>Easier to handle technical debt. You will not be forced to do long and risky migrations to get rid of your legacy code or technologies. You can start using new practices in your services incrementally, by letting the old and the new way coexist for a while. You will then be able to validate the new methods as well as stabilizing them before applying them globally. It is like doing canary deployment but for your internal infrastructure.</li></ul><h4>#2 Avoid dependencies between services</h4><p>This is not new to microservices, but dependencies have a much bigger impact. Having a microservices architecture with dependencies everywhere is an anti-pattern.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*9C0N6Q81n0j3WfXs84fUMQ.png" /><figcaption>Be like Jon Snow</figcaption></figure><p>You may think splitting your architectures in many pieces is not compatible with making them independent, a little clarification is required: of course your services will call each others and will be interdependent. The dependencies you must avoid are strong dependencies. Your services must only be aware of the interface of other services, and never assume its inner mechanism.</p><ul><li>As much as possible, your microservices should only be available through an API. Sometimes it can make sense to add other interfaces, like a Pub/Sub system (SNS) or a trigger (S3) to avoid adding some useless layer. It&#39;s OK to do that way, as long as you keep the same input/output format for all your interfaces so you don&#39;t need to assume the request origin in your lambda.</li><li>The way data are stored should only be known by the service itself, and no other service should assume a specific format. That way, you can completely change the inner mechanism overnight. It is one advantage I see using DynamoDB to store your data: your tables will belong to your service, will be deployed at the same time and you will implicitly segregate your data per service, instead of having a single database where any service can access any data.</li></ul><h4>#3 One language to rule them all</h4><p>This is of course very subjective, but I believe JavaScript is the best language for the cloud, and here are the reasons:</p><ul><li>Using the same language for frontend, backend, testing and scripting is a really powerful asset. It means all your team members speak the same language, can share code, and can potentially work on anything.</li><li>Modern JavaScript has a really simple and clean syntax. <em>async/await</em>, among other things, revolutionized my daily life. <a href="http://www.skrenta.com/2007/05/code_is_our_enemy.html">Less code is better!</a> If the last time you have used JavaScript was to do some jQuery in 2010, then you should have a look at <a href="https://github.com/mbeaudru/modern-js-cheatsheet">this cheatsheet</a>.</li><li>It has <a href="http://www.modulecounts.com/">the biggest packages repository</a>. You can focus on writing code that will benefit your business not reinventing the wheel.</li></ul><h4>#4 Organize your code in a monorepo</h4><p>This debate will probably stay alive for a while, but:</p><ul><li>It helps you thinking globally when developing a feature. You update all your services within a single PR (easier to test and review).</li><li>It makes it easier to share code (at least for developing). There are various ways to share a library:<br>- By publishing each package on npm. <a href="https://github.com/lerna/lerna">Lerna morepos</a> may help getting some coherence. I personally think this makes a lot of sense for open source projects (see <a href="https://github.com/babel/babel">babel</a>, <a href="https://github.com/Turfjs/turf">turf</a>), but versioning is constraining and not always necessary.<br>- By encapsulating your shared logic in a service. I suggest you to read <a href="https://theburningmonk.com/2018/02/aws-lambda-how-best-to-manage-shared-code-and-shared-infrastructure/">this article</a> for more details.<br>- By linking a folder locally (using `npm link`). It can have some side effects (more complicated webpack setup), but it is speeding up development time so much that I will probably stick with this option for a while.</li><li>It helps building a strong common CI. Microservices architecture makes functional and integration tests a key of your stability. I strongly encourage you to read <a href="https://medium.freecodecamp.org/the-best-ways-to-test-your-serverless-applications-40b88d6ee31e">this article</a> that redefines the testing paradigm for serverless.</li></ul><p>Once again, this may not work for everyone. Splitting in a few repositories may be wise depending on the size/number of your team(s)/product(s), as long as you can find the good partitioning criteria.</p><h4>#5 Apply a naming convention</h4><p>Of course the microservices architecture gives you some freedom on how each service behaves inside, but still as the number of your services will grow, you will need some consistency. I would suggest naming <strong>all</strong> your CloudFormation resources (even meaningless resources, like IAM roles) following a pattern, for instance {service-name}-{stage}-{resource-name}:</p><ul><li>You will easily recognize the service using this resource, as well as resources that aren’t part of any stack and could probably be cleaned up.</li><li>Having the stage in the name makes it possible to deploy the same service multiple times on the same account, so you can easily test features without breaking your environment.</li></ul><p>Protip: If you are using a framework, like Serverless, you can assign the prefix ({service-name}-{stage}-) to a variable to be less verbose in your declarations.</p><h4>#6 Serverless is not (only) FaaS</h4><p>The fully managed combo Lambda/APIGateway/DynamoDB/S3 is obviously a really good starting point and should be enough for simple use cases. However you may quickly have longer or more complex processes:</p><ul><li>How to manage states in a stateless environment (like waiting for a manual task)?</li><li>How to reuse your basic functions across your services, and how to chain them within a process?</li><li>How to encapsulate non-serverless tasks (like a Batch job) in a serverless process?</li></ul><p>If these questions sound familiar to you, you probably need an <strong>orchestrator</strong> and should give a try to<a href="https://aws.amazon.com/step-functions/"> <strong>Step Functions</strong></a>. It is based on the state machine pattern, and includes common blocks (wait, parallel branches, conditions, tasks). I recently wrote <a href="https://christophebougere.github.io/awesome-stepfunctions/">a presentation about that service</a> and I invite you to have a look for a good overview.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*cp32yMDPCQMLSPwG.jpg" /><figcaption>A state machine execution on AWS Step Functions</figcaption></figure><h4>#7 Equip yourself</h4><p>I couldn’t recommend you a better framework than <a href="https://serverless.com/">Serverless Framework</a>:</p><ul><li>The most popular, has quite a stable behavior.</li><li>It creates all your resources in CloudFormation stacks when using AWS.</li><li>You will find <a href="https://github.com/serverless/plugins">plugins to do almost everything</a>.</li><li>The core is multi cloud providers (currently 8), which help standardizing the serverless architecture (FaaS services are quite similar is working almost the same way regardless of the provider).</li></ul><p>You should also keep an eye to <a href="https://github.com/serverless/components">Serverless Components</a>, which allows you to build by assembling components rather than configuring a single service. Not sure where it is going yet, but it looks promising!</p><p>AWS is also seriously starting to catch up with <a href="https://github.com/awslabs/serverless-application-model">AWS Serverless Application Model</a> and its <a href="https://github.com/awslabs/aws-sam-cli">CLI</a>. It is obviously only focused on AWS and doesn&#39;t offer an ecosystem of plugins yet, but you will benefit from <a href="https://aws.amazon.com/serverless/serverlessrepo/">AWS Serverless Application Repository</a> which lets you deploy third party applications in a few clicks.</p><h4>#8 Adapt your tooling</h4><p>Regardless of all its advantages, serverless is still early stage and this new paradigm also brings new problematics.</p><blockquote>As developers begin to see the value of serverless architecture, a whole new startup ecosystem could begin to develop around it. — <a href="https://techcrunch.com/2018/07/08/serverless-computing-could-unleash-a-new-startup-ecosystem/">Techcrunch</a></blockquote><p>Here are 2 products I have been working with that I highly recommend you to give a try. In addition to be really focused on the serverless paradigm, in contrast with most well known tools, they are also listening to our needs and adapting really fast.</p><p><a href="https://epsagon.com/"><strong>Epsagon</strong></a></p><p>This service aims to answer one of the biggest challenge in the serverless world: observability. As a single request can easily go threw a dozen of entities (FaaS, database, API, State Machine, 3rd party service, …) it becomes extremely hard to track a whole request at once. Epsagon proposes to solve that by adding a simple wrapper in all your FaaS and offers a superb graph visualisation of your transactions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*prH3DI-OWZj0IxwA6DQYZg.png" /><figcaption><a href="https://www.epsagon.com/blog/distribute-messages-effectively-in-serverless-applications">An example of transaction in Epsagon</a></figcaption></figure><p>There are now <a href="https://serverless.com/blog/best-tools-serverless-observability/">several monitoring and observability products for serverless</a> and most of them provide nice features around Lambda, but I believe Epsagon is the only one supporting Step Functions. By combining a powerful serverless orchestrator with a powerful serverless observability tool, it is a real pleasure to develop and monitor complex processes.</p><p><a href="https://seed.run/"><strong>Seed.run</strong></a></p><p>Microservices architecture can makes your deployments more messy because they will be split in many parts. Indeed a framework like Serverless will help keeping a consistency in the process to deploy (a single command). But when you will have dozens of services and different stages, you may want something more automated to manage your deployments. Seed.run let’s you configure services within a monorepo as well as stages. You can automate deployment (deploy in staging when you merge on develop, deploy in production when you merge on master), or you can keep it manual and trigger a deployment in one click.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OulgAm7y2VPMzahd.png" /><figcaption>The seed.run console</figcaption></figure><h4>#9 Keep It Simple, Stupid (<a href="https://en.wikipedia.org/wiki/KISS_principle">KISS principle</a>)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/303/0*R0L_KgNBCQmrAd16.gif" /><figcaption>Simple is sexy</figcaption></figure><p>This principle should be your leitmotiv. You may wonder why it is particularly important here? Building small things will be easier, more testable, more evolutive. But it also implies you will have interactions between different parts of your architecture, whose you may not always know the inner mechanism. But as long as your services respect a predictable and simple interface, it shouldn’t be an issue because you will be able to learn how to use them in no time. If you are not able to explain in a sentence how to use a service, you should probably make it simpler.</p><h4>#10 Make it easy to create a new environment</h4><p>We can never say it enough, but automating everything in your deployments is the key. You should never create a resource outside of CloudFormation. It may take more time to develop, but you won&#39;t regret it. Here a a few reasons you may have to setup new environments:</p><ul><li>Creating a testing environment from scratch everyday for your nightly tests.</li><li>Creating an environment per developer or per feature.</li><li>Expanding on other AWS regions.</li></ul><p>Some resources can be complicated to deploy automatically (custom EC2 AMI, RDS cluster, 3rd party services) and this is where CloudFormation custom resources become useful. Custom resources are requesting a custom endpoint to update the status of a resource, which means you can create a Lambda that will handle the life cycle of your resources. Have a look <a href="https://apimeister.com/2017/06/18/extending-cloudformation-with-custom-resources.html">here</a> to get started, and if you need to create an EC2 AMI after having executed a User Data script, there is <a href="https://medium.com/poka-techblog/managing-amis-using-cloudformation-a097f86a3622">a really good ready-to-use stack</a> that will do everything for you (and with a step function).</p><p>Also, my advice is to separate your resources in 2 parts:</p><ul><li>What should be part of your infrastructure and should be unique per environment, like your VPC, domains on Route53, maybe RDS (if you choose to share a cluster and create a database per stage). These elements should be rarely updated and there isn&#39;t much added value to use a framework for that, so sticking with pure CloudFormation stacks may be the best.</li><li>What is part of your services and could be deployed many times in a single environment (to try a feature, or for a developer environment). As your resources contains the stage in their names it&#39;s now easy to make them coexist in the same AWS account.</li></ul><h4>#11 Have fun coding awesome things</h4><p>Oh wait, is it already the 11th commandment? It doesn&#39;t matter, serverless is making your life such cooler that it becomes a duty to push the boundaries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*Meq6tc5L-u3ndNFGtm9cXA.jpeg" /></figure><p>Thank you for reading this article (and many thanks to <a href="https://medium.com/u/8968b2037d42">Hervé Nivon</a> for the review), I hope it will be helpful! If you went so far, you are curious and here are my top 3 sources of inspiration about Serverless to get further:</p><ul><li><a href="https://serverless.com/blog/">Serverless Blog</a></li><li><a href="https://theburningmonk.com/">theburningmonk.com | serverless, AWS, functional programming, cloud</a></li><li><a href="https://medium.com/netflix-techblog">Netflix TechBlog</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5525e2c4bb6" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>