<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dmitri Zimine on Medium]]></title>
        <description><![CDATA[Stories by Dmitri Zimine on Medium]]></description>
        <link>https://medium.com/@dzimine?source=rss-63a761e7cf69------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*6_dDSjTq2K4S-yBu.jpg</url>
            <title>Stories by Dmitri Zimine on Medium</title>
            <link>https://medium.com/@dzimine?source=rss-63a761e7cf69------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 22:33:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dzimine/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AWS Greengrass Pro Tips]]></title>
            <link>https://medium.com/a-cloud-guru/aws-greengrass-pro-tips-4ef231821f9?source=rss-63a761e7cf69------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ef231821f9</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[iot]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Dmitri Zimine]]></dc:creator>
            <pubDate>Wed, 06 Mar 2019 21:41:15 GMT</pubDate>
            <atom:updated>2019-03-18T15:40:46.272Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>10 tips</strong> to help developers get the best out of AWS Greengrass</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*37wiW8Zd8mSzaTKZCyu9lw.png" /></figure><p>AWS Greengrass is a fantastic technology: it extends AWS cloud to the edge and let us build serverless solutions that span cloud, edge, and on-prem compute.</p><p>But as any new tech — it’s still a bit rough around some edges and far from being easy to use. While our AWS friends are working hard to make it more civilized, here are <strong>10 tips</strong> to help you get the best out of AWS Greengrass right now. Quick reference:</p><ol><li><a href="#4d5c">Nail your development setup</a></li><li><a href="#0d06">Greengrass in Docker: proceed with caution (with 5 steps to get it going)</a></li><li><a href="#4a22">Basic troubleshooting hints — logs, deployments &amp; lambda locations, etc</a></li><li><a href="#cc09">Checking connection</a></li><li><a href="#1487">Troubleshooting Lambda functions</a></li><li><a href="#1973">Getting inside Lambda container</a></li><li><a href="#13ea">Name your Lambda for happy troubleshooting</a></li><li><a href="#0665">Hack a quick code change in Lambda on local Greengrass</a></li><li><a href="#360f">Lambda on-demand? Lambda long-lived? Both!</a></li><li><a href="#d425">Use systemd to manage greengrass lifecycle</a></li></ol><blockquote>These are PRO tips, and assume a basic grasp on AWS Greengrass which includes some hands-on experience. This tips are best suited for those running a real IoT project with AWS. If you are new to this services, please bookmark this and come back after reviewing <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-gs.html">Greengrass tutorial</a> and <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-troubleshooting.html">AWS Greengrass Troubleshooting</a>.</blockquote><h4>Tip #1 — Nail your development setup</h4><p>There are many ways to set up a dev environment for different tastes — but some approaches are more efficient than others. Greengrassing via AWS web console is fine for a first tutorial, but after you fat-finger your subscriptions a few times you’ll be looking for a better way.</p><p>I prefer old-fashion local development, and I’m fanatical about GitOps and infrastructure-as-code. This influenced my setup:</p><p>The editor of choice, git, AWS CLI with named profiles to jump between testing and production accounts — all run on my Mac. Greengrass core software is installed on a Vagrant VM; all the pre-requisites and installations are codified in a Vagrantfile.</p><p>I use <a href="http://greengo.io/">greengo.io</a> to operate Greengrass Group as code. Greengo lets me define the entire group (core, subscriptions, policies, resources, etc) as a simple YAML. It then creates it in AWS as defined, takes care of creating Lambda functions in AWS from the local code. It downloads the certificates and configuration file that I put straight to Greengrass VM thanks to helper scripts. As a bonus, greengo also knows to cleans it all up!</p><p>With all that in place, I edit lambda functions and greengrass definitions alike with convenience of my favorite editor. I make changes, update, deploy, rinse, repeat. I jump onto Greengrasss VM with vagrant ssh to check on greengrass well-being via logs, start/stop the daemon, explore it and pull various tricks described below.</p><p>All the work is captured as code, tracked by <em>git</em>, goes to GitHub right off my laptop, and can be easily reproduced. A complete code example — a codified AWS ML inference tutorial — is on GitHub for you to check out and reproduce: <a href="https://github.com/dzimine/aws-greengrass-ml-inference">dzimine/aws-greengrass-ml-inference</a>.</p><p>This setup serves me well, but this is of course not the only approach. You can take <a href="https://medium.freecodecamp.org/aws-lambda-offering-developers-ultimate-flexibility-d8939ff4220">a three common ways to create your Lambda functions with AWS</a> and creatively extend them to your Greengrass development. You might like Cloud9 + Greengrass on AmazonVM, as <a href="https://twitter.com/janmetzner">Jan Metzner</a> demonstrated on <a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=89681">IOT402 Workshop</a> at re:Invent 2018.</p><p>If you’re a master of CloudFormation templates, check out the elegant <a href="https://github.com/Neosperience/GrassFormation">GrassFormation</a> and the development flow that comes with it. Whichever you choose, do yourself a favor — nail your development setup first.</p><blockquote>Update March 18/2019: AWS Greengrass <a href="https://aws.amazon.com/about-aws/whats-new/2019/03/aws-iot-greengrass-adds-new-connector-aws-iot-analytics-support-aws-cloudformation-templates-integration-fleet-indexing/">added support for CloudFormation templates</a>. This is great, now you can use it raw, or with the tool of your choice like <a href="https://serverless.com/framework">serverless framework</a> or <a href="https://github.com/awslabs/aws-sam-cli">SAM CLI</a>.</blockquote><h4>Tip #2 — Greengrass in Docker: proceed with caution</h4><p>I managed Greengrass in Docker before the official support in v.1.7, using a couple of docker tricks: devicemapper storage driver instead of default overlay2, and --privileged flag. It worked, and had no limitations on Greengrass functionality.</p><p>But with newer versions of Docker, Linux kernel, and Greengrass, it gradually stopped working. I abandoned the case, but you always try your luck at <a href="https://github.com/dzimine/aws-greengrass-docker">GitHub - dzimine/aws-greengrass-docker</a>).</p><p>Now we can officially <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/run-gg-in-docker-container.html">run AWS IoT Greengrass in a Docker Container</a>. It is a great convenience, but a close look shows that it is more of a work-around. Making greengrass containers run inside docker container is admittedly difficult and unlikely ever suite for production. So AWS introduced a configuration option to run Lambdas outside of a container, as OS processes — on per-lambda or per-group basis.</p><p>Choosing this option brings <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/run-gg-in-docker-container.html">limitations</a>: Connectors, local resource access, and local machine learning models are not supported. Hey, not a big loss! Connectors are in their infancy — and for that matter I’d much rather want them for Lambdas and StepFunctions.</p><p>When lambdas run as OS processes they can access any local resources — so no need for configuring local resources. And a local ML stack can be easily custom-built into a Greengrass image to your liking.</p><blockquote><strong>Design considerations<br></strong>A bigger concern with this approach, and overall with <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/lambda-group-config.html#lambda-containerization-considerations">optional containerization for Lambdas</a>, is that it smells hidden dependencies — making things fragile.</blockquote><blockquote>The IoT workflow is about defining deployments in the cloud and pushing it to a fleet of devices. Some devices don’t support containerized Lambdas: be it greengrass running inside Docker or on a constrained OS with no containerization. From v1.7, Greengrass says “It’s OK, it will run as long as a group is opted out from containerization”. But devices don’t advertise their capabilities, nor do they have a guard to reject an unsupported group deployment. The knowledge of what group is safe to run on what devices must reside outside the system and inside designer’s head — admittedly not the most reliable store.</blockquote><blockquote>While enabling containerization for Lambda is a seemingly legit change in a group definition , it can break the deployment or cause function failures. Not to mention that running code with and without containerization may differ in many ways— I did hit weird code bugs that manifested only when containerized.</blockquote><blockquote>AWS Docs warn to <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/lambda-group-config.html#lambda-containerization-considerations">use this option with caution</a>, and prescribe use cases where running without containerization may worth the tradeoff. If you want to adopt Docker with Greengrass, I’d recommend goling all the way: drop containerization, run in Docker in dev and production, use the same containers.</blockquote><p><strong>5 Steps for Greengrass in Docker <br></strong>If the limitations outlined above don’t deter you from this approach, here’s how to run Greengrass in docker in 5 simple steps:</p><p>1. Get the zipped repo with Dockefile and other artefacts from the <a href="https://d1onfpft10uf5o.cloudfront.net/greengrass-core/downloads/1.7.1/aws-greengrass-docker-1.7.1.tar.gz">Cloud Front.</a> <em>Why not GitHub? May I clone it to my GitHub, or is it against the license? Ask AWS</em>. For now, CloudFront:</p><pre>curl <a href="https://d1onfpft10uf5o.cloudfront.net/greengrass-core/downloads/1.7.1/aws-greengrass-docker-1.7.1.tar.gz">https://d1onfpft10uf5o.cloudfront.net/greengrass-core/downloads/1.7.1/aws-greengrass-docker-1.7.1.tar.gz</a> | tar -zx</pre><p>2. Build the docker image with docker-compose</p><pre>cd aws-greengrass-docker-1.7.1<br>PLATFORM=x86-64 GREENGRASS_VERSION=1.7.1 docker-compose build</pre><p>3. Use greengo to create a group: it will place the certificates and configuration for your into the /certs/ and /config. Or die-hard with AWS console, download and sort out the certificates and config. <strong>Attention:</strong> The deployment must have all Lambdas opted out of containers: or pinned=True in API, CLI, or greengo.yaml definition.</p><p>4. Run Docker with docker-compose:</p><pre>PLATFORM=x86–64 GREENGRASS_VERSION=1.7.1 docker-compose up</pre><p>5. Profit. For any complications, like a need for ARM image or a misfortune of running on Windows — open README.md and enjoy the detailed instructions.</p><p>You might ask “why not using Docker in a development setup instead of a heavy Vagrant VM”? You may: I do it myself sometimes, like a testbed for greengo development.</p><p>But for a production-ready IoT deployment I prefer an environment to be the most representative of the target deployment — which is not always the case with GG Docker limitations.</p><h4>Tip # 3 — Basic troubleshooting hints</h4><p>When something goes wrong — which it likely will — here are the things to check:</p><ol><li><strong>Make sure </strong><strong>greengrassd is started</strong>. <br>Look for any errors in the output.</li></ol><pre>/greengrass/ggc/core/greengrassd start</pre><p>If it doesn’t start, it most likely misses some pre-requisites, which you can quickly check by greengrass-dependency-checker script. You can get the checker for the most recent version from <a href="https://github.com/aws-samples/aws-greengrass-samples">aws-greengrass-samples repo</a>.</p><p>There is also <a href="https://aws.amazon.com/greengrass/device-tester/">AWS Device Tester</a>, a heavier option typically used by device manufactures to certify their device as Greengrass-capable.</p><p><strong>2. Check system logs for any error</strong>. <br>Look under /greengrass/ggc/var/log/system/, start with runtime.log. There are few other logs with various aspects of greengrass functionality, <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-troubleshooting.html#troubleshooting-logs">described in AWS docs</a>. I mostly look at runtime.log:</p><pre>tail /greengrass/ggc/var/log/system/runtime.log</pre><p>It helps to set the logs to DEBUG level (the default is INFO). It is set up at Greengrass Group level via AWS console, CLI, API, or greengo will do it for you. A group must be successfully deployed to apply the new log configuration.</p><p>But what if I am troubleshooting a failure in the initial deployment? Tough luck! There is a trick is to fake a deployment locally with just a logging config in it but it is quite cumbersome; it would be much better to set it up in config.json file. Join my ask to AWS to do so:</p><h3>Dmitri Zimine on Twitter</h3><p>To troubleshoot a failing AWS Greengrass group deployment, I need DEBUG log level. To set it to DEBUG, I need to set it in a group definition and deploy. Deployment is failing. To troubleshoot a failing deployment... CATCH 22! May I please set log level in config? #awswishlist</p><p><strong>3. Check the deployment</strong><br>Once you trigger deployment and reaches “In Progress” state, a blast of messages pops up in runtime.log as the greengrass handshakes, downloads the artefacts, and restarts the subsystems. The deployment artifacts are stored locally at /greengrass/ggc/deployment:</p><ul><li>group/group.json - a deployment definition. Explore on your own:</li></ul><pre>cd /greengrass/ggc/deployment/group/<br>cat group.json |  python -m json.tool</pre><ul><li>/greengrass/ggc/deployment/lambdas - lambda code is deployed here. The content there is an extracted Lambda zip package. Just as you had it on your Dev machine before shipping up to AWS.</li></ul><p><strong>Key hint:</strong> these artifacts are what the local greengrass takes as an input. Toying with them is a great way to creatively mess with Greengrass.</p><blockquote>I enthusiastically persuade you to write a profound logging in your Lambda code and set log levels to DEBUG for both system and user logging configuration.</blockquote><h4>Tip #4 — Checking connection</h4><p>Greengrass daemon starts, but nothing happens. Deployment is stuck forever with status “In progress’. What is wrong? Most likely, it is a connection problem.</p><p>Greengrass relies on MQTT (port 8883) for messaging and HTTPS (port 443) for downloading deployment artifacts. Make sure that your network connection allows the traffic.</p><p>Checking HTTPS is obvious, here is how to test your MQTT connection (<a href="https://docs.aws.amazon.com/iot/latest/developerguide/diagnosing-connectivity-issues.html:">source</a>):</p><pre>openssl s_client -connect \<br>YOUR_ENDPOINT.iot.eu-west-1.amazonaws.com:8883 \<br>-CAfile root.ca.pem \<br>-cert CERTIFICATE.pem \<br>-key CERTIFICATE.key</pre><p>In a typical corporate network port 8883 might be blocked, or your device may be behind a proxy. Starting from version 1.7, Greengrass can deal with a proxy. AWS docs also describe <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-core.html#alpn-network-proxy">how to reconfigure Greengrass to pass MQTT traffic through port 443</a>, but there is a confusing note that HTTPS traffic still goes over 8883.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/1*cSytREeoTQ4xASv-h2Mn1A.png" /><figcaption>Source: <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-core.html#alpn-network-proxy">AWS IoT Greengrass docs</a></figcaption></figure><p>Doesn’t this defeat the purpose? Greengrass needs HTTPS to get the deployment artifacts. If there is no way to switch it off 8443, how is it supposed to work? I didn’t experiment with this; if you suffer from over-secured networks please try and report your findings.</p><blockquote>Update March 10/2019: AWS released <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/what-is-gg.html#gg-core-download-tab">version 1.8</a> which has a the discrepancy above: now you can configure HTTPS to use 443, too. And it works! I happen to have firewall issue this very day, applied the fix, and it did wonders. Hurray!</blockquote><h4>Tip #5 — Troubleshooting Lambda functions</h4><p>When deployment is succeeded but nothing seems to be happening as expected, it’s time to look at Lambdas.</p><p>Check if the lambda functions are running:</p><pre>ps aux | grep lambda_runtime</pre><p>Look at the Lambda logs: dive under /greengrass/ggc/var/log/user/... until you find them.</p><p>A function may crash before it creates a log file: for example, the core failed to even start the Lambda for reasons like missing dependencies or misconfigured runtime.</p><p>Try and run the lambda manually: now that you know where it is stored locally on the greengrass device — yes, /greengrass/ggc/deployment/lambdas/... — go there and just run it in place, observe results.</p><h4><strong>Tip #6 — Getting inside Lambda container</strong></h4><p>Greenness runs lambdas in containers (unless you <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/lambda-group-config.html#lambda-containerization-considerations">explicitly opted out</a>). As such, it sometimes happen that a function that runs fine when you launch it on device manually, still fails when running under greengrass.</p><p>Find out the PID of your lambda and get inside the container with nsenter:</p><pre>sudo nsenter -t $PID -m /bin/bash</pre><p>Now you are in the containerized environment of your Lambda function and can see and check mounted file systems, access to resources, dependencies, environment variables — everything.</p><p>This trick is easy for pinned (long-lived) functions, but common event-triggered lambdas don’t live long enough to jump in their container: you might need to temporarily pin them to be able to get in.</p><p>If modifying the lambda definitions in a Greengrass group, redeploying the group, and remembering to switch it back later feels like too much trouble, hack it on the device: set pinned=True in the function section of group.json under /greengrass/ggc/deployment/group and restart the greengrassd. The next deployment will reset your local hacks.</p><h4>Tip #7 — Name your Lambda for happy troubleshooting</h4><p>What convention to use for naming Lambda functions and handlers? I used to call all my lambda entry points handler and place it in function.py files, like this:</p><pre># MyHelloWorldLambda/function.py</pre><pre>def handler():<br>    return {&#39;green&#39;: &#39;grass&#39;}</pre><p>But a check for running Lambda functions produced output like this:</p><pre>ps aux | grep lambda_runtime<br>20170 ?        Ss     0:01 python2.7 /runtime/python2.7/lambda_runtime.py --handler=function.handler<br>20371 ?        Ss     0:01 python2.7 /runtime/python2.7/lambda_runtime.py --handler=function.handler</pre><p>Not too informative, eh? Instead, consider naming either handler or function entry-point by the Lambda function name for more insightful output and easier troubleshooting. Then, finding Lambda process PID for the tricks in <strong><em>Tip 5</em></strong> will be a breeze.</p><h4>Tip #8 — Hack a quick code change in Lambda on Greengrass</h4><p>What is a typical development workflow for greengrass goes?</p><ol><li>Make a change to your Lambda function</li><li>Test it locally</li><li>Deploy the function to the AWS Lambda</li><li>Update/increment the alias and make sure the greengrass lambda definition points to this very alias</li><li>Deploy the group to your greengrass device</li></ol><p>After these steps, the lambda is ready to be called. This is great when deploying final production code to a fleet of field devices.</p><p>But during developing, especially in debugging, we often need to try a quick code change, as tiny as printing out a value. The full workflow feels ways too long and cumbersome. While our AWS friends are hopefully working on integrating a local debugger (why not if <a href="https://docs.microsoft.com/en-us/azure/iot-edge/how-to-vs-code-develop-module#debug-a-module-with-the-iot-edge-runtime">Azure IoT edge does it</a>?), here’s a quick hack I use:</p><ul><li>I ssh to my Vagrant VM with greengrass core</li><li>Go to the deployment folder</li><li>Find the lambda in question and edit it in place</li><li>Restart greengrassd, and test the change</li></ul><p>If I don’t like the update, the next deploy will revert the change. If I do like the updated, I copy the code over the function code to my local development environment and deploy.</p><p>Here <a href="http://greengo.io/">greengo.io</a> comes handy again, taking care of cumbersome routines like repackaging &amp; uploading lambda, incrementing versions, updating aliases, and repointing IoT lambda definitions to the right alias: all behind a simple command greengo update-lambda MY_FUNCTION</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/755/1*oMgjy2g4asy0gVfQiEBL-A.gif" /><figcaption>Update and deploy Lambda with greengo — quick and easy.</figcaption></figure><h4>Tip #9 — Lambda on-demand? Lambda long-lived? Both!</h4><p>AWS Greengrass supports <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/lambda-functions.html#lambda-lifecycle">two lifecycle models for Lambdas</a>: “on-demand” and “long-lived”.</p><p><strong>Lambda On-Demand<br></strong>The default behavior is “on-demand” which is the same as Lambda in the cloud — this is where the function is invoked by an event.</p><p>To trigger Greengrass “on-demand” lambda on MQTT event, you configure a subscription on a topic with the lambda function as a target. Greengrass will spawn the function execution on each topic message, pass a message as a parameter to a function handler, which then runs for the time not exceeding a configured timeout and shuts down. Variables and execution context is not retained between invocations.</p><p><strong>Lambda Long-Lived<br></strong>A “long-lived” function starts when Greengrass daemon starts which allows us to `pin` a function — make it long-running. This is a blessing for many use cases, like a video stream analysis, or protocol proxy where the function listens to the signals from BTLE or ZigBee devices and transfers them over MQTT.</p><p>The best kept secret? You can combine both. Yes, a long-lived function can be triggered on-demand. I can use this for good causes, like for keeping a non-critical state in memory between invocations.</p><p>For example: my function runs anomaly detection on the stream of device data with PEWMA algorithm. It must keep several previous data points to compute the averages. These data must persist between function executions triggered by data messages.</p><p>To achieve this, combine “long-lived” with “on-demand”:</p><ol><li>configure a subscription to fire the function on receiving the device data: greengrass dutifully calls the handler with the data payload as messages arrive</li><li>make the function long-living: now I can just keep the state in memory, between handler invocations. If the Lambda restarts and looses the state, it’s OK: the algorithm will quickly recover so a true persistence doesn’t worth the trouble.</li></ol><p>For a complete example, take a look at the <a href="https://github.com/dzimine/gg-protocol-conversion/blob/master/lambdas/AnomalyCatcher/function.py">AnomalyCatcher function code </a>in my Greengrass <a href="https://github.com/dzimine/gg-protocol-conversion">IIoT prototype</a>.</p><p>Remember that once the Lambda is configured “long-lived”, the functions won’t run in parallel: invocation are queued and handler is invoked one at a time. Check if it’s appropriate for your message volume, and mind your handler code from anything that might block a queue.</p><h4>Tip #10 — Use systemd to manage greengrass lifecycle</h4><p>When your Greengrass device reboots, you would want greengrass to start automatically, wouldn’t you? Well, it won’t until you make it so. Use systemd to manage the life cycle of greengrass daemon process.</p><p>You’ll need to modify the useSystemd flag to true in config.json, and set up a <a href="https://www.freedesktop.org/software/systemd/man/systemd.service.html">systemd</a> script, like the one in this simple example:</p><pre>[Unit]<br>Description=Greengrass Daemon</pre><pre>[Service]<br>Type=forking<br>PIDFile=/var/run/greengrassd.pid<br>Restart=on-failure<br>ExecStart=/greengrass/ggc/core/greengrassd start<br>ExecReload=/greengrass/ggc/core/greengrassd restart<br>ExecStop=/greengrass/ggc/core/greengrassd stop</pre><pre>[Install]<br>WantedBy=multi-user.target</pre><p>Here are <a href="http://www.andyfrench.info/2018/08/automatically-starting-aws-greengrass.html">detailed step-by-step instructions for Raspberry-PI</a>. If your greengrass device supports another init system — like upstart, SystemV, or runit — check the manual and configure the greengrassd as a daemon according to it.</p><p>I hope you enjoyed my 10 Greengrass tips and put them to a good use in your IoT projects. It took some pain to figure them out: your claps 👏 would be the best reward. Be sure to visit <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-troubleshooting.html">AWS Greengrass Troubleshooting</a> page, and check <a href="https://forums.aws.amazon.com/forum.jspa?forumID=254">AWS IoT Greengrass Forum</a> for more.</p><p>Please add <strong>your own tips</strong> in the comments here for the rest of us. For more stories and discussions on or follow me<a href="https://twitter.com/dzimine"> </a>on IoT, DevOps, and Serverless, follow me on Twitter <a href="https://twitter.com/dzimine">@dzimine</a>.</p><figure><a href="https://acloud.guru/membership?utm_source=medium&amp;utm_medium=blog&amp;utm_content=&amp;utm_campaign="><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OyGo8x0Zg516xq40xhji1Q.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ef231821f9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/a-cloud-guru/aws-greengrass-pro-tips-4ef231821f9">AWS Greengrass Pro Tips</a> was originally published in <a href="https://medium.com/a-cloud-guru">A Cloud Guru</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS Greengrass’ New Features, Announced & Hidden — from re:Invent 2018]]></title>
            <link>https://medium.com/@dzimine/aws-greengrass-new-features-announced-hidden-from-re-invent-2018-a34895d3536c?source=rss-63a761e7cf69------2</link>
            <guid isPermaLink="false">https://medium.com/p/a34895d3536c</guid>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[iot]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-greengrass]]></category>
            <category><![CDATA[awsreinvent]]></category>
            <dc:creator><![CDATA[Dmitri Zimine]]></dc:creator>
            <pubDate>Sun, 02 Dec 2018 05:24:45 GMT</pubDate>
            <atom:updated>2018-12-03T18:04:52.357Z</atom:updated>
            <content:encoded><![CDATA[<h3>AWS Greengrass’ New Features, Announced &amp; Hidden — from re:Invent 2018</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jeRpyiaP7kYaQKrTYLuj-g.jpeg" /></figure><p>This week at re:Invent 2018, AWS Greengrass team revealed a handful of new great features. It is probably the largest batch of Greengrass updates ever. So large, that some goodies have not been mentioned, so please <em>read to the end </em>to learn about them, too.</p><p><strong>Top highlights: Lambda container opt-in/opt-out, Greengrass in Docker, security tightening, ML performance optimizations, and many small yet practical improvements.</strong></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fl0tmTaIqAK8%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dl0tmTaIqAK8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fl0tmTaIqAK8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/378566532bfc4edc62b1e7f694a3829e/href">https://medium.com/media/378566532bfc4edc62b1e7f694a3829e/href</a></iframe><p>Making containerization optional for Lambdas is a big deal for installing Greengrass core on a broader variety of hardware. Too many devices run older Linux, or don’t support containers. Upgrading is not always an option. Disabling Lambda containerization works around these cases.</p><p>Running Greengrass in Docker simplified development flow and opens opportunities for integration with dev tool (VSCode debug support? any taker?)</p><p>Major security enhancements via integration with AWS Secret Manager keeps secrets encrypted in-wire and at-rest. Hardware security integration support helps protect our IoT projects from <a href="https://news.ycombinator.com/item?id=16844462">fish tank-like hacks</a>. Pre-compiled better-performing ML inference models. Proxy support, bulk deployment and similar seemingly little improvements show that AWS listens to us, the users. Ready-to-use connectors may be not as ready-to-use as one would expect, but it is part of a <a href="https://acloud.guru/series/serverlessconf-sf-2018/view/e6835b4c-d4cd-c72d-3d6e-d05a534fb595">bigger problem of serverless/FaaS reuse</a>. Read about all the new green goodies on <a href="https://aws.amazon.com/blogs/iot/aws-iot-greengrass-now-enables-simplified-deployments-enhanced-security-and-greater-flexibility/">AWS blog</a>, watch the <a href="https://youtu.be/l0tmTaIqAK8">video</a> from the Greengrass New features” session IOT365 at re:Invent, or quickly flip through the <a href="https://www.slideshare.net/AmazonWebServices/introducing-the-new-features-of-aws-greengrass-iot365-aws-reinvent-2018">slides</a>.</p><p>But there’s more! There are at least two little yet very dear extra features that were not mentioned on the blog or at the session. But here they are:</p><ul><li>Greengrass <strong>Core</strong> SDK for Python is <a href="https://pypi.org/project/greengrasssdk/">now on PyPi</a>. Remember there are two SDKs per runtime, one is IOT device SDK to run on devices, another is Greengras Core SDK. While device SDKs have always been on PyPi/NPM/Maven, the core SDK was not, which was a big pain. Great, it is no more! No more ugly hacks <a href="https://github.com/dzimine/gg-protocol-conversion/blob/master/scripts/build_py.sh#L13-L16">like this</a> to get this most common dependency to our Greengrass projects.</li></ul><blockquote>JavaScript fans is out of luck for now but I hold high hopes that it will come soon. For Java crowd — no sympathy, not from me.</blockquote><ul><li>Greengrass binary download is now available from a normal public download <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/what-is-gg.html#gg-core-download-tab">link</a>. Before one had to log in to AWS Web Console, drag &amp; click the mouse to download the binaries. No automation was possible, which caused pain of self-hosting the binaries or anti-pattern of putting it on Github. No more! The path is now clear for easy &amp; proper CI/CD for your Greengrass projects.</li></ul><p>Upgrade to Greengrass v1.7, hack more cool IoT projects. Like mini beer-store, a concept shopping experience inspired by Amazon Go, my re:Invent favorites:</p><h3>Dmitri Zimine on Twitter</h3><p>The coolest IoT setup by @rpedigoni and Renato Barbosa - retail checkout experience inspired by Amazon Go. Seriously awesome! Makers at the best. #reinvent @AWSreInvent #AWSreInvent2018 #IoT https://t.co/vHHr0I5wGC</p><p>Stay tuned to an upcoming <a href="https://greengo.io/">greengo.io</a> release with support to all the newly announced goodies.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a34895d3536c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS Greengrass — the Missing Manual]]></title>
            <link>https://medium.com/a-cloud-guru/aws-greengrass-the-missing-manual-2ac8df2fbdf4?source=rss-63a761e7cf69------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ac8df2fbdf4</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[iot]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[internet-of-things]]></category>
            <dc:creator><![CDATA[Dmitri Zimine]]></dc:creator>
            <pubDate>Thu, 12 Apr 2018 16:47:00 GMT</pubDate>
            <atom:updated>2018-06-07T18:56:29.587Z</atom:updated>
            <content:encoded><![CDATA[<h4>The Greengrass API is a pile of spare parts without wiring instructions — so here’s a map, an explanation, and some code</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MJqyV-FlufQ9pWq0qNvnMw.png" /></figure><p><a href="https://aws.amazon.com/greengrass/">AWS Greengrass</a> is a great technology. If you’re reading this, you likely already think so too. If you don’t — check out <a href="https://youtu.be/1IxDLeFQKPk?t=2h33m35s%20or%20https://www.youtube.com/watch?v=4JuyF2zwFp0">re:Invent keynote</a> and <a href="https://www.youtube.com/watch?v=1rLxPOxVJoQ">AWS Greengrass intro</a>, and you’ll come back convinced and inspired to hack.</p><p>To master Greengrass beyond the <a href="https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-gs.html">tutorials</a>, to define and deploy setups in automated, reliable, repeatable way, one must hit the AWS API or CLI — we long live in the bright DevOps world.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4KxRIDDmhgVfx2xdVGtE1g.png" /></figure><p>There is a big problem here: while <a href="http://boto3.readthedocs.io/en/latest/reference/services/greengrass.html">AWS documentation on Greengrass</a> lists individual calls, it gives no clue on where to start. There is no map to show how to wire the calls together. All parts are in place, but the assembly manual is missing. Or, <em>was</em> missing — until this post.</p><h4>Getting Started</h4><p>I’ll begin with the high level: concept definitions, a map of the model, some theory on how things work … but my millennial readers are eager to learn by doing, so let’s get started with some working code. No worries, you’ll still find the map and concepts below.</p><p>This is the code to create a Greengrass group with a single core.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f24334900f8a4601cf00d8005e9f8c02/href">https://medium.com/media/f24334900f8a4601cf00d8005e9f8c02/href</a></iframe><blockquote>As a bonus, get this code in a <a href="https://github.com/dzimine/greengo/blob/master/misc/greengrass_create_core.ipynb">Jupyter Notebook</a> for your convenience and immediate gratification!</blockquote><p>When you click a single “Create Group” button in the AWS console to create <em>Greengrass Core, </em>the code provided below is what happens bend the scenes. It’s quite a lot … 7 calls across 3 AWS services — 5 entities created explicitly, 2 implicitly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*zoVU04FHApFcNtH-nCEUyA.png" /></figure><p>Creating the group is just the beginning — you’ll still need to:</p><ul><li>create Lambda functions with AWS Lambda</li><li>register them with IoT via Lambda definitions</li><li>define resources for your Lambdas to access</li><li>configure loggers and register devices</li><li>setup MQTT subscriptions for every point-to-point communication</li><li>deploy the whole shebang to a Greengrass core</li></ul><p>And by the way, all of this must be properly installed and configured, carry the right certificates, and run on your edge device. If it sounds a bit complex, it is because it <em>is</em> a bit complex. Time to elevate to the concepts and get oriented.</p><h4>The Lifecycle Workflow</h4><p>First, let’s review the steps of a Greengrass operation lifecycle workflow.</p><ol><li><strong>Create Greengrass group definition in AWS IoT </strong>— a number of calls to AWS API to create and connect all necessary entities.</li><li><strong>Setup Greengrass core “on the edge” </strong>— configure a box Raspberry Pi or other <a href="https://aws.amazon.com/greengrass/faqs/#Greengrass_Core_Platform_Compatibility">compatible device</a>, or VM / Docker for dev &amp; testing. Download and install a proper version of greengrass, put certificates in place, adjust `config.json`, and launch the Greengrass daemon.</li><li><strong>Deploy </strong>— tell AWS to push Greengrass group definition to the Greengrass Core runtime(s).</li><li><strong>Clean up</strong> — the steps 1–3 produce a slew of artifacts that would pollute your AWS account if not properly recycled.</li></ol><h4><strong>The Model</strong></h4><p>Next let’s understand the model. The group definition — step 1 above — is by far the most complex. To help you navigate it, I created a “Greengrass Entity Map”. While the map is not a topologically precise representation of AWS API, it would help you grasp the model and not be lost.</p><p>Greengrass is intertwined with other AWS services — most notably IoT, Lambda, and IAM &amp; security which are color-coded on the map. Take a look at the map. Spot the objects created by the code above. Explore the rest. Enjoy!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HlRxVIZnld5EWNxi7rsnjg.png" /><figcaption>Please help make this map better — report <em>errors and omissions on the map </em><a href="https://github.com/dzimine/aws-greengrass-map/issues"><em>here</em></a><em>.</em></figcaption></figure><blockquote>The LoggerDefinition and ResourceDefinition blocks are not shown on the map to reduce clutter — they are relatively straightforward so you can extrapolate now that you see the pattern.</blockquote><p>Greengrass is conceptually a part of AWS IoT, and visually it’s a section of IoT in AWS console. But at the API level it is a separate service with distinct model — quite different from IoT in the concept, conventions, and even naming.</p><h4>The Entities</h4><p>In Greengrass, each entity is versioned and immutable. Any modification is done via updating an entity with a newly created version. Versions carry all the data and references to the other AWS services.</p><p><strong>Group Version<br></strong>The focal point of the model is a <em>Group Version</em>. A group manifests in the AWS console only once a <em>Group Version</em> is created. But to create a <em>Group Version</em>, you must first create <a href="http://boto3.readthedocs.io/en/latest/reference/services/greengrass.html#Greengrass.Client.create_group_version">a pinch of other entities</a> and supply them to <a href="http://boto3.readthedocs.io/en/latest/reference/services/greengrass.html#Greengrass.Client.create_group_version">create_group_version</a> call.</p><p><strong>Greengrass Core<br></strong>The <em>Greengrass Core</em> has dual nature: on the IoT side, it as a “<em>thing</em>”, or “device” for connection and communications, and is represented by IoT “<em>thing</em>” in the <a href="https://docs.aws.amazon.com/iot/latest/developerguide/thing-registry.html">IoT registry</a>, along with attached certificates, role and policies. On the Greengrass side, it is exposed as a <em>CoreDefinition/Version</em>.</p><p><strong>Lambda Functions<br></strong>Lambda functions are first created in AWS Lambda, and linked to Greengrass via <em>FunctionDefinition/Version</em>. It also adds some extra Greengrass-specific Lambda parameters — like pinned flag to make Lambda long running, device access policies, or extra environment variables.</p><p>To be linked, Lambdas must be published, and shall be referred by a <a href="https://docs.aws.amazon.com/lambda/latest/dg/versioning-intro.html">qualified ARN</a> — an ARN with a version or an alias (recommended). I wish AWS API doc told me about this— it doesn’t, and API throws a puzzling “functions definition is invalid or corrupted” message if you supply an unqualified ARN as FunctionArn.</p><p><strong>Subscriptions<br></strong><em>Subscriptions</em> are straightforward but too verbose to define: I find it irritating as it takes a bunch of them to define all communication between devices, lambdas, and cloud. How to deal with it? See “file-based approach to Greengrass setup” down below.</p><p><strong>Device Definition<br></strong><em>Device Definition</em> represent devices connected to the core as links to IoT Things — these Things must be set up in AWS IoT. Consider <a href="https://medium.com/@dzimine/two-ways-to-create-a-thing-in-the-internet-of-things-53504512b209">using ELF to quickly simulate a device</a> for you dev environment. <em>Logger Definition</em> tells Greengrass core what to log locally and what to send to CloudWatch. <em>Resource Definitions</em> are there to define Lambda access to resources of Greengrass core device group-wide, to attach to Lambda <em>Function Definitions.</em></p><blockquote>Oh, and did you notice Machine Learning resources in Greengrass AWS Console now that <a href="https://www.youtube.com/watch?v=EiQ8kqnAkno">AWS ML Inference is generally available?</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/690/1*7STCaWc-YF-unpeoRQRTiQ.png" /></figure><p>Enough said about the map: explore it, track and find the correspondent AWS API calls to create and connect the entities. I considered putting the exact methods on the map but decided to favor aesthetics, leaving you some space and joy of discovery.</p><h4>Deploying Greengrass</h4><p>Finally, a word on file-based approach to Greengrass deployment. As I played with Greengrass, I wanted — but missed—the three things I am so used to enjoy with Ansible, Terraform, or <a href="https://serverless.com/framework/">Serverless framework</a>:</p><ol><li>File-based Greengrass group definition</li><li>One-command to create, deploy, and clean things up</li><li>Simple, representable, repeatable dev boilerplate with Greengrass Core on a VM (Docker? even better!)</li></ol><p>I missed it, so I wrote one. Please welcome <a href="https://github.com/dzimine/greengo"><strong>greengo</strong></a> — a simple file based Greengrass configuration tool.</p><figure><a href="https://github.com/dzimine/greengo"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qrmEf986VtQEPyif-J-0Pw.png" /></a><figcaption><a href="https://github.com/dzimine/greengo">Greengo — a simple Greengrass configuration tool</a></figcaption></figure><p>Take it as your Greengrass dev boilerplate — adjust greengo.yaml to your liking and hit greengo create. Or use is as a cheat-sheet to find out how to call the right boto3 methods in the right order with the right parameters, as you track the chains on the map above.</p><p><strong>That’s it. </strong>I hope that the map above, the code examples, the explanations and greengo tool/cheat-sheet will fill out the blanks in AWS doc and help you operate Greengrass in automated, repeatable way, DevOps style.</p><p>Happy green-grassing!</p><figure><a href="https://github.com/dzimine/greengo"><img alt="" src="https://cdn-images-1.medium.com/max/100/1*tUwVEtT-gtH7wAYsbkiy_w.png" /></a></figure><p><em>If you find this manual helpful or interesting, I take your claps 👏 for tips . Please share your thoughts in the comments here, or follow me</em><a href="https://twitter.com/dzimine"><em> </em></a><em>on Twitter </em><a href="https://twitter.com/dzimine"><em>@dzimine</em></a><em> for discussions and stories on IoT, DevOps, and Serverless.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ac8df2fbdf4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/a-cloud-guru/aws-greengrass-the-missing-manual-2ac8df2fbdf4">AWS Greengrass — the Missing Manual</a> was originally published in <a href="https://medium.com/a-cloud-guru">A Cloud Guru</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a community sign-up app with Serverless, StepFunctions, and StackStorm Exchange — Episode…]]></title>
            <link>https://medium.com/free-code-camp/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-7c5f0e93dd6?source=rss-63a761e7cf69------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c5f0e93dd6</guid>
            <category><![CDATA[aws-step-functions]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[faa]]></category>
            <dc:creator><![CDATA[Dmitri Zimine]]></dc:creator>
            <pubDate>Wed, 03 Jan 2018 02:03:56 GMT</pubDate>
            <atom:updated>2018-01-18T21:48:25.727Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a community sign-up app with Serverless, StepFunctions, and StackStorm Exchange — Episode 4</h3><p>Build a real-world serverless application on AWS with <a href="https://serverless.com/framework/">Serverless framework</a> and ready-to-use functions from <a href="https://exchange.stackstorm.org/">StackStorm Exchange</a> open-source catalog.</p><p><a href="https://medium.com/@dzimine/tutorial-building-a-community-on-boarding-app-with-serverless-stepfunctions-and-stackstorm-b2f7cf2cc419">Episode One</a> | <a href="https://medium.com/@dzimine/building-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-2-b1efeb1b9bd6">Episode Two</a> | <a href="https://medium.com/@dzimine/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-6efb9c102b0a">Episode Three</a> | Episode Four</p><p>It took us three exciting episodes to create a non-trivial serverless app. The back-end now works, but two things are still missing: a Web UI, and a final discussion to sum up the lessons. Let’s continue… after a quick recap of the previous episodes:</p><ul><li>In <a href="https://medium.com/@dzimine/tutorial-building-a-community-on-boarding-app-with-serverless-stepfunctions-and-stackstorm-b2f7cf2cc419">Episode One</a>, I described the application we are building, walked you through setting up the development environment and creating a Serverless project, and showed how to build your first Lambda function from a <a href="https://exchange.stackstorm.org/">StackStorm Exchange</a> action with <a href="https://serverless.com/framework">Serverless Framework</a>.</li><li>In <a href="https://medium.com/@dzimine/building-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-2-b1efeb1b9bd6">Episode Two</a>, we added more actions: one native Lambda to record user info to DynamoDB, and another one from StackStorm Exchange to make a call to ActiveCampaign CRM system. You learned more of serverless.ymlsyntax and practiced the development workflow with Lambda functions.</li><li>In <a href="https://medium.com/@dzimine/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-6efb9c102b0a">Episode Three</a>, we wired the actions with StepFunctions, learned tricks on passing the data between the steps, struggled debugging StepFunction executions, and finally got the backend working end-to-end.</li></ul><p>You can get the final code for this episode from <a href="https://github.com/dzimine/slack-signup-serverless-stormless/tree/DZ/3-add-stepfunction">GitHub</a>.</p><h3>Add Web UI</h3><p>We need a web form that takes user names and emails, and posts to our StepFunction API Gateway endpoint. I just copied the one I used while <a href="https://github.com/dzimine/slack-signup-serverless">exploring Serverless with Python, StepFunctions, and Web Front-end Serverless</a>. Which, I confess, I grabbed from the old <a href="https://github.com/serverless-london/serverless-slack-invite">Serverless Slack Invite</a> project.</p><p>Those of you who are Web developers can surely create something more elegant with ReactJS — PRs welcome! Whether you build your own or grab mine, do this: create a directory web and place the static content there.</p><p>I use <a href="https://www.npmjs.com/package/http-server">http-server</a> for a quick look that at my static form:</p><pre>cd web<br>http-server<br>Starting up http-server, serving ./<br>Available on:<br>  <a href="http://127.0.0.1:8080/">http://127.0.0.1:8080</a></pre><p>Open a browser, check the form at <a href="http://127.0.0.1:8080/">http://127.0.0.1:8080</a> , see that it comes out right, and let’s move on to the next step.</p><p>How is a serverless web front end deployed to AWS? Typically, you put the static content on S3, configure it to serve the web, make your web app call your backend endpoint, and remember to enable CORS on API Gateway endpoint.</p><p>Alternatively, you add a path to your API Gateway resource to serve your static web content from the S3 bucket.</p><p><strong>Pros:</strong> 1) no mess with CORS, 2) no adjusting your web app to point to the correct backend endpoint. Extra bonus: easily done with <a href="https://github.com/sdd/serverless-apig-s3">serverless-apig-s3</a> plugin.</p><p><strong>Cons:</strong> 1) paying API Gateway charges for web requests 2) the plugin is a bit shaky: fine for examples and small apps, but I wouldn’t use it for anything resembling production.</p><p><strong>PRO TIP:</strong> For serious web front-ends with high loads: deploy them to CloudFront. Or use something like Netlify — check out the recent post <a href="https://serverless.com/blog/how-built-static-serverless-website-netlify/">“How to build a static Serverless site with Netlify”</a> from serverless.com.</p><p>I’ll use the API Gateway approach with <a href="https://github.com/sdd/serverless-apig-s3">serverless-apig-s3</a> plugin here: we don’t expect massive load. And it is simple. Watch:</p><p>Install the plugin (been there, done that):</p><pre>npm install --save-dev serverless-apig-s3</pre><p>Change serverless.yml. Add the plugin (been there, done that). Add the <a href="https://github.com/sdd/serverless-apig-s3">plugin’s configurations</a>:</p><pre>...<br>custom:<br>  private: ${file(env.yml):private}<br>  stage: ${opt:stage, self:provider.stage}<br>  region: ${opt:region, self:provider.region}<br>  apigs3:<br>    dist: web<br>    topFiles: true</pre><pre>...</pre><pre>plugins:<br>  - serverless-plugin-stackstorm<br>  - serverless-step-functions<br>  - serverless-apig-s3</pre><p>We are telling the plugin to pick our web directory and put it to S3. The topFiles flag tells API Gateway to expose our index.html and formsubmit.js at our endpoint, under /web/</p><p>Now deploy the service, and, separately, the client side, with two commands:</p><pre>sls deploy</pre><pre>sls client deploy</pre><p>Deploying the service will update the API Gateway with the paths to access our web content. Deploying the client puts the web content up to an S3 bucket. Now you can reason that if you add or remove web files and client deploy is not enough, a full sls deploy is required. But if you only change the web files, quick sls client deploy will put them up on AWS.</p><p>That’s it! <br>Go to https://YOUR-ENDPOINT.amazonaws.com/dev/web/index.html.The form is right there. The button is orange. You type in the user info and email, hit the orange button. The green message tells you everything is OK. Soon you receive an invitation from Slack. You check the StepFunction executions in AWS console and see everything is green.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/543/1*1T0kQDlFVqmDrWsJ1hCy9A.png" /></figure><p><strong>No?!</strong> Most likely, you hit this <a href="https://github.com/sdd/serverless-apig-s3/issues/16">apig-s3 bug</a>, just like I did. And just like I did, you got to do a little final step manually.</p><p>Open AWS Console -&gt;API Gateway-&gt;Resources-&gt;Actions-Deploy API. <br>Or use AWS CLI:</p><pre># Find out your rest-api-id first<br>aws apigateway get-rest-apis</pre><pre># Deploy the changes to the stage<br># Don&#39;t COPY-PASTE, it needs YOUR ID!<br>aws apigateway create-deployment --rest-api-id  YOUR_API_ID  --stage-name dev</pre><p><strong>Now it works for me.</strong></p><p><strong>PRO TIP: </strong>The plugin works when everything is deployed from scratch. But if you’re tempted to sls remove and redeploy again, you hit at least one <a href="https://github.com/sdd/serverless-apig-s3/issues/11">other plugin bug</a> on removing the stack. See Tips and Tricks at the end of the post on how to properly clean-up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jiZOQLvfX2257LHxe3n4cg.png" /></figure><p>Congratulations! You made it. You might have followed along and built your own app. You might have dropped it at some point out of frustration or fatigue. You might have just skimmed the text and examples. But if you made it to the end, you learned enough of serverless to be dangerous.</p><h3>Reflection</h3><p>After wrestling through all the 4 episodes, you likely sensed it: serverless is not simple. Many things can go wrong as you build a serverless app.</p><p>You can blame some on me: I surely made a few mistakes and omissions (BTW reports and comments much appreciated!)</p><p>But serverless complexity is not all my fault. We wire together many different services — AWS and 3rd party — with many different wires — frameworks and tools. The services are of different quality and [in]convenience (don’t get me started on AWS StepFunction CLI). The frameworks and tools are still maturing (which serverless plugin made you struggle the most?).</p><p>Why get into serverless? Because for some classes of apps, it is substantially cheaper. This community signup is a case in point: we used to run StackStorm’s with StackStorm. It was simple to build and solid like a rock, but it took a m3.large instance at $100/month, without even getting to multi-zone reliability. With serverless it runs at less than $1/month, thanks to the occasional load pattern. $100 would buy us ~5,000,000 sign-ups — that’s more that we’ll ever need!</p><p>On the other extreme, high volume apps with high reliability requirements might be better as serverless too, leveraging “infinite” elasticity which may be expensive to build and operate yourself. I carefully use “<em>might</em>” and “<em>would</em>”, as the equation varies greatly from case to case. Do your math upfront, watch your load patterns, explore your bill.</p><p>As for taming complexity, <a href="https://serverless.com/framework">Serverless framework</a> helps. To fully appreciate it, try and redo the same app without it. Make it reliable, repeatable, reviewable infrastructure-as-code. While Serverless framework is not the only game in town, I specifically like it for it’s pluggable architecture and the plugin ecosystem.</p><p>Plugins help, picking up in areas not covered by the core Serverless framework. Using the plugins, we enjoyed the simplicity of building StepFuncions and adding simple web front-ends (and if you haven’t, try and redo it without the plugins). There are more: skim and bookmark the <a href="https://github.com/serverless/plugins">list of official plugins</a> to keep them in mind when for your next project.</p><p><a href="https://exchange.stackstorm.org/">StackStorm Exchange</a> — the recent addition to the serverless ecosystem — brings a catalog of reusable integrations. While it is not hard to scrape APIs for ActiveCampaign or find and use undocumented Slack API calls, it is low-value work that takes away from building the application logic.</p><p>Explore what is already there: while DevOps integrations dominate due to StackStorm roots in IT automation, I expect that variety will grow now that Serverless community is joining to contribute and co-pilot the catalog.</p><p>With that, <strong>we are officially done. </strong>Enjoy!</p><h3>A few more Tips and Tricks</h3><ul><li>Removing everything will not remove the DynamoDB table. A reasonable default, but when you try and re-deploy the service, it’ll cry that the table can’t be created as it already exists. Delete it: aws dynamodb delete-table --table-name signup-stormless-dev</li><li>Due to an apig-s3 bug, sls remove will fail complaining that the bucket is not empty. Remove the web s3 bucket manually when removing a stack. aws s3 rb s3://bucketname --force</li><li>Removing the stack does not remove the DynamoDB table. If you really want to start from scratch, delete it manually (export the data first):<br> aws dynamodb delete-table --table-name slack-signup-dev</li><li>Sometimes sls delete fails to clean up. Reasons might be various, but the place to look is the same. Did I tell you to master CloudFormation? Go there, find a stack that failed to delete, find the reason, fix, and delete the stack.</li></ul><p>Hope this helped you learn something new, find something interesting, or provoked some good thoughts. Please share your thoughts in the comments here, or tweet me<a href="https://twitter.com/dzimine"> @dzimine</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c5f0e93dd6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/free-code-camp/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-7c5f0e93dd6">Building a community sign-up app with Serverless, StepFunctions, and StackStorm Exchange — Episode…</a> was originally published in <a href="https://medium.com/free-code-camp">We’ve moved to freeCodeCamp.org/news</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a community sign-up app with Serverless, StepFunctions, and StackStorm Exchange — Episode…]]></title>
            <link>https://medium.com/free-code-camp/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-6efb9c102b0a?source=rss-63a761e7cf69------2</link>
            <guid isPermaLink="false">https://medium.com/p/6efb9c102b0a</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws-step-functions]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[faa]]></category>
            <dc:creator><![CDATA[Dmitri Zimine]]></dc:creator>
            <pubDate>Thu, 21 Dec 2017 07:06:59 GMT</pubDate>
            <atom:updated>2018-01-04T00:02:15.357Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a community sign-up app with Serverless, StepFunctions, and StackStorm Exchange — Episode 3</h3><p>Build a real-world serverless application on AWS with <a href="https://serverless.com/framework/">Serverless framework</a> and ready-to-use functions from <a href="https://exchange.stackstorm.org/">StackStorm Exchange</a> open-source catalog.</p><p><a href="https://medium.com/@dzimine/tutorial-building-a-community-on-boarding-app-with-serverless-stepfunctions-and-stackstorm-b2f7cf2cc419">Episode One</a> | <a href="https://medium.com/@dzimine/building-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-2-b1efeb1b9bd6">Episode Two</a> | Episode Three | <a href="https://medium.com/@dzimine/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-7c5f0e93dd6">Episode Four</a></p><p>We are at Episode Three. Quick recap:</p><ul><li>In <a href="https://medium.com/@dzimine/tutorial-building-a-community-on-boarding-app-with-serverless-stepfunctions-and-stackstorm-b2f7cf2cc419">Episode One</a>, I described the application we are building, walked you through setting up the development environment and creating a Serverless project, and showed how to build your first Lambda function from a <a href="https://exchange.stackstorm.org/">StackStorm Exchange</a> action with <a href="https://serverless.com/framework">Serverless Framework</a>.</li><li>In <a href="https://medium.com/@dzimine/building-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-2-b1efeb1b9bd6">Episode Two</a>, we added more actions: one native Lambda to record user info to DynamoDB, and another one from StackStorm Exchange to make a call to ActiveCampaign CRM system. You learned more of serverless.yml syntax and practiced the development workflow with Lambda functions.</li></ul><p>In this third episode, I’ll show how to use AWS StepFunction to wire the actions into a workflow.</p><p>You can get the final code for this episode from <a href="https://github.com/dzimine/slack-signup-serverless-stormless/tree/DZ/3-add-stepfunction">GitHub</a>.</p><h3>Wiring functions together with StepFunction</h3><p>Now that our building blocks — Lambda functions — are all lined up, it’s time to string them together. An <a href="https://aws.amazon.com/step-functions/">AWS StepFunction</a> will define the sequence of calls, maintain the state of the sign-up workflow, and carry the data between the steps. I’ll use the<a href="https://github.com/horike37/serverless-step-functions">serverless-step-functions</a> plugin from Serverless Champion @horike37, give him a heart:</p><h3>Serverless on Twitter</h3><p>🙌 Celebrating Serverless Champion @horike37 - https://t.co/LgRnBFjxfK - #serverless #aws https://t.co/S0Mb4Wvs2r</p><p>Let’s get busy. Install the plugin:</p><pre>npm install --save-dev serverless-step-functions</pre><p>Add the plugin to the serverless.yml file:</p><pre>plugins:<br>  - serverless-plugin-stackstorm<br>  - serverless-step-functions</pre><p>The Step Function definition will require my accountID. As it is something I want to keep to myself, I add it to env.yml, which now looks like this:</p><pre># ./env.yml<br># Don&#39;t commit to Github :)</pre><pre>slack:<br>  admin_token: &quot;xoxs-111111111111-...&quot;<br>  organization: &quot;your-team&quot;<br>active_campaign:<br>  url: &quot;https://YOUR-COMPANY.api-us1.com&quot;<br>  api_key: &quot;1234567a9012a12345z12aa2aaa...&quot;<br>private:<br>  accountId: &quot;000000000000&quot;</pre><p>Go back to serverless.yml and add the following two blocks:</p><pre># ./serverless.yml<br>...<br>...<br>custom:<br>  private: ${file(env.yml):private}<br>  stage: ${opt:stage, self:provider.stage}<br>  region: ${opt:region, self:provider.region}</pre><pre>stepFunctions:<br>  stateMachines:<br>    signup:<br>      events:<br>        - http:<br>            path: signup<br>            method: POST<br>            cors: true<br>      definition: ${file(stepfunction.yml)}</pre><p>In the custom block, I assigned the private object from the private key in env.yml. I also defined variables for stage and region so that the values are picked from CLI options, if provided, or default to the current AWS settings.</p><p>The stepFunctions block is here to define - you have already guessed - StepFunctions. Mine is called &quot;signup&quot;.</p><p>The events section here is doing exactly what events sections do in function definitions: it configures an API Gateway endpoint for invoking StepFunction from outside of AWS. We&#39;ll use it later to call the back-end from a Web form.</p><p>The definition can be written as YAML right here in serverless.yml, but I prefer to include it from a separate file, keeping the logic separate from the configuration. Here it is:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dc5a7c366ed2ce6e28c7465e7eb7d465/href">https://medium.com/media/dc5a7c366ed2ce6e28c7465e7eb7d465/href</a></iframe><p>StepFunction definitions are written in <a href="https://states-language.net/spec.html">Amazon States Language</a>. The spec is short, well written and worth a read. Using YAML instead of JSON is a nice perk from the plugin — it reads better and allows comments. But if you want JSON — no problem, help yourself.</p><ul><li>Resource refers to Lambda functions by ARN. I used the variables we defined earlier to construct the ARNs matching account ID, region, and stage with the function name: arn:aws:lambda:${self:custom.region}:${self:custom.private.accountId}:function:${self:service}-${self:custom.stage}-RecordDB</li><li>ResultPath is used to pass data between steps. By default, StepFunctions work on a &quot;need-to-know&quot; basis: the step downstream receives only the output from the step directly upstream. If you think it logical, think again: if only RecordDB receives the workflow input, how will RecordAC and InviteSlack get it? RecordDB may just return &quot;200 OK&quot;, not email. Changing the code of functions to return their input would make them <a href="https://refactoring.guru/smells/inappropriate-intimacy">inappropriately intimate</a>. The trick is to use ResultPath to write the function output under a function-specific key, like ResultPath: $results.RecordDB. This preserves initial workflow input in the step output for downstream Lambda steps, while appending the output of each Lambda. Like this:</li></ul><pre>{<br>  &quot;body&quot;: {<br>    &quot;name&quot;: &quot;Vasili Terkin&quot;,<br>    &quot;email&quot;: &quot;dmitri.zimine+terkin@gmail.com&quot;,<br>    &quot;first_name&quot;: &quot;Vasili&quot;,<br>    &quot;last_name&quot;: &quot;Terkin&quot;<br>  },<br>  &quot;results&quot;: {<br>    &quot;RecordDB&quot;: {<br>      &quot;statusCode&quot;: 200<br>    },<br>    &quot;RecordAC&quot;: ...<br>    ...<br>  }<br>}</pre><p>To fully grasp it, read the <a href="https://states-language.net/spec.html#filters">“Input and Output” section</a> in the spec. Ansd entertain youself with a video from “<a href="https://foobar123.com/aws-step-functions-tutorial-76b9f5a7b9c8">AWS Step Functions Tutorial</a>” by <a href="https://medium.com/u/bd0dde1d3812">Marcia Villalba</a>.</p><p><strong>PRO TIP:</strong> I made the workflow sequential to demonstrate the data passing trick. It is more proper to run all three steps in parallel: it is faster and more resilient: the failure of one step will not prevent the other function invocations. Go ahead change the StepFunctions to parallel.</p><p>That is it. Time to try. Deploy, invoke, check the logs.</p><p>You CURL fans know what to do with the new API Gateway endpoint for our StepFunction. If you forgot the endpoint, sls info to the rescue. I’ll show off again with <a href="https://httpie.org/">httpie</a>:</p><pre># DON&#39;T COPY! Use YOUR ENDPOINT!</pre><pre>http POST https://YOUR.ENDPOINT.amazonaws.com/dev/signup \<br>body:=&#39;{&quot;email&quot;:&quot;santa@mad.russian.xmas.com&quot;, &quot;first_name&quot;:&quot;Santa&quot;, &quot;last_name&quot;: &quot;Claus&quot;}&#39;</pre><p>Ok, http or curl, either way it returns the StepFunction execution ARN so that we can check on it to see how our StepFunction is being executed. How do we check on it? I’m afraid you gotta open a browser and login to your AWS Console. If you want to use AWS CLI first, fine, don’t say I didn’t show you how:</p><pre>aws stepfunctions describe-execution --execution-arn arn:aws:states:us-east-1:00000000000:execution:SignupStepFunctionsStateMac-seo9CrijATLU:cbeda709-e530-11e7-86d3-49cbe4261318 --output json<br>{<br>    &quot;status&quot;: &quot;FAILED&quot;, <br>    &quot;startDate&quot;: 1513738340.18, <br>    &quot;name&quot;: &quot;cbeda709-e530-11e7-86d3-49cbe4261318&quot;, <br>    &quot;executionArn&quot;: &quot;arn:aws:states:us-east-1:00000000000:execution:SignupStepFunctionsStateMac-seo9CrijATLU:cbeda709-e530-11e7-86d3-49cbe4261318&quot;, <br>    &quot;stateMachineArn&quot;: &quot;arn:aws:states:us-east-1:00000000000:stateMachine:SignupStepFunctionsStateMac-seo9CrijATLU&quot;, <br>    &quot;stopDate&quot;: 1513738370.481, <br>    &quot;input&quot;: &quot;{\&quot;body\&quot;:{\&quot;email\&quot;:\&quot;santa@mad.russian.xmas.com\&quot;,\&quot;first_name\&quot;:\&quot;Santa\&quot;,\&quot;last_name\&quot;:\&quot;Claus\&quot;}}&quot;<br>}</pre><p>This is the output for an execution that failed because the RecordAC function timed out. Can you infer this from the output? The only valuable info here is FAILED. No kidding! I must say AWS don&#39;t give StepFunction the love it deserves. Not in CLI. If you think I missed something, check <a href="http://docs.aws.amazon.com/cli/latest/reference/stepfunctions/index.html">the CLI docs</a>, find it and tell me.</p><p>The most irritating part is that the CLI doesn’t tell me which step failed. They make me call the logs on every Lambda, one by one. Luckily I only have 3 functions, what if there were more?</p><p>Or, open a browser and hop on AWS Console.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/984/1*0xq8ywxkBv-oIe6b-6H9ug.png" /></figure><p>Even there, debugging some Lambda failures, like timeouts, is tricky. StepFunction execution “Exception” report says &quot;The cause could not be determined because Lambda did not return an error type.&quot;Go to the lambda logs to see what happened there.</p><p>There I find the line which I wish I saw in the StepFunction exception:</p><pre>2017-12-20T04:21:44.298Z 4230a73b-e53d-11e7-be6b-bff82b9b3572 Task timed out after 6.00 seconds</pre><p><strong>PRO TIP:</strong> For debugging, invoke the StepFunction with sls invoke stepf: it creates the execution, waits for completion, and prints the output to the terminal. Three AWS CLI commands in one.</p><pre>sls invoke stepf --name signup \<br>--data  &#39;{&quot;body&quot;: {&quot;email&quot;:&quot;<a href="/service/mailto:santa@mad.russian.xmas.com">santa@mad.russian.xmas.com</a>&quot;, &quot;first_name&quot;:&quot;Santa&quot;, &quot;last_name&quot;: &quot;Clause&quot;}}&#39;</pre><p>Your StepFunction executions may work just fine — we already adjusted the timeouts. I took you on this debugging detour for a taste of StepFunction troubleshooting, admittedly a bit bitter. On the sweet side, once debugged, StepFunctions run reliably like Toyota cars.</p><p>As a back-end developer, I am tempted to call it done here. But to make it a <br>“complete example” we need one more thing. <strong>The Web front-end.</strong></p><p>Let’s call it a day and save the web part and conclusions for the next and final episode.</p><p><a href="https://medium.com/@dzimine/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-7c5f0e93dd6"><strong>Episode 4</strong></a><strong>: Adding Web Front-end, Reflection and Summary</strong></p><p>Hope this helped you learn something new, find something interesting, or provoked some good thoughts. Please share your thoughts in the comments here, or tweet me<a href="https://twitter.com/dzimine"> @dzimine</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6efb9c102b0a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/free-code-camp/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-6efb9c102b0a">Building a community sign-up app with Serverless, StepFunctions, and StackStorm Exchange — Episode…</a> was originally published in <a href="https://medium.com/free-code-camp">We’ve moved to freeCodeCamp.org/news</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building community sign-up app with Serverless, StepFunctions and StackStorm Exchange — Episode  2]]></title>
            <link>https://medium.com/free-code-camp/building-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-2-b1efeb1b9bd6?source=rss-63a761e7cf69------2</link>
            <guid isPermaLink="false">https://medium.com/p/b1efeb1b9bd6</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[lambda]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Dmitri Zimine]]></dc:creator>
            <pubDate>Tue, 19 Dec 2017 19:41:18 GMT</pubDate>
            <atom:updated>2018-01-04T00:00:56.991Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a community sign-up app with Serverless, StepFunctions, and StackStorm Exchange — Episode 2</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*-0KcbVF9M6t_UvLpObeNYw.jpeg" /></figure><p>Build a real-world serverless application on AWS with <a href="https://serverless.com/framework/">Serverless framework</a> and ready-to-use functions from <a href="https://exchange.stackstorm.org/">StackStorm Exchange</a> open-source catalog.</p><p><a href="https://medium.com/@dzimine/tutorial-building-a-community-on-boarding-app-with-serverless-stepfunctions-and-stackstorm-b2f7cf2cc419">Episode One</a> | Episode Two | <a href="https://medium.com/@dzimine/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-6efb9c102b0a">Episode Three</a> | <a href="https://medium.com/@dzimine/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-7c5f0e93dd6">Episode Four</a></p><p>In this second episode, I’ll add two more actions: create a contact in ActiveCampaign, and record a user in Dynamo DB. The full code for this episode is on Github at <a href="https://github.com/dzimine/slack-signup-serverless-stormless/tree/DZ/2-add-more-actions">2-add-more-actions branch</a>.</p><h3>Add more actions</h3><p>To continue to the next step, you’ll need an <a href="https://www.activecampaign.com/">ActiveCampaign</a> account. Fear not: it takes 2 minutes — I just tried — they didn’t ask me for a credit card or anything stupid. Just email. Once in, go to “My Settings”, and select “Developer”.</p><p>Note the URL and Key fields in the &quot;API access&quot; section. Add them into env.yml like this:</p><pre># Copy to env.yml and replace with your values.<br># Don&#39;t commit to Github :)<br>slack:<br>  admin_token: &quot;xoxs-111111111111-111111111111-...&quot;<br>  organization: &quot;your-team&quot;<br>active_campaign:<br>  url: &quot;https://YOUR-COMPANY.api-us1.com&quot;<br>  api_key: &quot;1234567a9012a12345z12aa2...&quot;</pre><p><strong>PRO TIP:</strong> If you’re not in the mood for Active Campaign, or any sign-ups, just mock up the API endpoint with something like <a href="https://www.mocky.io/">Mocky</a> or <a href="https://www.mocky.io/">mockable.io</a>and adjust the examples accordingly. For bonus points, create another serverless function with API Gateway endpoint in your serverless.yml and use it to mock the ActiveCampaign API call.</p><p>Now I am ready to add an <a href="https://exchange.stackstorm.org/#activecampaign">Active Campaign</a> lambda function. Same routine as in <a href="https://medium.com/@dzimine/tutorial-building-a-community-on-boarding-app-with-serverless-stepfunctions-and-stackstorm-b2f7cf2cc419">Episode One</a>: start with finding a new action in a pack with sls stackstorm info --pack activecampaign (hint: the action you&#39;re looking is contact_sync which adds and updates a contact). Inspect the action with sls stackstorm info --action activecampaign.contact_sync:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cf2d7190c3dc87697701d78425cbf397/href">https://medium.com/media/cf2d7190c3dc87697701d78425cbf397/href</a></iframe><p>Bunch of parameters, but besides the config, only email is required. I also want to use first_name and last_name. The function definition in serverless.yml will look like this:</p><pre>RecordAC:<br>    timeout: 10<br>    memorySize: 128<br>    stackstorm:<br>      action: activecampaign.contact_add<br>      input:<br>        email: &quot;{{ input.body.email }}&quot;<br>        first_name: &quot;{{ input.body.first_name }}&quot;<br>        last_name: &quot;{{ input.body.last_name }}&quot;<br>      config: ${file(env.yml):activecampaign}</pre><p>I had to bump up the timeout as the ActiveCampaign API is taking longer than Lambda&#39;s default 6 seconds during Christmas season. But I can reclaim invocation cost by lowering memory use from default 512Mb. This time I don&#39;t bother to expose it to AWS API Gateway - we can conveniently test it with sls.</p><p>The function is ready to fly to AWS. We could do it at once with sls deploybut let&#39;s take it slow again and repeat the deployment steps just like in <a href="https://medium.com/@dzimine/tutorial-building-a-community-on-boarding-app-with-serverless-stepfunctions-and-stackstorm-b2f7cf2cc419">episode 1</a> of this tutorial to engrave the workflow in your mind:</p><ol><li>Build and package the bundle:</li></ol><pre>sls package</pre><p>2. Test locally (note I’m using --passthrough to only test transformations, remove it to make an actual call):</p><pre>sls  stackstorm docker run --function RecordAC \<br>--passthrough \<br>--verbose <br>--data &#39;{&quot;body&quot;:{&quot;email&quot;:&quot;<a href="/service/mailto:santa@mad.russian.xmas.com">santa@mad.russian.xmas.com</a>&quot;, &quot;first_name&quot;:&quot;Santa&quot;, &quot;last_name&quot;: &quot;Claus&quot;}}&#39;</pre><p>3. Deploy to AWS:</p><pre>sls deploy</pre><p>4. Test on AWS with sls invoke:</p><pre>sls invoke --function RecordAC --logs \ <br>--data &#39;{&quot;body&quot;:{&quot;email&quot;:&quot;santa@mad.russian.xmas.com&quot;, &quot;first_name&quot;:&quot;Santa&quot;, &quot;last_name&quot;: &quot;Claus&quot;}}&#39;</pre><p>5. Check the logs:</p><pre>sls logs --function  RecordAC</pre><p>and I checked, it works: Santa Claus appeared in ActiveCampaign’s contact list. How do I know it’s our Lambda? Because I no longer believe Santa is real enough to subscribe to community without our little Lambda function.</p><p><strong>PRO TIP:</strong> If you hit a bug or want a feature in a pack you are using from <a href="https://exchagne.stackstorm.org/">StackStorm Exchange</a>, you can fix it on the spot. Packs are cloned under ./~st2/packs. Find your action there, modify the code, and use a local run to debug and validate. <br>Big bonus for pushing fixes back to the Exchange: each pack is already a git clone that makes it naturally easy to contribute back to the community.</p><p>Let’s add the final action, RecordDB, that writes contact info into a DynamoDB table. I could have used the aws.dynamodb_put_item action from the <a href="https://exchange.stackstorm.org/#aws">AWS pack on StackStorm Exchange</a> - the pack is heavily used and well maintained. But I decided to make it a native Lambda: it&#39;s just 30 lines of code, with no extra Python dependencies, since the Boto library is already in the AWS Lambda environment.</p><p>The function’s code goes into ./record_db/handler.py:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/11774c2a8f24827af2fc9395bdadfc0a/href">https://medium.com/media/11774c2a8f24827af2fc9395bdadfc0a/href</a></iframe><p>The final serverless.yml with all three actions now looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/aec4918f4b510459c5bfd5f9df3926dc/href">https://medium.com/media/aec4918f4b510459c5bfd5f9df3926dc/href</a></iframe><p>Wow! The file doubled. The function itself is just 2 lines (35:36), but quite a few interesting additions took place. Let’s review them:</p><ol><li>Line 9 — Table name generated to avoid collisions between regions and environments.</li><li>Lines 10:13 — IAM Role created to give the function access to the DynamoDB table.</li><li>Line 39:56 — AWS CloudFormation template defining the table.</li></ol><p>The last point brings up an important observation: <a href="https://medium.com/(https://serverless.com/framework)">Serverless framework</a> simplifies mainly the FaaS part of serverless. But because servereless is more than FaaS, you’ll find yourself writing quite a lot of CloudFormation to provision other services. Serverless framework leaves space for provider specific resources. To be seriously serverless on AWS, master CloudFormation. Looking for “provider-agnostic serverless”? Double-check your assumptions.</p><p>Also, I moved memorySize and timeout to apply to all functions (lines 6:7). And the API Gateway endpoint is gone from the InviteSlack function: it served the demonstration role in the first episode, but now we&#39;ve learned to test Lambda functions directly. We&#39;ll return to API Gateway later to invoke the final StepFunction end-to-end workflow.</p><p>Let’s get the RecordDB function up to the sky and running. <br>Deploy, invoke, logs. Rinse repeat.</p><pre># Deploy<br>sls deploy <br>...</pre><pre># Invoke<br>sls invoke --function RecordDB --logs --data &#39;{&quot;body&quot;:{&quot;email&quot;:&quot;santa@mad.russian.xmas.com&quot;, &quot;first_name&quot;:&quot;Santa&quot;, &quot;last_name&quot;: &quot;Claus&quot;}}&#39;<br>...</pre><pre># Check logs.<br>sls logs --function RecordDB.</pre><p>Now all three actions are here, waiting to be wired together into a final workflow with a StepFunction. But I’m just reminded not to be Mr. Scrooge: it’s the holiday season, let’s take it easy and put it off to the next episode. Until then, keep the spirits high!</p><p>The complete code example for this episode is on Github at <a href="https://github.com/dzimine/slack-signup-serverless-stormless/tree/DZ/2-add-more-actions">2-add-more-actions</a> branch.</p><p><a href="https://medium.com/@dzimine/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-6efb9c102b0a"><strong>Episode 3</strong></a><strong>: Wiring the actions with a StepFunction</strong></p><p>Hope this helped you learn something new, find something interesting, or provoked some good thoughts. Please share your thoughts in the comments here, or tweet me<a href="https://twitter.com/dzimine"> @dzimine</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b1efeb1b9bd6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/free-code-camp/building-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-2-b1efeb1b9bd6">Building community sign-up app with Serverless, StepFunctions and StackStorm Exchange — Episode  2</a> was originally published in <a href="https://medium.com/free-code-camp">We’ve moved to freeCodeCamp.org/news</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tutorial: Building a community on-boarding app with Serverless, StepFunctions, and StackStorm…]]></title>
            <link>https://medium.com/free-code-camp/tutorial-building-a-community-on-boarding-app-with-serverless-stepfunctions-and-stackstorm-b2f7cf2cc419?source=rss-63a761e7cf69------2</link>
            <guid isPermaLink="false">https://medium.com/p/b2f7cf2cc419</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[slack]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Dmitri Zimine]]></dc:creator>
            <pubDate>Mon, 18 Dec 2017 18:15:36 GMT</pubDate>
            <atom:updated>2018-04-27T20:03:45.222Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a community sign-up app with Serverless, StepFunctions, and StackStorm Exchange</h3><p>Build a real-world serverless application on AWS with <a href="https://serverless.com/framework/">Serverless framework</a> and ready-to-use functions from <a href="https://exchange.stackstorm.org/">StackStorm Exchange</a> open-source catalog.</p><p>Episode One | <a href="https://medium.com/@dzimine/building-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-2-b1efeb1b9bd6">Episode Two</a> | <a href="https://medium.com/@dzimine/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-6efb9c102b0a">Episode Three</a> | <a href="https://medium.com/@dzimine/building-a-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-7c5f0e93dd6">Episode Four</a></p><p>Read on if you are:</p><ul><li>A serverless developer using <a href="https://serverless.com/framework/">Serverless framework</a> who wants to check out ready-to use functions from the StackStorm Exchange open-source catalog,</li><li>A <a href="https://stackstorm.com/">StackStorm</a> user who lives in AWS and misses the breadth of <a href="https://exchange.stackstorm.org/">StackStorm Exchange</a> integrations there.</li><li>Anyone who has 2 hours to follow-along and learn serverless with something more elaborate and real-life than a <a href="https://serverless.com/framework/docs/providers/aws/examples/hello-world/">Hello-world example</a>.</li></ul><p>If you only got 8 minutes to spare, skim the text and examples, spend an extra 30 seconds browsing <a href="https://exchange.stackstorm.org/">StackStorm Exchange</a> to see the potential, and bookmark this post to get back to it when you need it.</p><h4>Intro</h4><p>When I <a href="https://medium.com/@dzimine/exploring-serverless-with-python-stepfunctions-and-web-front-end-8e0bf7203d4b">explored Serverless with Python, StepFunctions, and Web Front-end</a>, one thing I missed is a catalog of reusable integrations. Something like <a href="https://docs.microsoft.com/en-us/azure/connectors/apis-list#popular-connectors">200 connectors for Azure Logic apps</a>. Or <a href="https://exchange.stackstorm.org/">130 integration packs for StackStorm</a>.</p><p>When we need to wire in Slack, Jira, Stripe, or Nest, could we skip digging into their APIs and authentication intrinsics, and just grab a ready-to-use function?</p><p>Now we can do exactly that: StackStorm just <a href="https://stackstorm.com/2017/12/14/stackstorm-exchange-goes-serverless">announced a plugin for Serverless framework</a> that turns integrations from StackStorm Exchange into AWS Lambda functions.</p><p>In this tutorial, I’ll show how to use the plugin and Exchange integrations, in the context of building a serverless community on-boarding application from a ground-up. Let’s make this conversational and fun.</p><p>I assume no familiarity with either <a href="https://serverless.com/framework/">Serverless framework</a> nor StackStorm. But you should know how to code, and be smart to compensate for mistakes and omissions I’ll inevitably make.</p><p>We will be going slow, with excruciating details, thus it is going to be four episodes.</p><p>In this first episode, I’ll set everything up and deploy my first StackStorm Exchange action.</p><p>In the next episode, we’ll add more actions.</p><p>In the third episode I’ll wire them together with a AWS StepFunction.</p><p>In the fourth episode we’ll add Web Front-end with the Reflection and Summary. Each episode will take about an hour to follow.</p><p>Ready? Let’s rock.</p><h3>The application</h3><p>We will build a community on-boarding application. Actually, rebuilding from scratch the one we run at StackStorm. It’s like <a href="http://rauchg.com/slackin/">SlackIn</a> with a multi-step customizable on-boarding workflow. The app presents a registration web-form, which passes new user info through API Gateway to the StepFunction workflow that carries on-boarding steps.</p><p>In my case, the steps are 1) invite users to Slack 2) create contact record in ActiveCampaign CRM tool and 3) put a user record into internal DynamoDB table. Here is how it looks:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QdErjJE8X0jg3SfR7QyBVQ.png" /></figure><p>You can find the previous implementation <a href="https://github.com/dzimine/slack-signup-serverless">on github</a>, or <a href="https://stackstorm.com/community-signup">use it and join StackStorm community</a> on Slack for questions about StackStorm Exchange integration.</p><h4>Getting Ready</h4><p>First, you’ll need AWS account, NodeJS, Docker, and Serverless framework. And Slack! As our first action will be inviting users to Slack.</p><ol><li>Make sure <a href="https://nodejs.org/">Node.JS</a> is installed, and it’s version is 8.4.0 or higher.</li><li><a href="https://serverless.com/framework/docs/providers/aws/guide/installation/">Install Serverless framework</a>, and setup AWS credentials for it <a href="https://serverless.com/framework/docs/providers/aws/guide/credentials/">following this guide</a>.</li><li><a href="https://docs.docker.com/engine/installation/">Install Docker</a>. The plugin uses it for build environment to make the lambdas binary compatible to AWS execution environment no matter what OS you’re using for development. There is a way to make it work without Docker but don’t take chances.</li><li>Slack! Our action will require admin access and will be using undocumented Slack API (docs here, pun intended) to invite new users. The easiest is to just go ahead <a href="https://slack.com/get-started">create a new team</a>. Takes 4 min. Slack won’t mind — they’ll show growth to their VC.</li><li>Once the workspace is created, time to get an authentication token. Go to <a href="https://api.slack.com/custom-integrations/legacy-tokens">api.slack.com/custom-integrations/legacy-tokens</a>. Fear not the “Legacy Warnings”: this tutorial will turn legacy before they do. Do what they say, get your token.</li></ol><p><strong>PRO TIP:</strong> Use <a href="https://github.com/StackStorm-Exchange/stackstorm-slack#obtaining-auth-token">this quick hack</a> to get and use your own user’s auth token. Much faster, good for playing and debugging. But please never, never use it for production!</p><h3>Create a project, add a first action</h3><p>Try sls --help to make sure that at least something works. sls is a shorthand for serverless, the Serverless framework CLI. Now put your coffee aside, time to create a project. Some folks like using templates that come with serverless: sls create --template. I prefer to start from scratch:</p><pre>mkdir slack-signup-serverless-stormless<br>cd slack-signup-serverless-stormless</pre><pre>npm init</pre><pre># Once you answer questions, the project is set up.</pre><p>Next, install <a href="https://github.com/StackStorm/serverless-plugin-stackstorm">serverless-plugin-stackstorm</a>, the one that plugs in the <a href="https://exchange.stackstorm.org/">StackStorm Exchange</a> actions.</p><pre>npm install --save-dev serverless-plugin-stackstorm</pre><p>… and create a minimal serverless.ym file so that sls command will pick up the plugin:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/225a96ee488c79074122574ee68738fc/href">https://medium.com/media/225a96ee488c79074122574ee68738fc/href</a></iframe><p>Now, <strong>create the first action</strong>. I’ll use a battle-tested <a href="https://exchange.stackstorm.org/#slack">Slack pack from StackStorm Exchange</a>. Which action, you say? Ok, StackStorm Exchange is not smart enough yet to show pack’s action list, but sls stackstorm will rescue us.</p><p>sls stackstorm info --pack slack</p><p>Oh my! There’re so many! what are they? I guess I need a PR to print action description. Meantime, | grep admin will get us the one we need: slack.users.admin.invite. Let&#39;s query the action for it&#39;s parameters:</p><pre>$ sls stackstorm info --action slack.users.admin.invite<br>slack.users.admin.invite ...... Send an invitation to join a Slack Org<br>Parameters<br>  attempts [integer]  ......... description is missing<br>  channels [string]  .......... Channels to auto join.<br>  email [string] (required) ... Email address to send invitation to.<br>  first_name [string]  ........ Users first name<br>  set_active [boolean]  ....... Should the user be active.<br>  token [string]  ............. Slack API token.<br>Config<br>  action_token [string]  ...... Slack Action token.<br>  admin [object]  ............. Admin-action specific settings.<br>  post_message_action [object]   Post message action specific settings.<br>  sensor [object]  ............ Sensor specific settings.</pre><p>Awesome! We can see that there is only one required parameter, email, but I&#39;ll add first_name to stay conversational. The token can be passed as parameters, or as config. And if I choose to use config, my prior tribal knowledge hints that the admin [object] requires only admin_token. The very one I asked you to remember when you were setting up Slack workspace.</p><p><strong>PRO TIP: </strong>While we are still polishing the plugin to expose all the Config details, you can find it out by exploring StackStorm Exchange pack config schema in config.schema.yaml file. For example, here is <a href="https://github.com/StackStorm-Exchange/stackstorm-slack/blob/master/config.schema.yaml#L47-L78">config.example.yaml</a> for our Slack pack.</p><p>Now we have all we need to create the heart of any Serverless project: the serverless.yml. Here it comes:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/079b28ddca56c1382d78b82b0bb4e9ae/href">https://medium.com/media/079b28ddca56c1382d78b82b0bb4e9ae/href</a></iframe><p>This is a good time to learn a bit of Serverless. Take a quick break to skim <a href="https://serverless.com/framework/docs/providers/aws/guide/intro/">Core Concepts</a> and bookmark <a href="https://serverless.com/framework/docs/providers/aws/guide/serverless.yml/">Serverless.yml Reference</a>.</p><p>I threw in the events section in lines 9:12 so that we can invoke the function with REST call through AWS API Gateway endpoint. Serverless framework will do all the Gateway magic.</p><p>Note that this default configuration instructs API Gateway to pass the REST POST call with POST body under the body key (<a href="https://serverless.com/framework/docs/providers/aws/events/apigateway#simple-http-endpoint">details here</a>. When we POST {&quot;first_name&quot;: &quot;Santa&quot;, &quot;email&quot;: &quot;santa@mad.russian.xmas.com&quot;}, the event passed to the Lambda is:</p><pre>...<br>&quot;body&quot;: {<br>    &quot;first_name&quot;: &quot;Santa&quot;, <br>    &quot;email&quot;: &quot;santa@mad.russian.xmas.com&quot;<br>}</pre><p>Knowing the input data structure is important to map it to the action input parameters. It’s intuitive: input represents event parameter of <a href="http://docs.aws.amazon.com/lambda/latest/dg/python-programming-model-handler-types.html">AWS Lambda programming model</a>(BTW should we call it event? Vote with a PR!).</p><p><a href="http://jinja.pocoo.org/">Jinja</a> is used to map the inputs; our JavaScript friends who&#39;re less familiar with this common Python tool find it intuitive enough in simple cases; and Stack-overflow is full of magic Jinja tricks.</p><p>In lines 16:17 of serverless.yml I map the two parameters from input body to desired action input parameters. Optionally, you can also form a function output from action results. I’ll keep it simple for now (line 20:22) and save more tricks for later.</p><p>To keep the config separate, I created a file env.yml that and put my config parameters in it:</p><pre># ./env.yml<br># WARNING: Don&#39;t commit to Github :)<br>slack:<br>  admin_token: &quot;xoxs-111111111111-...&quot;<br>  organization: &quot;my-team&quot;</pre><p>Then I used it in serverless.yml like this: admin: ${file(env.yml):slack}. Note how this syntax puts the object from the key in the file to the key in serverless.yml.</p><p>Ok, that’s it! The function is ready to fly to AWS with sls deploy. But I take it <strong>sloooow</strong>. Step by step. First, I’ll package it locally.</p><pre>sls package</pre><p>The very first time takes a long time as this is the time when the plugin installs its runtime dependencies. It pulls the Docker images from the Hub. It installs StackStorm runners — the code that knows how to run StackStorm Exchange packs. It pulls the slack pack from Exchange. It installs slack pack python dependencies. It does a lot of work for us, and it takes time. Good news: it&#39;s only the first time.</p><p>Oh, did I mention that you must be connected? Or do we assume internet connection a basic commodity like breathing air and electric power? At least before FCC repeals Network Neutrality? So yes, you need internet connection to live in the serverless world.</p><p>Now let’s run this locally.</p><pre>sls stackstorm docker run --function InviteSlack --verbose \<br>--data \<br>&#39;{&quot;body&quot;: {&quot;first_name&quot;: &quot;Santa&quot;, <br>&quot;email&quot;: &quot;santa@mad.russian.xmas.com&quot;}}&#39;</pre><p>Local runs happen in a container — you’ll see Spin Docker container to run a function CLI output. It takes a bit longer, but ensures that the execution environment matches AWS lambda very closely, so better safe than sorry.</p><p>When I debug input and output parameter transformations, I may not want to call the actual function, like in case of Slack rate-limiting API. Use --passthrough parameter that tells the plugin to do the dry-run and skip the action invocation.</p><p>Now we are really ready. Let’s deploy the function to AWS, and run it “serverless”.</p><pre>sls deploy</pre><p>It will take some while — now it’s serverless (and honestly, our bundle is a bit bloated, patience! plugin developers are currently busy solving other problems, we will optimize it as soon as we can)</p><p><strong>PRO TIP:</strong> if something goes wrong at this point, most likely something is not right with your AWS setup. Go back to “Getting Ready, step 2”. Read <a href="https://serverless.com/framework/docs/providers/aws/guide/installation/">Serverless Installation</a> doc. Google, Stack-overflow, Serverless <a href="https://gitter.im/serverless/serverless">Gitter channel</a> or <a href="https://forum.serverless.com/">Forum</a>.</p><p>You might be curious to see how it looks in the AWS console. If the PRO in you is saying “no, you should stay cool and use CLI”, don’t listen. Go indulge yourself, open a browser and take a good look at your Lambda. While there, you might also inspect the API Gateway endpoint that sls created for you.</p><p>But to test it, we’ll go back to terminal. Here is how to run your Lambda with sls:</p><pre>sls invoke --function InviteSlack --log --data &#39;{&quot;body&quot;: {&quot;first_name&quot;: &quot;Santa&quot;, &quot;email&quot;: &quot;santa@mad.russian.xmas.com&quot;}}&#39;</pre><p>Finally, let’s POST to the API endpoint. The endpoint was printed at the end of sls deploy and you should have taken notice, but it&#39;s OK if you didn&#39;t: you can always get it by typing sls info.</p><p>You curl lovers go ahead use it to POST; be sure to set Content-Type=application/json header. Me - I&#39;ll show off with <a href="https://httpie.org/">httpie</a>, aka CURL for humans:</p><pre># DON&#39;T copy-paste! Use YOUR endpoint!</pre><pre>http --json POST  https://YOUR-ENDPOINT.amazonaws.com/dev/invite \<br>email=test@example.com first_name=Dmitri</pre><p>How did it go? Everything worked, at least for me. Let’s fire yet another most useful slscommand to check the CloudWatch logs:</p><pre>sls logs --function InviteSlack</pre><p>Success! And the end of the first episode. Enough for now: Christmas time is here, take it slow, enjoy!</p><p>The code example so far is on Github at <a href="https://github.com/dzimine/slack-signup-serverless-stormless/tree/DZ/1-add-first-action">1-add-first-action</a>.</p><p><a href="https://medium.com/@dzimine/building-community-sign-up-app-with-serverless-stepfunctions-and-stackstorm-exchange-episode-2-b1efeb1b9bd6"><strong>Episode 2</strong></a>: <strong>Adding more actions</strong></p><p>Hope this helped you learn something new, find something interesting, or provoked some good thoughts. Please share your thoughts in the comments here, or tweet me<a href="https://twitter.com/dzimine"> @dzimine</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b2f7cf2cc419" width="1" height="1" alt=""><hr><p><a href="https://medium.com/free-code-camp/tutorial-building-a-community-on-boarding-app-with-serverless-stepfunctions-and-stackstorm-b2f7cf2cc419">Tutorial: Building a community on-boarding app with Serverless, StepFunctions, and StackStorm…</a> was originally published in <a href="https://medium.com/free-code-camp">We’ve moved to freeCodeCamp.org/news</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Two Ways to Create a “Thing” in the Internet of Things]]></title>
            <link>https://medium.com/@dzimine/two-ways-to-create-a-thing-in-the-internet-of-things-53504512b209?source=rss-63a761e7cf69------2</link>
            <guid isPermaLink="false">https://medium.com/p/53504512b209</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[internet-of-things]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[iot]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Dmitri Zimine]]></dc:creator>
            <pubDate>Fri, 20 Oct 2017 19:20:14 GMT</pubDate>
            <atom:updated>2018-06-07T18:55:28.446Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Quick &amp; dirty notes and hints while learning AWS IOT</em></p><p>To play with Internet of Things, we need things. To play with AWS IoT, we need to create and configure things. Yes, eventually they’ll come in many <a href="https://medium.com/@monkeytypewritr/amazon-echo-esp8266-iot-a42076daafa5">interesting form factors</a>. But what is the best and fastest to get things going? How do I quickly create a “<strong>Thing</strong>” on my computer, register and connect it to AWS IoT, send messages and tinker with SDK?</p><p>There is a hard way and a fast way. The hard is actually not hard at all, so start with it.</p><h4><strong>Create a thing, take 1: Device SDK with AWS console and local docker</strong></h4><p>I can get the “thing” SDK on my laptop. But rather than turning my laptop into a “thing”, I’d rather have it isolated. Containerized, I mean :) Yes, a quick docker container to contain a “thing” with AWS IoT SDK. Or a few of them. Here is a step-by-step:</p><p>1. Go to AWS Web console’s <a href="https://us-west-2.console.aws.amazon.com/iotv2/home?region=us-west-2#/connIntro">Onboard</a> page, select “Configuring a device” “Connect to AWS IoT”, click “Get Started” — it will get you to <a href="https://us-west-2.console.aws.amazon.com/iotv2/home?region=us-west-2#/connectdevice/">Connect to AWS IoT wizard</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f28WVpEsz1Te7Rhb1f7M2g.png" /></figure><p>2. Pick OS and SDK, give your “thing” a name, and download deployment package — it contains certificates, and a script that installs the SDK and runs a sample at the end to send messages to the test topic. I selected Linux/Python. Download the package and and unzip it to local directory:</p><pre>mkdir ~/thing<br>cd ~/thing <br>mv ~/Downloads/connect_device_package.zip .</pre><p>3. Create a simple docker file:</p><pre># ~/thing/Dockerfile <br>FROM python:2.7<br>ADD . /code<br>WORKDIR /code<br>CMD [“/bin/bash”]</pre><p>4. Build docker image</p><pre>build -t dzimine/thing .</pre><p>Run docker container from the image, mapping the current directory to enjoy your favorite editor on the host laptop while exploring the IoT SDK.</p><pre>run -it -v “$(pwd)”:/code dzimine/thing</pre><p>Now in the container, run the ./start.sh. The script will download and setup the AWS IoT SDK, and run the sample app to send messages to AWS IoT testing topic. You’ll see the messages at the last step of your “Register a thing” wizard, or go to <a href="https://us-west-2.console.aws.amazon.com/iotv2/home?region=us-west-2#/test">IoT Test Page on AWS console</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*izlMnnDx9wX2tTR7P08bOA.png" /></figure><p>Care to build a better docker image? Look at start.sh. and fancy yourself with the Dockerfile using it as a hint.</p><h4><strong>Create a thing, take 2: ELF</strong></h4><p>2) Now, the easy way. Get this awesome “thing” client <a href="https://github.com/awslabs/aws-iot-elf">https://github.com/awslabs/aws-iot-elf</a>. ELF stands for “Extremely Low Friction”, and indeed the friction is extremely low. Assuming you already have aws CLI installed, “it just works”. Hit this…</p><pre>python <a href="http://elf.py/">elf.py</a> create</pre><p>… and go see your thing. Hit that…</p><pre>python elf.py send --topic foo/bar</pre><p>… and see the messages popping up at AWS console IoT Test page.</p><h4>What is next?</h4><p>Pick your poison and explore the IoT device SDK and MQTT protocol. ELF example does more — in addition to “being a thing”, it uses boto3 library to create things on AWS side and updates the shadows.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53504512b209" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Serverless is cheaper, not simpler]]></title>
            <link>https://medium.com/free-code-camp/serverless-is-cheaper-not-simpler-a10c4fc30e49?source=rss-63a761e7cf69------2</link>
            <guid isPermaLink="false">https://medium.com/p/a10c4fc30e49</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Dmitri Zimine]]></dc:creator>
            <pubDate>Mon, 28 Aug 2017 14:16:15 GMT</pubDate>
            <atom:updated>2017-12-09T00:52:47.756Z</atom:updated>
            <content:encoded><![CDATA[<p>The <a href="http://www.emitconference.com/">(Emit)</a> conference last week featured a lineup of excellent talks, an engaging panel discussion, and plenty of time to meet and exchange notes with the awesome fellows of the serverless community.</p><p><strong>Cost</strong> is unanimously cited as the key driver for serverless adoption. On-demand execution and built-in elasticity optimizes utilization while keeping uptime and reliability even higher. Pay-per-use billing makes cost directly quantifiable. The <a href="https://techbeacon.com/economics-serverless-computing-real-world-test">savings</a> can be <a href="https://aws.amazon.com/solutions/case-studies/bustle/">insane</a>. <a href="https://twitter.com/AnneWoof">Anne Thomas</a>, a Gartner analyst, shared at the panel discussion that her enterprise clients are attracted to serverless by “cost” benefits.</p><p>But there is no free lunch in a closed system: to gain a benefit something must be sacrificed. In technology, the most common currency to pay for benefits is “complexity”.</p><p>When microservices replaced monoliths to gain the benefit of scale reliably under massive load,<a href="http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html"> it was not a free lunch</a>. Dealing with eventual consistency, handling asynchronicity, latency and fault tolerance, managing APIs and message schemas, load balancing, running rolling upgrades — we paid with a great deal of extra <a href="https://martinfowler.com/articles/microservice-trade-offs.html">complexity for the gains</a>.</p><p>Where does the complexity reside, and how do we manage it?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jIpMoYfpftPWmx-x." /><figcaption>From monolith to microservices to serverless, complexity increases to gain benefits.</figcaption></figure><p>Take a look at the picture. The orange blocks represents the code. As we move to micro-services to serverless, the individual blocks of code become smaller and simpler, while the total complexity grows. Much of this extra complexity is “wiring”: configurations, deployment scripts, devops tooling artifacts — templates, recipes, playbooks, dockerfiles… All the stuff used to forklift the solution up to the cloud, and keep it there. The gray stuff between the orange code blocks. This reminds me of an old Russian joke:</p><blockquote>- What is air made of?<br>- Well, it’s made from molecules!<br>- And what is between the molecules?<br>- Well, of course air!</blockquote><p>This wiring between the code: it better be code! And this is what DevOps was all about with it’s “Infrastructure as code” mantra. Devops — to give yet another definition — is a framework and tooling to tame the extra wiring complexity introduced by micro services.</p><p>When Serverless replaces micro-services, it is not going to be free lunch either. We are paying by introducing more complexity, now for the benefit of massive cost savings.</p><p>Some parts of the serverless system do become simpler. Not dealing with virtual machines, networking, storage, and operation systems is a huge relief from the massive operational burden of provisioning, configuring, monitoring and remediating infrastructure. Functions provide a simple programming model and mostly focus on business logic. The execution environment is tight, controlled and under “continuous X-Ray”. <a href="https://twitter.com/crandycodes">Chris Anderson</a>, a PM for <a href="https://twitter.com/AzureFunctions">@AzureFunctions</a>, highlighted how much easier it became to pinpoint problems with function executions.</p><p>But the law of conservation still applies, and when function are simplified, the released complexity moves somewhere else. Where does the complexity go, and how do we manage it?</p><p>Part of complexity is delegated to cloud PaaS services (may I call it JPaaS? [3]) . AWS API Gateway, S3, Kinesis, SNS, DynamoDB, StepFunctions, or their Azure and GCP siblings — are at play with any serverless solution. They are readily available, fully managed, feature rich, “infinitely” scalable, and pay-per use. To gain their benefits, we happily trade control — another common tech currency <em>— </em>to be liberated from infrastructure and boilerplate concerns. Note the return from micro-services’ <a href="https://martinfowler.com/articles/microservices.html#SmartEndpointsAndDumbPipes">“Smart endpoints and dumb pipes”</a> back to “dumb endpoints and smart pipes” and even the workflows; spend a minute to reflect on this pendulum sway.</p><p>Sometimes, the surrender of control backfires and complexity unexpectedly spikes where things used to be simple. Setting up binary headers in AWS API Gateway via CloudFormation templates makes you miss the clarity of handling it in your microservice’s code. Using feature-rich JPaaS services requires deep expertise, and unlike regular programming and CS this expertise doesn’t translate between platforms: knowing DymanoDB will be little help in learning BigTable.</p><p>Nevertheless, the delegation to JPaaS is a net positive and covers a good part of complexity. But not all of it. The other part of complexity lies, again, in “wiring”.</p><p>Functions are smaller, but there are an order of magnitude more relations to understand and dependencies to manage. Functions are simpler, but they no longer contain the logical flow. The logic and the flow of the end-to-end solution is driven by events. It no longer resides in any coding artifacts, and becomes very difficult to reason about.</p><p>How is this side of complexity handled today? Not too well. It takes uber-architect(s) to keep it under control. The processes and tools are yet to catch up. Serverless frameworks are growing like mushrooms after the rain but are mostly stuck with FaaS — with slight variations on the same, an already solved problem. Even the most advanced ‘serverless’ framework, and the one that recognizes that Serverless is more than FaaS, and provides plugins and native resource support for other services — even ‘serverless’ doesn’t cover the end-to-end wiring complexity of a serverless solution. Take the canonical <a href="https://read.acloud.guru/serverless-event-sourcing-at-nordstrom-ea69bd8fb7cc">HelloRetail</a> serverless <a href="https://github.com/Nordstrom/hello-retail">project code</a> and inspect how much complexity is left-out. The “project”, the services’ dependencies on JPaaS and each other, the deployment order, event types and schemas and many more are not covered by the framework. They are partially scripted, partially documented, but mainly residing in the heads of the “uber-architects” or as tribal knowledge (which, in software, is also known as “trouble knowledge”).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JaGG-grB6Xbpc1n7." /></figure><p>What about DevOps? Shouldn’t it help? The answer is NO, or, at least, not entirely. For one thing, DevOps folks obviously don’t flock around serverless nearly as much as they do around kubernetes. Why so? Maybe our maker-persona DevOps friends just don’t have enough infrastructure knobs and levers to tinker with? Or maybe the reason is more fundamental: DevOps is a response to another problem space — microservices — and not fully applicable to serverless as many microservice assumptions no longer hold. Taking away “ssh to the box” alone retires half of DevOps tooling. Even Terraform, built to codify cloud infrastructures, is somehow not nearly in as much favor in the Serverless community as it is in microservices DevOps.</p><p>As the result, serverless today lacks the established operational frameworks, patterns, and tooling that are required to tame it’s complexity. It requires an uber-architect to invent the end-to-end solution and tame complexity. These uber-architects are blazing the path and show success and helping the patterns emerge. But as Anne from Gartner pointed out at the (Emit) conference panel, there will be no widespread serverless adoption until the frameworks and tooling catch up.</p><p>Like many times before, challenge is an opportunity. DevOps came to tame complexity of micro-services and made cloud apps “possible”. Something is coming to tame complexity of serverless and make cloud apps “economical”. Whatever it is, it’s gonna be big. When it comes to $, economical forces work flawlessly. The cost saving benefits of serverless are so clear and so substantial, and the push from all major cloud vendors towards serverless is so strong, that it won’t be long before the complexity is tamed and serverless becomes mainstream in all the areas where it applies.</p><p>Thanks for reading. Please share your comments and your thoughts on the future of serverless here and on Twitter. And if you enjoyed reading this and want me to write more articles like it, just tap the “clap” button.</p><p>PS. You may be insterested in <a href="https://news.ycombinator.com/item?id=15326553">Hackernews discussion and comments on this article</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a10c4fc30e49" width="1" height="1" alt=""><hr><p><a href="https://medium.com/free-code-camp/serverless-is-cheaper-not-simpler-a10c4fc30e49">Serverless is cheaper, not simpler</a> was originally published in <a href="https://medium.com/free-code-camp">We’ve moved to freeCodeCamp.org/news</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Serverless with Python, StepFunctions, and Web Front-end]]></title>
            <link>https://medium.com/@dzimine/exploring-serverless-with-python-stepfunctions-and-web-front-end-8e0bf7203d4b?source=rss-63a761e7cf69------2</link>
            <guid isPermaLink="false">https://medium.com/p/8e0bf7203d4b</guid>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Dmitri Zimine]]></dc:creator>
            <pubDate>Wed, 02 Aug 2017 23:26:22 GMT</pubDate>
            <atom:updated>2017-08-02T23:33:54.777Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Serverless slack invite and community signup project. Serverless Python project boilerplate. Story of my serverless exploration. Tips, tricks, links.</em></p><p><a href="https://github.com/serverless-london/serverless-slack-invite">Slack invite</a> has been a canonical example of a serverless application. But what if you want to do something more involved? Like our <a href="https://stackstorm.com/community-signup">StackStorm community sign-up</a>, where we ask Slack to invite a user, but also record it into ActiveCampaign CRM with appropriate tags, and create a user record in the database? A perfect case for a serverless starter project, more realistic<em> </em>than <a href="https://github.com/serverless/examples/tree/master/aws-python-rest-api-with-dynamodb">aws-python-rest-api-with-dynamodb</a> yet less daunting than <a href="https://github.com/Nordstrom/hello-retail">“Hello Retail” showcase</a>. And, for a change, it’s in Python!</p><p>The code is at <a href="https://github.com/dzimine/slack-signup-serverless">https://github.com/dzimine/slack-signup-serverless</a>. There, you will find:</p><ul><li>Python boilerplate project for Lambda functions with lean dependency management, unit testing consistent with the lambda deployment, and atox-based build for packaging and continuous integration.</li><li>AWS StepFunctions to define the logic of Lambda invocations.</li><li>Web front-end deployed on S3 with API Gateway.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sTI6doZYPBkf2Suahhc72g.png" /><figcaption>Serverless community signup application diagram.</figcaption></figure><p>Deployment is fully-automated: you can use it “as is”: clone the code, set up your credentials, and run (enjoy <a href="https://medium.com/u/36584a5b84a">Evan Powell</a>!). Or use the code as a starting point: modify and add steps, re-style the web form — adjust to your liking or completely repurpose it.</p><p>Read on for the story of my serverless exploration. Enjoy and use the tips, tricks, and links spread along the way. Or jump straight to the <strong>Summary</strong> and comment on my takeaways.</p><h3>1. Lambda</h3><p>Getting the Lambda functions is a breeze with <a href="http://serverless.com/">serverless.com</a>. To appreciate it, I first tried to implement the <a href="https://github.com/serverless-london/serverless-slack-invite">original serverless-slack-invite</a> in Python with raw AWS Lambda. I spent a day on it, and API Gateway configuration was so painful that &lt; <em>some serious swearing removed from here</em>&gt;. It took me an hour to redo it with Serverless — a 10x productivity boost! and a joy of having a clear, reproducible infrastructure as code. A few notes on effective Lambda with Python:</p><p><strong>Project structure. </strong>Should you follow <a href="https://github.com/Nordstrom/hello-retail">“hello-retaile”</a> and create an uber-project, where each lambda is individual serverless service? Or go for a single serverless project (== single service) with fine-grained management of the parts? Trying both, I figured that a single project wins for 3–7 services. As the number of services grows beyond a dozen, with different developers working on different services with different deployment cadence, it’s time to breake it up. Be aware that all the managing the uber-project — build, deployment, service dependencies, versioning — is not covered by the serverless framework and will be your fun to invent and maintain.</p><p><strong>Environment variables</strong>: an elegant way for managing environment variables is a single <a href="https://github.com/dzimine/slack-signup-serverless/blob/master/env.yml.example"><em>env.yml</em></a> YAML file with a section for each service. You an then use environment: ${file(env.yml):my_service to pass the service-specific section of the <em>env.yml</em> to Lambda as a single set of environment variables. The filename is “.gitignored” to prevent committing your secrets to GitHub.</p><p><strong>Python Dependencies: </strong>Making Lambda packages lean with just enough dependencies keeps you safely within <a href="http://docs.aws.amazon.com/lambda/latest/dg/limits.html">Lambda package size limits</a>, and makes deployments enjoyably fast.</p><p>The first step is to tell serverless to manage Lambda functions individually. It is achieved by setting `package/individually: true`, and defining a `package` section per function in the `serverless.yml`.</p><p>Next, prepare Python dependencies. There are couple of serverless plugin to help with this. For simple cases I recommend <a href="https://github.com/UnitedIncome/serverless-python-requirements">serverless-python-requirements</a>. If your Lambda is called as a web service, and you configure API Gateway as <a href="https://serverless.com/framework/docs/providers/aws/events/apigateway/#lambda-proxy-integration">lambda-proxy</a>, taking the request-response management onto the Lambda function code, a <a href="https://github.com/logandk/serverless-wsgi">https://github.com/logandk/serverless-wsgi</a> plugin is a good way to go.</p><p>I preferred to not use plugins and manage dependencies from the code, building upon <a href="http://forum.serverless.com/t/aws-python-function-dependencies-load/451/4">this serverless forum hint</a>. “Lean” lambda packages make deployment fast and keep you safe within AWS lambda package size limits. Most importantly, this setup makes local unit testing match the remote execution. The <a href="https://github.com/dzimine/slack-signup-serverless/blob/master/build.sh">build.sh</a>script automates the dependency building, and <a href="https://tox.readthedocs.io/en/latest/">tox</a> is used for a full Python build &amp; test. Note that setup.cfg is required per <a href="http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html">this AWS hint</a>.</p><p><strong>Programming models and invocation discrepancies. </strong>In my development and testing process, I tried and called my Lambda functions in different ways — as a separate service, as part of testing from the serverless cli and AWS console, and as a step in StepFunction. A gotcha to watch out for here: the type and structure of the input and return values, as well as exception handling, are all different for different kinds of invocations. When Lambda is called from the API Gateway, Serverless framework <a href="https://serverless.com/framework/docs/providers/aws/events/apigateway/">encourage the </a><a href="https://serverless.com/framework/docs/providers/aws/events/apigateway/">lambda-proxy method</a>: <em>&quot;We highly recommend using the lambda-proxy method if it supports your use-case, since the lambda method is highly tedious&quot;.</em> With “lambda-proxy”, a full HTTP request is passed to the handler in the event input parameter, leaving the the handler to parse the body and return a proper HTTP response for success and failures, e.g. 500 in case of service errors. (If you take this route, consider the <a href="https://github.com/logandk/serverless-wsgi">serverless-wsgi</a> plugin mentioned above).</p><p>You can stick to the <a href="http://docs.aws.amazon.com/lambda/latest/dg/python-programming-model-handler-types.html">AWS mainstream</a> of doing string-to-JSON transformation on API Gateway, and signaling failures by throwing Python exceptions. This simplifies the Python lambda handler, but complicates API gateway configuration part in serverless.yaml. Pick your poison.</p><p>When Lambda is called from a StepFunction, an event contains an input to StepFunction step. The handler returns an object for the step output, and throws exception on failures. Special care is needed for passing the API input to Lambdas down the StepFunction (more on this gotcha in “StepFunction” section below).</p><p>Finally, we often call Lambda from serverless CLI, as part of convenient dev workflow offered by serverless:</p><pre>sls invoke stepf --name signup --data &#39;{&quot;foo&quot;:&quot;bar&quot;}&#39;</pre><p>Serverless first tries to parse the input as JSON and passes an dict, falling back to string if parsing fails. It’s a bummer if you have followed Serverless recommendation of using lambda-proxy and expect the event as a JSON string in your handler. No worries, though: if you tried it you already know it, if you haven’t, <a href="https://github.com/serverless/serverless/issues/3788">this bug</a> be likely fixed by the time you try.</p><h3>2. Wire logic with Step Function workflow.</h3><p>Workflows are instrumental in stitching the logic of individual Lambda invocations into complete serverless solutions. The serverless framework doesn’t support StepFunctions in the “core”, but there is a <a href="https://github.com/horike37/serverless-step-functions">serverless-step-functions plugin</a> that does. I opted for using it over the <a href="https://github.com/Nordstrom/hello-retail/tree/master/product-photos/process">raw CloudFormation approach shown in HelloRetail</a>.</p><p>AWS StepFunctions deserves a dedicated review, but I’ll give you a <strong>BIG BOLD WARNING</strong> right here. On any change, the execution history is gone. That is: step functions are immutable; any change creates a new instance, the previous instance is deleted (ok so far), and the execution history of the previous instance is gone with deleted instance (Really? WTF!!!)</p><p>If this doesn’t stop you from using StepFunctions, read on to pick up a few tricks.</p><p><strong>Passing workflow input to steps.</strong> The StepFunction data flow designed on “need-to-know” basis: no extra data is passed by default. The first Lambda receives the workflow input (the body of REST API call). The second Lambda only gets an output of the first Lambda. But I want all the Lambda steps to access the workflow input! The trick here is to use <a href="http://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-input-output-processing.html">StepFunction’s </a><a href="http://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-input-output-processing.html">ResultPath</a>: it is a way to control the key where the result of the Lambda execution is placed in the workflow state. The default ResultPath=$ sets the key to the root context so it overwrites the initial workflow input. But if we append the result of each lambda under some key, like ResultPath: $.results.InviteSlack (<a href="https://github.com/dzimine/slack-signup-serverless/blob/master/serverless.yml#L64-L67">like this</a>), the workflow input stays preserved and can be used by all Lambda steps.</p><p><strong>Invoking step-functions </strong>from serverless is a joy: the plugin starts the workflows, watches the execution and prints out the results of each step as it progresses:</p><pre>sls invoke stepf --name signup --data &#39;{&quot;email&quot;:&quot;your@email.com&quot;, &quot;first_name&quot;:&quot;Donald&quot;, &quot;last_name&quot;:&quot;Trump&quot;}&#39;</pre><p><strong>Exposing via API, calling from Web, and dealing with CORS.</strong> Exposing step function methods with the API Gateway is just as easy with a plugin as with serverless lambda. But I need to call it from the WebUI hosted on s3. This requires CORS configuration that the plugin didn’t support at a time (now it does). I wondered what to do. Have “lambda” in front of StepFunction just to invoke it? And pay for it? No-o-o way. Go to the dark side and use CloudFormation to do all API Gateway configuration? No-o-o fun. The answer? Read on to “Add Web, no CORS”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_nceOvoxc6VfG4w5oPXCpg.jpeg" /><figcaption>What to choose? <a href="https://www.google.com/culturalinstitute/beta/asset/-/wAFehP4o2JFtMA?hl=en">Knight at the Crossroads</a>, the classic Russian epic on False Choice Fallacy.</figcaption></figure><h3>3. Add Web, no CORS.</h3><p>The third, winning way was presented by <a href="https://github.com/sdd/serverless-apig-s3">serverless-apig-s3</a> plugin. The plugin does two things. It puts web content on S3, as pointed in serverless.yml (<a href="https://github.com/dzimine/slack-signup-serverless/blob/master/serverless.yml#L7-L9">./web in my case</a>) and enables the web access. It also configures API Gateway to serve the static content from the same domain, sidestepping any CORS issues. There are some other plugins for handling the Web Frontend but to me sidestepping CORS wins.</p><p><strong>Couple of gotchas to watch out for:</strong></p><p>When you add or remove web files, client deploy is not enough, a full sls deploy is required. It makes sense when you know how things work: the APIGateway part of the plugin changes the CloudFormation stack which needs to be updated, and the sls client deploy part puts the files on s3 bucket.</p><p>I added an image to the form while customizing it for StackStorm community signup. The image didn’t go through. Took me a frustrating while to figure why: the <a href="http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-payload-encodings-configure-with-control-service-api.html">binary payloads is not enabled by default and need to be configured</a>. If you happen to need it, configure it manually, via CloudFormation, try <a href="https://github.com/maciejtreder/serverless-apigw-binary">serverless-apigw-binary</a> plugin, or do a PR to fix <a href="https://github.com/sdd/serverless-apig-s3/issues/12">https://github.com/sdd/serverless-apig-s3/issues/12</a>.</p><h3>Summary</h3><p>Serverless framework works like a charm for Lambda. As you expand <a href="https://twitter.com/dzimine/status/890207097803096065">beyond “FaaS” to broader “Serverless” set of services</a>, things get harders. You either rely on plugins, where milage varies, or go to the dark side of AWS CloudFormation. Yet the <a href="https://twitter.com/goserverless">Serverless</a> framework is ways ahead of the competition, recognizing and supporting the fact that Serverless &gt; FaaS.</p><h3>Dmitri Zimine on Twitter</h3><p>Serverless != Lambda Lambda == FaaS Serverless &amp;gt; FaaS Keep this math in mind #serverless #AWS #lambda #faas</p><p>The <a href="https://github.com/serverless/serverless">serverless framework</a> is well-designed. It makes the common tasks and configurations really easy, provides solid extensibility to modify and expand the “core” function set via plugins. For advanced configurations and things not yet supported, the user can fall back to cloud-native configurations. You <em>will</em> fall to writing CloudFormation templates for any but a trivial project, but the framework helps to place it in the code and weave it into the development lifecycle.</p><p>The plugin ecosystem is dynamic — many issues I hit had been fixed over the time of writing the blog. Check the <a href="https://github.com/serverless/plugins">curated list of serverless plugins</a> when you miss something: it may have it already, or give you a clue. The ecosystem still feels “wild” and “early”, would benefit from more attention, curation and contributions from @goserverless core team. I am sure they’ll get there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/1*DOxQqNQk6m667TrgDkYplA.png" /><figcaption>Hmm… What’s the difference between the two plugins, really?</figcaption></figure><p>The user community is large, lively and helpful. Between <a href="https://serverless.com/framework/docs/">documentation</a>, <a href="https://gitter.im/serverless/serverless">Gitter chat</a>, <a href="http://forum.serverless.com/">forum</a>, and Github, I quickly found answers to my questions.</p><p>Linguistic nit: With <em>serverless</em>, the framework name, and <em>serverless</em>, the big trend, writing about using <em>serverless</em> for <em>serverless</em> gets close to poetry: “A serverless is a serverless is a serverless”<a href="https://en.wikipedia.org/wiki/Rose_is_a_rose_is_a_rose_is_a_rose">[*]</a>.</p><p>The best part? Once the code is written, it deploys like a charm with serverless. Clone the code, configure credentials, build dependencies, deploy:</p><pre>serverless deploy<br>serverless client deploy</pre><p>and enjoy the benefits of #serverless computing.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e0bf7203d4b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>