<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Xavier Clements on Medium]]></title>
        <description><![CDATA[Stories by Xavier Clements on Medium]]></description>
        <link>https://medium.com/@xavierclements?source=rss-96bd84dd7f5f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Z8raPo1bXR6Oy6UDD5VxxQ.jpeg</url>
            <title>Stories by Xavier Clements on Medium</title>
            <link>https://medium.com/@xavierclements?source=rss-96bd84dd7f5f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 13:36:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@xavierclements/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AWS re:Invent Recap — Wernel Vogel’s Keynote]]></title>
            <link>https://medium.com/weareservian/aws-re-invent-recap-wernel-vogels-keynote-c3d24f00a4cb?source=rss-96bd84dd7f5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/c3d24f00a4cb</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[lambda]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[dynamodb]]></category>
            <dc:creator><![CDATA[Xavier Clements]]></dc:creator>
            <pubDate>Thu, 29 Nov 2018 21:51:45 GMT</pubDate>
            <atom:updated>2019-07-08T05:54:44.058Z</atom:updated>
            <content:encoded><![CDATA[<h3>AWS re:Invent Recap — Wernel Vogel’s Keynote</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*zVz0S_39am7Dl-9SLaaTTQ.png" /></figure><p>Werner gave another passionate keynote address to a packed crowd here at re:Invent 2018.</p><p>I am not going to cover all of the release announcements he made however here are couple of the big hitters:</p><ul><li><a href="https://aws.amazon.com/about-aws/whats-new/2018/11/introducing-amazon-managed-streaming-for-kafka-in-public-preview/">Amazon Managed Streaming for Kafka</a> — this is huge! Fully managed and highly available Apache Kafka Service — potentially removing or at least lessening the steep learning curve and operational overhead with running one’s own Kafka.</li><li><a href="https://aws.amazon.com/about-aws/whats-new/2018/11/aws-lambda-now-supports-custom-runtimes-and-layers/">Lambda Runtime API</a> — enables developers to develop Lambda functions in their language of choice (as long as it is a language supported by linux)</li><li><a href="https://aws.amazon.com/about-aws/whats-new/2018/11/aws-lambda-now-supports-custom-runtimes-and-layers/">Lambda Layers</a> — This is exciting as packaging up the libraries for Lambda functions and deploying them can be really painful. Per the documentation:</li></ul><blockquote>With layers, you can centrally manage common components across multiple functions enabling better code reuse. To use layers, you simply put your common code in a zip file, and upload it to Lambda as a layer.</blockquote><ul><li><a href="https://aws.amazon.com/blogs/aws/new-aws-toolkits-for-pycharm-intellij-preview-and-visual-studio-code-preview/">AWS Toolkits for popular IDES</a> — Generally available for PyCharm with IntelliJ and VisualStudio Code in preview.</li><li><a href="https://aws.amazon.com/blogs/aws/new-aws-well-architected-tool-review-workloads-against-best-practices/">AWS Well-Architected Tool </a>— The process of requesting a well-architected review by AWS or its partners has now been automated to a great extent. Per the documentation:</li></ul><blockquote>The <a href="https://aws.amazon.com/well-architected-tool/">AWS Well-Architected Tool</a> helps you to define your workload, answer questions designed to review the workload against the best practices specified by the five pillars, and to walk away with a plan that will help you to do even better over time. The review process includes educational content that focuses on the most current set of AWS best practices.</blockquote><ul><li><a href="https://aws.amazon.com/about-aws/whats-new/2018/11/alb-can-now-invoke-lambda-functions-to-serve-https-requests/">Application Load Balancer Support for Lambda</a> — Werner hinted that this would be a great mechanism for transferring workloads that might currently be done upon servers to Lambda functions without affecting the end user.</li><li>AWS Step Functions is no longer just about orchestrating one’s Lambda Functions. Other service integrations which were announced include: <a href="https://aws.amazon.com/batch/">AWS Batch</a>, <a href="https://aws.amazon.com/fargate/">AWS Fargate</a>, <a href="https://aws.amazon.com/glue/">AWS Glue</a>, <a href="https://aws.amazon.com/sagemaker/">Amazon Sagemaker</a>, <a href="https://aws.amazon.com/sqs/">Amazon SQS</a>, <a href="https://aws.amazon.com/dynamodb/">Amazon DynamoDB</a> and 2 others that the resolution of the photo I am looking at does not let me read.</li><li>WebSocket Support for API Gateway coming soon.</li></ul><p>I likely missed a couple but my colleagues <a href="https://medium.com/u/7d0c95526ad4">Marat Levit</a>, <a href="https://medium.com/u/8ff660c7c041">Charles Sutikno</a> and <a href="https://medium.com/u/51b1e536b27e">Woratana Perth</a> will help to fill in the gaps.</p><p>Find out more about Servian’s AWS capability <a href="https://www.servian.com/convergent/">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c3d24f00a4cb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/weareservian/aws-re-invent-recap-wernel-vogels-keynote-c3d24f00a4cb">AWS re:Invent Recap — Wernel Vogel’s Keynote</a> was originally published in <a href="https://medium.com/weareservian">Cognizant Servian</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS re:Invent Recap — AWS Ground Station as a Service(Wednesday 2018–11–28)]]></title>
            <link>https://medium.com/weareservian/aws-re-invent-recap-aws-ground-station-as-a-service-wednesday-2018-11-28-a5a60ecc5032?source=rss-96bd84dd7f5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/a5a60ecc5032</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[satellite]]></category>
            <dc:creator><![CDATA[Xavier Clements]]></dc:creator>
            <pubDate>Thu, 29 Nov 2018 03:23:28 GMT</pubDate>
            <atom:updated>2019-07-08T05:59:19.478Z</atom:updated>
            <content:encoded><![CDATA[<h3>AWS re:Invent Recap — AWS Ground Station as a Service(Wednesday 2018–11–28)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*zVz0S_39am7Dl-9SLaaTTQ.png" /></figure><p><em>Note: </em>I am going to post the blogs out of order, the next being the summary of security sessions that I attended on Tuesday. They were extremely interesting, interrelated security sessions that I would like to synthesize into something more consumable.</p><p>Today, beyond attending the re:Invent 2018 keynote in one of the spillover rooms, I attended a session on <a href="https://aws.amazon.com/neptune/">Amazon Neptune</a> and one on the recently released <a href="https://aws.amazon.com/ground-station/">AWS Ground Station as a Service</a>. I won’t deal with Neptune in this blog as I want to look into the service and the associated tools and then perhaps put something together — targeting newbies like myself.</p><h3>Introducing AWS Ground Station — Fully managed Ground Station as a Service</h3><p>Speakers:</p><ul><li>Lowell Hawthorne — General Manager</li></ul><h4><a href="https://aws.amazon.com/ground-station/">AWS Summary</a></h4><p>Have you ever wanted to set up your own Ground Station so as to tap into satellite data and then batch or stream said data into your own AWS environment? No, well, neither have I. But now with the advent of this service it becomes an option.</p><p>From the above link, here is how the service is supposed to work at a high level:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/0*Adw4H4O5zhMTm_5Q.png" /></figure><p><strong>Customer Challenges:</strong></p><p>Lowell outlined the major complaints that customers have brought to him in regarding setting up their ground stations:</p><ul><li>Expensive to build and difficult to maintain their own ground stations.</li><li>Require high capital expenditure investment to scale</li><li>experience network latency and scheduling conflicts</li><li>have opaque pricing</li></ul><p><strong>AWS Ground Station as a Service (GSaaS) — IN PREVIEW</strong></p><p>AWS’ answer to the customers’ needs. Major points from the presentation include:</p><ul><li>There will be an onboarding process, to ensure that not just anyone can access the infrastructure. This is to allay concerns of those who own the satellite.</li><li>No ground station infrastructure commitments.</li><li>Usage-based, per-minute pricing. Reserved minutes also an option to decrease costs.</li><li>Self-service schedulting.</li><li>12 ground stations collocated with AWS Regions (in 2019)</li><li>Direct access to AWS services, infrastructure and global network.</li><li>Provide 2 different data rates — likened to the difference between 4G (low data rate) and 5G (high data rate).</li><li>Each ground station will have two antennas for redundancy.</li><li>Propagation and Location Sync Service — give predictions about availability of satellite.</li></ul><p><strong>Preview</strong></p><ul><li>Supported orbits: Low Earth Orbit (LEO), Medium Earth Orbit (MEO)</li><li>Supported frequencies: simultaneous narrowband S-band, X-band, UHF downlink.</li><li>2 ground stations — Oregon and Ohio</li></ul><p><strong>General Availability in 2019</strong></p><ul><li>Supported orbits: Low Earth Orbit (LEO), Medium Earth Orbit (MEO)</li><li>Supported frequencies at Launch: simultaneous narrowband S-band, X-band, UHF downlink and <strong>uplink</strong>.</li><li>Global coverage from 12 regions</li></ul><h4>Use Cases</h4><p>During the session Lowell canvassed a couple of the use cases that are envisaged or already in play for customers. Below are three with the associated AWS image outlining how it is envisaged that it will work:</p><ul><li>Natural Disaster Response.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iFJTEzz4bi5lx4TO.png" /></figure><ul><li>Accurate Weather Prediction</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/971/0*pXV2gYI44avj71Gq.png" /></figure><ul><li>Business Trends Assessment</li></ul><p>Also, if you like a bit of ML in your life have a gander, squiz or even a look at <a href="https://medium.com/u/8ff660c7c041">Charles Sutikno</a>’s article about <a href="https://medium.com/weareservian/ml-at-the-edge-3a9f05fb9ea3">Machine Learning at the Edge</a>.</p><p>Speaking of which have a look at <a href="https://medium.com/u/7d0c95526ad4">Marat Levit</a>’s round up of today’s releases <a href="https://medium.com/weareservian/new-product-launches-from-aws-re-invent-2018-keynote-featuring-andy-jassy-ceo-aws-dca4ce6896be">here</a>.</p><p>Find out more about Servian’s AWS capability <a href="https://www.servian.com/convergent/">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a5a60ecc5032" width="1" height="1" alt=""><hr><p><a href="https://medium.com/weareservian/aws-re-invent-recap-aws-ground-station-as-a-service-wednesday-2018-11-28-a5a60ecc5032">AWS re:Invent Recap — AWS Ground Station as a Service(Wednesday 2018–11–28)</a> was originally published in <a href="https://medium.com/weareservian">Cognizant Servian</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS re:Invent Recap — Keynote (Wednesday 2018–11–28)]]></title>
            <link>https://medium.com/weareservian/aws-re-invent-recap-keynote-wednesday-2018-11-28-62172da98316?source=rss-96bd84dd7f5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/62172da98316</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[awsreinvent]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Xavier Clements]]></dc:creator>
            <pubDate>Wed, 28 Nov 2018 20:09:52 GMT</pubDate>
            <atom:updated>2018-11-28T20:09:52.848Z</atom:updated>
            <content:encoded><![CDATA[<h3>AWS re:Invent Recap — Keynote (Wednesday 2018–11–28)</h3><p>Some brief notes about the services that have been released.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*zVz0S_39am7Dl-9SLaaTTQ.png" /></figure><h3>Enterprise Level Services</h3><h4><a href="https://aws.amazon.com/controltower/">AWS Control Tower</a></h4><ul><li>Automated Landing Zone with best-practice blueprints</li><li>Guardrails for policy enforcement</li><li>Dashboard for continuous visibility</li></ul><h4><a href="https://aws.amazon.com/security-hub/">AWS Security HUB</a></h4><ul><li>Save time by aggregating alerts (normalises inputs)</li><li>View summary of prioritised alerts</li></ul><h4><a href="https://aws.amazon.com/lake-formation/">AWS Lake Formation</a></h4><ul><li>data lake in days instead of months</li><li>Move, store, catalog, and clean your data faster with machine learning</li><li>enforce security policies and encryption</li><li>gain and manage new insights</li></ul><h3>Database Freedom</h3><h4>Amazon Aurora</h4><ul><li>fastest growing service in the history of AWS</li><li>2x growth of customers</li><li>Aurora Serverless</li><li>Aurora Global Database: lag of &lt;1s for replicas</li></ul><h4>Amazon DynamoDB</h4><ul><li>provisioning capacity for DynamoDB difficult for certain customers as unsure</li><li><a href="https://aws.amazon.com/blogs/aws/amazon-dynamodb-on-demand-no-capacity-planning-and-pay-per-request-pricing/"><strong>DynamoDB read/write capacity on demand</strong></a></li><li>No capacity planning</li><li>Ideal for unpredictable workloads — don’t use for known workloads</li></ul><h4><a href="https://aws.amazon.com/timestream/">Amazon Timestream</a></h4><p>Fast, scalable fully managed time series database</p><ul><li>multiple orders of magnitude performance improvement</li><li>adaptive querying</li><li><strong>1000x</strong> times faster than relational databases for time series data</li><li>trillions events daily handled</li><li>serverless</li></ul><h4><a href="https://aws.amazon.com/qldb/">Amazon Quantum Ledger Database</a> (Centralised Trust)</h4><ul><li>Backdrop — QLDB built for tracking S3 data plane changes</li><li>Immutable — append-only, immutable journal tracks history of all changes</li><li>Cryptographically verifiable</li><li>Transparent — full visibility into entire data set</li><li>Fast — execute 2–3x more transactions</li><li>highly scalable</li><li>Easy to use — query with familiar SQL operators</li></ul><h4><a href="https://aws.amazon.com/managed-blockchain/">Amazon Managed Blockchain</a> (Decentralised Trust)</h4><ul><li>Choose between Hyperledger Fabric or Etherium (2019 release)</li><li>Scales to support thousands of applications and running millions of transactions</li><li>Create blockchain networks with a few clicks; manage them with simple API calls</li><li>Easy to move data into QLDB for further analysis</li></ul><h3>Machine Learning</h3><h3>Infrastructure and frameworks level</h3><h4><a href="https://aws.amazon.com/blogs/aws/amazon-elastic-inference-gpu-powered-deep-learning-inference-acceleration/">Amazon Elastic Inference</a></h4><ul><li>up to 75% savings</li><li>attached (like EBS) to an instance — identifies the frameworks that are supported and then optimises the use of the GPUs on the associated machines</li><li>VPC bound</li></ul><h4><a href="https://aws.amazon.com/machine-learning/inferentia/">AWS Inferentia</a></h4><ul><li>ML inference chip</li><li>2019 release</li><li>support multiple data types and most major frameworks</li><li>available on most EC2s</li><li>with Elastic inference expected to see 10x drop in cost</li></ul><h3>ML Services</h3><h4><a href="https://aws.amazon.com/blogs/aws/amazon-sagemaker-ground-truth-build-highly-accurate-datasets-and-reduce-labeling-costs-by-up-to-70/">Amazon Sagemaker Ground Truth</a></h4><ul><li>highly accurate labelling service for data</li><li>reduce cost up to 70%</li></ul><h4><a href="https://aws.amazon.com/blogs/aws/new-machine-learning-algorithms-and-model-packages-now-available-in-aws-marketplace/">AWS Marketplace for machine learning</a></h4><ul><li>browse algorithms</li><li>subscribe in a single click</li><li>available in Sagemaker afterwards</li></ul><h4><a href="https://aws.amazon.com/blogs/aws/amazon-sagemaker-rl-managed-reinforcement-learning-with-amazon-sagemaker/">Amazon Sagemaker RL</a></h4><ul><li>fully managed RL algos</li><li>tensorflow, MXNet, Intel Coach, and Ray RL</li><li>example notebooks and tutorials</li></ul><h4><a href="https://aws.amazon.com/deepracer/">AWS DeepRacer</a></h4><ul><li>3D simulator with virtual car and track</li><li>apply simple or complex reward functions</li><li>Sagemaker starts training on the RL algorithm</li><li>Once trained deploy to DeepRacer</li><li>Sports League — autonomous racing league</li></ul><h3>AI Services</h3><p>Built models rather than developing one’s own</p><h4><a href="https://aws.amazon.com/textract/">Amazon Textract</a></h4><ul><li>OCR++ service</li><li>can identify more difficult to establish objects e.g. tables / columns within text</li><li>recognise that certain data objects are in order to address updating format issues overtime</li><li>no need to write the rules / templates</li></ul><h4><a href="https://aws.amazon.com/blogs/aws/amazon-personalize-real-time-personalization-and-recommendation-for-everyone/">Amazon Personalize</a></h4><ul><li>real time personalization based on Amazon personalisation algorithms</li><li>uses EMR under the hood</li><li>choose from 6 models</li></ul><h4><a href="https://aws.amazon.com/forecast/">Amazon Forecast</a></h4><ul><li>Accurate time series forecasting based on same forecasting mechanisms in Amazon</li><li>choose from 8 models under the hood</li></ul><h3>On Prem to Cloud</h3><h4>VMware Cloud on AWS (previously announced)</h4><ul><li>intention is that for every region of AWS, VMWare will be there by 2019 end</li><li>Amazon RDS on VMWare environments (previously released) creating customer success</li></ul><h4><a href="https://aws.amazon.com/outposts/">AWS Outposts</a> (2019)</h4><p>Run AWS infrastructure on premises for a truly consistent AWS</p><ul><li>Options 1 — VMWare Cloud on AWS</li><li>Options — AWS Native</li><li>Same hardware as AWS</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=62172da98316" width="1" height="1" alt=""><hr><p><a href="https://medium.com/weareservian/aws-re-invent-recap-keynote-wednesday-2018-11-28-62172da98316">AWS re:Invent Recap — Keynote (Wednesday 2018–11–28)</a> was originally published in <a href="https://medium.com/weareservian">Cognizant Servian</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS re:Invent Recap — Serverless & Security (Monday 2018–11–26)]]></title>
            <link>https://medium.com/weareservian/aws-re-invent-recap-serverless-security-monday-2018-11-26-888087f5c4a9?source=rss-96bd84dd7f5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/888087f5c4a9</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[awsreinvent]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Xavier Clements]]></dc:creator>
            <pubDate>Tue, 27 Nov 2018 15:37:44 GMT</pubDate>
            <atom:updated>2018-11-27T15:37:44.193Z</atom:updated>
            <content:encoded><![CDATA[<h3>AWS re:Invent Recap — Serverless &amp; Security (Monday 2018–11–26)</h3><p>Below are just some highlights from 2 of the sessions that I attended yesterday at re:Invent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*zVz0S_39am7Dl-9SLaaTTQ.png" /></figure><h3>Building Serverless Analytics Pipelines with AWS Glue</h3><p>Speakers:</p><p>Mehul Shah — Sr. Software Development Manager<br>Arup Ray — VP Data Engineering, realtor.com</p><h4>Announcement:</h4><p>A new AWS Glue ETL primitive to be released in December 2018: <strong>AWS Glue Python shell. </strong>AWS described this new feature a “new cost-effective ETL primitive for small to medium tasks”.</p><p>Arup Ray from <a href="https://www.realtor.com/">realtor.com</a> likened the new ETL primitive to the role of the Edge Node in traditional Hadoop ecosystems i.e. the edge will be able to execute lightweight transformations or push jobs out to the cluster (<strong>AWS Glue ETL Jobs</strong>).</p><p>The breakdown about this new feature:</p><ul><li>Python 2.7 environment with many of the usual suspects installed: boto3, awscli, numpy, scipy, pandas, scikit-learn etc.</li><li>Cold spin up is <strong>under</strong> <strong>20 seconds</strong></li><li>Supports VPCs</li><li><strong>No runtime limit</strong></li><li><strong>Sizes</strong>: 1 Data Processing Unit (DPU — see <a href="https://aws.amazon.com/glue/pricing/">here</a> for details) with 16 Gb memory or 1/16 DPU (includes 1Gb)</li><li><strong>1 minute minimum duration </strong>compared to 10 minutes for regular AWS Glue ETL jobs.</li><li><strong>Pricing: USD$0.44 </strong>per DPU-hour with per second billing</li></ul><h4>AWS Summary</h4><p>Overall performance improvements with AWS Glue made over the last 12 months were summarised:</p><ul><li>Crawlers had a <strong>7x</strong> performance improvements running over <strong>90M+ </strong>files.</li><li>AWS Glue’s Dynamic Frames optimised for ETL as compared to Apache Spark Dataframes. Where the latter requires schema to be provided, no need for same with the Dynamic Frames.</li><li>Optimising for small files: Dynamic Frames automatically group files into fewer tasks to aid in lessening the performance degradation that comes with a large number of small files. Further improvements in the future were hinted at.</li><li>Optimizing parallelism: With the requisite logging enabled, the optimum number of DPUs required can be established quickly with the “needed executors” metric.</li></ul><h4>Business Case</h4><p>Arup Ray from realtor.com described the data ingestion journey the company had been on with AWS.</p><p>An early iteration of their data platform saw them using <a href="https://aws.amazon.com/datapipeline/">AWS Data Pipeline</a> with <a href="https://aws.amazon.com/ecs/">Amazon ECS</a> to extract and coordinate updates to the metadata that was used to orchestrate and lock the AWS Glue ETL jobs that would transform the raw data into one of their Processed Data Transactional curated data sets.</p><p>In a new iteration, <a href="https://airflow.apache.org/">Airflow</a> was used to orchestrate the individual application’s data movement and, as an early beta adopter, realtor.com replaced ECS with AWS Glue Python Shell. It was noted that one of the major benefits was reduced operational overhead.</p><h3>Security &amp; Compliance for Modern Serverless Applications</h3><p>Speakers:</p><ul><li>David Behroozi — Software Development Engineer, AWS</li><li>Daniel Lees — Principal Cloud Architect, HSBC</li></ul><h4>AWS Summary</h4><p>High level takeaways from the session include:</p><ul><li>Use <a href="https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-paramstore.html">AWS Simple Systems Manager Parameter Store</a> to organise configurations and <a href="https://aws.amazon.com/secrets-manager/">AWS Secrets Manager</a> for secrets associated with Lambda functions. Enables the ability to control the rotation of these credentials and can ensure that Lambda environment variables are only sparingly used.</li><li>Ensure (and we knew it was coming) that the policies tied to each IAM role for each Lambda function are (wait for it) least privilege.</li><li>Use <a href="https://aws.amazon.com/waf/">AWS WAF</a> liberally and integrate same with<a href="https://aws.amazon.com/api-gateway/"> Amazon API Gateway</a> for your applications</li><li>Avoid inconsistent security hygiene with users of the application by using services like <a href="https://aws.amazon.com/cognito/">Amazon Cognito</a> to authenticate users and then enable advanced features of that service e.g. prevent sign up if any combination of the username / password has been externally compromised.</li><li>Use only one token type across one’s platform particularly if there are multiple authorisation enforcement points.</li><li>Only use short-lived tokens, be they for Amazon Cognito User Pools (read Identity Provider service) or Identity Pools (for Federated Access to AWS Services).</li><li>Do not implement a mechanism of checking tokens — use services like API Gateway.</li><li>To detect vulnerabilities in 3rd party libraries, be they in Lambda packages or not, use the <a href="https://www.owasp.org/index.php/OWASP_Dependency_Check">Open Web Application Security Project (OWASP) Dependency Checker</a>.</li><li>Log the “RequestId” from all AWS API calls made within Lambda functions so as to trace through to those services.</li><li>Enable <a href="https://aws.amazon.com/xray/">AWS X-Ray</a></li><li>Ensure that CloudTrail is enabled and capturing data events.</li></ul><h4>Business Case</h4><p>Daniel Lees showcased how HSBC developed a “framework of continuous compliance” so as to monitor and control the services and interactions within their <strong>&gt;300</strong> AWS accounts.</p><p>At a high level this framework involved:</p><ol><li>Selecting AWS Services</li><li>Defining and creating secure patterns</li><li>Design controls.</li><li>Build and Implement</li><li>Monitor</li></ol><p>The controls were of three different types:</p><ul><li><strong>Corrective</strong>: The service or feature that has been spawned is not configured properly or not permitted and Lambda functions are triggered to disable that service and send an email to the necessary parties about the exemption process that needs to be followed.</li><li><strong>Detective</strong>: As I understood, this was purely for investigating accounts and their services.</li><li><strong>Preventative</strong>: A requested service is simply not allowed and will not be created.</li></ul><p>The architecture that was shown had a central <strong>AWS Compliance Account</strong> which was connected to multiple solutions accounts via strict AWS Organisation controls.</p><p>The flow of the solution, as far as the grainy photo I took of the architecture slide indicates is as follows:</p><ol><li>A CloudWatch Event Bus (see <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/CloudWatchEvents-CrossAccountEventDelivery.html">here</a> for details) in the solution accounts pushes logs to another Event Bus within the Compliance Account.</li><li>This then triggers one of many Lambda functions which will first check whether the relevant breach is permitted within the Exemptions Service which is housed in <a href="https://aws.amazon.com/dynamodb/">Amazon DynamoDB</a>.</li><li>If no exemption is found, a Lambda function in the Compliance Account will trigger a remediative action in the offending solution account whilst at the same time a SNS notification is sent (though likely SES). The <em>really</em> impressive part is that an email is sent to the relevant recipients of the solution account advising of the exemption process that needs to be followed for the relevant service that triggered the initial event.</li></ol><p>The team at HSBC have a dashboard collating and categorising the risk that each of the current accounts pose for a one stop view of the current state of play.</p><p>In my next blog I’ll endeavour to put a few images in. This was a tad dry.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=888087f5c4a9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/weareservian/aws-re-invent-recap-serverless-security-monday-2018-11-26-888087f5c4a9">AWS re:Invent Recap — Serverless &amp; Security (Monday 2018–11–26)</a> was originally published in <a href="https://medium.com/weareservian">Cognizant Servian</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>