<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ilana Polonsky on Medium]]></title>
        <description><![CDATA[Stories by Ilana Polonsky on Medium]]></description>
        <link>https://medium.com/@ilanap25?source=rss-78bff0c5cc4f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*FVCHHRJmxTr8Oxw0KrtAIA.jpeg</url>
            <title>Stories by Ilana Polonsky on Medium</title>
            <link>https://medium.com/@ilanap25?source=rss-78bff0c5cc4f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 09:35:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ilanap25/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[URL Redirection in Serverless]]></title>
            <link>https://medium.com/meliopayments/url-redirection-in-serverless-d1fc2fa99319?source=rss-78bff0c5cc4f------2</link>
            <guid isPermaLink="false">https://medium.com/p/d1fc2fa99319</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[lambda-edge]]></category>
            <category><![CDATA[melioengineering]]></category>
            <dc:creator><![CDATA[Ilana Polonsky]]></dc:creator>
            <pubDate>Tue, 05 Jul 2022 09:32:00 GMT</pubDate>
            <atom:updated>2023-08-01T09:24:34.880Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*xVmW7JcGbIO5C9Q4Ar6iXw.png" /></figure><p>In this blog post I’m going to introduce a use case we had at Melio, in which we needed to create a redirection from an old website with custom URL paths to a new website with a different path structure. We’ll look at the serverless components that are part of the solution, and finish with implementation details so you can go ahead and create this solution right away.</p><p>But first of all, what does it mean to create a redirection? <br>When a user asks for a certain URL, this request travels across multiple DNS servers, each holding different subdomain records pointing to the next layer of naming — for example, a “.com” server will point to the DNS server holding example.com record. The last DNS server on the list will return the IP address of the actual web server.</p><p>The simplest example of redirection is to change the pointing to a different IP address, this way all requests are forwarded as they are to a different IP. But what if we want to customize the request before forwarding it to the new IP? Let’s take a look at the use case we had…</p><h3>Challenging use case</h3><p>Melio is customer facing, and uses an external tool for its service desk. Part of this tool includes creating articles, like popular Q&amp;A regarding our product or technical support. <br>This product is a hosted service, and was not in our domain. So we created a subdomain ‘help-desk.mycompany.com’ in our domain, and used a DNS record to point to any access to an external website. It’s also important to mention that this website has a unique URL path for all our articles:`htps://old-desk.com/en/articles/&lt;ARTICLE-NUMBER&gt;</p><p>After some time using the old desk tool, we decided to move to another service desk for several reasons.<br>This new service desk is also a hosted service, with a different URL path for articles: “https://new-desk.com/hc/en-us/articles/&lt;NEW-NUMBER&gt;”.</p><p>Disclaimer: We can’t change this URL path or control the number associated with each article. We can only change the DNS record to point to the new website. The downside is that the URL path does not match on both ends, and any customer who has saved any original technical guide will be redirected to a nonexistent URL path, and we will lose our credibility.</p><p><strong>Page ranking — </strong>moving a website from one URL to another can affect page ranking (<a href="https://searchengineland.com/guide/what-is-seo">SEO</a>), so we need to include 301 code messages in our redirection.</p><p>The 301 code message is an HTTP response status code, saying “Page Moved Permanently”. The response also includes Location headers that indicate the new location for this page and pass all ranking scores from the old URL to the new URL.</p><p>So how do we work around these challenges?</p><h3>Doing it serverless</h3><p>Using AWS (or any other cloud provider) is already the first step into serverless. No more traditional way of buying servers, storing them in big cold rooms and taking a trip just to change a broken disk.</p><p>In our case, we’re going to use AWS services that enable us to force the action configuration and functionality of the solution, and not the provisioning and management of it.</p><p>Serverless services are automatically scaled, and don’t require us to actively provision new resources. For example, in Route 53 you only need to to create a Hosted zone and a DNS record, all network traffic behind the scenes is managed by AWS. So you don’t need to worry about managing DNS servers, and scaling up and down. Fun fact: AWS is committed to making Route 53 100% available (<a href="https://aws.amazon.com/route53/sla/#:~:text=AWS%20will%20use%20commercially%20reasonable%20efforts%20to%20make%20Amazon%20Route%2053%20100%25%20Available">Route 53 SLA</a>).</p><h3>Component overview</h3><p>Before jumping into the implementation details, I’d like to explain the components and services that we are going to use:</p><ul><li><strong>Route 53</strong> is a scalable and reliable DNS service, giving a simple and easy way to route traffic. It is also used for domain registration and management of any subdomain.</li><li><strong>CloudFront</strong> is a content delivery network, helping users to request data. CF uses distribution for setting the content to be delivered, resulting in low latency and low network trafficCloudFront that uses a cache mechanism at the edge location. This is effective in reducing latency by being closer to the user and reducing the load on your origin server.</li><li><strong>Lambda@Edge</strong> is a feature of CloudFront that allows you to run custom code in response to CloudFront events, in the form of AWS Lambda functions.</li><li><strong>CloudFormation </strong>lets you model, provision and manage AWS and third-party resources, by treating infrastructure as code.</li></ul><h3>Solution overview</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Q8dVP5jSzc7ffd5g" /></figure><p>The following diagram shows the path a request goes through in our architecture, starting from the user request leaving the computer.</p><p>As a request arrives at our local managed subdomain “help-desk.mycompany.com” in Route 53, this DNS record converts the name to a CloudFront distribution address (an alias record pointing to AWS service-Cloudfront). <br>For each request to the CloudFront distribution, there are two options: the content is cached on the edge location and delivered directly to the user, or the CloudFront needs to retrieve it from the origin we set up.</p><p>When setting the origin, and the behavior for this origin, we use the Function associations feature. So when we’re retrieving the data from the origin, we’re invoking the Lambda function and storing it in the cache. So the next request will be automatically answered, and the Lambda function won’t be invoked (lowering the cost).</p><p>The Lambda function is responsible for parsing the input request, and returning a response to the user, based on what was asked. The responsible body will contain the 301 error message, with location header for the new URL.</p><h3>Implementation Details</h3><p>Let’s take a look at how to configure each of the services introduced:</p><h4>Lambda function</h4><p>Create a Lambda function, and use Node.js 14.x runtime.</p><p>The following function was used in our case, redirecting to a different URL path:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e63268113d2c950bdb379642e63195a7/href">https://medium.com/media/e63268113d2c950bdb379642e63195a7/href</a></iframe><p>The function uses a dictionary to convert the old article number to the new one, and passes it to the new URL path. Any other URL requests are redirected to the new home page.</p><p>You can customize the return URL value for your specific use case. Create a version for your Lambda and save the ARN of it.</p><p>You can use the “Test” feature, and configure it as a template to input a “CloudFront Access Request In Response”, and test different scenarios of requests by changing the “url” value.</p><p>For the Lambda function permission, follow this <a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-edge-permissions.html">guide</a>. After deploying the Lambda function permission, create a version and save the ARN.</p><h4>CloudFront distribution</h4><p>When creating a new distribution use an HTTPS only protocol, and under Function: association: Viewer request enter the Lambda ARN. Add “CNAME: help-desk.mycompany.com”, and make sure you add an SSL certificate for it (<a href="https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request-public.html">Create a certificate</a>).</p><p>* If you’re using another distribution for the old website, you’ll need to remove the CNAME and only then add it (two distributes can’t use the same CNAME).</p><p>After creating the distribution, you can test how the Lambda and the distribution are behaving — using cURL as the command line tool to create http requests for the distribution. Your distribution has an address that can be reached at, for example, “abcdef.cloudfront.net”.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7486c17fbf401fcc0e563c5c65c0313b/href">https://medium.com/media/7486c17fbf401fcc0e563c5c65c0313b/href</a></iframe><h4>Route 53</h4><p>After setting and testing the distribution, all that is left is to change the DNS record to point to the new distribution — this DNS record is an alias type using CloudFront. After changing the record, everything should be synced within seconds. You can test your new configuration using your web browser or cURL, another cool CLI tool is the command “dig”, which collects data about Domain Name Servers.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f883e4f92ebbc51a2b3176b2411927fc/href">https://medium.com/media/f883e4f92ebbc51a2b3176b2411927fc/href</a></iframe><h3>Wrapping up</h3><p>As you can see, the solution in a serverless way is super easy to create and get started (just a few steps). <br>I highly recommend you start using more AWS serverless services, to make the process of provisioning and managing as easy as possible. <br>Added bonus, it also frees up the team’s time and resources, so they can invest more time on the application side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3qSivDZI-3fRph6erM7GtQ.png" /><figcaption><a href="https://bit.ly/jobs-medium">Visit our career website</a></figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d1fc2fa99319" width="1" height="1" alt=""><hr><p><a href="https://medium.com/meliopayments/url-redirection-in-serverless-d1fc2fa99319">URL Redirection in Serverless</a> was originally published in <a href="https://medium.com/meliopayments">Melio’s R&amp;D blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cross-Account Kinesis delivery stream to S3 Bucket destination in a different…]]></title>
            <link>https://medium.com/@ilanap25/cross-account-kinesis-delivery-stream-to-s3-bucket-destination-in-a-different-63d2740e15a3?source=rss-78bff0c5cc4f------2</link>
            <guid isPermaLink="false">https://medium.com/p/63d2740e15a3</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cross-account]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kinesis]]></category>
            <category><![CDATA[s3]]></category>
            <dc:creator><![CDATA[Ilana Polonsky]]></dc:creator>
            <pubDate>Tue, 14 Jun 2022 17:32:11 GMT</pubDate>
            <atom:updated>2022-06-14T17:32:11.191Z</atom:updated>
            <content:encoded><![CDATA[<h3>Cross-Account Kinesis delivery stream to S3 Bucket destination in a different account</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O2v0b73CUHPAhtC_CxMeeg.jpeg" /></figure><p>Create a Kinesis Firehose delivery stream in account A and set this destination as an S3 bucket in account B. In order to configure this cross-account communication, we need to allow the deliver stream write access to the bucket in a different account. Follow those steps:</p><ol><li>In Account B — Create S3 Bucket as the destination for the delivery stream<br>ARN: arn:aws:s3:::bucket-name</li><li>In account A — Create an IAM role to be used by the delivery stream <br>AWS Service: kinessis and choose Kinesis Firehose<br>Create a new policy and attach it to the role:</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/37a8d0e973ab17e7809830826f2ff604/href">https://medium.com/media/37a8d0e973ab17e7809830826f2ff604/href</a></iframe><p>IAM Role ARN: arn:aws:iam::ACCOUNT-A:role/Delivery-Role-S3-To-Account-B</p><p>3. In Account B — Edit the S3 access policy to allow the stream to write to this bucket:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0ae683b56909959b623177986e8d0a6c/href">https://medium.com/media/0ae683b56909959b623177986e8d0a6c/href</a></iframe><p><strong>4. </strong>In Account A: We need to create a temporary S3 bucket in order to create the delivery stream. if we try to enter account B bucket name we will get the following error:</p><p>Invalid URI. Choose an S3 bucket or enter a valid URI of an S3 bucket.</p><p>Bucket ARN: arn:aws:s3:::temp-bucket</p><p><strong>5.</strong> In Account A: Create a Kinesis Data Firehose delivery stream. <br>Let’s go with the basic configuration.</p><ul><li>Source: Direct Put</li><li>Destination: Amazon S3</li><li>Delivery stream name: Delivery-Stream-To-AccountB</li><li>Destination settings: s3://temp-bucket</li><li>Advanced Settings: Permission: Choose existing IAM role: <br>Delivery-Role-S3-To-Account-B</li></ul><p>You now have a delivery stream created, but we want our destination to be to the bucket that is at account B, AWS is now allowing us to select or enter a bucket that is not in the account so we need to use the CLI command to change the destination.</p><p><strong>6. </strong>Run those CLI commands to change the S3 destination bucket :</p><p>a. Get information about your stream:</p><pre>$ aws firehose describe-delivery-stream --delivery-stream-name Delivery-Stream-To-AccountB</pre><p>You will need VersionId, DestinationId</p><p>b. Change destination bucket:</p><pre>$ aws firehose update-destination --delivery-stream-name    Delivery-Stream-To-Account --current-delivery-stream-version-id $VersionId --destination-id ‘DestinationId’ --extended-s3-destination-update ‘{“BucketARN”:”arn:aws:s3:::bucket-name”}’</pre><p>Notice, that from now on every change made to the ‘destination Settings’ will need to be done via CLI command, else you will not be able to save the changes since AWS doesn’t recognize the bucket name.</p><p>For example, you can run the same command to change the S3 bucket prefix:<br>--extended-s3-destination-update ‘{“Prefix”:”subfolder/”}’ and don’t forget to update the version/destination-id.</p><p>7. Test the delivery with Demo data: <br>You can use the built-in feature to simulate data and test if the results are written to the S3 bucket in the other account.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aYZKMK1DPGQjjwfxp2hbuw.png" /></figure><p>Give it a few seconds and then Stop the sending, go to the destination bucket and check if new files were created.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=63d2740e15a3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Debugging terraform schema version changes made to a provider]]></title>
            <link>https://medium.com/@ilanap25/debugging-terraform-schema-version-changes-made-to-a-provider-1012c27f3d22?source=rss-78bff0c5cc4f------2</link>
            <guid isPermaLink="false">https://medium.com/p/1012c27f3d22</guid>
            <category><![CDATA[terraform-provider]]></category>
            <category><![CDATA[json]]></category>
            <category><![CDATA[error]]></category>
            <category><![CDATA[hashicorp]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Ilana Polonsky]]></dc:creator>
            <pubDate>Sat, 11 Jun 2022 12:08:42 GMT</pubDate>
            <atom:updated>2022-06-14T14:37:56.444Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XrXy7ya8mOAqyQhSPjLsCw.png" /></figure><p>The last time I ran terraform applyI got an error that google search couldn’t help much, so after solving it, I decided to share my debugging process.</p><pre>$ terraform apply<br>.<br>.<br>.<br>Plan: 1 to add, 0 to change, 0 to destroy.<br><br>Failed generating plan JSON<br>Exit code: 1</pre><pre>Operation failed: failed generating plan JSON: failed running command (exit 1)</pre><p>terraform apply include in itself the <a href="https://www.terraform.io/cli/commands/plan">terraform plan</a> command which does 3 main actions:</p><ul><li>Check the current state of the existing infrastructure</li><li>Read the correct configuration and compares it with the previous state</li><li>Based on those two actions proposes a list of changes that need to be taken to match the infrastructure to the configuration.</li></ul><p>The cool thing is that you can do only part of the command. for example, you just want to update the state file to match and reflect your current infrastructure, running terraform plan -refresh-only will do it.</p><p>The error indicates that it couldn&#39;t generate a JSON plan, so terraform failed on his third task.</p><h3>Debugging</h3><p>The command terraform show -json will show a JSON representation of the state.</p><pre>$ terraform show -json<br>Failed to marshal state to json: schema version 1 for random_password.master_password in state does not match version 2 from the provider</pre><p>The Failed to marshal state to json suggest the current provider (with their current schema version) is different than the provider who created and encoded the state file in the first place (version 1 vs version 2).</p><p>To take a close look at the details about the provider run:</p><pre>$ terraform providers schema -json | jq . &gt; schema.json</pre><p>OR</p><pre>$ terraform providers schema -json | jq &#39;.&quot;provider_schemas&quot;.&quot;registry.terraform.io/hashicorp/random&quot;.&quot;resource_schemas&quot;.&quot;random_password&quot;.&quot;version&quot;&#39;</pre><p>This command prints to a file and shows schemas for the providers used in the configuration. We are looking for the random_password providers in that file:</p><pre>..<br>&quot;random_password&quot;: {<br>  <strong>&quot;version&quot;: 2</strong>,<br>  &quot;block&quot;: {<br>   &quot;attributes&quot;: {<br>     &quot;bcrypt_hash&quot;: {<br>       &quot;type&quot;: &quot;string&quot;,<br>        &quot;description&quot;: &quot;A bcrypt hash of the generated random..</pre><p>We can also list the current providers versions:</p><pre>$ terraform version<br>Terraform v1.1.9<br>on darwin_amd64<br>+ provider registry.terraform.io/hashicorp/aws v3.75.2<br>+ provider registry.terraform.io/hashicorp/random v3.3.1<br>+ provider registry.terraform.io/sullivtr/graphql v2.5.1<br>+ provider registry.terraform.io/twingate/twingate v0.1.7<br>+ provider registry.terraform.io/winebarrel/mysql v1.10.6</pre><p>So as we understand random_password provider version v3.3.1 using version 2 for the schema, we would like to it to use version 1.</p><h3>Research</h3><p>In my case, I know that a week ago I was able to run terrafrom apply without any problem, so it means that there is a new version for this provider, and I need to find the least version that still works for me.</p><p>A quick google search lead me to the release list of random_password provider:</p><p><a href="https://github.com/hashicorp/terraform-provider-random/releases">Releases · hashicorp/terraform-provider-random</a></p><p>And as I can see, a new version was released 4 days ago.</p><h3>Solution</h3><p>By default terraform should be able to update the schema without you even noticing, but in some cases, the update doesn’t work so there is a workaround for it.</p><p>A possible solution is to pinpoint your provider version:</p><pre>$ vim terraform.tf<br>terraform {<br>    ....<br>    required_providers {<br>    random = {<br>      source = &quot;hashicorp/random&quot;<br>      <strong>version = &quot;3.2.0&quot;</strong><br>    }<br>  }<br>}</pre><p>and reinstall all your providers:</p><pre>$ rm .terraform.lock.hcl<br>$ terraform init</pre><p>You can repeat all the commands listed in the blog and see that the current and wanted version is 1 for random_password provider.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1012c27f3d22" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transform existing Lambda based stack to CloudFormation stack using SAM template— Part 3]]></title>
            <link>https://medium.com/@ilanap25/transform-existing-lambda-based-stack-to-cloudformation-stack-using-sam-template-part-3-68b61083b858?source=rss-78bff0c5cc4f------2</link>
            <guid isPermaLink="false">https://medium.com/p/68b61083b858</guid>
            <category><![CDATA[sam]]></category>
            <category><![CDATA[deploy]]></category>
            <category><![CDATA[stack]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloudformation]]></category>
            <dc:creator><![CDATA[Ilana Polonsky]]></dc:creator>
            <pubDate>Sun, 01 May 2022 12:14:09 GMT</pubDate>
            <atom:updated>2022-05-01T12:47:31.451Z</atom:updated>
            <content:encoded><![CDATA[<p>In this part, I will show how to convert all of the components that we have introduced to a SAM and CloudFormation template to pack and deploy everything at once via SAM commands.</p><p>In <a href="https://medium.com/@ilanap25/collect-parse-and-store-sso-events-from-cloudtrail-using-lambda-and-s3-bucket-part-2-78f07d85b889">part 2</a>, we used the python parser and ran it using Lambda Function, so we can have it running monthly, collect all the logs from the previous month, and store the results in an S3 bucket.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5K_WAWgR1Hv3BvCfIpL1UQ.png" /></figure><h3>Transform component to template file</h3><p>Our workflow includes the following components:</p><p><strong>EventBridge, Lambda Function, IAM Policy, IAM Execution Role, Resource-Based Policy, and S3 bucket.</strong></p><p>We created all those components manually using the AWS console UI in the previous part. Our template will include SAM resources and CloudFormation resources.</p><p>Let’s split the template into <strong>two</strong> parts and explain each one:</p><ol><li>CloudFormation resources: Since SAM is more oriented towards serverless applications, we will use the CloudFormation resources to create the IAM role and policy and the S3 bucket.</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d989b38a855adb44fc5ff54d9f65f3e7/href">https://medium.com/media/d989b38a855adb44fc5ff54d9f65f3e7/href</a></iframe><p>Let’s explain things:</p><ul><li>Transform: &#39;AWS::Serverless-2016-10-31 — a macro hosted by CF, takes a template written in the AWS SAM syntax and transforms and expands it into a compliant CloudFormation template.</li><li>Parameters — used to input custom values to your template each time you create or update a stack. In this case using it as a name convention prefix for all our services.</li><li>AWS::IAM::Role — using the AssumeRolePolicyDocument to defines which entities can assume the role. In this case we want lambda to use this role.</li><li>AWS::IAM:Policy —using inline polices document to list our actions and custom resource names. We are using Roles to attach those policies to our role.</li><li>AWS::S3::Bucket — creates our destination S3 bucket to store the results, adding configuration to ensure that the bucket is not public.</li></ul><p>Using !Sub and !Ref enable us to write more generic templates.<br>!Sub allows us to change part of the value string and !Ref enables us to reference other resources in the template.</p><p><strong>2. </strong>SAM resources: SAM is a framework for building serverless applications. It provides shorthand syntax to express functions, APIs, databases, and event source mappings. In addition, SAM transforms and expands the SAM syntax into AWS CloudFormation syntax.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d793f16bf8eaeff380aea50e65677445/href">https://medium.com/media/d793f16bf8eaeff380aea50e65677445/href</a></iframe><p>As you can see, we are using a different Type value. Let dive in:</p><ul><li>AWS::Serverless:Function — Creates an AWS Lambda function, an AWS IAM execution role, and event source mappings that trigger the function. This <strong>one</strong> block (13 lines) of code will create <strong>three</strong> resources: Lambda Function, EventBridge, and Resource-Based Policy.</li></ul><p>Most parameters are straightforward, but we do have a few points to notice:</p><ul><li>CodeUri — If the path to a local folder is provided, for the code to be transformed properly the template must go through the workflow that includes sam build followed by either sam deploy or sam package.</li><li>Role we are referencing the Role that was created to attack our Lambda</li><li>Events with this we are creating the EventBridge event that will trigger the Lambda to run monthly, using the cron tab syntax. This section created the EventBridge and Resource-Based Policy.</li></ul><p>*notice that we could use the Policies parameter to directly inside the inline policies we created using AWS::IAM:Policy .</p><p>When the stack is created, AWS SAM automatically adds a lambda:createdBy:SAM tag to this Lambda function, and to the default roles that are generated for this function.</p><p>The folder tree should have the template.yaml and our code inside src/ folder with lambda_function.py .</p><h3>Checking and Testing the template file</h3><p><strong>SAM Validate</strong><br>This command validates that the template provided is valid JSON / YAML.</p><pre>$ sam validate -t template.yaml<br>2022-01-19 10:19:30 Loading policies from IAM...<br>2022-01-19 10:19:36 Finished loading policies from IAM.<br>full_path/template.yaml is a valid SAM Template</pre><p><strong>CFN-lint<br></strong>This open source project validate AWS CF yaml/json templates against the Resource Specification and additional checks. SAM is is supported by the linter. (<a href="https://github.com/aws-cloudformation/cfn-lint">Github project</a>).</p><pre>$ cfn-lint template.yaml</pre><p>will return a non zero exit if there are any issues with your template</p><h3>Use SAM commands to build and deploy template</h3><p>After all this hard work of writing and testing, let’s get into deployment and see our Stack created in AWS.</p><p><strong>sam build</strong> <br>Processes your AWS SAM template file, application code, and any applicable language-specific files and dependencies.</p><pre>$ sam build<br>Build Succeeded</pre><pre>Built Artifacts  : .aws-sam/build<br>Built Template   : .aws-sam/build/template.yaml</pre><pre>$ tree .aws-sam<br>.aws-sam<br>├── build<br>│   ├── LambdaSSOProcessor<br>│   │   └── lambda_function.py<br>│   └── template.yaml<br>└── build.toml</pre><p>Build everything into staging folders so that everything is ready to be packaged and deployed.</p><p><strong>sam deploy</strong></p><p>Deploys an AWS SAM application.</p><pre>$ sam deploy --template-file template.yaml \<br>             --s3-bucket &lt;S3-NAME&gt; \<br>             --s3-prefix &lt;S3-IN-Folders&gt; \<br>             --stack-name &lt;STACK-NAME&gt; \<br>             --confirm-changeset \<br>             --capabilities CAPABILITY_IAM</pre><p>The deploy command firstly perform the sam package command and uploads the artifact to the s3 bucket. You can see output like Uploading to .. X/X after running deploy , then SAM deploys the application using AWS CloudFormation.</p><p>About the flags:</p><ul><li>Providing the S3 bucket name to store our artifact and processed template</li><li>confirm-changeset — super helpful flag to allow you to review the change set before deploying it.</li><li>capabilities CAPABILITY_IAM —our stack affect permissions in the AWS account (we crate an IAM role), we must explicitly acknowledge their capabilities.</li></ul><p><strong>Done!</strong> Go to the AWS CloudFormation console to see our stack, or to any other service that we created a resource on (Lambda, IAM, S3).</p><p><strong>Updating the Stack</strong></p><p>If you wish to update the stack and change the configuration (like update the runtime or update the Lambda code), just make the change and run the build and deploy commands again.</p><p><strong>sam delete</strong></p><p>Deleting the CloudFormation stack, the artifacts that were packaged and deployed to S3, and the AWS SAM template file.</p><pre>$ sam delete --stack-name &lt;NAME&gt;</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=68b61083b858" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Collect, parse and store SSO events from CloudTrail using Lambda and S3 bucket — Part 2]]></title>
            <link>https://medium.com/@ilanap25/collect-parse-and-store-sso-events-from-cloudtrail-using-lambda-and-s3-bucket-part-2-78f07d85b889?source=rss-78bff0c5cc4f------2</link>
            <guid isPermaLink="false">https://medium.com/p/78f07d85b889</guid>
            <category><![CDATA[eventbridge]]></category>
            <category><![CDATA[boto3]]></category>
            <category><![CDATA[aws-iam]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[lambda]]></category>
            <dc:creator><![CDATA[Ilana Polonsky]]></dc:creator>
            <pubDate>Sun, 27 Mar 2022 16:34:09 GMT</pubDate>
            <atom:updated>2022-03-27T16:34:09.371Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Collect, parse and store SSO events from CloudTrail using Lambda and S3 bucket — Part 2</strong></h3><p>In <a href="https://awstip.com/multiple-ways-to-query-sso-events-from-cloudtrail-events-history-part-1-fcc9c4971054">part 1</a>, we saw how to query events from CloudTrail event history in three ways: console, CLI and Python SDK. The result was parsed with Python to extract the SSO event information from the metadata of the event. Check it out if you missed it!</p><p>In this part, we will take the Python parser and run it using Lambda Function, so we can have it running monthly, collect all the logs from the previous month, and store the results to an S3 bucket.</p><h3>Components overview</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k9xyedvBsCCBXaF-Nf-b9g.jpeg" /></figure><p><em>Our workflow includes:</em></p><ul><li><strong>EventBridge</strong> — Delivers a stream of real-time data from event sources. For example: CloudWatch event that has a schedule expression to invoke a Lambda Function.</li><li><strong>Lambda Function</strong> — Lambda is a compute service that lets you run code without provisioning or managing servers. This way we will run our code on AWS automatically.</li><li><strong>Policy</strong> — Documents that control what actions an identity can perform, on which resources, and under what conditions. Our policy includes access to CloudTrail Events, put action on S3 and log Lambda execution.</li><li><strong>Execution Role</strong> — An IAM role that grants the function permission to access AWS services and resources. This role is used by Lambda and enables the policies listed.</li><li><strong>Resource-Based Policy </strong>— Grant an account permission to invoke or manage a function. We use it to allow an EventBridge to invoke your function on your behalf.</li><li><strong>S3 Bucket </strong>— Ability to store, retrieve, access, and back up any amount of data at any time and place. We will use it to store our parsed result.</li></ul><h3>Creating the resources</h3><p>We will use the convention of sso-processor-&lt;Service&gt; .</p><p><strong>S3 Bucket<br></strong>Head to the S3 service &gt; Buckets &gt; Create Bucket .</p><pre>#create you unique name and leave all the other fields default.<br>Bucket name: cloudtrail-sso-events-files </pre><p><strong>IAM Policy<br></strong>Head to the IAM service &gt; Policies &gt; Create Policy &gt; JSON and enter the following json policy:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5b6b34e3ac8c468c3cb812cf182be1a0/href">https://medium.com/media/5b6b34e3ac8c468c3cb812cf182be1a0/href</a></iframe><p>Add a new name to the policy: sso-processor-policy and create the policy.</p><p><strong>IAM Roles<br></strong>Head to the IAM service &gt; Roles &gt; Create Role</p><pre>Select type of trusted entity: AWS Service<br>Choose a use case:  Lambda <br>&gt; Next: Permissions<br>Attach permissions policies: search for sso-processor-policy<br>Role name:  sso-processor-role</pre><p><strong>Lambda Function<br></strong>Head to the Lambda service &gt; Functions &gt; Create function</p><pre>Function name: sso-processor-lambda<br>Runtime: Python 3.9<br>Execution role:  sso-processor-role   #role we create<br>&gt; create function </pre><pre># add timeout<br>Configuration &gt; General configuration &gt; timeout &gt; set to 60 sec</pre><p>Copy the following code into lambda_function.py file under the Code tab.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/74f4364bc7de7bf11f981638d986e215/href">https://medium.com/media/74f4364bc7de7bf11f981638d986e215/href</a></iframe><p>Deploy and check the function by running Test , you should see function logs in the Execution results and when it’s done a new file will be stored to the S3 bucket.</p><p><strong>The lambda_handler doing the following:</strong></p><ul><li>Get the first and last date of the previous month to use for the query</li><li>Query the CloudTrail service using boto3 and extract the CloudTrailEvents field from the event history json (more info about it in <a href="https://awstip.com/multiple-ways-to-query-sso-events-from-cloudtrail-events-history-part-1-fcc9c4971054">Part 1</a>).</li><li>Save the result into a json file on the local file system.</li><li>Upload the json file to S3 bucket name.</li></ul><p><strong>EventBridge Rule<br></strong>Head to the Amazon EventBridge service &gt; Rules&gt; Create Rule</p><pre>Name: SSOProcessorSchedule<br>Define Pattern: Schedule<br>Cron expression: cron(0 12 1 * ? *) #on the first of each month at 12:00.  </pre><pre>Select event bus: AWS default event bus <br>Target: Lambda<br>Function: sso-processor-lambda<br>&gt; Create<br># save the ARN of this rule - &lt;EVENT-ARN&gt;</pre><p>Add Resource-Based policy that will enable this event to invoke our function.</p><p>Head back to our Lambda configuration, Configuration Permission -&gt; Resource-based policy -&gt; Add permissions :</p><pre>AWS service<br>Service: EventBridge (CloudWatch Events)<br>Statement ID: &lt;Enter Unique ID&gt;  # create your own ID<br>Principal: events.amazonaws.com<br>Source ARN: &lt;EVENT-ARN&gt;<br>Action: lambda:InvokeFunction</pre><p>In part 3, I will show how to convert all of the components that we have introduced today to a SAM and CloudFormation template, and create a united stack, so we can deploy everything at once via simple command.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=78f07d85b889" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multiple ways to query SSO events from CloudTrail events history — Part 1]]></title>
            <link>https://awstip.com/multiple-ways-to-query-sso-events-from-cloudtrail-events-history-part-1-fcc9c4971054?source=rss-78bff0c5cc4f------2</link>
            <guid isPermaLink="false">https://medium.com/p/fcc9c4971054</guid>
            <category><![CDATA[aws-cli]]></category>
            <category><![CDATA[cloudtrail]]></category>
            <category><![CDATA[aws-sso]]></category>
            <category><![CDATA[aws-sdk]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Ilana Polonsky]]></dc:creator>
            <pubDate>Wed, 02 Mar 2022 14:34:15 GMT</pubDate>
            <atom:updated>2022-04-03T07:14:35.647Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Multiple ways to query SSO events from CloudTrail events history — Part 1</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/1*eG8itB_NrLqvU_QFMZ55oQ.png" /></figure><p>From a security and compliance point of view, we would like to have auditing for actions made to permission sets in our main account.</p><p>CloudTrail is already enabled on your AWS by default, and you don’t have to do anything else to use it. Every activity in the SSO is recorded in a CloudTrail event and the Event history service.</p><h4>Querying the events history via the console:</h4><p>In the AWS console, search for CloudTrail service &gt; Event history:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*i6vI8kh3P3FWg5ya" /><figcaption>Event history main console</figcaption></figure><p>We can filter by different filters like:</p><pre>Event Source : sso.amazonaws.com OR<br>Event Name: AttachManagedPolicyToPermisionSet</pre><p><a href="https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events-console.html">Here is more info</a> about all the possible<a href="https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events-console.html"> </a>filtering events.</p><p>We are interested in querying four different event names that are related to SSO configuration changes made to permission sets:</p><ul><li>AttachManagedPolicyToPermissionSet</li><li>DetachManagedPolicyFromPermissionSet</li><li>PutInlinePolicyToPermissionSet</li><li>DeleteInlinePolicyFromPermissionSet</li></ul><p>Let’s filter by one of them and limit the results to December only:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5Ql0pj-nnyKC6TMe" /><figcaption>Filtering by date</figcaption></figure><p>For each event, you can have information like: user who performed this action, IP source, region, and the event record that contains all of the data about that SSO event.</p><p><strong>Note </strong>As you can see, we can only filter one field and can’t use multiple filters. This leads us to the following query method — AWS CLI.</p><h4>Querying the events history via the AWS CLI:</h4><p>We want to filter by different filters. <br>Using the AWS CLI we can pass multiple attributeKeys:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3792f5db4da74801266d6e6c9fb2e969/href">https://medium.com/media/3792f5db4da74801266d6e6c9fb2e969/href</a></iframe><p>This will return all the AttachManagedPolicyToPermissionSet events that user performed in that month. <a href="https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events-cli.html">Here is more info</a> about the arguments.</p><p>The results will look like something like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5def4ef44d14ab3dc60b207aeb66c610/href">https://medium.com/media/5def4ef44d14ab3dc60b207aeb66c610/href</a></iframe><p>The results show the metadata for each event as we saw in the console.</p><p>Inside the CloudTrailEvent is the actual data we need; it contains information about the SSO event like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1efa0f6bf6a541a3286d3f140a6b9174/href">https://medium.com/media/1efa0f6bf6a541a3286d3f140a6b9174/href</a></iframe><p>Notice that the default number of results is 50, with a maximum of 50.</p><p>If we have more than 50, we need to perform another query with the same command line parameters, include a new parameter, and enter the token string that we see at the end of the result, like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7cbea5d96c34f9dde72f3c9a9608ef16/href">https://medium.com/media/7cbea5d96c34f9dde72f3c9a9608ef16/href</a></iframe><p><strong>Note: </strong>Notice that you can’t use the same AttributeKey twice in the AWS command.</p><p>This can be a problem for us since we want to query <strong>four</strong> different EventName values and also we get a different token string each time. That’s a lot of work…</p><p>So let’s take it to the next level and use the AWS Python SDK.</p><h4>Querying the events history via the AWS SDK Python:</h4><p>We are going to use boto3 to work with AWS, and query all <strong>four</strong> EventName:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6f0b150fd1f1b42b5a8f432d072053c1/href">https://medium.com/media/6f0b150fd1f1b42b5a8f432d072053c1/href</a></iframe><p>To get the next 50 events, we need to loop the results and add the token string returned in event_history_results (until the token is empty string).</p><h3>Bonus!</h3><p>We have everything that we need, but there is also too much metadata information about the cloud trail event. Therefore, we would like only data about the SSO event in the final result. This information is a key/value in each json we have in the list. I wrote some code to extract this information and parse it, since the format includes backslashes.</p><pre>&quot;{\&quot;eventVersion\&quot;:\&quot;1.08\&quot;,\&quot;userIdentity\&quot;:{\&quot;type\&quot;:\&quot;AssumedRole\&quot;,\&quot;principalId\&quot;:\&quot;XXXXXXXXXH:ilana@email.com\&quot;,\&quot;arn\&quot;:\&quot;arn:aws:...\&quot;,\&quot;accountId\&quot;:\&quot;999999998\&quot;,\&quot;accessKeyId\&quot;:\&quot;AXXXXXXXXZUX\&quot;,\&quot;sessionContext\&quot;:{\&quot;sessionIssuer\&quot;:{\&quot;type\&quot;:\&quot;Role\&quot;,\&quot;principalId\&quot;:\&quot;AROXXXXXXXXXXXXNH\&quot;,\&quot;arn\&quot;:\&quot;arn:aws.....&quot;:\&quot;cloudtrail.us-east-1.amazonaws.com\&quot;}}&quot;</pre><p>We want to get beautiful json format.</p><p>Use this function to extract the data in a json format:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9e3654fdb481bb5c7d9ba6ef5246c219/href">https://medium.com/media/9e3654fdb481bb5c7d9ba6ef5246c219/href</a></iframe><h3>Stay tuned!</h3><p>In the <a href="https://medium.com/@ilanap25/collect-parse-and-store-sso-events-from-cloudtrail-using-lambda-and-s3-bucket-part-2-78f07d85b889">next part</a>, I will show you how to automate this as a lambda function that automatically runs monthly, collects all the previous month’s events, and stores it into the s3 bucket.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fcc9c4971054" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/multiple-ways-to-query-sso-events-from-cloudtrail-events-history-part-1-fcc9c4971054">Multiple ways to query SSO events from CloudTrail events history — Part 1</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>