<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Deependra Saraswat on Medium]]></title>
        <description><![CDATA[Stories by Deependra Saraswat on Medium]]></description>
        <link>https://medium.com/@deependra-saraswat?source=rss-10cb688cf928------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*0NE3cNaydrF9-GUq.jpg</url>
            <title>Stories by Deependra Saraswat on Medium</title>
            <link>https://medium.com/@deependra-saraswat?source=rss-10cb688cf928------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 04:31:55 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@deependra-saraswat/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Terraform — The Power of Dynamic Block]]></title>
            <link>https://deependra-saraswat.medium.com/terraform-the-power-of-dynamic-block-6d876ffe088?source=rss-10cb688cf928------2</link>
            <guid isPermaLink="false">https://medium.com/p/6d876ffe088</guid>
            <category><![CDATA[ec2-instance]]></category>
            <category><![CDATA[modules]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[dynamic-block]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Deependra Saraswat]]></dc:creator>
            <pubDate>Mon, 11 Jan 2021 11:55:46 GMT</pubDate>
            <atom:updated>2021-01-11T11:55:46.851Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Terraform — The Power of Dynamic Block</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5eWI1ucNf-XklZ8b.jpg" /></figure><p>When I started writing terraform modules, the idea was to write modules in such a way that the terraform code is highly reusable and encompasses all the edge cases giving the end user capability to perform multiple operations or skip some as per the requirement.</p><p>Say for example, there can be a case where you want to provision few EC2 instances which just have only the root volume attached whereas there may also be a requirement to provision EC2s which are backed by EBS. You may also want to attach multiple EBS volumes to the same EC2. One more example can be of a situation where you want to create private and public hosted zones in Route53. The terraform module for a private hosted zone will mandatorily need a VPC parameter whereas a public hosted zone would not. Achieving all this with the conventionally written terraform code won’t be possible until unless we introduce redundancy which will eventually violate the basic principle of programming i.e always write the DRY code.</p><p>The solution is to use dynamic block. The dynamic block in terraform helps to write the DRYest possible code. Let’s try to understand all this by building an EC2 module which handles instance creation.</p><p>Below is the terraform module for provisioning EC2.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ab4f396863cd3863a87cb5f5432965f5/href">https://medium.com/media/ab4f396863cd3863a87cb5f5432965f5/href</a></iframe><p>This terraform module will take care of creating EC2 instances in AWS based on user inputs. The <em>count</em> acts like a for loop and executes the code <em>var.number_of_instances</em> number of times. Meaning which if there are <em>N</em> number of instances to be provisioned then variable <em>number_of_instances</em> is assigned value <em>N</em> while calling the module and the count block executes the code <em>N</em> times. The rest of the parameters covers things like AMI, instance type, subnet id , root block device etc. which are required for spinning up EC2s.</p><p>That was just a high level overview of what this module does. Now let’s talk about the dynamic block(ebs_block_device) mentioned in the above module in detail.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7e86e1b13a4b2d8ef0087861c0444244/href">https://medium.com/media/7e86e1b13a4b2d8ef0087861c0444244/href</a></iframe><p>The dynamic block acts much like a for expression, but produces nested blocks instead of a complex typed value. It iterates over a given complex value, and generates a nested block for each element of that complex value.</p><p>Here the ebs_block_devicedynamic block will be executed for every map passed in ebs_block_device variable which accepts a list of map. By default, the ebs_block_device is kept blank meaning which no EBS volume will be provisioned if no maps are passed.</p><blockquote>The for_each meta-argument accepts a map or a set of strings, and creates an instance for each item in that map or set</blockquote><blockquote>lookup retrieves the value of a single element from a map, given its key. If the given key does not exist, the default value is returned which is null in this case for all the keys.</blockquote><blockquote><em>delete_on_termination </em>accepts a boolean value and enables/disables termination protection for the EBS volume.</blockquote><blockquote><em>device_name </em>is the device name to expose to the instance. For exmaple, /dev/sdh etc.</blockquote><blockquote><em>snapshot_id </em>The Snapshot ID to mount.</blockquote><blockquote>iops The amount of IOPS to provision for the disk. Only valid for type of io1 or io2.</blockquote><blockquote><em>volume_size</em> The size of the volume in gibibytes (GiB).</blockquote><blockquote><em>volume_type</em> The type of volume. Can be &quot;standard&quot;, &quot;gp2&quot;, &quot;gp3&quot;, &quot;io1&quot;, &quot;io2&quot;, &quot;sc1&quot;, or &quot;st1&quot;. (Default: &quot;gp2&quot;).</blockquote><blockquote><em>kms_key_id</em> Amazon Resource Name (ARN) of the KMS Key to use when encrypting the volume. Must be configured to perform drift detection.</blockquote><blockquote><em>encrypted</em> Enables EBS encryption on the volume. Accepts boolean value. Cannot be used with snapshot_id</blockquote><p>Below is an example of how this EC2 module can be called to provision instances in AWS.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/754815495c663a9fee1cbd2ce4827697/href">https://medium.com/media/754815495c663a9fee1cbd2ce4827697/href</a></iframe><p>In the above code <em>ebs_block_device </em>variable accepts a list of maps. The map contains a key value pairs. The first map in the list contains details for <em>“/dev/sdf”</em> device and the second map contains details for <em>“/dev/sdg”</em> device. For every EC2 instance provisioned there will be 2 EBS volumes provisioned as per above code. The default values of keys for which no value is passed is <em>null</em> as mentioned in the module.</p><p>If we do not wish to provision EBS backed instances, just omit <em>ebs_block_device </em>variable. Its default value is an empty list and hence no EBS will be provisioned.</p><p>In this way, we can use the dynamic block to write DRY terraform code to build various modules — Route53 zones, s3, elastic bean stalk etc. etc. which are highly reusable and cover all the edge cases.</p><p><strong><em>Happy Learning!!!</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6d876ffe088" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Prometheus — EC2 Service Discovery]]></title>
            <link>https://deependra-saraswat.medium.com/prometheus-ec2-service-discovery-ea6dbda50b72?source=rss-10cb688cf928------2</link>
            <guid isPermaLink="false">https://medium.com/p/ea6dbda50b72</guid>
            <category><![CDATA[aws-ec2]]></category>
            <category><![CDATA[prometheus]]></category>
            <category><![CDATA[service-discovery]]></category>
            <dc:creator><![CDATA[Deependra Saraswat]]></dc:creator>
            <pubDate>Tue, 05 Jan 2021 14:59:40 GMT</pubDate>
            <atom:updated>2021-01-05T14:59:40.791Z</atom:updated>
            <content:encoded><![CDATA[<h3>Prometheus — EC2 Service Discovery</h3><p>Manually Managing list of targets in prometheus yaml file can be irritating over a period of time and if you have a huge infra with <em>n</em> number of micro-services and servers which come and go on routine basis, then manually managing the static list of targets in <em>prometheus</em>.yaml can be a big pain.</p><p>In this blog, I will walk you through as to how you can dynamically manage all these targets using the EC2 service discovery mechanism in prometheus which will ease things to a great extent.</p><p>For the demonstration purpose, I spun up 2 EC2 instances with Netdata installed<em>(Netdata is another monitoring tool/agent that generates prometheus metrics and works similar to node exporter. The added advantage is that it has got support for a wide range of plugins to monitor multiple other applications like elasticsearch, redis, kafka etc. It’s in the league of tools like telegraf)</em>. And an EC2 instance onto which my prometheus would run.</p><p>Next I setup an IAM role with the AmazonEC2ReadOnlyAccess policy. This policy gives the role permission to list EC2 instances.</p><p>Attach the IAM role we created earlier with the prometheus EC2 instance. This is where all the magic happens. Read on to discover how.</p><p>Below is the prometheus configuration detailing EC2 service discovery block:</p><pre>- job_name: &#39;dummy&#39;<br>    metrics_path: &#39;/api/v1/allmetrics?format=prometheus&#39;<br>    ec2_sd_configs:<br>    - region: ap-south-1<br>      port: 19999<br>    relabel_configs:<br>    - source_labels: [__meta_ec2_tag_Name]<br>      regex: myserver-.*<br>      action: keep<br>    - source_labels: [__meta_ec2_tag_Name,__meta_ec2_availability_zone]<br>      target_label: instance</pre><blockquote><strong>metric_path: </strong>This is the http api endpoint over which netdata will expose prometheus metrics for the prometheus server to scrape. By default, its /metrics.</blockquote><blockquote><strong>ec2_sd_configs:</strong> EC2 SD configurations allow retrieving scrape targets from AWS EC2 instances.</blockquote><blockquote><strong>region</strong>: Region where you AWS VPC is located.</blockquote><blockquote><strong>port:</strong> The port to scrape metrics from on discovered EC2 instances.</blockquote><blockquote><strong>__meta_ec2_tag_Name:</strong> This is metadata information retrieved from the EC2 instance by prometheus. You can use relabelling to define regex for targeting instances with specific name and tag. Here we are targeting all instances with myserver tag name.</blockquote><p>Reload you prometheus.</p><pre>curl -X POST http://localhost:9090/-/reload</pre><p>Let’s now talk about the magic. The prometheus server would automatically discover this IAM role from the instance it is running on and use this as default to discover our resources in the VPC. No headache of maintaining and rotating access key ids and secret keys or the role arns in the prometheus yaml.</p><p>Go to your prometheus console &gt; status &gt; targets and all you discovered EC2 instances are registered and being scraped for metrics. Plus any new instance with specified tagging gets discovered on its own now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9GCrdWUzpgghhTE8.png" /></figure><h4>Problem Solved!!!!</h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea6dbda50b72" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>