<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Bharat Singh on Medium]]></title>
        <description><![CDATA[Stories by Bharat Singh on Medium]]></description>
        <link>https://medium.com/@bharat-singh-06?source=rss-52932ec5027d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*JfofcOUJNiO99zPbT4Snxg@2x.jpeg</url>
            <title>Stories by Bharat Singh on Medium</title>
            <link>https://medium.com/@bharat-singh-06?source=rss-52932ec5027d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 04:06:40 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bharat-singh-06/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[S3 Vector Bucket as a Vector Database for Agentic AI Systems]]></title>
            <link>https://bharat-singh-06.medium.com/s3-vector-bucket-as-a-vector-database-for-agentic-ai-systems-5033cfc52c3a?source=rss-52932ec5027d------2</link>
            <guid isPermaLink="false">https://medium.com/p/5033cfc52c3a</guid>
            <category><![CDATA[vector-embeddings]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[amazon-s3-vectors]]></category>
            <category><![CDATA[amazon-bedrock]]></category>
            <category><![CDATA[vector-database]]></category>
            <dc:creator><![CDATA[Bharat Singh]]></dc:creator>
            <pubDate>Wed, 11 Nov 2026 13:58:35 GMT</pubDate>
            <atom:updated>2026-11-11T13:58:35.085Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Why Do Agentic AI Systems Need a Vector Database?</blockquote><p>One of the major challenges in Generative AI and Agentic AI systems is hallucination, where Large Language Models (LLMs) generate incorrect or unsupported information. One approach to reducing hallucinations is Retrieval-Augmented Generation (RAG).</p><p>RAG is an architectural pattern that enables LLMs to retrieve relevant information from external knowledge sources before generating a response.</p><p>However, RAG is an architectural concept, not a database or storage technology. To implement RAG, we need supporting components such as document storage, chunking, embedding models and vector databases.</p><p>Traditionally, developers have used vector databases such as Pinecone, Weaviate, Chroma and OpenSearch to store and retrieve embeddings.</p><p><em>AWS introduced Amazon S3 Vectors to provide a managed, serverless and cost-efficient solution for storing, indexing and querying vector embeddings.</em></p><p>An important distinction: S3 Vectors is not a complete RAG solution. It provides the vector storage and similarity-search layer required to implement RAG applications.</p><p>What is Amazon S3 Vectors?</p><p>Amazon S3 Vectors is a vector storage capability within Amazon S3, designed to store, index and query vector embeddings.</p><p>It provides a managed vector storage layer that can be integrated into RAG-based applications and Agentic AI systems.</p><blockquote><em>Key Features</em><br>Serverless vector storage.<br>Managed vector indexing and similarity search.<br>Cost-efficient storage for large vector datasets.<br>Semantic search capabilities.<br>Metadata filtering.</blockquote><p>Integration with AWS services and third-party Agentic AI frameworks.</p><p>Platform independence: S3 Vectors is not restricted to Amazon Bedrock. You can integrate it with other Agentic AI frameworks and LLM providers, including LangChain, LangGraph and custom Python applications, using AWS APIs and SDKs.</p><p><strong>Vector Bucket: <em>This is a kind of S3 that stores a vector index.</em></strong></p><p>Vector Index: <em>A vector index stores vector embeddings and associated metadata. It supports similarity searches to identify vectors that are semantically close to a query vector</em></p><p>Vector embeddings: Converting text, images, or video into a numeric format called an embedding.</p><p>e. g.“AWS provides cloud computing services.” [1,2,2,3,4334311,998999,…..097777]</p><p>What are embeddings:</p><p>Machine learning models generally operate on numerical representations rather than directly understanding raw text. Therefore, we convert text into numerical representations called embeddings.<br>Previously, traditional NLP applications commonly used techniques such as:</p><p>Bag of Words (BoW).<br>TF-IDF. Word2Vec.<br>GloVe.</p><p>Modern GenAI applications commonly use transformer-based embedding models to generate contextual semantic representations</p><p>Similarity search: we convert text to an embedding vector and check the distance between vectors. Minimum-distance vectors are considered similar.</p><p>Cosine similarity<br>Euclidean distance</p><p><strong><em>Implementing S3 Vectors using Python and Boto3</em></strong></p><p>Vector Bucket <br>Vector Index <br>Vectors: Add, Query, Retrieve, Delete</p><p>Create an S3 Vector bucket. Log in to the AWS Account and search for the S3 Vector bucket</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wGHogWQOj9I82T3Ojk8J4Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MRxuBzLQWltxd8uN8fwCoA.png" /></figure><p>Create a Python project and install dependencies</p><pre>pip install boto3 pydantic python-dotenvs langchain-openai langchain-text-splitters</pre><p>S3 Vector bucket Boto3 client:<em> Before you get</em></p><pre>import os<br>import boto3<br>from dotenv import load_dotenv<br>from langchain_openai import OpenAIEmbeddings<br><br>load_dotenv(override=True)<br><br>SEP = &quot;=&quot; * 20<br><br>AWS_REGION = os.getenv(&quot;AWS_REGION&quot;, &quot;us-east-1&quot;)<br>ACCESS_KEY = os.getenv(&quot;AWS_ACCESS_KEY_ID&quot;)<br>SECRET_KEY = os.getenv(&quot;AWS_SECRET_ACCESS_KEY&quot;)<br><br>if not ACCESS_KEY or not SECRET_KEY:<br>    raise ValueError(&quot;AWS credentials are missing&quot;)<br><br>session = boto3.Session(<br>    aws_access_key_id=ACCESS_KEY,<br>    aws_secret_access_key=SECRET_KEY,<br>    region_name=AWS_REGION<br>)<br><br>def s3vectors_client():<br>    print(SEP + &quot; S3-VECTOR-CLIENT &quot; + SEP)<br>    return session.client(&quot;s3vectors&quot;)<br><br>def bedrock_client():<br>    print(SEP + &quot; BEDROCK-CLIENT &quot; + SEP)<br>    return session.client(&quot;bedrock-runtime&quot;)<br><br>def embedd_client():<br>    print(SEP + &quot; OPENAI-EMBEDDING &quot; + SEP)<br>    return OpenAIEmbeddings(model=&quot;text-embedding-3-small&quot;, dimensions=500)</pre><p>Defined Model: <em>We can use Pydantic to define the structure of a vector.</em></p><p>data is a float32 array dictionary. The array dimension should match the batch with the S3 vector index.</p><pre>from pydantic import BaseModel, Field<br><br>class S3Vectors(BaseModel):<br>    key: str<br>    data: dict = Field(default_factory=dict) #Numerical embedding representation<br>    metadata: dict = Field(default_factory=dict) #Additional information associated with the vector</pre><p>Vectors: Add, Query, Retrieve, Delete.</p><p>Embedding</p><pre>from dotenv import load_dotenv<br><br>from src.clients import embedd_client<br>from src.model import S3Vectors<br><br>load_dotenv(override=True)<br><br><br>def generate_embeddings(docs):<br>    print(&quot;Generating embeddings...&quot;)<br><br>    if not docs:<br>        raise ValueError(&quot;No documents found&quot;)<br><br>    embedding_client = embedd_client()<br>    embeddings = embedding_client.embed_documents(docs)<br><br>    data_vectors = []<br>    for i, (doc, embedding) in enumerate(zip(docs, embeddings)):<br>        vector = S3Vectors(<br>            key=f&quot;bank-report-{i}&quot;,<br>            data={&quot;float32&quot;: embedding},<br>            metadata={<br>                &quot;source&quot;: f&quot;finance-report-{i}&quot;,<br>                &quot;data_type&quot;: &quot;bank&quot;,<br>                &quot;content_preview&quot;: doc[:100]<br>            }<br>        )<br>        data_vectors.append(vector)<br><br>    print(f&quot;Generated vectors: {len(data_vectors)}&quot;)<br><br>    return data_vectors</pre><p>S3VectorBucket Operations:</p><pre>import json<br>import os<br>from dotenv import load_dotenv<br>from src.clients import SEP, s3vectors_client, embedd_client<br>from src.doc_loader import load_docs<br>from src.embedding import generate_embeddings<br><br>load_dotenv(override=True)<br><br>def add_embedding():<br>    print(SEP * 2)<br>    print(&quot;Saving vectors to S3 Vectors...&quot;)<br>    docs = load_docs()<br>    vectors = generate_embeddings(docs)<br>    client_obj = s3vectors_client()<br><br>    payload = [vector.model_dump() for vector in vectors  ]<br><br>    response = client_obj.put_vectors(<br>        vectorBucketName=os.getenv(&quot;S3VECTORS_BUCKET&quot;),<br>        indexName=os.getenv(&quot;S3VECTORS_INDEX&quot;),<br>        vectors=payload<br>    )<br><br>    print(&quot;Vector ingestion completed successfully!&quot;)<br><br>    return response<br><br>def query_embedding(query):<br>    print(SEP * 2)<br>    print(&quot;Fetch vector data.....&quot;)<br><br>    embedding_client = embedd_client()<br>    query_vector = embedding_client.embed_query(query.strip())<br><br>    client_obj = s3vectors_client()<br><br>    response = client_obj.query_vectors(<br>        vectorBucketName=os.getenv(&quot;S3VECTORS_BUCKET&quot;),<br>        indexName=os.getenv(&quot;S3VECTORS_INDEX&quot;),<br>        queryVector={&quot;float32&quot;: query_vector},<br>        topK=3,<br>        returnDistance=True,<br>        returnMetadata=True<br>    )<br><br>    print(&quot;Vector search completed successfully!&quot;)<br>    print(json.dumps(response[&quot;vectors&quot;], indent=2))<br><br>    return response</pre><p>Amazon S3 Vectors provides a serverless, scalable and cost-efficient vector storage layer for RAG-based applications</p><p>Key points to remember<br><em>RAG is an architectural pattern.<br>Embedding is the technique to convert content to a numeric representation.<br>Chunking divides a big document into small pieces.<br>Vector buckets provide vector storage.<br>A vector index organises embeddings and enables similarity search<br>Agentic AI frameworks orchestrate retrieval, reasoning and tool execution.<br>LLMs generate responses using retrieved context.</em></p><p><a href="https://github.com/everythingisdata1/s3-vector-bucket-rag">I have also updated the same in my GitHub project</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5033cfc52c3a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Amazon Bedrock Python Tutorial for Beginners (2026)]]></title>
            <link>https://bharat-singh-06.medium.com/getting-started-with-amazon-bedrock-85ac3423cfe8?source=rss-52932ec5027d------2</link>
            <guid isPermaLink="false">https://medium.com/p/85ac3423cfe8</guid>
            <category><![CDATA[amazon-bedrock]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <dc:creator><![CDATA[Bharat Singh]]></dc:creator>
            <pubDate>Sun, 07 Jun 2026 06:50:12 GMT</pubDate>
            <atom:updated>2026-06-28T10:54:00.727Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong><em>Invoke Your First Foundation Model Using Python</em></strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J6b-4OHuKTtnH9_o5hwH-w.png" /></figure><p>Welcome to the<strong> #1</strong> tutorial in my Amazon Bedrock series on building Generative AI applications with AWS.</p><p>Generative AI is rapidly transforming how organisations build products, automate business processes, and deliver innovative customer experiences. As adoption continues to grow across industries, developers need practical and scalable ways to integrate AI into their applications.</p><p>Amazon Bedrock is a fully managed AWS service that enables developers to build and deploy Generative AI applications using foundation models from leading AI providers without managing the underlying infrastructure. With Amazon Bedrock, you can access models from multiple providers through a unified API and focus on building business value rather than managing complex AI infrastructure.</p><p>In this article, we will build our first Generative AI application using Amazon Bedrock and Python. By the end of this tutorial, you will be able to successfully invoke a foundation model, send prompts, and generate AI-powered responses directly from your local machine.</p><h3>Prerequisites</h3><p>Before getting started, make sure you have:</p><ul><li>An AWS account (<strong>You can use a Free Tier</strong> )</li><li>An IAM user with programmatic access (avoid using the root account)</li><li>Basic knowledge of Python</li><li>Python 3.10 or later is installed on your machine</li><li>AWS CLI configured locally</li></ul><h3>What You Will Learn</h3><p>In this article, we will cover:</p><ul><li>Identify Amazon Bedrock model IDs</li><li>Configure AWS credentials for Bedrock access</li><li>Integrate Amazon Bedrock with Python using Boto3</li><li>Invoke an Amazon Nova model from your local environment</li><li>Send prompts and generate AI responses</li></ul><p><strong>Let’s get started</strong></p><p>Amazon Bedrock provides access to both Amazon foundation models and models from leading third-party providers. This gives developers the flexibility to choose the most appropriate model based on their application’s requirements, performance expectations, and budget.</p><p>For this tutorial, we will use <strong>Amazon Nova</strong> models. They are cost-effective, easy to get started with, and well-suited for building Generative AI applications on AWS.</p><p><strong><em>Add Amazon Bedrock Permissions</em></strong></p><p>Before accessing Bedrock locally, add the required permission to the IAM user. For production, we should follow the least privilege principle.</p><p><strong><em>Configure the AWS account locally</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/1*7T0iLxfyW_czcdiNkGeS1Q.png" /></figure><p>To verify, you can execute the command below</p><pre>aws sts get-caller-identity</pre><p><strong><em>Python Boto3 Client for Amazon Bedrock</em></strong></p><h4>Verify the AWS Region:</h4><p><em>Some Amazon Bedrock models are not available in every AWS Region.</em></p><pre>import logging<br>import boto3<br><br>logging.basicConfig(level=logging.INFO,<br>                    format=&quot;%(asctime)s %(levelname)s %(name)s - %(message)s&quot;,<br>                    force=True)<br><br>log = logging.getLogger(&quot;BedrockAgent&quot;)<br>log.setLevel(logging.INFO)<br><br>class ChatService:<br><br>    def __init__(self):<br>        log.info(&quot;Initializing ChatService&quot;)<br>        self.modelId = &quot;us.amazon.nova-lite-v1:0&quot;<br>        self.bedrock_client = boto3.client(&quot;bedrock-runtime&quot;,<br>                                           region_name=&quot;us-east-1&quot;)<br><br>    def chat(self):<br>        log.info(f&quot;[INVOKED]::  Amazon Bedrock&quot;)<br>        response = self.bedrock_client.converse(<br>            modelId=self.modelId,<br>            messages=[{&quot;role&quot;: &quot;user&quot;,<br>                       &quot;content&quot;: [<br>                           {&quot;text&quot;: &quot;Explain python? - response should not exceed more then 10 word&quot;}<br>                       ]}])<br><br>        answer = response[&quot;output&quot;][&quot;message&quot;][&quot;content&quot;][0][&quot;text&quot;]<br>        log.info(&quot;Model response: %s&quot;, answer)<br><br>        return answer<br><br>if __name__ == &#39;__main__&#39;:<br>    ChatService().chat()</pre><p>Model Response</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*QgrDpBGkHtB6RED-YBCUFw.png" /></figure><p><strong><em>Thanks for Reading</em></strong></p><p>Congratulations! We have successfully built our first Generative AI application using Amazon Bedrock and Python Boto3.</p><p><strong><em>What’s Next?</em></strong></p><p>In the next article, we will take this application a step further and deploy it to AWS using serverless services, including:</p><ul><li>AWS Lambda</li><li>Amazon API Gateway</li><li>Amazon Bedrock</li></ul><blockquote>Reference:</blockquote><p><a href="https://docs.aws.amazon.com/nova/latest/userguide/what-is-nova.html">What is Amazon Nova?</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=85ac3423cfe8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How To Pass AWS Certified AI Practitioner in 2026 (Using AI)]]></title>
            <link>https://bharat-singh-06.medium.com/how-to-pass-aws-certified-ai-practitioner-in-2026-using-ai-for-smarter-preparation-6f22723704bf?source=rss-52932ec5027d------2</link>
            <guid isPermaLink="false">https://medium.com/p/6f22723704bf</guid>
            <category><![CDATA[aws-certification]]></category>
            <category><![CDATA[aws-bedrock]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[aws-sagemaker]]></category>
            <dc:creator><![CDATA[Bharat Singh]]></dc:creator>
            <pubDate>Sat, 23 May 2026 08:46:55 GMT</pubDate>
            <atom:updated>2026-06-28T10:47:46.408Z</atom:updated>
            <content:encoded><![CDATA[<h3><em>AWS Certified AI Practitioner (AIF-C01) Study Guide 2026: How I Passed in 8 Weeks Using AI</em></h3><p>This is my<strong> #4 blog </strong>on Medium. I passed the AWS Certified AI Practitioner (<strong>AIF-C01</strong>) on May 26 and wanted to share my preparation strategy, learning journey and how I used <strong>AI </strong>in my<strong> preparation.</strong></p><p>If you are planning to take this exam, I hope this article will help you in planning and preparing for your certification journey.</p><p><strong>Preparation Time:</strong> 8 weeks <br><strong>Resources Used:</strong> Udemy + Practice Tests + AI + AWS Documentation</p><h3><strong>Why I Attended This Exam</strong></h3><p>AWS is my all-time favourite cloud platform, and industries are rapidly adopting AI. I wanted to strengthen my understanding of both the AI and Cloud perspectives.</p><p>This exam helped me:</p><ul><li><strong><em>Build a strong understanding of AI and ML concepts.</em></strong></li><li><strong><em>Improve my ability to design scalable AI solutions.</em></strong></li><li><strong><em>Understand the concept of Generative AI and build a Generative AI solution on AWS Cloud</em></strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*gvDFQCzjadrR6ZgTMI5Zvw.png" /></figure><p><strong>AWS Certified AI Practitioner Exam Domains</strong></p><p><strong>Fundamentals of AI &amp;ML (20%)</strong> learning ML, AI concepts and algorithms ( Supervised, unsupervised and clustering).</p><p><strong><em>Fundamentals of Generative AI(28%):</em></strong> Explore Generative AI concepts(Chucking, Embedding, vectordb).</p><p><strong><em>Applications of Foundation Models (28%):</em></strong> Model selection, understanding Business use case.</p><p><strong><em>Guidelines for Responsible AI (14%):</em></strong> Exploring GenAI Responsible principles (Fairness, Security and Privacy, model explainability).</p><p><strong><em>Security, Compliance, and Governance for AI Solutions(14%):</em></strong> IAM fundamentals, SageMaker/Bedrock governance, compliance.</p><h3><strong>My Study Approach</strong></h3><p><strong>Week 1–5: </strong>Learn GenAI Concepts and AWS services.</p><ul><li><strong><em>Registered for an AWS Free Tier account.</em></strong></li><li><strong><em>Completed Stephane Mearek’s Udemy course.</em></strong></li><li><strong><em>Took notes for each domain.</em></strong></li><li><strong><em>Instead of memorising services, focus on understanding services and their use case.</em></strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8_nEpwrP6N9RqIMW-8SarA.png" /></figure><p><strong>Week 5 -8:</strong> Practices Exams</p><ul><li><strong><em>Attended all 4 sets of Stephane Mearek’s practice exams.</em></strong></li><li><strong><em>Re-reviewed all the failed ones after every attempt.</em></strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GZYtF_V-5OIBlh_AUleiKw.png" /></figure><h3>How I Used AI for Exam Preparation</h3><p>I have used AI to clarify GenAI concepts and explain its use case around AWS AI Services. Some sample prompts.</p><blockquote>“Explain AWS Textract vs Polly with examples”<br>“Give me scenario-based AIF-C01 questions around Bedrock”<br>“AWs audit manager vs AWs artefacts vs AWs trusted advisor”</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/664/1*60nRYzv9MDh-vGkzi1W-dg.png" /></figure><p><strong><em>Final Tips:</em></strong></p><blockquote><em>Understand use case around AWS services- don’t memorise.</em></blockquote><blockquote><em>Avoid learning completely new concepts on the last day of the exam.</em></blockquote><p>If you are preparing for the AWS AI Practitioner exam, I hope this guide helps. Best of luck with your certification journey.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*euL5gafeh3nQGZ7J8s_rBw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6f22723704bf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Common Commands]]></title>
            <link>https://bharat-singh-06.medium.com/kubernetes-common-commands-462dab37af3c?source=rss-52932ec5027d------2</link>
            <guid isPermaLink="false">https://medium.com/p/462dab37af3c</guid>
            <category><![CDATA[iac]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[kubectl]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Bharat Singh]]></dc:creator>
            <pubDate>Sun, 07 Jan 2024 02:15:20 GMT</pubDate>
            <atom:updated>2024-01-07T02:23:17.545Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*gz_DJ337IlG2ztLCkzkNew.png" /><figcaption>kubectl common commands</figcaption></figure><p>For the last 3,4 months, I have been exploring Kubernetes. I come across some common commands, which I will share. I feel they are useful for the CKA certification exam.</p><p>Pods Creation with — dry-run flag</p><pre>kubectl run –image=redis redis --dry-run=client -o yaml</pre><p>Create Deployment</p><pre>kubectl create deployment redis --image=redis --dry-run=client -o yamlsh</pre><p>Edit Already Created Deployment</p><pre>kubectl edit deployment redis    </pre><p>Edit Image in Already Created Deployment</p><pre>kubectl set image deployment redis --image=redis:1.8</pre><p>Create Kubernetes Objects with Yaml</p><pre>kubectl apply -f redis-definitaiton.yaml<br>kubeclt create -f redis-definitation.yaml<br><br>Update the running deployments <br><br>Kubectl replace -f redis-definitation.yaml</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=462dab37af3c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Access S3 Bucket through STS (Security Token Service)]]></title>
            <link>https://bharat-singh-06.medium.com/access-s3-bucket-through-sts-security-token-service-f6c613b5db5f?source=rss-52932ec5027d------2</link>
            <guid isPermaLink="false">https://medium.com/p/f6c613b5db5f</guid>
            <category><![CDATA[aws-iam-role]]></category>
            <category><![CDATA[aws-sts]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-s3]]></category>
            <category><![CDATA[iam-roles]]></category>
            <dc:creator><![CDATA[Bharat Singh]]></dc:creator>
            <pubDate>Sun, 30 Jul 2023 07:04:11 GMT</pubDate>
            <atom:updated>2023-07-30T07:10:26.284Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/996/1*toWnm0MxVD6Qel0fYmkfIg.png" /><figcaption>Security Token Service (Temporary Access Service)</figcaption></figure><p>STS is an AWS service, which provides temporary credentials for accessing the AWS resources to the user.</p><p>Security Token Service is time bound service. It works on least privilege policies. After the expiry of the secure token, the user will not be able to access the AWS resources. If the user wants to access AWS resources after token expiry, the user needs to generate a token again.</p><p>STS expiry is configurable from 15 minutes to 36 hours.</p><p>STS is a Global service. (<a href="https://sts.amazonaws.com/">https://sts.amazonaws.com</a>). But it also supports the region endpoint.</p><p><strong>How STS works.</strong></p><p>Login users have to invoke AWS STS API for accruing the access secret key, a secure token. These are temporary credentials and expire after some time. So, the user can not use the expired token for accessing the AWS Service/resources.</p><p><strong>Access S3 with STS</strong></p><p><strong>IAM User(sts_user)</strong></p><p>Create an AWS IAM user and also generate an Access key for user authentication. Also, We need user ARN for trust relationship in STS policy. <strong>arn:aws:iam::&lt;AccountId&gt;:user/sts_user</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T4gXdKg2f3qQy3-TIS-zew.png" /><figcaption>sts_user</figcaption></figure><p><strong>AWS S3 Bucket</strong></p><p>Create an AWS S3 bucket that you want to read/access through the STS token. <strong>Stsdemobuckeet</strong></p><p><strong>S3 Read Only IAM Policy</strong></p><p>Create an AWS S3 Read-only policy. (<strong>sts_s3_read_only_policy</strong>). Which we will assign to AWS IAM Role.</p><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;s3:Get*&quot;,<br>                &quot;s3:List*&quot;,<br>                &quot;s3-object-lambda:Get*&quot;,<br>                &quot;s3-object-lambda:List*&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;<br>        }<br>    ]<br>}</pre><p><strong>AWS IAM Role with S3 ReadOnly Access</strong></p><p>Create an AWS IAM role, which will have aws s3 read-only access with the previous policy. From IAM console select roles and create an IAM role with a Trusted entity as AWS Account (it supports other accounts/ third-party to perform actions in your account).(<strong>Sts_s3_access_role</strong>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ld6SQBzgTH2hV3l3fDlFow.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dlTfzAWFreKOP30j-Ku0ww.png" /><figcaption>IAM Role with s3 read-only policy</figcaption></figure><p>Edit Trust Relationship and add user ARN to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HHxmEzzcseZNFT-IbX28_Q.png" /><figcaption>Add user ARN as a Principal in the trust relationship</figcaption></figure><p><strong>STS Policy</strong></p><p>Create an STS policy from the AWS IAM policy console and assign it to the user.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T1nR5RBunqvfOU9ZmpASqw.png" /><figcaption>STS policy</figcaption></figure><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Sid&quot;: &quot;VisualEditor0&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;sts:GetSessionToken&quot;,<br>                &quot;sts:DecodeAuthorizationMessage&quot;,<br>                &quot;sts:GetAccessKeyInfo&quot;,<br>                &quot;sts:GetCallerIdentity&quot;,<br>                &quot;sts:GetServiceBearerToken&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;<br>        },<br>        {<br>            &quot;Sid&quot;: &quot;VisualEditor1&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: &quot;sts:*&quot;,<br>            &quot;Resource&quot;: &quot;arn:aws:iam::&lt;ACCOUNT_ID&gt;:user/sts_user&quot;<br>        }<br>    ]<br>}</pre><p><strong>Attach STS Policy to the STS user</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eH70XmW7HVAvt8_gnW99Nw.png" /><figcaption>attach STS Policy to user</figcaption></figure><p><strong>Login with STS and access S3.</strong></p><p>With the help of AWS CLI, we can now access S3</p><p>a. Configure user</p><p>b. Execute STS command</p><p>c. Set STS token in the environment</p><p>d. Execute the AWS s3 command</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8qu9giodjgYcKKULyc_MEw.png" /><figcaption>aws sts token access and use in s3 access.</figcaption></figure><p><strong>Next:</strong> USE STS with Java or Python</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f6c613b5db5f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create VPC Gateway Endpoint Terraform]]></title>
            <link>https://bharat-singh-06.medium.com/create-vpc-gateway-endpoint-terraform-75dbe43d1059?source=rss-52932ec5027d------2</link>
            <guid isPermaLink="false">https://medium.com/p/75dbe43d1059</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[vpc-endpoint]]></category>
            <category><![CDATA[aws-vpc-endpoints]]></category>
            <category><![CDATA[aws-vpc-terraform]]></category>
            <dc:creator><![CDATA[Bharat Singh]]></dc:creator>
            <pubDate>Thu, 29 Jun 2023 08:48:48 GMT</pubDate>
            <atom:updated>2023-06-29T08:48:48.700Z</atom:updated>
            <content:encoded><![CDATA[<p>VPC Gateway Endpoint is a service, which is provided by AWS for privately accessing the aws services. It secure and reliable service. As the request is travelling within the AWS network not over over the internet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*ULH6LPEGrq682LV8zyXoxA.png" /><figcaption>VPC Gateway Endpoint</figcaption></figure><ul><li><strong>VPC: </strong>create the VPC</li></ul><pre>resource &quot;aws_vpc&quot; &quot;network_vpc&quot; {<br>  cidr_block           = &quot;10.10.0.0/16&quot;<br>  enable_dns_hostnames = true<br>  enable_dns_support   = true<br>  tags = {<br>    &quot;Name&quot; = &quot;AWS_NETWORKING&quot;<br>  }<br>}</pre><ul><li><strong>Public Subnet</strong></li></ul><pre>resource &quot;aws_subnet&quot; &quot;public_subnet&quot; {<br>  vpc_id                  = aws_vpc.network_vpc.id<br>  cidr_block              = &quot;10.10.0.0/24&quot;<br>  map_public_ip_on_launch = true<br>  availability_zone       = &quot;ap-south-1a&quot;<br><br>  tags = {<br>    &quot;Name&quot; = &quot;PublicSubnet_1&quot;<br>    &quot;VPC&quot;  = &quot;AWS_NETWORKING&quot;<br>  }<br>}  </pre><ul><li>Route Table for Public Subnet with Association</li></ul><pre>resource &quot;aws_route_table&quot; &quot;RT_PUBLIC&quot; {<br>  vpc_id = aws_vpc.network_vpc.id<br>  route {<br>    cidr_block = &quot;0.0.0.0/0&quot;<br>    gateway_id = aws_internet_gateway.network_vpc_igw.id<br>  }<br>  tags = {<br>    &quot;Name&quot; = &quot;RT_PUBLIC&quot;<br>    &quot;VPC&quot;  = &quot;AWS_NETWORKING&quot;<br>  }<br>}<br><br>resource &quot;aws_route_table_association&quot; &quot;subnet_route_table_association&quot; {<br>  route_table_id = aws_route_table.RT_PUBLIC.id<br>  subnet_id      = aws_subnet.public_subnet.id<br>}</pre><ul><li>Internet Gateway</li></ul><pre>resource &quot;aws_internet_gateway&quot; &quot;network_vpc_igw&quot; {<br>  tags = {<br>    &quot;Name&quot; = &quot;AWS_NETWORKING&quot;<br>  }<br>}<br>resource &quot;aws_internet_gateway_attachment&quot; &quot;igw_vpc_attachement&quot; {<br>  internet_gateway_id = aws_internet_gateway.network_vpc_igw.id<br>  vpc_id              = aws_vpc.network_vpc.id<br>}</pre><ul><li>Private subnet</li></ul><pre>resource &quot;aws_subnet&quot; &quot;private_subnet&quot; {<br>  cidr_block        = &quot;10.10.1.0/24&quot;<br>  vpc_id            = aws_vpc.network_vpc.id<br>  availability_zone = &quot;ap-south-1a&quot;<br>  tags = {<br>    &quot;Name&quot; = &quot;PrivateSubnet_1&quot;<br>    &quot;VPC&quot;  = &quot;AWS_NETWORKING&quot;<br>  }<br>}</pre><ul><li>Private Route Table and Private Subnet Association.</li></ul><pre>resource &quot;aws_route_table&quot; &quot;RT_PRIVATE&quot; {<br>  vpc_id = aws_vpc.network_vpc.id <br>  tags = {<br>    &quot;Name&quot; = &quot;RT_PRIVATE&quot;<br>    &quot;VPC&quot;  = &quot;AWS_NETWORKING&quot;<br>  }<br>}<br><br>resource &quot;aws_route_table_association&quot; &quot;privateRouteTable1_association&quot; {<br>  subnet_id      = aws_subnet.private_subnet.id<br>  route_table_id = aws_route_table.RT_PRIVATE.id<br>  depends_on     = [aws_subnet.private_subnet]<br>}</pre><ul><li>EC2 Instance in Private Subnet</li></ul><pre>resource &quot;aws_instance&quot; &quot;private_ec2&quot; {<br>  ami             = var.aws_ami_id<br>  instance_type   = &quot;t2.micro&quot;<br>  key_name        = var.tf_key_name<br>  subnet_id       = var.private_subnet_id<br>  security_groups = [var.private_sg_id]<br>  iam_instance_profile = aws_iam_instance_profile.instance_profile.name<br>  tags = {<br>    &quot;Name&quot; = &quot;PRIVATE_EC2_HOST&quot;<br>    &quot;VPC&quot;  = &quot;AWS_NETWORKING_EXAM_IGW&quot;<br>  }<br>}<br><br>resource &quot;aws_iam_instance_profile&quot; &quot;instance_profile&quot; {<br>  name = &quot;PRIVATE_INSTNACE_INSTANCE_PROFILE&quot;<br>  tags = {<br>    &quot;Name&quot; : &quot;PRIVATE_INSTNACE_INSTANCE_PROFILE&quot;<br>  }<br>  role = aws_iam_role.s3_iam_role.name<br>}<br><br>resource &quot;aws_iam_role&quot; &quot;s3_iam_role&quot; {<br>  assume_role_policy = data.aws_iam_policy_document.assume_doc.json<br>}<br><br>data &quot;aws_iam_policy_document&quot; &quot;assume_doc&quot; {<br>  statement {<br>    actions = [&quot;sts:AssumeRole&quot;]<br>    effect  = &quot;Allow&quot;<br>    principals {<br>      type        = &quot;Service&quot;<br>      identifiers = [&quot;ec2.amazonaws.com&quot;]<br>    }<br>  }<br>}<br><br>resource &quot;aws_iam_role_policy&quot; &quot;role_policies&quot; {<br>  role   = aws_iam_role.s3_iam_role.id<br>  policy = data.aws_iam_policy_document.iam_policy_doc.json<br>}<br><br>data &quot;aws_iam_policy_document&quot; &quot;iam_policy_doc&quot; {<br>  statement {<br>    actions   = [&quot;s3:*&quot;]<br>    effect    = &quot;Allow&quot;<br>    resources = [&quot;*&quot;]<br>  }<br>}</pre><ul><li>Bastion Host in Public Subnet with EIP</li></ul><pre>resource &quot;aws_instance&quot; &quot;BASTION_HOST&quot; {<br>  ami                         = var.aws_ami_id<br>  instance_type               = &quot;t2.micro&quot;<br>  key_name                    = var.tf_key_name<br>  subnet_id                   = var.public_subnet_id<br>  security_groups             = [var.public_sg_id]<br>  associate_public_ip_address = true<br>  tags = {<br>    &quot;Name&quot; = &quot;BASTAIN_HOST&quot;<br>    &quot;VPC&quot;  = &quot;AWS_NETWORKING&quot;<br>  }<br>}<br># EIP<br>resource &quot;aws_eip&quot; &quot;BASTAIN_HOST_IP&quot; {}<br><br><br>resource &quot;aws_eip_association&quot; &quot;BASTION_HOST_EIP_MAPPING&quot; {<br>  allocation_id = aws_eip.BASTAIN_HOST_IP.id<br>  instance_id   = aws_instance.BASTION_HOST.id<br>}</pre><ul><li><strong>Create a VPC Gateway Endpoint for accessing the AWS Services privately.</strong></li></ul><pre>resource &quot;aws_vpc_endpoint&quot; &quot;gateway_endpoint&quot; {<br>  vpc_id            = aws_vpc.network_vpc.id<br>  service_name      = &quot;com.amazonaws.ap-south-1.s3&quot;<br>  route_table_ids   = [aws_route_table.RT_PRIVATE.id] <br>  tags = {<br>    &quot;Name&quot; = &quot;GATEAGY_INTERFAC&quot;<br>  }<br>}</pre><ul><li>Security Groups for Private and Public Subnet</li><li>Terraform Apply</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GHsf8UxM6H8OH1P2k1WB8w.png" /></figure><ul><li>Access Private EC2 Instance Putty</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EFtOtpu086gaYe6U3PNRXg.png" /></figure><ul><li>Test Result</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*3uXKwHiAr7TJFHMFalD2cQ.png" /><figcaption>Access S3 Buckets over the VPC Gateway EndPoint</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75dbe43d1059" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Terraform Rename Module Without Destroying]]></title>
            <link>https://bharat-singh-06.medium.com/terraform-rename-exisiting-resoruce-dcf7416677ee?source=rss-52932ec5027d------2</link>
            <guid isPermaLink="false">https://medium.com/p/dcf7416677ee</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[terraform-modules]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[terraform-command]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <dc:creator><![CDATA[Bharat Singh]]></dc:creator>
            <pubDate>Thu, 15 Jun 2023 11:58:01 GMT</pubDate>
            <atom:updated>2024-01-07T02:28:29.957Z</atom:updated>
            <content:encoded><![CDATA[<p>Sometimes, we need to rename the existing resources.</p><p>Maybe the name of the resource was not correct or mistakenly we created it incorrectly. In that case, we need to change the resource name. like I was creating the resources for the “<strong>Bastion ” </strong>host but mistakenly gave the name <strong>“public_ec2”.</strong></p><p>Renaming the name of AWS resources, we can do it two ways.</p><p>1- Update the terraform resource name in the configuration and execute the apply command. <strong>(Apply command will destroy the resource and recreate it.)</strong></p><pre>terraform apply -auto-approve</pre><p>2- Update the terraform resource name in the configuration and use the rename command. <strong>(The rename (MV) command will just update the state file, it’s not recreating the resources again)</strong></p><pre>terraform mv  EXISTING_RESOURCE_NAME OLD_RESOURCE_NAME</pre><p><strong>Terraform Configuration </strong>with wrong resource name<strong> “public_ec2”.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/1*Bjrl6PwgeD8Q8KEP7aypBQ.png" /><figcaption>TF configuration with the wrong resource name</figcaption></figure><p><strong>Terraform Configuration </strong>with updated resource name<strong> “bastion_ec2”.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/532/1*3HmM4uAyRFaiADRtoelOsA.png" /><figcaption>TF with updated resource name</figcaption></figure><p><strong>terraform.tfstate </strong>after executing the rename command</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sVn8fHq5OGsXbj_hO_yQDg.png" /></figure><p><strong>terraform.tfstate.backup </strong>state file backup after renaming the resource</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EZJBFnMN_RxINrOFlZzYQQ.png" /></figure><p>So, as terraform.tfstate is metadata of current infrastructure linking, the rename command just updates the terraform.tfstate file, it is not recreating the resources.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dcf7416677ee" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Terraform import and refresh-only command]]></title>
            <link>https://bharat-singh-06.medium.com/terraform-import-and-refresh-only-command-7dda86da3915?source=rss-52932ec5027d------2</link>
            <guid isPermaLink="false">https://medium.com/p/7dda86da3915</guid>
            <category><![CDATA[aws-certification]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[hashicorp-terraform]]></category>
            <dc:creator><![CDATA[Bharat Singh]]></dc:creator>
            <pubDate>Sun, 11 Jun 2023 09:41:18 GMT</pubDate>
            <atom:updated>2023-06-11T09:45:24.169Z</atom:updated>
            <content:encoded><![CDATA[<p>As terraform is infrastructure as code, it has multiple commands. Here we are going to discuss the use of <strong>import </strong>and <strong>refresh-only </strong>commands<strong>.</strong></p><p>When and how to use.</p><blockquote>&gt;terraform import &lt;resource reference &gt; &lt;resoruce_id&gt;</blockquote><p>Import command help in importing the existing resources from infrastructure. So, If we have a resource created by the console, and the user wants to manage the same resources from Terraform. we can import an existing state file.</p><p>Note. It will add the imported resource to the state file. But it will not create a configuration. So, the user has to add configuration.</p><blockquote>resource “aws_nat_gateway” “private_nat_gateway” {</blockquote><blockquote>subnet_id =aws_subnet.public_subnet.id</blockquote><blockquote>connectivity_type = “public”</blockquote><blockquote>}</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*00l8liidzKSGLj5iX46MFQ.png" /><figcaption>Terraform import command</figcaption></figure><blockquote>&gt; Terraform apply -refresh-only</blockquote><p>the <strong><em>refresh-only</em></strong> command is the replacement of the <strong><em>refresh </em></strong>command.</p><p>In case, if the user has modified any resources from the AWS console and you want to add that change to your terraform. we can use this command.</p><p>Note: It will update the state file, but not the configuration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*brsQdrLZYOuoKYjm_F44og.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/655/1*ri4fs3hywkVHyaR4HlgEuw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7dda86da3915" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Integrate AWS Lambda with RDS through RDS Proxy.]]></title>
            <link>https://bharat-singh-06.medium.com/integrate-aws-lambda-with-rds-through-rds-proxy-b9e032ca5def?source=rss-52932ec5027d------2</link>
            <guid isPermaLink="false">https://medium.com/p/b9e032ca5def</guid>
            <dc:creator><![CDATA[Bharat Singh]]></dc:creator>
            <pubDate>Sun, 27 Nov 2022 15:18:28 GMT</pubDate>
            <atom:updated>2022-11-27T15:18:28.769Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/761/0*M2rJ3Zh2UfNp0XY1" /></figure><p><strong>Problem Statement</strong></p><p>Recently, I was going through RDS connection issues. We were accessing the RDS with Lambda through RDS URL, and we observed that too many connections were getting opened and they were taking more time in connection initiation. Because every time whenever a new instance of lambda starts, it creates a new connection for every instance.</p><p>For overcoming this connection management problem, AWS introduced another service called RDS Proxy. It is a fully managed AWS service</p><p>Lambda-&gt; RDS Proxy-&gt;Database</p><p><strong>Advantages</strong></p><ol><li>RDS Proxy is a fully managed Connection Pooling AWS Service.</li><li>Used for Low-Latency and High-concurrency applications.</li><li>Proxy is a Highly Available service as it supports multi-AZ.</li><li>Secure connection with RDS as RDS Proxy access DB credentials from the Secret manager.</li></ol><p><strong>Steps to Setup</strong></p><p>We need to follow the below steps to set up the RDS proxy and integrate the lambda function.</p><p><strong>Step 1: RDS DB Creation</strong></p><p>Create a Database from AWS RDS Console.</p><p>db_username=&lt;DB_USER&gt;</p><p>db_passowrd=&lt;DB_PASSWORD&gt;</p><p><strong>Step 2: Security Groups for RDS, Lambda, and RDS Proxy </strong><a href="https://www.linkedin.com/feed/hashtag/awssecurity"><strong>#awssecurity</strong></a></p><p>Create three lambda Security Groups for integration of lambda with proxy and RDS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YQhngyMUtmNjtm6u" /></figure><p>Three Security Group.</p><p>DEV_LAMBDA_SG: Create a default Security group and attach it to lambda. DEV_RDS_PROXY_SG: Modify the inbound rules to allow lambda security group on MySQL port 3306 and Attach to RDS Proxy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*h956DXKD0MNRmmOf" /></figure><p>Security Group for RDS Proxy, Add Inboud Rule for Lambda SGroup</p><p>DEV_RDS_SG: Modify the inbound rules to allow the RDS PROXY security group on MySQL port 3306 and attach it to RDS Database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OUA1MqxKqWQv7cAf" /></figure><p>Security Group for RDS, Add Inbound Rule for Proxy Security Group</p><p><strong>Setup 3: Secrets Manager</strong></p><p>For Securely connecting the RDS proxy with RDS, we need to create a Secret Manager. The secret mange will keep the DB password and username. Go to the Secret Magner wizard, Provide the Database username/password and select the DB.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*e1XPL0sPFdnkVpkQ" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2yNomCwF-nZh9paM" /></figure><p>Secrets Manager ARN will be used for Secret manager policy creation. Which will provide RDS Proxy Role permission to access Database Secrets</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1NAxpGB0nKIK2QSk" /></figure><p><strong>Step 4: Secrets Manager Policy</strong></p><p>Resources-Base Policy for Secret manager, This Policy will allow the RDS proxy to access Secret Manger and Decrypt the Database credentials from Secret Manager.</p><p>DEV_ACCESS_SECRET_MANAGER_POLICY</p><pre>{&quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Sid&quot;: &quot;GetSecretValue&quot;,<br>            &quot;Action&quot;: [<br>                &quot;secretsmanager:GetSecretValue&quot;<br>            ],<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Resource&quot;: [<br>                &quot;arn:aws:secretsmanager:us-east-1: &lt;ACCOUNT_ID&gt;:secret:DEV_SECRET_MANAGER-Z5cfVO&quot;<br>            ]<br>        },<br>        {<br>            &quot;Sid&quot;: &quot;DecryptSecretValue&quot;,<br>            &quot;Action&quot;: [<br>                &quot;kms:Decrypt&quot;<br>            ],<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Resource&quot;: [<br>                &quot;arn:aws:kms:us-east-1:&lt;ACCOUNT_ID&gt;:key/b04cef60-f65a-4302-b2d9-aceb825a850d&quot;<br>            ],<br>            &quot;Condition&quot;: {<br>                &quot;StringEquals&quot;: {<br>                    &quot;kms:ViaService&quot;: &quot;secretsmanager.us-east-1.amazonaws.com&quot;<br>                }<br>            }<br>        }<br>    ]<br>}</pre><p><strong>Step 5: IAM Role</strong></p><p>AWS IAM Role, which will allow the RDS proxy to access Secret Manager and connect with RDS.</p><p>RDS_PROXY_ROLE</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xq6Tp53_7Tm7OdHk" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SjBr8ir_-xHwZg8D" /></figure><p><strong>Step 6: RDS Proxy </strong><a href="https://www.linkedin.com/feed/hashtag/rds"><strong>#rds</strong></a></p><p>Navigate to the RDS Proxy console and initiate Proxy creation. This proxy will do the connection management for our lambda functions. It will work as Connection Pool for RDS. It has multiple properties like connection idle and percentage of connections pool.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/997/0*EQqbZlEA5HnVaHK1" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1019/0*zkPfHfa3L4IL0mUK" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*S5mfbSrlfhss3Hdh" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Wl9TTBMouuor2qUm" /></figure><p><strong>Setup 7: AWS lambda — Python </strong><a href="https://www.linkedin.com/feed/hashtag/awslambda"><strong>#awslambda</strong></a></p><p>For accessing the DB in lambda over RDS proxy. Create a lambda function with python with AWS lambda console and use RDS proxy ENDPOINT_URL.</p><p>NOTE: AWS lambda and DB should be in the same VPC Subnets.</p><p>lambda_funcation.py</p><pre>import json<br>import logging<br>import sys<br>import rds_config<br>import pymysql<br><br># rds settings<br>rds_proxy_host = rds_config.rds_proxy_host<br>db_username = rds_config.db_username<br>db_password = rds_config.db_password<br>db_name = rds_config.db_name<br><br><br>logger = logging.getLogger()<br>logger.setLevel(logging.INFO)<br><br>try:<br>    conn = pymysql.connect(host=rds_proxy_host, user=db_username, <br> passwd=db_password, db=db_name, connect_timeout=5)<br>except Exception as err:<br>    logger.error(&quot;ERROR: Unable to connect with  MySQL instance.&quot;)<br>    logger.error(err)<br>    sys.exit()<br><br>logger.info(&quot;SUCCESS: Connected&quot;)<br><br><br>def lambda_handler(event, context): <br>    with conn.cursor() as cur:<br>        sql = &quot;select * from user&quot;<br>        cur.execute(sql)<br>        body = cur.fetchall()<br><br>    return {<br>        &quot;statusCode&quot;: 200,<br>        &quot;body&quot;: json.dumps(body)<br>    }<br></pre><p>rds_confg.py</p><pre>rds_proxy_host = &quot;&lt;RDS_PROXY_ENDPOINT_URL&gt;<br>db_username = &lt;DB_USER&gt;<br>db_password = &lt;DB_PASSWORD&gt;&quot;</pre><p>Step 8: Associate the RDS Proxy with lambda</p><p>As we need to access the RDS proxy in our lambda function. Lambda-&gt; Configuration-&gt; RDS proxy</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hyhET6MhD-FPue5b" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-BWH0XQVeyzeip8i" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*t9nDkBVXhiCRybl5" /></figure><p>Step 9: Test Lambda integration</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*R7jywY7v8UYTPcyY" /></figure><p><strong>Execute lambda and Test Resonse</strong></p><p>Troubleshooting In case of any issue, please use the below command for troubleshooting the Proxy availability</p><p><strong>Show Proxy Details</strong></p><pre>aws rds describe-db-proxies --db-proxy-name &lt;DB_PROXY_NAME&gt;</pre><p>Show Proxy Target details and status of Target</p><pre>aws rds describe-db-proxy-target-groups --db-proxy-name &lt;DB_PROXY_NAME&gt;</pre><p>References</p><p><a href="https://github.com/BharatSingh0686/awslambda-rdsproxy">GitHub</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b9e032ca5def" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>