<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Bhargav on Medium]]></title>
        <description><![CDATA[Stories by Bhargav on Medium]]></description>
        <link>https://medium.com/@bhargav.dev01?source=rss-a425b718d8d2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*DKUb8zkXP_RXVpo9by6IPg.png</url>
            <title>Stories by Bhargav on Medium</title>
            <link>https://medium.com/@bhargav.dev01?source=rss-a425b718d8d2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 08:31:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bhargav.dev01/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[I Used AI to Find Free ChatGPT Plus Methods So You Don’t Have To]]></title>
            <link>https://medium.com/@bhargav.dev01/i-used-ai-to-find-free-chatgpt-plus-methods-so-you-dont-have-to-e5141bde491b?source=rss-a425b718d8d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5141bde491b</guid>
            <dc:creator><![CDATA[Bhargav]]></dc:creator>
            <pubDate>Sat, 09 May 2026 08:17:05 GMT</pubDate>
            <atom:updated>2026-05-09T08:17:05.544Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ywJio5LYK8qfrM29yixIWA.png" /><figcaption>Prompt Spine(core idea)</figcaption></figure><p>YouTube has a problem. Search for anything related to “free ChatGPT Plus” and you get flooded — dozens of thumbnails, clickbait titles, suspiciously happy faces. Most of it is noise.</p><p>So instead of watching 50 videos myself, I did something different. I wrote a prompt and handed the whole job to an AI browser agent.</p><p>Here’s exactly how it works — and what I found.</p><h3>The Comments Don’t Lie</h3><p>YouTube comments are unfiltered. Nobody is being paid to write them. And when someone actually gets something to work, they say so.</p><p>Comments like “It worked bro”, “Still working in 2026”, or “You saved me” are the real signal. On the other hand, “Fake”, “Patched”, “Scam” — that tells you everything you need to know.</p><p>So the idea was simple: build a filter around comments, not titles.</p><h3>The Prompt I Gave the Agent</h3><p>Three searches. One filter. Zero manual scrolling.</p><p>Here’s the exact prompt I used — you can copy and paste this directly into any AI browser agent like Gemini, ChatGPT with browsing, or Comet:</p><pre>Open YouTube and search for:<br>&quot;How to get ChatGPT Plus free&quot;<br>&quot;ChatGPT free premium&quot;<br>&quot;Free ChatGPT Plus method&quot;<br><br>Apply the filter:<br>Upload Date -&gt; This Month<br><br>Then do the following for each relevant video:<br><br>1. Open the video in a new tab.<br>2. Check the comments section carefully.<br>3. Scroll through a good number of comments and prioritise:<br>   - Top comments<br>   - Most liked comments<br>   - Relevant recent comments<br><br>4. Look for comments that strongly indicate the method actually worked:<br>   &quot;It worked&quot; / &quot;Thanks bro&quot; / &quot;This still works&quot; /<br>   &quot;You saved me&quot; / &quot;Confirmed working&quot; / &quot;Worked in 2026&quot;<br><br>IF multiple users are confirming success:<br>   -&gt; KEEP the tab open<br>   -&gt; Pause the video<br><br>IF comments mostly say &quot;Fake&quot; / &quot;Not working&quot; / &quot;Patched&quot; / &quot;Scam&quot;<br>or there are no meaningful confirmation comments:<br>   -&gt; Close the tab immediately<br><br>Continue until you have at least 5 tabs open where comments<br>strongly suggest the method is genuinely working.<br><br>Prioritise:<br>- Videos with high engagement<br>- Positive user feedback<br>- Recent uploads<br>- Clear demonstrations<br>- Less clickbait style thumbnails/titles<br><br>GOAL: End with at least 5 paused YouTube tabs open that have<br>strong positive confirmation from commenters that the method works.</pre><h3>What Makes This Actually Smart</h3><p>You’re not trusting the creator. You’re trusting the crowd.</p><p>The agent doesn’t watch a single second of video. It goes straight to comments, reads the sentiment, and makes a binary call: keep or close. That’s it.</p><p>It also prioritises the right things:</p><p>High engagement over empty views. Real demos over talking heads. Recent uploads, not year-old content. Less clickbait in the title = more trust.</p><p>By the end, you’re not watching 50 videos. You’re watching 5 — the ones the internet already vouched for.</p><h3>The Bigger Point</h3><p>This isn’t really about ChatGPT Plus.</p><p>It’s about the realisation that once you can describe a research process in plain language, you can delegate it. Completely.</p><p>Product research. Price comparisons. Scam detection. Review analysis. Competitor monitoring. All of it can be handed off to an agent with the right prompt.</p><p>I just happened to start with the most ironic version of it: using AI to find ways to get AI for free.</p><p>That’s the experiment. Make of it what you will.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5141bde491b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Claude : Subprocess Initialisation Error]]></title>
            <link>https://medium.com/@bhargav.dev01/claude-subprocess-initialisation-error-b681a7bbcb43?source=rss-a425b718d8d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/b681a7bbcb43</guid>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[claude]]></category>
            <category><![CDATA[subprocess-error]]></category>
            <category><![CDATA[anthropic-claude]]></category>
            <dc:creator><![CDATA[Bhargav]]></dc:creator>
            <pubDate>Sun, 26 Apr 2026 14:03:58 GMT</pubDate>
            <atom:updated>2026-04-26T14:03:58.528Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/676/1*X8ca3szE3nIBFIXu5inFQQ.png" /><figcaption>Error in vscode claude chat</figcaption></figure><p>fix this:<br>take backup of the root level Claude settings json file</p><p>Delete the settings json file<br>uninstall Claude extension</p><p>reinstall extension <br>and try:<br>if it works keep the backup file in same root level in .claude folder</p><p>this is one of the real Root cause for my set up</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n3oIbZ_wewnpwNb_h0_7FQ.png" /></figure><p>Thanks.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b681a7bbcb43" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Someone Almost Hacked Every Linux Server on the Planet]]></title>
            <link>https://medium.com/@bhargav.dev01/how-someone-almost-hacked-every-linux-server-on-the-planet-3037d6e53f1b?source=rss-a425b718d8d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/3037d6e53f1b</guid>
            <category><![CDATA[hacking]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Bhargav]]></dc:creator>
            <pubDate>Sat, 28 Feb 2026 07:26:11 GMT</pubDate>
            <atom:updated>2026-02-28T07:26:11.526Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b5keI1amxMv8Xi5fq2k_1g.png" /></figure><p>In early 2024, someone almost hacked millions of Linux servers around the world.</p><p><strong><em>Not by breaking a firewall. Not by guessing a password.</em></strong></p><p>They did it by being patient for 2.5 years — and hiding a tiny piece of malicious code inside a tool that almost nobody was watching.</p><p>This is the story of the XZ Utils backdoor.</p><h4>What is XZ Utils?</h4><p>XZ Utils is a small program that compresses files. Think of it like a zip tool.</p><p>It runs quietly in the background on almost every Linux server in the world. Nobody thinks about it. Nobody watches it.</p><p>But here’s the problem — OpenSSH, the tool you use to remotely log into Linux servers, indirectly depends on XZ Utils.</p><p>So if XZ Utils is compromised, the attacker gets access to SSH logins.</p><h3><strong>That means: full control of the server.</strong></h3><h4>The Attack Flow — Simple Version</h4><p>Step 1: Attacker spends 2.5 years slowly gaining trust in the open source community<br>Step 2: Gets added as a maintainer of XZ Utils<br>Step 3: Hides malicious code inside the project<br>Step 4: The code intercepts SSH logins<br><strong>Step 5: Attacker could log into any affected server as root</strong></p><h4>How It Was Stopped:<br>Step 1: One developer noticed SSH was 500ms slower than usual<br>Step 2: He investigated and found the hidden code<br>Step 3: All Linux distributions rolled back XZ to a safe version</h4><h4>Phase 1 — Earning Trust (The Social Engineering)</h4><p>The attacker didn’t write any malicious code at first.</p><p><strong><em>They just helped.</em></strong></p><p>A developer named Lasse Collin had been maintaining XZ Utils alone since 2005. For 20 years. Unpaid. Nights and weekends.</p><p>By 2022, he was exhausted.</p><p>Then a person called “Jia Tan” appeared. They started submitting helpful code fixes. They were polite, fast, and reliable.</p><p>After a year of good work, Lasse trusted Jia enough to give them full access to the project.</p><p><strong><em>No hacking needed. Just patience and pretending to be helpful.</em></strong></p><blockquote>This is the part no security tool can catch.</blockquote><h4>Phase 2 — Hiding the Malicious Code</h4><p>Jia couldn’t just push bad code to GitHub — anyone can read source code.</p><p>So instead, Jia hid the malicious code inside binary test files. These are compressed data files used for testing. They look like random garbage — nobody reads them.</p><p>Jia also made a small change to the build script. When someone compiled XZ from source, the script would secretly unpack and inject the malicious code into the final program.</p><blockquote>The malicious code was invisible in plain sight.</blockquote><h4>Phase 3 — How the Hack Actually Worked</h4><p>This part is technical, but let’s keep it simple.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/944/0*oYZRlvpR9bH-qTuM.png" /></figure><p>When OpenSSH starts, Linux creates a “lookup table” — like a phone book — that maps function names to their memory addresses.</p><p>Normally:<br>OpenSSH wants to run RSADecrypt<br>→ Looks up the phone book<br>→ Finds the real address → Runs the real function</p><p>After the backdoor:<br>OpenSSH wants to run RSADecrypt<br>→ Looks up the phone book<br>→ The address has been swapped to the attacker’s code → Runs attacker’s function first</p><p>The swap happened at a very precise moment during startup — after the address was written, but before the system locked the table as read-only.</p><p>Jia timed it perfectly using two obscure Linux startup features.</p><h4>Phase 4 — What the Backdoor Did</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yeVMNBzzqzcXhgDd2-xBbg.png" /></figure><p><strong><em>Once the swap was in place, every SSH connection was intercepted.</em></strong></p><p>But the backdoor was smart — it didn’t let everyone in.</p><p><strong>If the connection had the attacker’s secret key → Grant silent root access (no </strong>logs)<br>If it was a normal user → Pass through normally, user sees nothing</p><p>Only the attacker, with their private key, could trigger it. Everyone else got a normal experience.</p><h4>How It Was Caught</h4><p>Andres Freund is a Microsoft engineer. He was not looking for a backdoor.</p><p>He was just setting up a test environment. And he noticed SSH logins were consistently 500ms slower than they should be.</p><p>He could have ignored it. Most people would have.</p><p><strong><em>Instead, he dug in.</em></strong></p><p>He found test files that were never actually used in tests. He kept pulling the thread. Eventually he found the backdoor — hidden inside what looked like random binary data.</p><p>Here’s the ironic part: all that hiding and obfuscation made the code slower. The 500ms slowdown was caused by the attacker’s own careful hiding.</p><blockquote>Their caution gave them away.</blockquote><p>The Red Hat security team was called on a Friday evening. Within hours, every major Linux distribution rolled back XZ to a safe version.</p><p>Jia Tan’s accounts went completely silent. They have never been heard from again.</p><h4>Who Was Jia Tan?</h4><p>Nobody knows for sure.</p><p>The operation ran for 2.5 years. Multiple fake accounts were used to pressure Lasse. The code commit times pointed to UTC+8 (China) and UTC+2 (Russia/Israel).</p><p>Some researchers believe it was APT29, a Russian government hacking group. Others think it was China.</p><p><strong>What’s clear: this was not some random hacker. This was a well-funded, <em>highly organized operation. Possibly a nation-state.</em></strong></p><h4>What You Should Learn From This</h4><p>1. Every dependency you use is a trust decision<br>You didn’t write XZ Utils. You didn’t audit it. But it ran on your server. Always pin your dependency versions. Use tools like npm audit, pip-audit, or Dependabot to scan for issues.</p><p>2. Open source projects need support<br>Lasse was one person, unpaid, maintaining software used by millions of servers. The community pressured him without helping him. That burnout made him vulnerable.</p><p>If your product depends on an open source library maintained by one tired volunteer — that is a real risk.</p><p>3. Small performance changes are worth investigating<br><strong><em>Andres noticed 500ms. He investigated. He saved the internet.</em></strong></p><p>If something is slightly slower than it should be, that’s worth 10 minutes of your time.</p><p>4. You don’t need a zero-day to hack the world<br>The attacker never exploited a software bug. They exploited trust, burnout, and an unmonitored dependency.</p><blockquote>We got lucky this time. The attacker made one small mistake.</blockquote><h3>Next time, they might not.</h3><p><strong><em>Stay curious. Know what’s running on your servers.</em></strong></p><p>— — —</p><h3>Want to Go Deeper?</h3><p>This article was inspired by the excellent Veritasium video: “The Internet Was Weeks Away From Disaster and No One Knew” — a 30+ minute deep dive into the full story of the XZ Utils backdoor.</p><p>Veritasium covers this story in incredible detail, including:</p><ul><li>The full psychological profile of “Jia Tan” — how a nation-state level attacker built a fake identity over 2+ years to earn the trust of the open source community</li><li>• How the malicious code was hidden inside binary test files that no one reads — one of the most clever obfuscation techniques ever found in open source</li><li>• The exact technical breakdown of how the backdoor hijacked the RSA key verification process in OpenSSH using a function pointer swap</li><li>• Why Andres Freund — a Microsoft engineer who wasn’t even doing security work — was the one who caught it, and what exactly made him suspicious</li><li>• The geopolitical angle: evidence pointing to APT29 (Russia’s SVR intelligence unit) and why this may have been a state-sponsored supply chain attack</li><li>• What this means for the future of open source security and why a single burned-out unpaid maintainer became a global vulnerability</li></ul><p>Full credit and thanks to Veritasium (Derek Muller) for the in-depth research and storytelling behind this story. If this article sparked your curiosity, the video goes 10x deeper.</p><p><strong>Watch: The Internet Was Weeks Away From Disaster and No One Knew → </strong><a href="https://www.youtube.com/watch?v=aoag03mSuXQ">https://www.youtube.com/watch?v=aoag03mSuXQ</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3037d6e53f1b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Netflix Delivers Billions of Hours of Streaming Flawlessly]]></title>
            <link>https://medium.com/@bhargav.dev01/how-netflix-delivers-billions-of-hours-of-streaming-flawlessly-d918ddd582c0?source=rss-a425b718d8d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/d918ddd582c0</guid>
            <dc:creator><![CDATA[Bhargav]]></dc:creator>
            <pubDate>Mon, 27 Jan 2025 01:46:50 GMT</pubDate>
            <atom:updated>2025-11-17T17:57:51.604Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/732/0*kB-Y3gyPFNmec57O.jpg" /></figure><p>Netflix has over 270 million users worldwide. While everyone knows Netflix for its shows and movies, the real magic is in the technology that makes streaming smooth for everyone.</p><p>Here’s how Netflix does it:</p><p>1. Smart Video Processing</p><p>Netflix takes original, high-quality videos and converts them into different formats to ensure you get the best experience, no matter your device or internet speed.</p><p>They use Amazon EC2 servers that automatically scale up or down based on demand. Big video encoding jobs are split into smaller tasks so everything gets done faster.</p><p>Automation checks for bad frames, black screens, or other video problems so you always get quality streaming.</p><p>As people moved to 4K and HDR, Netflix switched to a modern, microservices-based system to handle heavier workloads.</p><p>— -</p><p>2. Switching to GraphQL for Mobile Apps</p><p>In 2022, Netflix updated its mobile apps to use GraphQL, which makes data handling faster and more flexible.</p><p>First, they built GraphQL on top of the old system (Falcor) and compared both in real time.</p><p>After testing and fine-tuning, they fully switched to GraphQL — without any interruption to users.</p><p>— -</p><p>3. Introducing Ads Smoothly</p><p>In 2022, Netflix rolled out an ad-supported plan.</p><p>Instead of using fake data, they copied real user traffic and replayed it in a test system. This helped them make sure ads wouldn’t slow things down or cause errors.</p><p>Their data scientists also estimated how many users would try the new ad plan so they could be ready for any surge.</p><p>— -</p><p>4. Chaos Engineering for Reliability</p><p>Netflix built Chaos Monkey, a tool that randomly shuts down servers to test system stability.</p><p>They also have advanced tools (FIT and others) to simulate errors and breakdowns in specific areas.</p><p>These strategies help Netflix prepare for anything going wrong, so users rarely notice issues.</p><p>— -</p><p>5. The Open Connect Program</p><p>Netflix created its own global server network called Open Connect.</p><p>Special Netflix servers (“Open Connect Appliances”) store shows and movies in over 1,000 locations — close to where users live.</p><p>This reduces buffering and keeps streams fast, even during peak hours.</p><p>Thanks to this system, Netflix delivers over 125 million hours of streaming every day.</p><p>— -</p><p>Final Thoughts</p><p>Netflix’s success isn’t just about great content — it’s about the powerful technology working behind the scenes. From smart video processing to chaos engineering, Netflix constantly finds new ways to improve.</p><p>If you want to learn more, check out the Netflix Tech Blog. It’s packed with fascinating engineering stories.</p><p>If you enjoyed this article, don’t forget to like, share, and comment!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d918ddd582c0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Build a Gemini 2.0-Powered Personalized Learning Assistant with React & Flask]]></title>
            <link>https://medium.com/google-cloud/how-to-build-a-gemini-2-0-powered-personalized-learning-assistant-with-react-flask-83b5466a1d69?source=rss-a425b718d8d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/83b5466a1d69</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[generative-ai]]></category>
            <category><![CDATA[gcp-app-dev]]></category>
            <dc:creator><![CDATA[Bhargav]]></dc:creator>
            <pubDate>Fri, 13 Dec 2024 10:23:52 GMT</pubDate>
            <atom:updated>2025-01-05T22:28:02.903Z</atom:updated>
            <content:encoded><![CDATA[<h3>AI Assistant Project: Web App Developed using Flask and React With An Gemini 2.0</h3><p>AI models like Gemini can transform how people interact with content. In this blog, I introduce the AI Assistant Project using Gemini api.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/1*KLUbbhKPlfz6DOTrlzl0ZA.png" /></figure><h3>Introduction | Overview</h3><h3><strong>The Problem</strong></h3><p>Generic text generation and search tools lack personalization, making tailored content creation and information retrieval inefficient without advanced AI like Gemini.</p><h3><strong>The Solution</strong></h3><p>The AI Assistant Project integrates Google’s Gemini 2.0 API with a React frontend and Flask backend to deliver personalized text generation and search capabilities.</p><h3>Design</h3><h3>High-Level Architecture</h3><p>The AI Assistant Project is built on a modular architecture that ensures seamless interaction between various components:</p><p><strong>Frontend (React):</strong></p><ul><li><strong>User Interface:</strong> Interactive dashboard for user input and display of AI-generated content.</li><li><strong>API Communication:</strong> Sends requests to the backend via RESTful APIs.</li><li><strong>State Management &amp; Routing:</strong> Handles application state and navigation.</li></ul><p><strong>Backend (Flask):</strong></p><ul><li><strong>API Endpoints:</strong> Manages requests for content generation, user management, and voice interactions.</li><li><strong>AI Integration:</strong> Interfaces with Google’s Gemini 2.0 API to generate AI-driven content.</li><li><strong>Database Interaction:</strong> Utilizes Firestore for data storage and retrieval.</li><li><strong>Authentication:</strong> Secures endpoints and manages user sessions.</li></ul><p><strong>Database (Firestore):</strong></p><ul><li><strong>User Data:</strong> Stores registered user information and authentication details.</li><li><strong>Content Storage:</strong> Maintains records of user inputs and AI-generated content for analytics and retrieval.</li></ul><p><strong>AI Integration (Gemini 2.0 API via Vertex AI):</strong></p><ul><li><strong>Content Generation:</strong> Processes user inputs to generate personalized educational content.</li><li><strong>Scalability:</strong> Efficiently handles multiple concurrent requests through Vertex AI’s managed services.</li></ul><p><strong>Deployment (Google Cloud Run):</strong></p><ul><li><strong>Containerization:</strong> Packages the Flask backend into Docker containers for consistent deployment.</li><li><strong>Scalability &amp; Reliability:</strong> Automatically scales backend services based on demand, ensuring high availability.</li></ul><h3>Rationale Behind the Design</h3><ul><li><strong>Modularity:</strong> Separating frontend and backend allows for independent development and scaling.</li><li><strong>Scalability:</strong> Leveraging Google Cloud Run ensures the application can handle increasing loads without manual intervention.</li><li><strong>Real-Time Data Handling:</strong> Firestore provides real-time synchronization, enhancing user experience with instant updates.</li><li><strong>AI-Powered Personalization:</strong> Integrating Gemini 2.0 API enables the generation of tailored content, addressing individual learning needs.</li></ul><h3>Prerequisites</h3><p>Before diving into the setup, ensure you have the following software and tools installed:</p><ul><li><strong>Node.js &amp; npm:</strong> <a href="https://nodejs.org/">Download Node.js</a></li><li><strong>Python 3.9+:</strong> <a href="https://www.python.org/downloads/">Download Python</a></li><li><strong>Git:</strong> <a href="https://git-scm.com/downloads">Download Git</a></li><li><strong>Docker:</strong> <a href="https://www.docker.com/get-started">Download Docker</a></li><li><strong>Postman:</strong> <a href="https://www.postman.com/downloads/">Download Postman</a></li><li><strong>Visual Studio Code or any preferred IDE:</strong> <a href="https://code.visualstudio.com/">Download VS Code</a></li><li><strong>Google Cloud Account:</strong> Set up a Google Cloud project and enable necessary APIs.</li></ul><h3>Step-by-Step Instructions</h3><h3>1. Clone the Repository</h3><p>Begin by cloning the project repository to your local machine:</p><pre>git clone https://github.com/bhragavvc/ai-assistant-project.git<br>cd ai-assistant-project/frontend</pre><h3>2. Frontend Setup</h3><h4>Install Dependencies</h4><p>Navigate to the frontend directory and install the necessary packages:</p><pre>npm install</pre><h4>Configure Environment Variables</h4><p>Create a .env file in the frontend directory:</p><pre>REACT_APP_BACKEND_URL=https://ai-assistant-backend-xyz.a.run.app</pre><p><em>Replace </em><em>https://ai-assistant-backend-xyz.a.run.app with your actual backend URL after deployment.</em></p><h4>Start the Development Server</h4><p>Launch the frontend application:</p><pre>npm start</pre><p>Access the application at <a href="http://localhost:3000/">http://localhost:3000</a>.</p><h3>3. Backend Setup</h3><h4>Navigate to Backend Directory</h4><pre>cd ../backend</pre><h4>Set Up a Virtual Environment</h4><pre>python3 -m venv venv<br>source venv/bin/activate</pre><h4>Install Dependencies</h4><pre>pip install -r requirements.txt</pre><h4>Configure Environment Variables</h4><p>Create a .env file in the backend directory:</p><pre>GOOGLE_CLOUD_PROJECT=your_project_id<br>GOOGLE_CLOUD_LOCATION=your_region<br>GOOGLE_GENAI_USE_VERTEXAI=True<br>GEMINI_API_KEY=your_gemini_api_key<br>FIREBASE_SERVICE_ACCOUNT_KEY=path/to/serviceAccountKey.json<br>PORT=8080</pre><p><em>Ensure you replace placeholders with your actual project details and API keys.</em></p><h4>Initialize Firebase Admin SDK</h4><p>Ensure you have a Firebase project set up and have downloaded the service account key JSON file. Update the path in main.py accordingly.</p><h4>Run the Flask Application Locally</h4><pre>python main.py</pre><p>The backend should be running at <a href="http://localhost:8080/">http://localhost:8080</a>.</p><h3>4. Deployment Instructions</h3><h4>Deploying the Backend to Google Cloud Run</h4><p><strong>Authenticate with Google Cloud:</strong></p><pre>gcloud auth login<br>gcloud config set project learning-assistant</pre><p><strong>Build and Push Docker Image:</strong></p><pre>docker build -t gcr.io/learning-assistant/ai-assistant-backend .<br>docker push gcr.io/learning-assistant/ai-assistant-backend</pre><p><strong>Deploy to Cloud Run:</strong></p><pre>gcloud run deploy backend-service \<br>  --image gcr.io/project-id/ai-assistant-backend \<br>  --platform managed \<br>  --region ur-region \<br>  --allow-unauthenticated \<br>  --port 8080</pre><p><em>After deployment, update the </em><em>REACT_APP_BACKEND_URL in the frontend </em><em>.env file with the provided Cloud Run URL.</em></p><h4>Deploying the Frontend</h4><p><strong>Build the React Application:</strong></p><pre>cd ../frontend<br>npm run build</pre><p>Choose Hosting Platform:</p><ul><li><strong>Firebase Hosting</strong></li><li><strong>Vercel</strong></li><li><strong>Netlify</strong></li><li><strong>Google Cloud Storage + Cloud CDN</strong></li></ul><p><strong>Deploy to Firebase Hosting (Example):</strong></p><pre>npm install -g firebase-tools<br>firebase login<br>firebase init hosting<br>firebase deploy --only hosting</pre><p><em>Access your frontend via the provided Firebase Hosting URL.</em></p><h3>5. <strong>Connecting Frontend and Backend</strong></h3><p>Ensure that the backend’s CORS middleware allows requests from the frontend’s origin (http://localhost:3000 or your deployed frontend URL).</p><h3>6. Additional Backend Enhancements</h3><ul><li><strong>Error Handling:</strong> Implement robust error handling to manage API failures gracefully.</li><li><strong>Secure API Keys:</strong> Ensure API keys are never exposed on the frontend. Store them securely using environment variables on the backend.</li><li><strong>Authentication:</strong> Implement user authentication (e.g., JWT) for secure access and personalized experiences.</li></ul><h3>Result / Demo</h3><p>After following the setup and deployment instructions, you should have a fully functional AI-powered personalized learning assistant. Here’s what you can expect:</p><p><strong>Interactive Dashboard:</strong></p><ul><li>Input your study topics or preferences.</li><li>Choose the type of content you want (e.g., study summaries, quizzes, flashcards).</li></ul><p><strong>AI-Generated Content:</strong></p><ul><li>Receive personalized educational materials instantly.</li><li>Engage with real-time content generation based on your inputs.</li></ul><p><strong>Voice Interaction (Optional):</strong></p><ul><li>Use voice commands to interact with the assistant.</li><li>Receive voice responses for a hands-free experience.</li></ul><p><strong>Export Features:</strong></p><ul><li>Export generated content and reports as PDF documents for offline use.</li></ul><p><em>High-Level Architecture of the AI Assistant Project</em></p><h3>What’s Next?</h3><p>While the AI Assistant Project offers a robust foundation for personalized learning, there are several avenues for future enhancements:</p><ul><li><strong>Advanced AI Features:</strong> Integrate more sophisticated AI models for deeper content personalization.</li><li><strong>Mobile Application:</strong> Develop a mobile version of the assistant for on-the-go access.</li><li><strong>Multilingual Support:</strong> Extend capabilities to support multiple languages.</li><li><strong>Gamification:</strong> Introduce gamified elements to make learning more engaging.</li><li><strong>Analytics Dashboard:</strong> Implement real-time analytics to track user interactions and improve services.</li></ul><h3>Call to Action</h3><p>To learn more about Google Cloud services and to create impact for the work you do, get around to these steps right away:</p><ol><li><strong>Explore Google Cloud Services:</strong> Dive into the comprehensive suite of tools and APIs that Google offers to enhance your projects.</li><li><strong>Join the Community:</strong> Engage with other developers and AI enthusiasts to share knowledge and collaborate on innovative solutions.</li><li><strong>Contribute to Open Source:</strong> Enhance projects like the AI Assistant Project by contributing your skills and ideas.</li></ol><h3>Summary</h3><p>This Medium blog post covers all essential aspects of the AI Assistant Project, from introduction and design to deployment and future enhancements, making it accessible and engaging for your audience.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=83b5466a1d69" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/how-to-build-a-gemini-2-0-powered-personalized-learning-assistant-with-react-flask-83b5466a1d69">How to Build a Gemini 2.0-Powered Personalized Learning Assistant with React &amp; Flask</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Serverless Architecture on AWS]]></title>
            <link>https://medium.com/@bhargav.dev01/serverless-architecture-on-aws-bfb867b6cdf9?source=rss-a425b718d8d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/bfb867b6cdf9</guid>
            <category><![CDATA[lambda]]></category>
            <category><![CDATA[sns]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[dynamodb]]></category>
            <category><![CDATA[s3]]></category>
            <dc:creator><![CDATA[Bhargav]]></dc:creator>
            <pubDate>Sun, 01 Dec 2024 09:11:29 GMT</pubDate>
            <atom:updated>2024-12-01T09:38:45.567Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*qBQHTFxvITWsprn4.png" /></figure><p>Index</p><ol><li><a href="#6461">Introduction</a></li><li><a href="#4c9e">Prerequisites</a></li><li><a href="#fb43">Creating S3 Bucket</a></li><li><a href="#9609">Create a Dynamo DB Table</a></li><li><a href="#52d1">Create a SNS Topic</a></li><li><a href="#637d">Creating Lambda Role for Read from S3 &amp; Update to the Dynamo DB</a></li><li><a href="#e140">Creating Lambda role for Read from Dynamodb &amp; Send SNS notification</a></li><li><a href="#406f">Create a Lambda Function to Read from S3 and Upload to Dynamo DB</a></li><li><a href="#6548">Create a Lambda Function to Read from DynamoDB and Send an SNS Notification for Low Inventory</a></li><li><a href="#ab67">Testing</a></li><li><a href="#75c1">Clean Up the setup</a></li><li><a href="#d686"><em>Infrastructure as Code</em>Introduction</a></li></ol><p><strong>Introduction</strong></p><p>Imagine you want to get regular notifications whenever specific events happen. In this example, let’s say you own multiple stores in different locations. At the end of each day, you want to know which products and store locations have inventory levels below 5. With a serverless setup, you can easily receive these notifications to stay informed. This approach has countless business applications. Plus, to make things even simpler, there’s a CloudFormation YAML template that sets up the entire infrastructure with just one click.</p><p><strong>Prerequisites</strong></p><ol><li><strong>AWS Account</strong>: Make sure you have an AWS account. If you don’t have one yet, you can <a href="https://chatgpt.com/c/674bf114-6958-8008-b203-73734cd06356#">sign up here</a> and take advantage of the Free-Tier benefits.</li><li><strong>Project Files</strong>: Access all the project files in my <a href="https://github.com/bhargavvc/aws-projects/tree/main/inventory">GitHub portfolio</a>.</li></ol><h4><strong>STEP 1: Creating an S3 Bucket</strong></h4><ol><li>Log in to the <strong>AWS Console</strong>, search for <strong>S3</strong>, and click on it to open the S3 Console.</li><li>In the <strong>Buckets</strong> section, click on <strong>Create bucket</strong>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*UQzrP2ABs5NUGIBa.png" /></figure><p>3. Fill in the details:</p><ul><li>Choose your preferred <strong>AWS Region</strong>.</li><li>Enter a unique <strong>Bucket Name</strong> (bucket names must be unique within the region).</li><li>Scroll down and click <strong>Create bucket</strong>.</li><li>Make a note of the <strong>ARN</strong> (Amazon Resource Name) of the created bucket. You’ll need this later.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*37kJMQUXIzJ5bxZo.png" /></figure><blockquote><strong>STEP2: Create a Dynamo DB Table</strong></blockquote><ol><li>Search for <strong>Dynamo DB on the AWS Console. Click Dyanmo Db and then click create Table</strong></li><li>Enter the Table name , Partition key and Sort Key . Scroll to the bottom and click on Create table</li><li>The table is created and appears on your list of tables. Note theDynamoDB ARN of the table. We will use it later.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u29NA68us2-Fl3ooQg3OKQ.png" /><figcaption>Step2: 1</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B6EHSM_fdl3qeWN6sLiqZg.png" /><figcaption>Step2: 2</figcaption></figure><blockquote><strong>STEP 3: Create SNS Topic</strong></blockquote><p><strong>Simple Definition for SNS:</strong><br> Amazon SNS (Simple Notification Service) is a fully managed messaging service for sending notifications to various endpoints like email, SMS, or applications.</p><p><strong>Real-World Example:</strong><br> A retail app uses SNS to send customers real-time order updates via SMS or email after they make a purchase.</p><p>SNS sends you a text message to let you know your pizza is on the way.</p><ol><li>Search for sns In the AWS Console</li><li>In the Amazon SNS dashboard Give topic name Click Create.</li><li>Under the Topics page, click on Create topic.</li><li>On the Subscriptions tab, click on Create subscription.</li><li>In the Create subscription page, set the Protocol as Email and enter your preferred email address as the Endpoint . Scroll down and click on Create subscription</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/929/1*5rLq32kszMBFhUdeGkfdNQ.png" /><figcaption>STEP3: 1</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yQixP0G3OPcFtEb3Hs0mqw.png" /><figcaption>STEP3: 2</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*P5qZ3o07gsbibEy3Z7W9yA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PpG9TYPntbbaWy98GOGmIg.png" /><figcaption>STEP3: 3</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R6fHWdyGlRsE-e0oQPl_NQ.png" /><figcaption>STEP 3: 4</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oXCWgIed_OGRh3RWQ79wjQ.png" /><figcaption>STEP 3: 5</figcaption></figure><blockquote>STEP 4: <strong><em>Creating a Role for Lambda to read from S3 and update the DynamoDB</em></strong></blockquote><ol><li>In the IAM Console , under Policies click on Create policy.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*DW3-kDEx3WR8WiJD.png" /></figure><p>2. Select the json tab of the policy and paste the json policy below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*zV6BMHbCrupWIPey.png" /></figure><pre>{<br> &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>      {<br>        &quot;Effect&quot;: &quot;Allow&quot;,<br>        &quot;Action&quot;: [<br>          &quot;s3:GetObject&quot;,<br>          &quot;s3:ListBucket&quot;<br>        ],<br>        &quot;Resource&quot;: [<br>          &quot;{S3-Bucket ARN}/*&quot;,<br>          &quot;{S3-Bucket ARN}&quot;<br>        ]<br>      },<br>      {<br>        &quot;Effect&quot;: &quot;Allow&quot;,<br>        &quot;Action&quot;: [<br>          &quot;dynamodb:PutItem&quot;,<br>          &quot;dynamodb:BatchWriteItem&quot;<br>        ],<br>        &quot;Resource&quot;: &quot;{DynamoDB ARN}&quot;<br>      }<br>    ]<br>}</pre><p>Replace {S3-Bucket ARN} with the ARN of the S3 bucket you created and {DynamoDB ARN} with the ARN of your DynamoDB Table</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*K4k6XqKLEflnQrmr.png" /></figure><p>Scroll to the bottom and click on Next</p><p>4. Give the policy a name then scroll down and click on Create policy</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*4QSazKvnRgLXLZeW.png" /></figure><p>5. On the IAM Console click on Roles then Create role</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*p3u38W-8rxT5Tq2b.png" /></figure><p>6. Under Trusted entity type , select AWS service , select Lambda as the Use case then click on Next</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*jIrcES_ubLlJgjPV.png" /></figure><p>7. In the Add permissions page, search for the policy we just created, select the check box then click on Next</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*mam1UqIucJ-lhAUv.png" /></figure><p>8. Give the Role a name, than scroll down and click on Create role</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*M7JyA-ZFxQRKsZRR.png" /></figure><blockquote>STEP 5: <strong><em>Creating a Role for Lambda to read from DynamoDB and send an SNS Notification</em></strong></blockquote><ol><li>Follow the steps followed in creating the previous role but use the JSON policy below</li></ol><pre>{<br> &quot;Version&quot;: &quot;2012-10-17&quot;,<br> &quot;Statement&quot;: [<br>  {<br>   &quot;Effect&quot;: &quot;Allow&quot;,<br>   &quot;Action&quot;: [<br>    &quot;dynamodb:PutItem&quot;,<br>    &quot;dynamodb:UpdateItem&quot;,<br>    &quot;dynamodb:GetItem&quot;,<br>    &quot;dynamodb:Scan&quot;,<br>    &quot;dynamodb:Query&quot;<br>   ],<br>   &quot;Resource&quot;: &quot;{DynamoDB Table ARN}&quot;<br>  },<br>  {<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;dynamodb:GetRecords&quot;,<br>                &quot;dynamodb:GetShardIterator&quot;,<br>                &quot;dynamodb:DescribeStream&quot;,<br>                &quot;dynamodb:ListStreams&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;{DynamoDB Table ARN}/Stream/*&quot;<br>        },<br>  {<br>   &quot;Effect&quot;: &quot;Allow&quot;,<br>   &quot;Action&quot;: &quot;sns:Publish&quot;,<br>   &quot;Resource&quot;: &quot;{SNS Topic ARN}&quot;<br>  }<br> ]<br>}</pre><p>Insert your Dynamo DB ARN and the SNS Topic ARN in the respective positions in the policy above</p><blockquote>STEP 6: <strong><em>Create a Lambda Function to Read from S3 and Upload to Dynamo DB</em></strong></blockquote><ol><li>Search for Lambda and open the Lambda Console. Click on Create function</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*fL8QGRtGiIDi-s6u.png" /></figure><p>2. Under Create function, choose Author from scratch, enter your preferred Function name and Runtime . Scroll down and click on Create function</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*oVc4UyVNeajfeIjE.png" /></figure><p>3. In the Code source section, enter the code below and click on Deploy</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*oW2neLZFlWpJAWaq.png" /></figure><pre>def lambda_handler(event, context):<br>   <br>    # Get the S3 bucket and object information from the event<br>    s3_bucket = event[&#39;Records&#39;][0][&#39;s3&#39;][&#39;bucket&#39;][&#39;name&#39;]<br>    s3_key = event[&#39;Records&#39;][0][&#39;s3&#39;][&#39;object&#39;][&#39;key&#39;]<br><br>    # Test lambda function manually<br>    # s3_bucket = &#39;productinventory240215&#39;<br>    # s3_key = &#39;inventory.csv&#39;<br><br>    # Set the DynamoDB table name<br>    dynamo_table_name = &#39;AppleInventoryTable&#39;<br><br>    # Create a DynamoDB client<br>    dynamodb = boto3.resource(&#39;dynamodb&#39;)<br>    table = dynamodb.Table(dynamo_table_name)<br><br>    # Create an S3 client<br>    s3 = boto3.client(&#39;s3&#39;)<br><br>    # Download the CSV file from S3<br>    temp_file_path = &#39;/tmp/temp.csv&#39;<br>    s3.download_file(s3_bucket, s3_key, temp_file_path)<br><br>    # Read and process the CSV file, skipping the header row<br>    with open(temp_file_path, &#39;r&#39;) as csvfile:<br>        csvreader = csv.reader(csvfile)<br><br>        # Skip the header row<br>        next(csvreader, None)<br>        <br>        for row in csvreader:<br>            store = row[0]<br>            product = row[1]<br>            count = int(row[2])<br><br>            # Update DynamoDB table<br>            table.put_item(<br>                Item={<br>                    &#39;StoreRegion&#39;: store,<br>                    &#39;Product&#39;: product,<br>                    &#39;ProductCount&#39;: count<br>                }<br>            )<br>  <br>    # Clean up the temporary file<br>    os.remove(temp_file_path)<br>    return {<br>        &#39;statusCode&#39;: 200,<br>        &#39;body&#39;: &#39;CSV processing complete&#39;<br>    }</pre><p>4. Unser the Configuration tab, select Triggers then click on Add trigger</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*63cYlHbTgGRK3-lE.png" /></figure><p>5. Select the Source as S3 . Select the Bucket you created. For Event types use All object create events. Add .csv as the Suffix. (We will be uploading csv files with the data). Select the Check box then click on Add</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*miL8czUtfd74kGjz.png" /></figure><p>6. Under the Permissions select Edit</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*gu1hzMRfUr-KM-Oh.png" /></figure><p>7. Scroll down. Increase the Timeout to 5 minutes . This will help deal with large files. Under Execution role select Use an existing role then select the role we created to read data from S3 and upload to DynamoDB. Click on Save</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*L0Jvos9Tr8ffgWyV.png" /></figure><blockquote><strong><em>STEP 7: Create a Lambda Function to Read from DynamoDB and send an SNS Notification for Low Inventory</em></strong></blockquote><ol><li>Navigate to the Lambda Console page and click on Create function</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*cBu0T_ci7xsd8OLC.png" /></figure><p>2. Select Author from scratch , give the function a name and select the Python runtime. Scroll down and click on Create function</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*MPvEGFTe2t5m6qJo.png" /></figure><p>3. Under the code tab, insert the Python code below then click on Deploy</p><pre>def lambda_handler(event, context):<br>    # Define DynamoDB and SNS clients<br>    dynamo_table_name = &#39;AppleInventoryTable&#39;<br>    dynamodb = boto3.resource(&#39;dynamodb&#39;)<br>    table = dynamodb.Table(dynamo_table_name)<br><br>    sns = boto3.client(&#39;sns&#39;)<br>    region = os.environ[&#39;AWS_REGION&#39;]<br>    account_id = context.invoked_function_arn.split(&quot;:&quot;)[4]<br><br>    # Scan DynamoDB table to get items with count of 5 or less than 5<br>    response = table.scan(<br>        FilterExpression=boto3.dynamodb.conditions.Attr(&#39;ProductCount&#39;).lt(6)<br>    )<br>    <br>    # Check if there are items with count less than 5<br>    items = response.get(&#39;Items&#39;, [])<br>    <br>    if items:<br><br>        # Prepare notification message<br>        notification_message = &quot;Low inventory alert:\n&quot;<br>        for item in items:<br>            notification_message += f&quot;StoreRegion: {item[&#39;StoreRegion&#39;]}, Item: {item[&#39;Product&#39;]}, Count: {item[&#39;ProductCount&#39;]}\n&quot;<br>        <br>        # Publish notification to SNS<br>        sns.publish(<br>            # TopicArn=&#39;arn:aws:sns:us-east-1:471285348599:LowInventoryTopic&#39;,<br>            TopicArn = f&#39;arn:aws:sns:{region}:{account_id}:LowInventoryTopic&#39;,<br>            Message=notification_message,<br>            Subject=&#39;Low Inventory Alert&#39;<br>        )<br>        <br>        return {<br>            &#39;statusCode&#39;: 200,<br>            &#39;body&#39;: &#39;Notification sent successfully.&#39;<br>        }<br>    else:<br>        return {<br>            &#39;statusCode&#39;: 200,<br>            &#39;body&#39;: &#39;No items with count less than 5.&#39;<br>        }</pre><p>4. Under the Configuration tab, select Permissions then click on Edit</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*RKwxf0uSInJN2rV1.png" /></figure><p>5. On the Edit basic settings page, scroll down and set the timeout at 5 min . Under Execution role, select Use an existing role and select the role you created to read from DynamoDB and send a notification of low inventory. Scroll to the bottom and click on Save</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*kytDgIk1la-irDnr.png" /></figure><p>6. Under the Configuration tab, select Triggers then click on Add trigger</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*ka7BcTLnEInm6nEV.png" /></figure><p>5. Select DynamoDB as the Source. Select the DynamoDB Table we created earlier . Check the box for Activate trigger. Enter 1,000 as the Batch size then scroll down and click on Add</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*V9sweq6RYfT-HDSm.png" /></figure><blockquote><strong><em>STEP 8: Testing</em></strong></blockquote><ol><li>Upload a csv file with 3 columns for Store, Product and Count. You can get a sample file <a href="https://github.com/bhargavvc/aws-projects/blob/main/inventory/inventory.csv">here</a></li><li>Navigate to S3 and select the bucket we created</li><li>Click on Upload</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*rtHKsdoVYAD2RuZm.png" /></figure><p>4. Click on Add files then navigate to your files and select the required file. Click on Upload at the bottom</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*8rSM_D-07Zm_3a1c.png" /></figure><p>5. Once uploaded, the lambda function should upload the items to DynamoDB</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*FGJoKgbsvjXxY9Ix.png" /></figure><p>6. An email should also be sent, showing the Products and Stores with less than 5 items in inventory</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/698/0*Nysgl2T7EBuV3lm6.png" /></figure><p>7. This ensures that the system works end to end.</p><blockquote><strong><em>STEP 9: Clean up</em></strong></blockquote><ol><li>Delete the uploaded file from the S3 bucket</li><li>Delete the S3 bucket</li><li>Delete the SNS Topic</li><li>Delete the Dynamo DB table</li><li>Delete the Lambda functions</li><li>Delete the IAM roles created</li><li>Delete the IAM policies created</li></ol><p>STEP 10: <strong><em>Infrastructure as Code</em></strong></p><p>The beauty of setting up infrastructure on AWS is that we have access to tools used for Infrastructure as Code (IaC).</p><ol><li>Navigate to CloudFormation on the AWS console. Under Stacks click on Create stack</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*RZqOSDHxWdiivw4N.png" /></figure><p>2. Select Template is ready and Upload a template file so that we can upload our <a href="https://github.com/bhargavvc/aws-projects/blob/main/inventory/inventory.yaml">yaml template</a>. You can get the yaml template in the <a href="https://github.com/bhargavvc/aws-projects/blob/main/inventory/inventory.yaml">GitHub repo</a>. Click Next</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*B7XHag_awxHwtwu2.png" /></figure><p>3. Provide a Stack name , the notification email and your preferred s3 bucket name</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*oOnU-imhexJ5cjQI.png" /></figure><p>4. click on Next on the next 2 pages then click on Submit at the bottom. Give time for the resources to create. You can monitor the creation of resources on the Resources tab. Once complete, the status will show CREATE_COMPLETE . In the event of a ROLL_BACK check to confirm what the error is and correct it</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*2J5n0GFH4YykNwCC.png" /></figure><p>5. Confirm subscription to the SNS topic from your email</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*KZc_s08ZQ6hVIMlS.png" /></figure><p>6. Proceed to the Testing section as done in the manual setup</p><blockquote><strong><em>Clean Up</em></strong></blockquote><ol><li>Once done, click on Delete on the CloudFormation template page. Theis will delete all the resources created</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*8vHtx4lcHEP74BCa.png" /></figure><p>2. On the pop up that appears, click on Delete and the infrastructure will be pulled down</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*bl86ZaDdjjfJe_AL.png" /></figure><blockquote><strong><em>Conclusion</em></strong></blockquote><p>This demo steps give a walkthrough on how to create a serverless on AWS. You could customize the configurations to fit your use case.</p><p>Follow for more demos and networking.<br><a href="https://www.linkedin.com/in/bhargav-chitteti/">Bhargav Chitteti</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bfb867b6cdf9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automatically Turning On/Off AWS EC2 Instances On Public Holiday and Events.]]></title>
            <link>https://medium.com/@bhargav.dev01/how-to-save-money-by-automatically-turning-off-aws-ec2-instances-when-not-in-use-f6b0991b5d3c?source=rss-a425b718d8d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/f6b0991b5d3c</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[lambda]]></category>
            <category><![CDATA[cloudwatch]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ec2]]></category>
            <dc:creator><![CDATA[Bhargav]]></dc:creator>
            <pubDate>Thu, 28 Nov 2024 05:12:01 GMT</pubDate>
            <atom:updated>2024-11-28T05:18:45.544Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>By Bhargav Chitteti</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*h47axkteDOn6o-GW.png" /></figure><p>Are you paying for Amazon Web Services (AWS) EC2 instances even when you’re not using them? Imagine if your virtual servers could automatically shut down during nights, weekends, or holidays, and then start up again exactly when you need them. This isn’t a futuristic dream — it’s something you can set up today to save money and make your cloud operations more efficient.</p><p>In this guide, we’ll walk you through the steps to schedule your EC2 instances to start and stop automatically. This means you won’t have to remember to turn them off, and you’ll only pay for what you actually use.</p><h3>What You’ll Need</h3><ol><li><strong>An AWS Account</strong>: If you don’t have one, you can <a href="https://aws.amazon.com/free/">sign up here</a> and take advantage of the Free Tier.</li><li><strong>Access to Code Files</strong>: Download the necessary files from this GitHub repository: <a href="https://github.com/bhargavvc/aws-projects/tree/main/start-stop-ec2">Start-Stop-EC2.</a></li></ol><p>Step 1: <strong>Launch EC2 Instances with Tags</strong></p><p>First, we’ll create four EC2 instances. We’ll use tags to organize them, which will help us control them later.</p><ol><li><strong>Create Two Instances for Development (</strong><strong>dev)</strong>:</li></ol><ul><li>Go to the EC2 dashboard and launch two new instances.</li><li>When configuring them, add a tag with the key env and the value dev.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*BXQ4gOIRBKBIA-d3.png" /></figure><p>2. <strong>Create Two Instances for Testing (</strong><strong>test)</strong>:</p><ul><li>Launch two more instances.</li><li>Add a tag with the key env and the value test.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yMen7ENBCEVO_oTn.png" /></figure><p>Now you should have four running instances — two tagged as dev and two tagged as test.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*G1Lkb-IM_6S4ounE.png" /></figure><h3>Step 2: Create a Role with Permissions</h3><p>We’ll create a role that allows our automation script to start and stop EC2 instances.</p><ol><li><strong>Open IAM Service</strong>:</li></ol><ul><li>In the AWS Management Console, search for <strong>IAM</strong> (Identity and Access Management) and open it.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QfDGK5Ga-spI8feJ.png" /></figure><p><strong>2. Create a Policy</strong>:</p><ul><li>Click on <strong>Policies</strong> in the left menu.</li><li>Click on <strong>Create Policy</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*52NdZd6_3vMz7u5c.png" /></figure><ul><li>Choose the <strong>JSON</strong> tab and paste the following code:</li></ul><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Sid&quot;: &quot;StartStopDescribeInstances&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;ec2:StartInstances&quot;,<br>                &quot;ec2:StopInstances&quot;,<br>                &quot;ec2:DescribeInstances&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;<br>        }<br>    ]<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vfrwWex0Kok1mloG.png" /></figure><ul><li>Click <strong>Next</strong>.</li><li>Give the policy a name, like start_stop_EC2.</li><li>Click <strong>Create Policy</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KA3e6iqy0icMF6au.png" /></figure><p>. Once the policy is created, click on ‘Roles’ on the Left sidebar, then click on ‘Create role’</p><p>3. Create a Role:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5ND9SIrJJVbijRUZ.png" /></figure><ul><li>Click on <strong>Roles</strong> in the left menu.</li><li>Click on <strong>Create Role</strong>.</li><li>Choose <strong>AWS Service</strong> and select <strong>Lambda</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NmVOOWMIBUqHktPA.png" /></figure><ul><li>Click <strong>Next</strong>.</li><li>On the ‘Add Permissions’ page, search for the policy name you used in step 5 then select it when it appears and click ‘Next’</li><li>On the next page, give the role a name, then scroll to the bottom of the page and click on ‘Create role’</li><li>Click <strong>Create Role</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IwGMz9UjsR0KBl7l.png" /></figure><h3>Step 3: Create a Lambda Function to Start Instances</h3><p>We’ll create a Lambda function (a small piece of code that runs in the cloud) to start our EC2 instances automatically.</p><ul><li>Make sure your dev instances are <strong>stopped</strong> before testing(before going to lambda fucntion creation)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*ZqXHczjAUE-P8RGN.png" /></figure><ol><li><strong>Open Lambda Service</strong>:</li></ol><ul><li>In the AWS Management Console, search for <strong>Lambda</strong> and open it.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tP1Nx8tBA-r9_OaP.png" /><figcaption>2</figcaption></figure><ol><li><strong>Create a New Function</strong>:</li></ol><ul><li>Click on <strong>Create Function</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-OvrKZ6n0WzEETeY.png" /></figure><ul><li>Choose <strong>Author from scratch</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MpuY4iEL0OGbOEgIVWUGYA.png" /></figure><ul><li>Give your function a name, like start_Ec2.</li><li>Under <strong>Permissions</strong>, expand <strong>Change default execution role</strong>.</li><li>Select <strong>Use an existing role</strong> and choose the role you created earlier (ec2_start_stop_role).</li><li>Click <strong>Create Function</strong>.</li></ul><p>2. Add the Code</p><p>. On the function page that appears, select the ‘Code’ tab and paste the following code in the ‘lamba_function’ code section</p><pre># Start the instances in the dev environment:<br><br>import boto3<br><br># AWS region and tag details<br>aws_region = &#39;us-east-1&#39;<br>tag_key = &#39;env&#39;<br>tag_value = &#39;dev&#39;<br><br># Create an EC2 client<br>ec2 = boto3.client(&#39;ec2&#39;, region_name=aws_region)<br><br>def lambda_handler(event, context):<br>    # Get instances with the specified tag and value<br>    instances_to_start = get_instances_with_tag(tag_key, tag_value)<br>    <br>    if instances_to_start:<br>        # Start the instances<br>        start_instances(instances_to_start)<br>        return {<br>            &#39;statusCode&#39;: 200,<br>            &#39;body&#39;: &#39;Started instances: {}&#39;.format(instances_to_start)<br>        }<br>    else:<br>        print(&#39;No instances&#39;)<br>        return {<br>            &#39;statusCode&#39;: 200,<br>            &#39;body&#39;: &#39;No instances found with tag &quot;{}&quot; and value &quot;{}&quot;.&#39;.format(tag_key, tag_value)<br>        }<br><br>def get_instances_with_tag(key, value):<br>    try:<br>        response = ec2.describe_instances(<br>            Filters=[<br>                {<br>                    &#39;Name&#39;: &#39;tag:&#39; + key,<br>                    &#39;Values&#39;: [value]<br>                }<br>            ]<br>        )<br>        <br>        instances = []<br>        for reservation in response[&#39;Reservations&#39;]:<br>            for instance in reservation[&#39;Instances&#39;]:<br>                instances.append(instance[&#39;InstanceId&#39;])<br>        <br>        print(instances)<br>        return instances<br>        <br>    except Exception as e:<br>        print(&#39;Error:&#39;, e)<br>        return []<br><br> <br>def start_instances(instance_ids):<br>    try:<br>        ec2.start_instances(InstanceIds=instance_ids)<br>        print(&#39;Starting instances:&#39;, instance_ids)<br>    except Exception as e:<br>        print(&#39;Error:&#39;, e)<br><br></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*6OG4AwGJOl8OQoSg.png" /></figure><p>This code will be used to start the instances in the ‘dev’ environment. Remember to set the region in which you are running the instances in</p><ol><li><strong>Deploy and Test the Function</strong>:</li></ol><ul><li>Click <strong>Deploy</strong> to save your changes.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*eXjjH42VrUEMuQrm.png" /></figure><ul><li>Click <strong>Test</strong> to create a test event (you can use the default settings).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*QLgfwX5ztkI02Bet.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*AMggiJP_yEJy-Hlf.png" /></figure><ul><li>Click <strong>Test</strong> again to run the function.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*Ol0DoTm7pL1JB_7R.png" /></figure><ul><li>Your dev instances should start automatically.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*wNYW-SFesenuhfmk.png" /></figure><h3>Step 4: Create a Lambda Function to Stop Instances</h3><p>Now we’ll create another Lambda function to stop the instances.</p><ol><li><strong>Repeat the Steps Above</strong>:</li><li>Create a new Lambda function named stop_dev_EC2.</li></ol><ul><li>Use the same execution role (start_stop_EC2_roles).</li><li>Replace the function code with the following:</li></ul><pre>#Stop instances in the dev environment<br><br>import boto3<br><br># AWS region and tag details<br>aws_region = &#39;us-east-1&#39;<br>tag_key = &#39;env&#39;<br>tag_value = &#39;dev&#39;<br><br># Create an EC2 client<br>ec2 = boto3.client(&#39;ec2&#39;, region_name=aws_region)<br><br>def lambda_handler(event, context):<br>    # Get instances with the specified tag and value<br>    instances_to_stop = get_instances_with_tag(tag_key, tag_value)<br>    <br>    if instances_to_stop:<br>        # Stop the instances<br>        stop_instances(instances_to_stop)<br>        return {<br>            &#39;statusCode&#39;: 200,<br>            &#39;body&#39;: &#39;Stopped instances: {}&#39;.format(instances_to_stop)<br>        }<br>    else:<br>        print(&#39;No instances&#39;)<br>        return {<br>            &#39;statusCode&#39;: 200,<br>            &#39;body&#39;: &#39;No instances found with tag &quot;{}&quot; and value &quot;{}&quot;.&#39;.format(tag_key, tag_value)<br>        }<br><br>def get_instances_with_tag(key, value):<br>    try:<br>        response = ec2.describe_instances(<br>            Filters=[<br>                {<br>                    &#39;Name&#39;: &#39;tag:&#39; + key,<br>                    &#39;Values&#39;: [value]<br>                }<br>            ]<br>        )<br>        <br>        instances = []<br>        for reservation in response[&#39;Reservations&#39;]:<br>            for instance in reservation[&#39;Instances&#39;]:<br>                instances.append(instance[&#39;InstanceId&#39;])<br>        <br>        return instances<br>        <br>    except Exception as e:<br>        print(&#39;Error:&#39;, e)<br>        return []<br><br>def stop_instances(instance_ids):<br>    try:<br>        ec2.stop_instances(InstanceIds=instance_ids)<br>        print(&#39;Stopping instances:&#39;, instance_ids)<br>    except Exception as e:<br>        print(&#39;Error:&#39;, e)</pre><ul><li><strong>Note</strong>: Again, replace &#39;us-east-1&#39; with your AWS region.</li><li><strong>Deploy and Test the Function</strong>:</li><li>Click <strong>Deploy</strong>.</li><li>Click <strong>Test</strong> to run the function.</li><li>Your dev instances should stop automatically.</li></ul><h3>Step 5: Schedule the Lambda Functions</h3><p>Now we’ll set up a schedule to run these functions automatically at specific times.</p><p>Search for ‘cloudwatch’ on the services search tab and select the service</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*A5DHI5K-f8OzMo35.png" /></figure><p>. On the Left panel, expand ‘Events’ then click on ‘Rules’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*9dDfWrXgB22crVYH.png" /></figure><ul><li>Click on <strong>Create schedule</strong>.</li><li>3. On the ‘Rules’ page that appears, click on ‘Create rule’</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*WWOKfxM1hqpnVp3D.png" /></figure><p>4. Give the Rule a name, a description and select ‘Schedule’ for the Rule type, then click on ‘Continue in EventBridge Scheduler’ at the bottom of the page</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*ScepweR5P64gPBRl.png" /></figure><p>5. on the ‘Create schedule’ page, scroll down to ‘Schedule pattern’ and set the following configurations:</p><p>Occurrence: Recurring schedule</p><p>Schedule type: Cron-based schedule</p><p>Cron expression: Input the cron for when you would like to run the function. Use <a href="https://crontab.guru/">crontab</a> to develop your cron expression. In my case, I will use 0 8 ?* 2–6 * to mean that I want the schedule to run every weekday (Mon — Fri) at 08:00 AM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*1VchmQ1E5Djz18Gc.png" /></figure><ul><li>The section at the bottom shows the next 10 triggers</li><li>6. Set the Flexible time window to ‘Off’ and scroll to the bottom of the page and click on ‘Next’</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*Bk4PaOa-UQwTlsOe.png" /></figure><ul><li>7. On the ‘Select target’ page, select ‘AWS Lambda’ as the Target API and select the function to start the EC2s from the dropdown</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/0*cDI23HIK4DKauDXC.png" /></figure><ul><li>Scroll to the bottom and click on ‘Skip review and create schedule’</li><li>On the Review and ‘create schedule’ page, scroll to the bottom and click on ‘Create schedule’</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/0*mgdOyHOOL3cQS6WV.png" /></figure><ul><li>This creates the schedule</li><li>11. Repeat steps from STEP 5 to schedule the Lambda function to stop the instances. The only thing we will be changing is the name of the schedule and the cron schedule</li><li>12. After creating the schedule, you should have both schedules on your ‘schedules’ dashboard</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*YCaLMNTufOxLNUpO.png" /></figure><ul><li>Now, your dev instances will automatically start at 8:00 AM and stop at 6:00 PM on weekdays.</li></ul><h3>Step 6: Repeat for Other Environments (Optional)</h3><p>If you want to automate your test instances as well, repeat the steps above:</p><ul><li>Create Lambda functions to start and stop instances tagged with env = test.</li><li>Adjust the code by changing tag_value = &#39;test&#39;.</li><li>Schedule them according to your needs.</li></ul><h3>Summary</h3><p>By following these steps, you’ve set up an automatic system to start and stop your EC2 instances based on a schedule. This helps you save money by ensuring you’re not paying for instances when they’re not in use.</p><h3>Let’s Connect!</h3><p>I hope this guide was helpful to you. If you have any questions or want to discuss more about cloud computing, feel free to connect with me:</p><ul><li><strong>LinkedIn</strong>: <a href="https://www.linkedin.com/in/bhargav-chitteti/">Connect with Bhrgav Chitteti</a></li><li><strong>GitHub</strong>: <a href="https://github.com/bhargavvc/aws-projects/tree/main/start-stop-ec2">Access the Code Repository</a></li></ul><p>Thank you for taking the time to read this guide. Let’s continue learning and improving together!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f6b0991b5d3c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Set Up a Flask App with Continuous Deployment Pipeline on AWS.]]></title>
            <link>https://medium.com/@bhargav.dev01/set-up-a-flask-app-with-continuous-deployment-pipeline-on-aws-1d4537dd0517?source=rss-a425b718d8d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/1d4537dd0517</guid>
            <category><![CDATA[elastic-beanstalk]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[deployment]]></category>
            <category><![CDATA[flask]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Bhargav]]></dc:creator>
            <pubDate>Wed, 27 Nov 2024 05:39:17 GMT</pubDate>
            <atom:updated>2024-11-27T09:53:40.105Z</atom:updated>
            <content:encoded><![CDATA[<h3>Set Up a Flask App with Continuous Deployment Pipeline on AWS</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M1C7ZmIEBFmzRt98EipHOQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fqA4oQsh1V_4hSXZ89nxxw.png" /></figure><p>If you’re new to AWS and want to automate the process of deploying your code, this guide is for you! We’ll walk you through setting up a <strong>continuous deployment pipeline</strong> step by step. Don’t worry — it’s beginner-friendly, and we’ll use free-tier AWS resources so it won’t cost you anything.</p><p>With this setup, your code will automatically deploy whenever you make changes to your GitHub repository. Let’s dive in!</p><h3>What Will You Learn?</h3><p>By the end of this guide, you’ll know how to:</p><ol><li><strong>Create a Deployment Target</strong>: Set up a server (using AWS Elastic Beanstalk) where your app will run.</li><li><strong>Create a Deployment Pipeline</strong>: Link your code from GitHub to the server so changes deploy automatically.</li></ol><h3>Step 1: Setting Up the Deployment Target (Where Your Code Will Run)</h3><p>A <strong>deployment target</strong> is the place where your application will run, such as a server or cloud environment. AWS Elastic Beanstalk is a great tool for beginners because it handles the underlying servers for you. This means less work managing servers and more time focusing on your app.</p><p>Here’s how to set it up:</p><ol><li><strong>Log in to AWS Console</strong>: Search for <strong>Elastic Beanstalk</strong> and open it.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/937/1*pPTHhN0rRKCD0PhJRN16mQ.png" /></figure><p><strong>1a.Create an Application</strong>:</p><ul><li>Click <strong>Create application</strong> (if you don’t see this, check the top-right corner).</li><li>Choose <strong>Web server environment</strong> (this is for web apps).</li><li>For the platform, select <strong>Python</strong>.</li><li>Under Application code, select <strong>Sample application</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QdviAYHtp8C4eYwPHueLBA.png" /><figcaption>Ignore the above warning message</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mz_XmZ83paGMMTgPTj0I2A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/965/1*37eAmjcUd_UBtNBSf8pVvw.png" /><figcaption>choose platform</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/965/1*dLtvdO2-vwgptFL2o_kFLg.png" /></figure><p>Enter an application name (e.g., “test-beanstalk-app”) and optional details like description or tags.</p><ul><li>Click <strong>Create</strong>.</li></ul><p><strong>1b. Create an Environment</strong>:</p><ul><li>We need to give service role and instance profile. So create one role with necessary permissions</li><li>I have created one. Choose Create and use role</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/964/1*Hp0F6JVeTo2poSvvEk-roA.png" /></figure><ul><li>Click Add permissions . Make sure select these.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zn-SxU4an51i7H5Cpfacjw.png" /></figure><ul><li>And create new IAM with EC2 and Beanstalk Necessary permissions role.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l6UaImv55LBQuTyaw66U1w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*r2hFz3MCwLz87DHJHqK8Tg.png" /></figure><p>Click <strong>next</strong>.</p><p>. Ignore VPC and Database Settings and Click Next</p><p>. Note : Check carefully Here</p><p>Choose root volume type as i selected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*tOZJtkrhADThjW0-VwErUQ.png" /></figure><ul><li>Choose Spot Instance in Fleet Composition</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*gqhp7Nt-QDuOHwuJSU9IIA.png" /></figure><p>Turn on this capacity rebalancing option too.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*ZSd44k5u8n-lIfBcBxtQVw.png" /></figure><ul><li>Click Next or Skip to Review</li><li>Click Submit.</li></ul><p><strong>Finish Setup</strong>:</p><ul><li>Leave all other settings as default.</li><li>Click <strong>Submit</strong> at the end. Elastic Beanstalk will set up your environment in about a minute.</li><li>Once setup is complete, you’ll see a green success message.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pkk19iGtYiIT8Pj3kHDTxw.png" /></figure><ul><li>Click the provided domain link to see your sample application (a basic webpage).</li><li>It Shows Like this</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xw8k5bIdiIJo5b7CXGoqQg.png" /></figure><p>Great! Your deployment target is ready.</p><h3>Step 2: Creating the Deployment Pipeline (Linking Your Code)</h3><p>Now, let’s set up a pipeline to automatically deploy your code from GitHub to the deployment target we just created.</p><ol><li><strong>Open AWS CodePipeline</strong>:</li></ol><ul><li>Search for <strong>CodePipeline</strong> in the AWS Console.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*Ng7Uh0khfCvrM7sfeH4Ugw.png" /></figure><ul><li>Click <strong>Create pipeline</strong>.</li></ul><ol><li><strong>Set Up the Pipeline</strong>:</li></ol><ul><li>Give your pipeline a name (e.g., “beanstalk-pipeline”).</li><li>If this is your first time, select <strong>New service role</strong>. AWS will handle permissions for you.</li><li>Click <strong>Next</strong>.</li></ul><p><strong>2. Add the Source (GitHub)</strong>:</p><ul><li>Select <strong>GitHub version 2</strong> as the source provider.</li><li>Connect your GitHub account.</li><li>Choose the repository and branch for your code.</li><li>Check the box to enable automatic updates when you push changes.</li><li>Leave the default artifact format and click <strong>Next</strong>.</li></ul><p>3. <strong>Add the Deploy Stage</strong>:</p><ul><li>Select <strong>AWS Elastic Beanstalk</strong> as the deployment provider.</li><li>Choose the application and environment you created earlier.</li><li>Click <strong>Next</strong>.</li><li><strong>Review and Create</strong>:</li><li>Review the details and click <strong>Create pipeline</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*Ki4Eei5nAd6a7srGDXCq1g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*FDeSfQ_a0kj_7gXJu_dSOw.png" /></figure><p>click next</p><p>Here choose github and complete authentication.</p><p>If you need new code to deploy . checkout to <strong>deploy branch</strong> clone this source <a href="https://github.com/bhargavvc/aws-projects.git">code</a>. skip the build stage .</p><p>Next choose branch you want deploy the code.</p><p>#<strong>Please check Configuration tab Elastic Beanstalk and check all specifed configured as this image</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IFhmZj40DVHuagRPNFrL_w.png" /></figure><p>check <strong>wsgi Path</strong> to be (your application.py and app function)</p><h3>Step 3: Testing Your Pipeline</h3><p>Now that your pipeline is ready, let’s test it:</p><ol><li>Go to your GitHub repository and make a small change to the code (e.g., change the background color or text on the page).</li><li>Commit and push your changes to the repository.</li><li>Wait a minute or two for the pipeline to deploy the changes.</li><li>Refresh the webpage (from Step 1or click beanstalk url) to see the updated version of your app.</li><li>I have done some changes will reflected in seconds to minutes…</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QEFDSuk9yabV4kOUSrI6vw.png" /></figure><h3>And That’s It!</h3><p>Congratulations! 🎉 You’ve successfully set up a <strong>continuous deployment pipeline</strong> on AWS. Now, every time you push changes to GitHub, your app will automatically update.</p><p>This is just the beginning — now you can explore more features like custom environments, build stages, and monitoring tools.</p><p>Happy coding and happy deployment! 🚀</p><p>Let me know if you’d like further tweaks!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1d4537dd0517" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hosting a Static Website on AWS S3]]></title>
            <link>https://medium.com/@bhargav.dev01/hosting-a-static-website-on-aws-s3-c41adb699ea8?source=rss-a425b718d8d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/c41adb699ea8</guid>
            <category><![CDATA[static-websites]]></category>
            <category><![CDATA[aws-s3]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Bhargav]]></dc:creator>
            <pubDate>Tue, 26 Nov 2024 10:58:10 GMT</pubDate>
            <atom:updated>2024-11-26T10:58:48.766Z</atom:updated>
            <content:encoded><![CDATA[<p>Do you want to host a static website quickly and cheaply? Maybe you have a simple site with information that doesn’t change often, and you’re looking for the easiest and most cost-effective way to get it online. AWS Simple Storage Service (S3) offers a great solution for this!</p><p>With AWS S3, you can host a static website easily and affordably. You only pay for the storage you use, and if you’re within the AWS Free Tier, you get up to 5GB of storage free for a year — plenty for a simple static site.</p><p>In this guide, we’ll walk you through the steps to host your static website on AWS S3.</p><h3>Steps to Host Your Static Website on AWS S3</h3><h3>1. Access the AWS S3 Service</h3><ul><li>Log in to your AWS Management Console.</li><li>In the search bar at the top, type <strong>“S3”</strong> and select <strong>“S3”</strong> from the results.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*_UnO60eyzQeXY-rH.png" /><figcaption>search s3 in</figcaption></figure><h3>2. Create a New S3 Bucket</h3><ul><li>Click on <strong>“Create bucket”</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*hrjrN1dczgbwlumu.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*amKWV6suXF4oqEpK.png" /></figure><h3>3. Configure Your Bucket</h3><ul><li><strong>Bucket Name</strong>: Enter a unique name for your bucket. Remember, bucket names must be unique across all of AWS.</li><li><strong>Region</strong>: Choose the AWS region closest to your audience.</li></ul><h3>4. Adjust Public Access Settings</h3><ul><li>Scroll down to <strong>“Block Public Access settings for this bucket”</strong>.</li><li><strong>Uncheck</strong> the option <strong>“Block all public access”</strong>. This allows users on the internet to access your website.</li><li>A warning will appear; acknowledge it to proceed.</li></ul><h3>5. Create the Bucket</h3><ul><li>Scroll to the bottom and click on <strong>“Create bucket”</strong>.</li></ul><h3>6. Enable Static Website Hosting</h3><ul><li>In your list of buckets, click on the one you just created.</li><li>Go to the <strong>“Properties”</strong> tab.</li><li>Scroll down to <strong>“Static website hosting”</strong> and click <strong>“Edit”</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*yGfa81_DeYPGlQha.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*Vflreo-usn9eOC0H.png" /></figure><h3>7. Configure Static Website Hosting</h3><ul><li>Select <strong>“Enable”</strong>.</li><li>Choose <strong>“Host a static website”</strong>.</li><li><strong>Index Document</strong>: Enter the name of your homepage file, usually <strong>index.html</strong>.</li><li><strong>Error Document</strong>: (Optional) Enter the name of your error page file, like <strong>error.html</strong>.</li><li>Click <strong>“Save changes”</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*cBOeqgrdyM1tPLjY.png" /></figure><h3>8. Note Your Bucket Endpoint</h3><ul><li>After enabling static website hosting, you’ll see your <strong>Bucket website endpoint</strong>. This is the URL where your website will be accessible.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*Kt5Sdypubh8McrPZ.png" /></figure><h3>9. Set Up Bucket Policy for Public Access</h3><ul><li>Navigate to the <strong>“Permissions”</strong> tab.</li><li>Scroll down to <strong>“Bucket policy”</strong> and click <strong>“Edit”</strong>.</li><li>Paste the following policy into the editor, replacing your-bucket-name with the name of your bucket:</li></ul><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Sid&quot;: &quot;StaticWebsiteHosting&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Principal&quot;: &quot;*&quot;,<br>            &quot;Action&quot;: &quot;s3:GetObject&quot;,<br>            &quot;Resource&quot;: &quot;arn:aws:s3:::your-bucket-name/*&quot;<br>        }<br>    ]<br>}</pre><ul><li>Click <strong>“Save changes”</strong>.</li></ul><h3>10. Upload Your Website Files</h3><ul><li>Go to the <strong>“Objects”</strong> tab in your bucket.</li><li>Click on <strong>“Upload”</strong>.</li><li>Add your website files (e.g., index.html, error.html).</li><li>Click <strong>“Upload”</strong> to add them to your bucket.</li></ul><h3>11. Sample HTML Files</h3><p><strong>a. </strong><strong>index.html</strong></p><pre>&lt;!DOCTYPE html&gt;<br>&lt;html&gt;<br>&lt;head&gt;<br>  &lt;meta charset=&quot;utf-8&quot;&gt;<br>  &lt;title&gt;Static Website&lt;/title&gt;<br>  &lt;style&gt;<br>    body {<br>      color: #ffffff;<br>      background-color: #3700ff;<br>      font-family: Arial, sans-serif;<br>      font-size: 14px;<br>    }<br>    h1 {<br>      font-size: 500%;<br>      font-weight: normal;<br>      margin-bottom: 0;<br>    }<br>    h2 {<br>      font-size: 200%;<br>      font-weight: normal;<br>      margin-bottom: 0;<br>    }<br>  &lt;/style&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>  &lt;div align=&quot;center&quot;&gt;<br>    &lt;h1&gt;AWS S3 Rocks!&lt;/h1&gt;<br>    &lt;h2&gt;This page is deployed on AWS S3&lt;/h2&gt;<br>    &lt;h2&gt;I am static&lt;/h2&gt;<br>  &lt;/div&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;</pre><p><strong>b. </strong><strong>error.html</strong></p><pre>&lt;!DOCTYPE html&gt;<br>&lt;html&gt;<br>&lt;head&gt;<br>  &lt;meta charset=&quot;utf-8&quot;&gt;<br>  &lt;title&gt;Error Page&lt;/title&gt;<br>  &lt;style&gt;<br>    body {<br>      color: #ffffff;<br>      background-color: #ff0000;<br>      font-family: Arial, sans-serif;<br>      font-size: 14px;<br>    }<br>    h1 {<br>      font-size: 500%;<br>      font-weight: normal;<br>      margin-bottom: 0;<br>    }<br>  &lt;/style&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>  &lt;div align=&quot;center&quot;&gt;<br>    &lt;h1&gt;ERROR 404!!! PAGE NOT FOUND.&lt;/h1&gt;<br>  &lt;/div&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;</pre><h3>12. Access Your Website</h3><ul><li>Open the <strong>Bucket website endpoint</strong> URL you noted earlier.</li><li>Your index.html page should load as the homepage.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*bClrKObpuUKIwK3f.png" /></figure><h3>13. Test the Error Page</h3><ul><li>To see the error page in action, try accessing a page that doesn’t exist or temporarily rename/delete your index.html file.</li><li>The error.html page should display.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*Jpg8vNqPqUlqn2bA.png" /></figure><h3>Congratulations!</h3><p>You’ve successfully hosted a static website on AWS S3! This method is quick, easy, and cost-effective — perfect for simple websites that don’t require server-side processing.</p><p><strong>Thank you, and have fun hosting your static websites!</strong></p><p><strong>About the Author</strong></p><p><em>Bhargav Chitteti is passionate about cloud computing and enjoys helping others learn about AWS services. Connect with him on </em><a href="https://www.linkedin.com/in/bhargav-chitteti"><em>LinkedIn</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c41adb699ea8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Visualizing Insights with AWS QuickSight]]></title>
            <link>https://medium.com/@bhargav.dev01/visualizing-insights-with-aws-quicksight-a916b94f9b59?source=rss-a425b718d8d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/a916b94f9b59</guid>
            <category><![CDATA[aws-quicksight]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Bhargav]]></dc:creator>
            <pubDate>Mon, 25 Nov 2024 15:28:08 GMT</pubDate>
            <atom:updated>2024-11-25T15:47:39.057Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*dAR8PwOQ4c2kbtyD.png" /></figure><p>AWS QuickSight helps you turn your raw data into meaningful insights through powerful visualizations. It works easily with data stored in Amazon S3, making it simple to analyze and understand. This blog explains how AWS QuickSight and S3 work together to create useful and beautiful visual reports, helping everyone — from business owners to data analysts — get the most out of their data.</p><blockquote><strong><em>Prerequisites</em></strong></blockquote><ol><li>Have an AWS account. If you don’t have one, <a href="https://aws.amazon.com/">sign up</a> here and enjoy the benefits of the <a href="https://aws.amazon.com/free/?trk=99f831a2-d162-429a-9a77-a89f6b3bd6cd&amp;sc_channel=ps&amp;ef_id=Cj0KCQjw06-oBhC6ARIsAGuzdw03qBfKD7AA8ejOZPfdI7Pg4XshpRd57Q1p-a8--id9VCM5Cjg78xEaAmJtEALw_wcB%3AG%3As&amp;s_kwcid=AL%214422%213%21645125273267%21e%21%21g%21%21aws+free+tier%2119574556890%21145779847112&amp;all-free-tier.sort-by=item.additionalFields.SortRank&amp;all-free-tier.sort-order=asc&amp;awsf.Free+Tier+Types=*all&amp;awsf.Free+Tier+Categories=*all">Free-Tier Account</a></li><li>Download the following 2 files: <a href="https://github.com/Kevin-byt/AWS-Projects/blob/main/Data%20Visualization%20with%20QuickSight%20and%20S3/listeners.csv">listeners.csv</a> and <a href="https://github.com/Kevin-byt/AWS-Projects/blob/main/Data%20Visualization%20with%20QuickSight%20and%20S3/listeners.json">listeners.json</a> (The listeners.csv is the dataset and the listeners.json is the <a href="https://docs.aws.amazon.com/quicksight/latest/user/supported-manifest-file-format.html">AWS QuickSight manifest file</a> used to configure the data source on AWS QuickSight</li></ol><blockquote><strong><em>Data Source</em></strong></blockquote><blockquote>Let’s first get our data to work with. You can get your date from <a href="https://www.kaggle.com/datasets">Kaggle</a>. They offer plenty of datasets that you can use. I chose a dataset that contains Spotify data showing artists and the number of listers they have. You can choose any of the datasets they provide. You can find the Spotify Dataset <a href="https://github.com/Kevin-byt/AWS-Projects/blob/main/Data%20Visualization%20with%20QuickSight%20and%20S3/listeners.csv">here</a>.</blockquote><p>Here is a peak to the first 10 records of the data:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*0ncDuKEerQSnc19x.png" /></figure><blockquote><strong><em>Creating an S3 Bucket and Uploading the Data</em></strong></blockquote><ol><li>Log into your AWS account</li><li>On the search box, search for “S3” and click on the first option that appears.</li><li>Give the bucket a name (the bucket name should be unique compared to all other active buckets within the region from all AWS users) and select the region to have your S3-bucket</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*1Zt-hmM922tOax9U.png" /></figure><p>3. On the Amazon S3 interface that appears, click on “Create Bucket”</p><p>4. Give the bucket a name and select the region to have your S3-bucket. All other settings can be left as default (Feel free to go through them and edit where you deem fit)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*tReL7Iprkq_f1IaM.png" /></figure><p>5. Click on “Create bucket” at the bottom of the page</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*oc6e85jfuXF_O64i.png" /></figure><p>6. The bucket should then appear in your “buckets” list</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*zli4Cz7zN3Cy7Ked.png" /></figure><p>7. Click on the bucket name</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*iSCQx1KWcfZd2Vhh.png" /></figure><p>8. Let’s have a look at the manifest file:</p><pre>{<br>   &quot;fileLocations&quot;: [<br>       {<br>           &quot;URIs&quot;: [<br>               &quot;s3://BUCKET-NAME/listeners.csv&quot;<br>           ]<br>       }<br>   ],<br>   &quot;globalUploadSettings&quot;: {<br>       &quot;format&quot;: &quot;CSV&quot;,<br>       &quot;delimiter&quot;: &quot;,&quot;,<br>       &quot;textqualifier&quot;: &quot;\&quot;&quot;,<br>       &quot;containsHeader&quot;: &quot;true&quot;<br>   }<br>}</pre><p>The manifest file is used by AWS QuickSight to get the data source. You can have a look at the description and structure of the manifest file<em> </em><a href="https://docs.aws.amazon.com/quicksight/latest/user/supported-manifest-file-format.html"><em>here</em></a>. Move to the URIs and edit it by replacing ‘BUCKET-NAME’ with the name of your bucket. Save the file</p><p>In our case, it will be as follows:</p><pre>{<br>   &quot;fileLocations&quot;: [<br>       {<br>           &quot;URIs&quot;: [<br>               &quot;s3://kev-quicksight-bucket/listeners.csv&quot;<br>           ]<br>       }<br>   ],<br>   &quot;globalUploadSettings&quot;: {<br>       &quot;format&quot;: &quot;CSV&quot;,<br>       &quot;delimiter&quot;: &quot;,&quot;,<br>       &quot;textqualifier&quot;: &quot;\&quot;&quot;,<br>       &quot;containsHeader&quot;: &quot;true&quot;<br>   }<br>}</pre><p>8. Click on “Upload” (They are 2, any can do)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*ZDsUFuBubxKLFzmQ.png" /></figure><p>9. On the upload page, Click on “Add files” then navigate your local storage to the <a href="https://github.com/Kevin-byt/AWS-Projects/blob/main/Data%20Visualization%20with%20QuickSight%20and%20S3/listeners.csv">listeners.csv</a> and <a href="https://github.com/Kevin-byt/AWS-Projects/blob/main/Data%20Visualization%20with%20QuickSight%20and%20S3/listeners.json">listeners.json</a> files that we downloaded. You may also drag and drop them onto the page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*B2A0Ex9RGQ9K2ikl.png" /></figure><p>10. Click on Upload at the bottom of the page</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*VamPWU5fcmp0lvNA.png" /></figure><p>11. The files are uploaded and are now visible on S3</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*zI-0UeqMS3r2J5YN.png" /></figure><blockquote><strong><em>Setting up AWS QuickSight</em></strong></blockquote><ol><li>On the Services search bar at the top, searck for ‘quicksight’ and select the first option that appears</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*37fHwZMChGNSXaug.png" /></figure><p>2. In the event you have not signed up for AWS QuickSight on your account, Click on ‘Sign up for QuickSight”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*O6FnKulScxQBtC-1.png" /></figure><p>3. Select the Enterprise edition to get a Trial period for 30 days (You can cancel the subscription before the 30 days are over to avoid unnecessary charges) Click Continue</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*TlVgQGG9QxkKBztp.png" /></figure><p>4. Select your Authentication method. Select the region to host AWS QuickSight, select a username and an email address that you can use for notifications</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*MYG2xj0tUtKetKQD.png" /></figure><p>5. Under “<em>Allow access and autodiscovery for these resources</em>”, Select Amazon S3 to allow access and select the bucket that you created with our data and manifest files. Then Click “Finish” at the bottom</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*4EqyxSFiAyWoy2Io.png" /></figure><p>6. Give QuickSight some time to create your account (Takes less than a minute).</p><blockquote><strong><em>Visualize your Data on AWS QuickSight</em></strong></blockquote><ol><li>Once AWS QuickSight loads, it presents a dashboard. Click on datasets, the “New dataset”</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*I1xAK06-eOrkiRzV.png" /></figure><p>2. On the Data sources, Select S3</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*DGXbM6Gv3wM3bIjm.png" /></figure><p>3. Give the Data source name (You can choose any name you wish)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*uoNANoRENj1xKack.png" /></figure><p>4. Give the url of the manifest file. To get the url of the manifest file, select “listeners.json” from your uploaded files on S3 and copy the S3 URI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*gAGLd2jmNX31g6Nv.png" /></figure><p>5. Paste the S3 URI of the manifest file then click ‘Connect’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*y3repC3LQ4PalHPC.png" /></figure><p>6. Click on “Visualize”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*WvUMQg8ed33FwO2g.png" /></figure><p>7. In some cases, AWS QuickSight may not access S3, if it fails, follow this guidelines on <a href="https://docs.aws.amazon.com/quicksight/latest/user/troubleshoot-connect-S3.html"><em>granting QuickSight access to S3</em></a></p><p>8. select a Tiled Layout then click on ‘Create’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*ZsI1ZgvYMnXGvwgz.png" /></figure><p>9. AWS QuickSight presents its workarea showing the Fields (Data Columns), 1 visual (You may choose to add more visuals) and the Visual Types for the data</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*XvYsC3n41JeW6nJe.png" /></figure><p>10. Let’s do a sample data visualization. We will create a visual of Artists against Listeners. Click on ‘Artists’ on the Fields list then click on Listeners. A visual of the data appears on the Visual space</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*JOeK2eZslcBy1tqh.png" /></figure><p>11. The data was too much, let’s do a filter, say we want to view the Artists with more than 75,000,000 listeners.</p><p>12. Click on Filter, then ‘ADD FILTER’ then select ‘Listeners’ as this is the Field we want to filter by</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*UpHs9UYoYMCJzI-w.png" /></figure><p>13. Select Listeners</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*XETiaXUkiZIpL8sV.png" /></figure><p>14. Set Aggregation to Sum, Filter condition to ‘Greater than’ and the Minimum value to ‘75000000’ (There are many more options that you can explore). Then click ‘APPLY’ at the bottom</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*9TetJfdJ6P7ht_l1.png" /></figure><p>15. The visual changes to show you the data that meets the criteria</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*KFjDfzM_9rkqXOYq.png" /></figure><p>16. You may also change the Visual type to show the data in different formats. The following are some examples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*cjyqCHysYIWZ-U47.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*3GGLw9mtvRIcGY2x.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*hbEc4-SOFafR_UEF.png" /></figure><blockquote><strong><em>Additional Practice</em></strong></blockquote><p>AWS QuickSight offers a wide range of features for data visualization and analysis. Here are some key features that you can practice and explore:</p><ol><li><strong>Data Source Connectivity</strong>: AWS QuickSight allows you to connect to various data sources, including Amazon S3, Amazon RDS, Redshift, SQL Server, and more. Practice connecting to different data sources to import data into QuickSight.</li><li><strong>Data Preparation</strong>: You can perform data cleaning, transformation, and shaping operations within QuickSight, such as filtering, pivoting, and aggregating data. Practice cleaning and preparing your data to make it suitable for visualization.</li><li><strong>Interactive Dashboards</strong>: Build interactive dashboards by combining multiple visualizations on a single canvas. Practice designing dashboards that convey insights and allow users to explore data interactively.</li><li><strong>Custom Calculations</strong>: Use QuickSight’s formula editor to create custom calculations and calculated fields. Practice creating calculated fields to derive new insights from your data.</li><li><strong>Usage and Performance Monitoring</strong>: Monitor usage and performance of your dashboards and analyses to optimize and scale your QuickSight deployment effectively.</li></ol><p>By practicing these features, you’ll gain a deeper understanding of how AWS QuickSight can help you analyze and visualize data, turning it into valuable insights for your organization.</p><p>Well that’s it for this article. We’ve seen how AWS QuickSight works smoothly with Amazon S3 to help us analyze data easily. Using the power of the cloud, AWS offers lots of tools to help us do even more with data. I encourage you to keep exploring AWS and QuickSight to find more ways they can help you in your work or projects. Lastly, a big thank you to <a href="https://www.linkedin.com/in/lucywang-/">Lucy Wang</a> from the <a href="https://www.youtube.com/@TechwithLucy">Tech with Lucy YouTube Channel</a> for the idea that got us started on this journey. It’s teamwork and ideas like these that make the AWS community so great. Together, we can keep learning and doing exciting things in the world of cloud computing.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a916b94f9b59" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>