<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Hector Santana on Medium]]></title>
        <description><![CDATA[Stories by Hector Santana on Medium]]></description>
        <link>https://medium.com/@hsantana_72806?source=rss-0c6aa2d11be7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Dic8c5R9Ln86EyvocgLIaA.jpeg</url>
            <title>Stories by Hector Santana on Medium</title>
            <link>https://medium.com/@hsantana_72806?source=rss-0c6aa2d11be7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 23:13:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@hsantana_72806/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[ Automate Your Job Search: How I Built an AWS Lambda Job Scraper in 30 Minutes]]></title>
            <link>https://awstip.com/automate-your-job-search-how-i-built-an-aws-lambda-job-scraper-in-30-minutes-4c420fbdbcd9?source=rss-0c6aa2d11be7------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c420fbdbcd9</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Hector Santana]]></dc:creator>
            <pubDate>Thu, 27 Feb 2025 17:24:39 GMT</pubDate>
            <atom:updated>2025-03-07T16:20:12.885Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>A guide to a real-world Lambda project that revolutionized my job hunt</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CyIG7CJBJ9QDXjATL39TqQ.png" /></figure><h3>Why I Built This</h3><p>As a job seeker and cloud enthusiast, I was spending hours manually refreshing job boards and wading through stale listings. I needed hands-on Lambda experience, and I needed a more efficient job search process. The solution? Build it myself.</p><p>This project wasn’t just theoretical — it was born from a real need. I created a serverless application that has saved me countless hours and provided valuable AWS experience I can discuss in interviews.</p><h3>What My Solution Does</h3><p>The Lambda function I built:</p><ul><li>✅ Pulls job listings from multiple RSS feeds every 20 minutes</li><li>✅ Saves job listings as a CSV in an S3 bucket</li><li>✅ Sends an email notification with fresh job postings</li><li>✅ Includes job source links to track where listings come from</li></ul><p>This practical solution helps me (and can help you) streamline your search and focus on real, active opportunities while gaining hands-on experience with AWS services.</p><h3>Tech Stack I Used</h3><ul><li><strong>AWS Lambda</strong> → My first serverless function implementation</li><li><strong>AWS S3</strong> → For storing job listings as CSV files</li><li><strong>AWS SES</strong> → To send myself email alerts with fresh jobs</li><li><strong>Python</strong> → To process RSS feeds efficiently</li><li><strong>EventBridge</strong> → To automate execution every 20 minutes</li></ul><h3>Step 1: Setting Up My AWS Environment</h3><p>Before deploying, I ensured I had:</p><ul><li>✅ AWS CLI configured</li><li>✅ An IAM role with permissions for Lambda, S3, and SES</li><li>✅ An S3 bucket (hs-my-jobsearch-bucket) for job data storage</li><li>✅ A verified email in AWS SES to send notifications</li></ul><h3>Step 2: Writing My Lambda Function</h3><p>Here’s a simplified version of the Python script I created:</p><pre>import json, requests, csv, boto3, xml.etree.ElementTree as ET<br>from datetime import datetime</pre><pre># AWS Services<br>s3 = boto3.client(&#39;s3&#39;)<br>ses = boto3.client(&#39;ses&#39;, region_name=&#39;us-east-2&#39;)</pre><pre># Configuration<br>RSS_FEED_URLS = [<br>    &quot;https://rss.app/feeds/YOUR_FEED_URL_1&quot;,<br>    &quot;https://rss.app/feeds/YOUR_FEED_URL_2&quot;<br>]<br>S3_BUCKET = &quot;hs-my-jobsearch-bucket&quot;<br>EMAIL_SENDER = &quot;your-email@example.com&quot;<br>EMAIL_RECIPIENT = &quot;your-email@example.com&quot;</pre><pre>def fetch_rss_jobs():<br>    jobs = []<br>    for feed_url in RSS_FEED_URLS:<br>        try:<br>            response = requests.get(feed_url, timeout=10)<br>            if response.status_code == 200:<br>                root = ET.fromstring(response.content)<br>                for item in root.findall(&quot;.//item&quot;):<br>                    try:<br>                        title = item.find(&quot;title&quot;).text or &quot;No Title&quot;<br>                        link = item.find(&quot;link&quot;).text or &quot;No Link&quot;<br>                        pub_date = item.find(&quot;pubDate&quot;).text or &quot;No Date&quot;<br>                        jobs.append([title, link, pub_date, feed_url])<br>                    except AttributeError as e:<br>                        print(f&quot;Error parsing item: {e}&quot;)<br>            else:<br>                print(f&quot;Failed to fetch {feed_url}, status code: {response.status_code}&quot;)<br>        except Exception as e:<br>            print(f&quot;Error fetching feed {feed_url}: {e}&quot;)<br>    return jobs</pre><pre>def save_to_csv(jobs):<br>    file_name = f&quot;jobs_{datetime.now().strftime(&#39;%Y-%m-%d_%H-%M-%S&#39;)}.csv&quot;<br>    local_file = f&quot;/tmp/{file_name}&quot;<br>    with open(local_file, &quot;w&quot;, newline=&quot;&quot;) as file:<br>        writer = csv.writer(file)<br>        writer.writerow([&quot;Job Title&quot;, &quot;Link&quot;, &quot;Published Date&quot;, &quot;Source&quot;])<br>        writer.writerows(jobs)<br>    s3.upload_file(local_file, S3_BUCKET, file_name)<br>    return f&quot;https://{S3_BUCKET}.s3.amazonaws.com/{file_name}&quot;</pre><pre>def send_email(jobs, s3_url):<br>    subject = &quot;🚀 New Job Listings Available!&quot;<br>    job_list_html = &quot;&quot;.join(f&quot;&lt;li&gt;&lt;strong&gt;{job[0]}&lt;/strong&gt; - &lt;a href=&#39;{job[1]}&#39; target=&#39;_blank&#39;&gt;Apply Now&lt;/a&gt; (Source: {job[3]})&lt;/li&gt;&quot; for job in jobs[:10])<br>    body_html = f&quot;&lt;html&gt;&lt;body&gt;&lt;h2&gt;New Job Listings&lt;/h2&gt;&lt;ul&gt;{job_list_html}&lt;/ul&gt;&lt;p&gt;&lt;a href=&#39;{s3_url}&#39;&gt;Download Full Job List&lt;/a&gt;&lt;/p&gt;&lt;/body&gt;&lt;/html&gt;&quot;</pre><pre>    ses.send_email(<br>        Source=EMAIL_SENDER,<br>        Destination={&quot;ToAddresses&quot;: [EMAIL_RECIPIENT]},<br>        Message={&quot;Subject&quot;: {&quot;Data&quot;: subject}, &quot;Body&quot;: {&quot;Html&quot;: {&quot;Data&quot;: body_html}}}<br>    )</pre><pre>def lambda_handler(event, context):<br>    jobs = fetch_rss_jobs()<br>    s3_url = save_to_csv(jobs)<br>    send_email(jobs, s3_url)<br>    return {&quot;statusCode&quot;: 200, &quot;body&quot;: json.dumps(&quot;Success&quot;)}</pre><h3>Step 3: Deploying My Lambda Function</h3><ol><li>I zipped and uploaded the function to AWS Lambda</li><li>Scheduled it using EventBridge (rate(20 minutes))</li><li>Tested it manually to confirm S3 uploads and email notifications worked</li></ol><p>This was my first production Lambda function, and watching it execute successfully was incredibly satisfying!</p><h3>Improvements I Made Along the Way</h3><p><strong>Deduplication</strong>: To avoid receiving duplicate job listings, I added code to read the previous CSV file from S3 and compare new listings before sending emails:</p><pre>def get_previous_jobs():<br>    try:<br>        # List objects in bucket and get most recent CSV<br>        response = s3.list_objects_v2(Bucket=S3_BUCKET, Prefix=&quot;jobs_&quot;)<br>        if response[&#39;KeyCount&#39;] &gt; 0:<br>            files = sorted(response[&#39;Contents&#39;], key=lambda x: x[&#39;LastModified&#39;], reverse=True)<br>            latest_file = files[0][&#39;Key&#39;]<br>            <br>            # Download the latest file<br>            s3.download_file(S3_BUCKET, latest_file, &quot;/tmp/previous_jobs.csv&quot;)<br>            <br>            # Read previous job links<br>            previous_jobs = set()<br>            with open(&quot;/tmp/previous_jobs.csv&quot;, &quot;r&quot;) as file:<br>                reader = csv.reader(file)<br>                next(reader)  # Skip header<br>                for row in reader:<br>                    if len(row) &gt;= 2:  # Ensure row has a link<br>                        previous_jobs.add(row[1])  # Add job link to set<br>            return previous_jobs<br>    except Exception as e:<br>        print(f&quot;Error getting previous jobs: {e}&quot;)<br>    return set()</pre><p><strong>Storage Management</strong>: I added a function to clean up old CSV files to keep costs down:</p><pre>def cleanup_old_files():<br>    try:<br>        response = s3.list_objects_v2(Bucket=S3_BUCKET, Prefix=&quot;jobs_&quot;)<br>        if response[&#39;KeyCount&#39;] &gt; 10:  # Keep only the 10 most recent files<br>            files = sorted(response[&#39;Contents&#39;], key=lambda x: x[&#39;LastModified&#39;])<br>            files_to_delete = files[:-10]  # All but the 10 newest files<br>            for file in files_to_delete:<br>                s3.delete_object(Bucket=S3_BUCKET, Key=file[&#39;Key&#39;])<br>                print(f&quot;Deleted old file: {file[&#39;Key&#39;]}&quot;)<br>    except Exception as e:<br>        print(f&quot;Error cleaning up old files: {e}&quot;)</pre><h3>What I Learned</h3><p>Building this Lambda function taught me:</p><ol><li>How to work with Lambda’s execution environment</li><li>Managing IAM permissions for a real-world application</li><li>Event-driven architecture with EventBridge</li><li>Practical usage of multiple AWS services together</li><li>Troubleshooting Lambda execution issues</li></ol><h3>The Results</h3><p>✅ A fully automated job search assistant that I use daily<br>✅ Fresh job listings from multiple sources delivered every 20 minutes<br>✅ No more wasted time on expired job postings<br>✅ Costs less than $1/month to run (often free within AWS Free Tier)<br>✅ A concrete AWS project for my portfolio and interviews</p><h3>Where to Find RSS Feeds for Jobs</h3><p>Most major job boards offer RSS feeds, including:</p><ul><li><a href="http://indeed.com/">Indeed.com</a></li><li>LinkedIn Jobs</li><li>Stack Overflow Jobs</li><li>GitHub Jobs</li><li><a href="http://remote.co/">Remote.co</a></li></ul><p>You can also create custom RSS feeds from almost any job board using services like:</p><ul><li>RSS.app</li><li>Feedity</li><li>Feed43</li></ul><h3>What’s Next?</h3><p>I’m considering expanding this project to include:</p><ul><li>A simple web UI to view and filter jobs</li><li>Machine learning to rank jobs by relevance</li><li>Integration with applicant tracking systems</li></ul><p>This was my first step in building a practical AWS serverless application. If you’re looking to get hands-on with Lambda or automate your job search, I hope this inspires you to build something similar!</p><p><em>Have questions about how I built this or suggestions for improvements? Drop a comment below or connect with me on LinkedIn.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c420fbdbcd9" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/automate-your-job-search-how-i-built-an-aws-lambda-job-scraper-in-30-minutes-4c420fbdbcd9">🚀 Automate Your Job Search: How I Built an AWS Lambda Job Scraper in 30 Minutes</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[   :       …]]></title>
            <link>https://medium.com/@hsantana_72806/-69599ad89343?source=rss-0c6aa2d11be7------2</link>
            <guid isPermaLink="false">https://medium.com/p/69599ad89343</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[job-hunting]]></category>
            <dc:creator><![CDATA[Hector Santana]]></dc:creator>
            <pubDate>Fri, 21 Feb 2025 01:53:41 GMT</pubDate>
            <atom:updated>2025-02-21T01:53:41.267Z</atom:updated>
            <content:encoded><![CDATA[<h3>𝗕𝗲𝘄𝗮𝗿𝗲 𝗼𝗳 𝗚𝗵𝗼𝘀𝘁 𝗝𝗼𝗯𝘀: 𝗛𝗼𝘄 𝘁𝗼 𝗦𝗽𝗼𝘁 𝗙𝗮𝗸𝗲 𝗼𝗿 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲 𝗝𝗼𝗯 𝗟𝗶𝘀𝘁𝗶𝗻𝗴𝘀</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ThnsQbfF3AAazb4jJWRp-Q.png" /></figure><p>If you’re job hunting, you’ve probably come across a promising listing — only to realize later it wasn’t real. Some recruiters post “𝗴𝗵𝗼𝘀𝘁 𝗷𝗼𝗯𝘀” to collect resumes, build candidate pipelines, or gauge salary expectations. Here’s how to spot and avoid them:</p><p>𝗥𝗲𝗱 𝗙𝗹𝗮𝗴𝘀 𝘁𝗼 𝗪𝗮𝘁𝗰𝗵 𝗙𝗼𝗿</p><p>🚩 𝗡𝗼 𝗠𝗲𝗻𝘁𝗶𝗼𝗻 𝗼𝗳 𝘁𝗵𝗲 𝗛𝗶𝗿𝗶𝗻𝗴 𝗖𝗼𝗺𝗽𝗮𝗻𝘆 — If the recruiter won’t disclose the client upfront, they may be fishing for candidates rather than filling a real job.</p><p>🚩 𝗚𝗲𝗻𝗲𝗿𝗶𝗰 𝗝𝗼𝗯 𝗗𝗲𝘀𝗰𝗿𝗶𝗽𝘁𝗶𝗼𝗻𝘀 — Listings that just list tech like 𝘉𝘎𝘗, 𝘖𝘚𝘗𝘍, 𝘝𝘓𝘈𝘕 but don’t mention real projects, vendors (Cisco, AWS, Juniper), or team structure often aren’t legit.</p><p>🚩 𝗥𝗲𝗾𝘂𝗲𝘀𝘁 𝗳𝗼𝗿 𝗣𝗲𝗿𝘀𝗼𝗻𝗮𝗹 𝗜𝗻𝗳𝗼 𝗧𝗼𝗼 𝗦𝗼𝗼𝗻 — If they ask for your 𝘃𝗶𝘀𝗮 𝘀𝘁𝗮𝘁𝘂𝘀, 𝗿𝗮𝘁𝗲, 𝗼𝗿 𝘀𝘁𝗮𝗿𝘁 𝗱𝗮𝘁𝗲 𝗯𝗲𝗳𝗼𝗿𝗲 𝗱𝗶𝘀𝗰𝘂𝘀𝘀𝗶𝗻𝗴 𝘁𝗵𝗲 𝗷𝗼𝗯, they may just be gathering data.</p><p>🚩 𝗡𝗼 𝗦𝗮𝗹𝗮𝗿𝘆 𝗥𝗮𝗻𝗴𝗲 — Real jobs usually list at least an estimate. Staffing agencies often avoid this to negotiate lower rates later.</p><p>🚩 𝗣𝗼𝗼𝗿 𝗥𝗲𝗰𝗿𝘂𝗶𝘁𝗲𝗿 𝗧𝗿𝗮𝗻𝘀𝗽𝗮𝗿𝗲𝗻𝗰𝘆 — If they can’t name the hiring manager or give vague answers about the role, proceed with caution.</p><p>𝗛𝗼𝘄 𝘁𝗼 𝗩𝗲𝗿𝗶𝗳𝘆 𝗕𝗲𝗳𝗼𝗿𝗲 𝗔𝗽𝗽𝗹𝘆𝗶𝗻𝗴</p><p>✔️ 𝗔𝘀𝗸 𝗱𝗶𝗿𝗲𝗰𝘁𝗹𝘆: “𝘊𝘢𝘯 𝘺𝘰𝘶 𝘤𝘰𝘯𝘧𝘪𝘳𝘮 𝘵𝘩𝘦 𝘩𝘪𝘳𝘪𝘯𝘨 𝘤𝘰𝘮𝘱𝘢𝘯𝘺 𝘢𝘯𝘥 𝘮𝘢𝘯𝘢𝘨𝘦𝘳’𝘴 𝘯𝘢𝘮𝘦?”<br>✔️ 𝗖𝗵𝗲𝗰𝗸 𝘁𝗵𝗲 𝗰𝗼𝗺𝗽𝗮𝗻𝘆’𝘀 𝘀𝗶𝘁𝗲 — If the job isn’t posted there, be skeptical.<br>✔️ 𝗦𝗲𝗮𝗿𝗰𝗵 𝗟𝗶𝗻𝗸𝗲𝗱𝗜𝗻 — See if others were hired for the same role recently.<br>✔️ 𝗟𝗼𝗼𝗸 𝘂𝗽 𝗿𝗲𝗰𝗿𝘂𝗶𝘁𝗲𝗿 𝗿𝗲𝘃𝗶𝗲𝘄𝘀 — Bad Glassdoor reviews? That’s a sign.</p><p>𝗙𝗶𝗻𝗮𝗹 𝗧𝗶𝗽</p><p>Don’t waste time on ghost jobs. 𝗣𝘂𝘀𝗵 𝗳𝗼𝗿 𝘁𝗿𝗮𝗻𝘀𝗽𝗮𝗿𝗲𝗻𝗰𝘆 and focus on real opportunities. Have you encountered fake job postings? Share your experience below! 🚀</p><p>#JobSearch #GhostJobs #CareerTips</p><p><strong>Beware of Ghost Jobs: How to Spot Fake or Pipeline Job Listings</strong></p><p>If you’re job hunting, you’ve probably come across a promising listing — only to realize later it wasn’t real. Some recruiters post <strong>“ghost jobs”</strong> to collect resumes, build candidate pipelines, or gauge salary expectations. Here’s how to spot and avoid them:</p><p><strong>Red Flags to Watch For</strong></p><p>🚩 <strong>No Mention of the Hiring Company</strong> — If the recruiter won’t disclose the client upfront, they may be fishing for candidates rather than filling a real job.</p><p>🚩 <strong>Generic Job Descriptions</strong> — Listings that just list tech like <em>BGP, OSPF, VLAN</em> but don’t mention real projects, vendors (Cisco, AWS, Juniper), or team structure often aren’t legit.</p><p>🚩 <strong>Request for Personal Info Too Soon</strong> — If they ask for your <strong>visa status, rate, or start date before discussing the job</strong>, they may just be gathering data.</p><p>🚩 <strong>No Salary Range</strong> — Real jobs usually list at least an estimate. Staffing agencies often avoid this to negotiate lower rates later.</p><p>🚩 <strong>Poor Recruiter Transparency</strong> — If they can’t name the hiring manager or give vague answers about the role, proceed with caution.</p><p><strong>How to Verify Before Applying</strong></p><p>✔️ <strong>Ask directly:</strong> <em>“Can you confirm the hiring company and manager’s name?”</em></p><p>✔️ <strong>Check the company’s site</strong> — If the job isn’t posted there, be skeptical.</p><p>✔️ <strong>Search LinkedIn</strong> — See if others were hired for the same role recently.</p><p>✔️ <strong>Look up recruiter reviews</strong> — Bad Glassdoor reviews? That’s a sign.</p><p><strong>Final Tip</strong></p><p>Don’t waste time on ghost jobs. <strong>Push for transparency</strong> and focus on real opportunities. Have you encountered fake job postings? Share your experience below! 🚀</p><p>#JobSearch #GhostJobs #CareerTips</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=69599ad89343" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying a Next.js Website on AWS with Terraform: Key Lessons]]></title>
            <link>https://medium.com/@hsantana_72806/deploying-a-next-js-website-on-aws-with-terraform-key-lessons-66f1bcd4876c?source=rss-0c6aa2d11be7------2</link>
            <guid isPermaLink="false">https://medium.com/p/66f1bcd4876c</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Hector Santana]]></dc:creator>
            <pubDate>Wed, 19 Feb 2025 17:47:19 GMT</pubDate>
            <atom:updated>2025-02-19T17:47:19.560Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>📌 Overview</strong></p><p>Ever tried deploying a Next.js site on AWS with Terraform? It’s a powerful approach, but not without challenges! Here’s what I learned while automating my setup with S3 and CloudFront. Along the way, I tackled <strong>GitHub version control challenges</strong>, Terraform state management, and AWS permission issues.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kA03-LROs86vm-VwEiA5CA.jpeg" /></figure><p><strong>🔧 Challenges &amp; Fixes</strong></p><p><strong>1️⃣ GitHub Submodule Issue</strong></p><p>• <strong>Problem:</strong> nextjs-blog/ was mistakenly tracked as a <strong>submodule</strong>, preventing commits.</p><p>• <strong>Fix:</strong> Removed submodule tracking and re-added it as a normal directory.</p><p><strong>2️⃣ Terraform S3 &amp; CloudFront Configuration</strong></p><p>• <strong>Problem:</strong> Misconfigured Terraform resources caused deployment failures.</p><p>• <strong>Fix:</strong> Used correct AWS resources for <strong>website hosting &amp; IAM policies</strong>.</p><p><strong>3️⃣ AWS S3 Access Denied (403 Error)</strong></p><p>• <strong>Problem:</strong> Website files were uploaded, but access was blocked.</p><p>• <strong>Fix:</strong> Adjusted S3 <strong>bucket policies and public access settings</strong>.</p><p><strong>4️⃣ GitHub Push Rejected (Large Terraform Files)</strong></p><p>• <strong>Problem:</strong> Terraform provider files exceeded GitHub’s <strong>100MB limit</strong>.</p><p>• <strong>Fix:</strong> Added .terraform/ to .gitignore and removed large files from history.</p><p><strong>🎯 Key Takeaways</strong></p><p>✅ <strong>Terraform automates cloud infrastructure</strong> — S3 + CloudFront deployment in minutes.</p><p>✅ <strong>GitHub version control matters</strong> — Managing submodules &amp; Terraform state correctly is crucial.</p><p>✅ <strong>AWS permissions require fine-tuning</strong> — IAM policies &amp; S3 settings make or break deployments.</p><p>✅ <strong>IaC is a game-changer</strong> — Easily scalable, repeatable infrastructure management.</p><p><strong>🚀 Next Steps</strong></p><p>• <strong>Optimize CloudFront caching</strong> for better performance.</p><p>• <strong>Automate deployments</strong> with a CI/CD pipeline.</p><p>• <strong>Expand Terraform automation</strong> to additional AWS services.</p><p>Check out the <strong>GitHub repo</strong>: <a href="https://github.com/hsantanatech/terraform-portfolio-project">Terraform + Next.js Deployment</a></p><p>Have you deployed a Next.js site with Terraform? <strong>Let’s connect and share insights!</strong> 🚀 #Terraform #AWS #Nextjs #CloudEngineering</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=66f1bcd4876c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Infrastructure as Code & Terraform: My Week 7 Cloud Engineering Journey]]></title>
            <link>https://medium.com/@hsantana_72806/from-zero-to-infrastructure-as-code-my-week-7-cloud-engineering-journey-5719a1843f36?source=rss-0c6aa2d11be7------2</link>
            <guid isPermaLink="false">https://medium.com/p/5719a1843f36</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Hector Santana]]></dc:creator>
            <pubDate>Thu, 13 Feb 2025 00:56:46 GMT</pubDate>
            <atom:updated>2025-02-13T01:18:05.565Z</atom:updated>
            <content:encoded><![CDATA[<p>The past week marked a significant milestone in my cloud engineering transition as I dove deep into Terraform. What started with basic configurations quickly evolved into deploying complex infrastructure using Infrastructure as Code (IaC). Here’s what I learned and where I’m headed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*odhxjI-F5lZTetD0_xdC_A.jpeg" /></figure><h3>Key Learnings</h3><p>My journey began with the fundamentals: installing Terraform via Homebrew and configuring AWS provider integration. I quickly progressed to implementing remote state management using S3 and DynamoDB for state locking — crucial elements for any production environment.</p><p>The real learning came from hands-on experience. I built a complete AWS VPC infrastructure, including subnets, internet gateways, and route tables. Like any learning journey, I encountered challenges: troubleshooting SSH configuration issues, managing state migrations, and resolving VPC misconfigurations. Each obstacle became a valuable learning opportunity.</p><h3>Portfolio Project Preview: Next.js on AWS</h3><p>With these fundamentals mastered, I’m embarking on an exciting portfolio project: deploying a Next.js application on AWS using Terraform. The architecture will leverage S3 for static assets, CloudFront for content delivery, and EC2 for application hosting — all orchestrated through infrastructure as code.</p><p>This project isn’t just about deployment; it’s about demonstrating how modern cloud infrastructure can be version-controlled, automated, and easily replicated. I’m particularly excited about implementing infrastructure that follows AWS best practices while maintaining simplicity and scalability.</p><h3>Looking Forward</h3><p>Week 7 has transformed my understanding of cloud infrastructure. Terraform isn’t just a tool — it’s a new way of thinking about infrastructure management. As I progress with my Next.js deployment project, I’ll be sharing more insights and learning experiences.</p><p>Are you on a similar journey or interested in infrastructure as code? Let’s connect and share experiences! Follow along as I document this cloud engineering adventure.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5719a1843f36" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I’m Making Thousands of Dollars Mining Monero on a Raspberry Pi]]></title>
            <link>https://medium.com/@hsantana_72806/how-im-making-thousands-of-dollars-mining-monero-on-a-raspberry-pi-e21c4469afb9?source=rss-0c6aa2d11be7------2</link>
            <guid isPermaLink="false">https://medium.com/p/e21c4469afb9</guid>
            <category><![CDATA[raspberry-pi]]></category>
            <category><![CDATA[crypto-mining]]></category>
            <category><![CDATA[ćrypto]]></category>
            <category><![CDATA[linux]]></category>
            <dc:creator><![CDATA[Hector Santana]]></dc:creator>
            <pubDate>Wed, 05 Feb 2025 15:47:06 GMT</pubDate>
            <atom:updated>2025-02-05T15:47:06.813Z</atom:updated>
            <content:encoded><![CDATA[<h3>💰 Only Takes 12 Years for My First $1K! 💰</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6_9ZTINqgTVsIjI9qg-9VQ.png" /></figure><h3>Disclaimer ⚠️</h3><ul><li>This guide is for educational purposes</li><li>Check local regulations on cryptocurrency mining</li><li>Electricity costs may exceed potential earnings</li><li>Overclocking may void Raspberry Pi warranty</li></ul><h3>Introduction: The Dream of Passive Income</h3><p>You’ve seen the stories — people striking it rich with crypto mining, retiring early, sipping coconut water on a beach while their rigs do all the work. Naturally, I thought, “Why not me?”</p><p>Armed with a Raspberry Pi 4 (4GB recommended), an external HDD, and an unwavering sense of optimism, I embarked on my Monero mining journey. Turns out, there’s a fine line between a profitable miner and a glorified space heater. 💪</p><h3>Prerequisites</h3><ul><li>Raspberry Pi 3 (1GB RAM minimum)</li><li>External HDD</li><li>Stable internet connection</li><li>Basic Linux command-line knowledge</li></ul><h3>Step 1: Setting Up XMRig on Raspberry Pi</h3><p>Monero (XMR) is one of the few cryptocurrencies where CPU mining remains viable — meaning even a humble Raspberry Pi can participate. Can doesn’t mean should, but hey, let’s do it anyway!</p><h3>Installing Dependencies</h3><pre># Update system<br>sudo apt update &amp;&amp; sudo apt upgrade -y</pre><pre># Install required dependencies<br>sudo apt install -y git build-essential cmake libuv1-dev libssl-dev libhwloc-dev</pre><h3>Downloading and Building XMRig</h3><pre>git clone https://github.com/xmrig/xmrig.git<br>cd xmrig<br>mkdir build &amp;&amp; cd build<br>cmake ..<br>make -j$(nproc)</pre><pre># Benchmark (Expect 5-20 H/s)<br>./xmrig --benchmark 60</pre><h3>Step 2: Overclocking &amp; Performance Optimization</h3><h3>Set CPU to Performance Mode</h3><pre>echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor</pre><h3>Careful Overclocking (Use at Own Risk)</h3><p>Edit /boot/config.txt:</p><pre>over_voltage=6<br>arm_freq=1350</pre><h3>Step 3: External HDD Configuration</h3><pre># Mount external HDD<br>sudo mkdir -p /mnt/externalhdd<br>sudo mount /dev/sda1 /mnt/externalhdd</pre><pre># Update XMRig log location in config.json<br>&quot;log-file&quot;: &quot;/mnt/externalhdd/mining_logs/xmrig.log&quot;</pre><h3>Step 4: Create Swap File</h3><pre>sudo dd if=/dev/zero of=/mnt/externalhdd/swapfile bs=1M count=4096<br>sudo chmod 600 /mnt/externalhdd/swapfile<br>sudo mkswap /mnt/externalhdd/swapfile<br>sudo swapon /mnt/externalhdd/swapfile</pre><pre># Make swap permanent in /etc/fstab<br>/mnt/externalhdd/swapfile none swap sw 0 0</pre><h3>Step 5: System Optimization</h3><pre># Remove unnecessary software<br>sudo apt purge -y libreoffice* wolfram-engine sonic-pi scratch* minecraft-pi<br>sudo apt autoremove -y &amp;&amp; sudo apt clean</pre><pre># Disable unnecessary services<br>sudo systemctl disable avahi-daemon bluetooth cups triggerhappy</pre><h3>Step 6: Background Mining Setup</h3><pre># Install screen<br>sudo apt install -y screen</pre><pre># Run XMRig in screen<br>screen -S xmrig<br>./xmrig --randomx-no-rdmsr --cpu-max-threads-hint=100</pre><pre># Detach with CTRL+A then D<br># Reattach later with: screen -r xmrig</pre><h3>Auto-Start on Boot</h3><p>Add to /etc/rc.local before exit 0:</p><pre>cd /home/pi/xmrig/build<br>./xmrig --randomx-no-rdmsr --cpu-max-threads-hint=100 &amp;</pre><h3>Realistic Earnings Projection</h3><p>Hashrate (H/s)Monero Earned Per YearUSD Value (at $150/XMR)10 H/s0.0004 XMR$0.0620 H/s0.0008 XMR$0.1250 H/s0.002 XMR$0.30</p><h3>Recommended Monero Mining Pools</h3><ul><li><a href="http://supportxmr.com/">supportxmr.com</a></li><li><a href="http://minexmr.com/">minexmr.com</a></li><li><a href="http://xmr.pool.minergate.com/">xmr.pool.minergate.com</a></li></ul><h3>Security Tips</h3><ul><li>Use strong SSH passwords</li><li>Enable SSH key authentication</li><li>Configure firewall (ufw)</li><li>Keep system updated</li><li>Monitor system temperature</li></ul><h3>Conclusion: Is It Worth It?</h3><p>✅ Great as an educational project<br>✅ Fun for hardware tinkering<br>❌ Poor financial returns<br>❌ High electricity consumption</p><p>Recommendation: Unless you’re passionate about optimization, consider purchasing Monero directly.</p><h3>Final Thoughts</h3><p>Mining Monero on a Raspberry Pi is more about the journey of learning than making money. Enjoy the process, but manage your expectations! 🚀</p><p>Have you tried mining on a Raspberry Pi? Share your experiences in the comments! 😆</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e21c4469afb9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hands-On AWS Learning: My Progress Over the Last Two Weeks]]></title>
            <link>https://medium.com/@hsantana_72806/hands-on-aws-learning-my-progress-over-the-last-two-weeks-a73765235a49?source=rss-0c6aa2d11be7------2</link>
            <guid isPermaLink="false">https://medium.com/p/a73765235a49</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[cloudformation]]></category>
            <dc:creator><![CDATA[Hector Santana]]></dc:creator>
            <pubDate>Mon, 03 Feb 2025 17:28:29 GMT</pubDate>
            <atom:updated>2025-02-03T17:28:29.499Z</atom:updated>
            <content:encoded><![CDATA[<p>Over the past two weeks, I’ve been diving deep into AWS fundamentals, gaining hands-on experience through practical exercises and projects. This journey has strengthened my understanding of core AWS services, networking concepts, security best practices, and infrastructure automation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LLycXelntwin80z979JNxQ.png" /></figure><h3>🚀 AWS Fundamentals</h3><p>Understanding AWS’s global infrastructure was the foundation of my learning. Key concepts explored:</p><ul><li><strong>AWS Global Infrastructure</strong> — Regions, availability zones, and edge locations.</li><li><strong>Amazon VPC (Virtual Private Cloud)</strong> — Subnets, route tables, and internet gateways.</li><li><strong>Public vs Private Subnets</strong> — Use cases and configurations.</li><li><strong>AWS IAM (Identity and Access Management)</strong> — Users, roles, policies, and groups.</li></ul><h3>🔥 Hands-On Exercises:</h3><ul><li><strong>VPC Design &amp; Architecture</strong> — Created a custom VPC design diagram.</li><li><strong>VPC Implementation</strong> — Built a VPC, including subnets, route tables, and security groups.</li><li><strong>EC2 and Bastion Host Setup</strong> — Launched EC2 instances and set up a bastion host.</li><li><strong>Security Groups &amp; Key Management</strong> — Managed security groups and key pairs.</li><li><strong>Project Wrap-Up</strong> — Cleaned up resources to optimize costs.</li></ul><h3>🏗 CloudFormation — Automating Infrastructure</h3><p>One of the most exciting parts of my journey was learning <strong>AWS CloudFormation</strong>, which enables infrastructure-as-code deployment.</p><h3>Key Lessons &amp; Exercises:</h3><ul><li><strong>Creating an S3 Bucket with CloudFormation</strong></li><li><strong>Building a VPC Using CloudFormation</strong></li><li><strong>Subnets, Routing, and Internet Gateway Setup</strong></li><li><strong>Deploying Bastion Hosts and EC2 Instances via CloudFormation</strong></li><li><strong>Testing Connectivity Through a Bastion Host</strong></li><li><strong>IAM with CloudFormation</strong> — Implemented IAM users, groups, and policies programmatically.</li></ul><h3>☁️ EC2 and Auto Scaling</h3><p>Scaling applications efficiently is critical in the cloud. This week, I gained hands-on experience with:</p><ul><li><strong>Elastic Load Balancer (ELB)</strong></li><li><strong>Auto Scaling Groups</strong></li><li><strong>Launch Templates and Auto Scaling Policies</strong></li><li><strong>Load Balancer &amp; Auto Scaling Exercises</strong> — Successfully set up and validated auto-scaling EC2 instances.</li></ul><h3>🛠 S3 — Object Storage and Static Website Hosting</h3><p>AWS S3 is a fundamental storage service that I explored in-depth.</p><h3>Key Activities:</h3><ul><li><strong>S3 Bucket Creation and Policy Management</strong></li><li><strong>Hosting a Static Website Using S3</strong></li><li><strong>Defining Bucket Policies and Validating Access Controls</strong></li><li><strong>Deleting S3 Stacks to Manage Resources Effectively</strong></li></ul><h3>📊 RDS — Relational Database Service</h3><p>Databases are an essential part of any cloud infrastructure. This week, I:</p><ul><li><strong>Created an RDS Instance via the AWS Console</strong></li><li><strong>Deployed an RDS Instance Using CloudFormation</strong></li><li><strong>Deleted RDS Stacks to Manage Costs</strong></li></ul><h3>🔑 Key Takeaways</h3><ul><li><strong>Networking</strong> — VPCs, subnets, routing, and security best practices are crucial for cloud architecture.</li><li><strong>Automation</strong> — CloudFormation simplifies infrastructure deployment.</li><li><strong>Security</strong> — IAM roles, policies, and security groups play a key role in cloud security.</li><li><strong>Scalability</strong> — Auto Scaling and Load Balancing ensure high availability.</li><li><strong>Storage &amp; Databases</strong> — Managing S3 and RDS efficiently optimizes cost and performance.</li></ul><h3>📌 Next Steps</h3><p>Moving forward, I’ll continue deepening my AWS expertise by exploring advanced networking, security best practices, and serverless architectures. I’ll also start working on a real-world project to solidify my learning.</p><p>Stay tuned for more updates on my cloud engineering journey! 🚀</p><p><em>If you’re also on a cloud learning journey, let’s connect and share insights.</em> 🙌</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a73765235a49" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Rollbacks to Resilience — Debugging an AWS VPC Deployment (Follow-up)]]></title>
            <link>https://medium.com/@hsantana_72806/from-rollbacks-to-resilience-debugging-an-aws-vpc-deployment-follow-up-3332d1d23211?source=rss-0c6aa2d11be7------2</link>
            <guid isPermaLink="false">https://medium.com/p/3332d1d23211</guid>
            <category><![CDATA[cloudformation]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Hector Santana]]></dc:creator>
            <pubDate>Wed, 29 Jan 2025 21:37:24 GMT</pubDate>
            <atom:updated>2025-01-29T21:37:24.589Z</atom:updated>
            <content:encoded><![CDATA[<h3>From Rollbacks to Resilience — Debugging an AWS VPC Deployment (Follow-up)</h3><p>A journey through AWS VPC deployment challenges and the lessons learned along the way.</p><p>Deploying an AWS VPC with CloudFormation seemed straightforward — until I ran into ROLLBACK_IN_PROGRESS… multiple times. What started as a simple YAML file quickly turned into a deep dive into AWS internals, dependency management, and network debugging.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_onXFLP9sb0kXyUyI3PiKQ.png" /></figure><h3>The Roadblocks</h3><p>Everything seemed correct: subnets, route tables, security groups, and instances. But as soon as I deployed, the errors started flying:</p><ul><li>“Invalid id: expecting sg-… but got subnet-…” — Security groups were incorrectly referenced.</li><li>“Instance is not in a public subnet” — The public subnet was missing an Internet Gateway route.</li><li>EC2 Instance Connect failed — No valid route to the instance.</li><li>SSH timeout on the bastion host — Security groups weren’t allowing inbound traffic from my public IP.</li><li>“No updates are to be performed” — CloudFormation didn’t detect changes, leading to deployment frustration.</li></ul><h3>What I Learned from the Debugging Process</h3><ol><li>Dependencies Matter — CloudFormation processes resources in a logical order, but explicit dependencies (e.g., DependsOn) prevent race conditions.</li></ol><p>2. Public vs. Private Subnets Must Be Explicitly Configured — Just assigning a public IP to an instance doesn’t make it public; it needs an Internet Gateway route.</p><p>3. Subnet ≠ Route Table — Referencing the wrong resource type in an EC2 instance leads to immediate failure.</p><p>4. Security Groups Must Allow the Right Traffic — An overly restrictive security group meant my bastion host wasn’t accessible. Adding an ICMP rule also helped with debugging.</p><p>5. Validate Early, Validate Often — Running ‘aws cloudformation validate-template’ before deployment would have prevented many issues.</p><h3>Final Fixes &amp; Deployment Success</h3><p>After multiple iterations, I finally got my AWS VPC working as intended. The final steps that led to a successful deployment included:</p><ul><li>Attaching an Internet Gateway and adding a 0.0.0.0/0 route to the public subnet.</li><li>Modifying the security group to allow inbound SSH and ICMP traffic from my public IP.</li><li>Verifying the subnet’s MapPublicIpOnLaunch setting to ensure auto-assigned public IPs.</li><li>Using ‘aws ec2 describe-route-tables’ to confirm the correct routes were in place.</li><li>Re-deploying the CloudFormation stack after making incremental changes to isolate and resolve errors.</li></ul><h3>Lessons Beyond AWS</h3><p>This experience reinforced some core cloud engineering principles:</p><ul><li>Patience and persistence are key when debugging infrastructure.</li><li>Understanding AWS networking beyond just creating resources helps in resolving complex issues.</li><li>Building in public and sharing lessons learned helps others facing similar challenges.</li></ul><p>This follow-up builds on my previous post about CloudFormation troubleshooting and highlights some deeper insights from the debugging process.</p><p>Have you ever battled CloudFormation rollbacks? What’s been your biggest VPC deployment challenge? Let’s share and learn together! 🚀</p><p>#AWS #CloudFormation #CloudEngineering #DevOps #VPC #LearningInPublic</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3332d1d23211" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Resilience Through Rollbacks: Lessons from AWS VPC]]></title>
            <link>https://medium.com/@hsantana_72806/building-resilience-through-rollbacks-lessons-from-aws-vpc-8c42536a1cb0?source=rss-0c6aa2d11be7------2</link>
            <guid isPermaLink="false">https://medium.com/p/8c42536a1cb0</guid>
            <category><![CDATA[vpc]]></category>
            <category><![CDATA[cloudformation]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Hector Santana]]></dc:creator>
            <pubDate>Fri, 24 Jan 2025 15:53:50 GMT</pubDate>
            <atom:updated>2025-01-24T15:53:50.799Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H4pjxejD9HJl32J1iAQy0w.jpeg" /></figure><p>As I continue transitioning into a Cloud Engineering role, I recently worked on deploying a VPC using CloudFormation. Let’s just say, it didn’t go perfectly on the first try.</p><p>From Internet Gateway attachments to route table associations, I ran into multiple ROLLBACK_IN_PROGRESS errors. Each one forced me to dig into the details of dependencies, properties like InternetGatewayId, and resource creation order.</p><p>Here’s what this experience taught me:</p><p>1️⃣ <strong>Validate your templates</strong> to catch syntax and logical errors early.</p><p>2️⃣ <strong>Dependencies are key</strong> — ensure resources like Internet Gateways are fully attached before creating routes.</p><p>3️⃣ <strong>Read the logs and stay curious</strong> — errors are just opportunities to learn.</p><p>In the end, I not only deployed the VPC successfully but also deepened my understanding of CloudFormation and VPC architecture.</p><p>For anyone else transitioning into cloud engineering, embrace the learning process — it’s worth it. What’s a recent cloud challenge that taught you something new? Let’s share and grow together!</p><p>#CloudEngineering #AWS #VPC #LearningInPublic #CloudFormation</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8c42536a1cb0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Hilarious Lesson in Cloud Networking “Design vs Create”]]></title>
            <link>https://medium.com/@hsantana_72806/a-hilarious-lesson-in-cloud-networking-design-vs-create-77fb7c083078?source=rss-0c6aa2d11be7------2</link>
            <guid isPermaLink="false">https://medium.com/p/77fb7c083078</guid>
            <dc:creator><![CDATA[Hector Santana]]></dc:creator>
            <pubDate>Tue, 21 Jan 2025 17:47:53 GMT</pubDate>
            <atom:updated>2025-01-21T17:47:53.252Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Blog Post: The Hilarious Moment I Skipped a Step in Cloud Engineering</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RXTTvBgfm8URMah3aljihA.jpeg" /></figure><p>Ever been so eager to jump into a project that you totally miss the point? That was me this week. I dove headfirst into <strong>creating</strong> a Virtual Private Cloud (VPC), thinking I was knocking it out of the park.</p><p>Subnets? Done. Route tables? Check. Gateways? Nailed it. I was crushing this — or so I thought.</p><p>Then, halfway through, it hit me: <strong>the next step wasn’t to create the VPC — it was to design it.</strong></p><p>Yep, I had enthusiastically skipped over the <em>design phase</em> and jumped straight into building. Instead of a well-thought-out, structured VPC, I ended up with what I can only describe as a “creative brainstorming session in AWS.”</p><p>And the punchline? As I wrapped up my Frankenstein VPC, I discovered a series of videos in the course titled <strong>“Designing Your VPC.”</strong> Yep, there it was — the steps I skipped in my rush to dive hands-on beforehand. The irony wasn’t lost on me.</p><p>If this experience taught me anything, it’s that:</p><p>1. <strong>Patience is a skill</strong> — and in cloud engineering, it’s a vital one.</p><p>2. The learning journey isn’t about being perfect; it’s about gaining clarity through both wins and missteps.</p><p>In the end, my “mistake” wasn’t a total loss. It gave me a chance to think critically about VPC components and laugh at myself for skipping the obvious.</p><p>To my fellow cloud learners: Have you ever been so excited to dive in that you skipped an important step? Share your funniest or most humbling moments in the comments — I’d love to hear them! 😊</p><p>Moral of the story: Sometimes, the best way to learn is to stumble a little — and always check for the next video in the series!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=77fb7c083078" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Designing a Scalable Video Sharing Platform Using Cloud Technologies]]></title>
            <link>https://medium.com/@hsantana_72806/designing-a-scalable-video-sharing-platform-using-cloud-technologies-c493b8dfb275?source=rss-0c6aa2d11be7------2</link>
            <guid isPermaLink="false">https://medium.com/p/c493b8dfb275</guid>
            <category><![CDATA[scalability]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[cloud]]></category>
            <dc:creator><![CDATA[Hector Santana]]></dc:creator>
            <pubDate>Wed, 15 Jan 2025 02:02:20 GMT</pubDate>
            <atom:updated>2025-01-15T02:02:20.631Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>🎯 Project Design: Video Sharing Platform</strong></p><p><strong>📊 Architecture Overview</strong></p><p>In today’s content-driven world, video-sharing platforms play a critical role in delivering engaging multimedia experiences. This project explores the design of a simplified video-sharing platform where users can upload videos, process them into multiple resolutions, analyze content for appropriateness, and ensure fast global delivery using modern cloud technologies. Dive into the architecture, tools, and design decisions that make this possible.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6bsx0pqrzQ1r4LgOF2q7Bg.png" /></figure><p><strong>Components:</strong></p><p>1. <strong>User Interface (Frontend):</strong></p><p>• Web application for video upload and playback.</p><p>• Framework: React.js.</p><p>2. <strong>Upload Service:</strong></p><p>• API Gateway + Lambda for handling video uploads.</p><p>• Videos stored in <strong>Amazon S3</strong> (Object Storage).</p><p>3. <strong>Metadata Handling:</strong></p><p>• <strong>Amazon DynamoDB</strong> (NoSQL) for storing metadata due to its scalability and schema flexibility.</p><p>4. <strong>Video Processing:</strong></p><p>• <strong>AWS Lambda</strong> + <strong>Elemental MediaConvert</strong> for serverless transcoding of videos into multiple resolutions (1080p, 720p, 480p).</p><p>5. <strong>Content Analysis:</strong></p><p>• AWS Rekognition for automated content analysis (inappropriate content detection).</p><p>• Rekognition flags metadata stored in <strong>DynamoDB</strong>.</p><p>6. <strong>Content Delivery Network (CDN):</strong></p><p>• <strong>Amazon CloudFront</strong> for global video delivery with caching.</p><p>• S3 bucket integrated with CloudFront for cached video delivery.</p><p>7. <strong>Monitoring and Logging:</strong></p><p>• <strong>Amazon CloudWatch</strong> for logs and monitoring.</p><p><strong>📦 Video Upload and Storage</strong></p><p>• <strong>User uploads a video</strong> via the web app.</p><p>• API Gateway receives the upload request and triggers <strong>Lambda</strong>.</p><p>• <strong>Lambda</strong> uploads the raw video to an <strong>S3 bucket</strong> for original content storage.</p><p>• Upload success is acknowledged to the user.</p><p><strong>📊 Metadata Handling</strong></p><p><strong>Why DynamoDB?</strong></p><p>• <strong>NoSQL Choice:</strong></p><p>• Fast read/write performance.</p><p>• Automatically scales for massive datasets.</p><p>• <strong>Stored Metadata:</strong></p><p>• VideoID (Primary Key)</p><p>• Title, Description, Tags</p><p>• Upload Timestamp</p><p>• Processing Status</p><p>• CDN URL for playback.</p><p><strong>🎥 Video Processing (Transcoding)</strong></p><p>1. <strong>Event Trigger:</strong></p><p>• <strong>S3 Event Trigger</strong> detects a new video upload.</p><p>• Event invokes <strong>AWS Lambda</strong> for processing.</p><p>2. <strong>Transcoding:</strong></p><p>• Lambda invokes <strong>AWS Elemental MediaConvert</strong>.</p><p>• Outputs: 1080p, 720p, 480p versions of the video stored in S3.</p><p>3. <strong>Storage:</strong></p><p>• Each transcoded version is stored in a dedicated S3 bucket.</p><p>4. <strong>Metadata Update:</strong></p><p>• Lambda updates the <strong>DynamoDB</strong> entry with transcoded video URLs.</p><p><strong>🔍 Content Analysis</strong></p><p>• After transcoding, a Lambda function calls <strong>AWS Rekognition</strong>.</p><p>• Rekognition flags content issues (violence, nudity, offensive content).</p><p>• If flagged, the <strong>DynamoDB</strong> metadata is updated with a <strong>flag</strong>.</p><p>• Videos can be reviewed by moderators.</p><p><strong>🌍 Video Delivery (CDN Strategy)</strong></p><p>• <strong>Amazon CloudFront</strong> caches the video files across global edge locations.</p><p>• <strong>URL Management:</strong> The CDN URL is stored in <strong>DynamoDB</strong> and served via the frontend.</p><p>• <strong>Caching Policy:</strong></p><p>• Cache Control Headers: Long TTL for static files, shorter TTL for dynamic metadata.</p><p>• Range Requests: Enabled for adaptive streaming.</p><p><strong>📈 Scalability &amp; Resilience Considerations</strong></p><p>1. <strong>Scalability:</strong></p><p>• <strong>S3:</strong> Highly scalable for video storage.</p><p>• <strong>DynamoDB:</strong> Serverless and horizontally scalable.</p><p>• <strong>CDN:</strong> CloudFront for global delivery and load balancing.</p><p>2. <strong>Resilience:</strong></p><p>• Multi-region <strong>S3 storage</strong>.</p><p>• <strong>Lambda</strong> has retry mechanisms for processing errors.</p><p>3. <strong>Latency Optimization:</strong></p><p>• <strong>CloudFront</strong> reduces latency via edge caching.</p><p>• <strong>Video Compression:</strong> Transcoding reduces file size.</p><p><strong>📜 System Design Summary</strong></p><p><strong>Technologies Chosen:</strong></p><p>• <strong>Frontend:</strong> React.js</p><p>• <strong>Storage:</strong> Amazon S3</p><p>• <strong>Processing:</strong> AWS Lambda, Elemental MediaConvert</p><p>• <strong>Metadata:</strong> DynamoDB (NoSQL)</p><p>• <strong>CDN:</strong> Amazon CloudFront</p><p>• <strong>Content Analysis:</strong> AWS Rekognition</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c493b8dfb275" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>