<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yojana Ghimire on Medium]]></title>
        <description><![CDATA[Stories by Yojana Ghimire on Medium]]></description>
        <link>https://medium.com/@yojanaghimire06?source=rss-038b64eb2970------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*YEJwakmULis0jhUbwGV8DQ.jpeg</url>
            <title>Stories by Yojana Ghimire on Medium</title>
            <link>https://medium.com/@yojanaghimire06?source=rss-038b64eb2970------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 09:28:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@yojanaghimire06/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[My First AWS re:Invent Experience]]></title>
            <link>https://medium.com/@yojanaghimire06/my-first-aws-re-invent-experience-ae475ab0a973?source=rss-038b64eb2970------2</link>
            <guid isPermaLink="false">https://medium.com/p/ae475ab0a973</guid>
            <category><![CDATA[aws-reinvent-2025]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Yojana Ghimire]]></dc:creator>
            <pubDate>Tue, 30 Dec 2025 18:01:32 GMT</pubDate>
            <atom:updated>2026-01-22T06:39:43.746Z</atom:updated>
            <content:encoded><![CDATA[<p>I still remember exactly where I was.</p><p>I was studying for my board exams. The pressure was at its peak .Brain was tired. Dreams loud.</p><p>Then an email arrived.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JftEZMWy1FOZYXJk4Sgx6A.png" /></figure><p>I read it again. And again.</p><p>I had received All Builders Welcome Grant for AWS re:Invent 2025 that covers all my travel, accommodation and conference ticket.</p><p>For a moment, I froze. I didn’t scream. I didn’t tell anyone.<br> I just sat there thinking, <em>Is this real?</em></p><p>A 3rd-year computer engineering girlie from a small town is going to attend the <strong>biggest cloud conference in the world</strong>.</p><p>Yeah. I was on cloud nine.</p><p>I was lucky enough because I built my luck.</p><p>And, It was also my <strong>first international trip</strong>.</p><p><strong>Every flight on the way to Las Vegas</strong>, every moment felt surreal. The fact that I was there <em>because someone believed I deserved to be there</em>.</p><p>and I kept reminding myself: <em>You belong here.</em></p><p>The <strong>All Builders Welcome (ABW) Grant</strong> doesn’t just open doors but it <strong>changes lives</strong>.</p><p>Many of us would only watch re:Invent through screens if it was not ABW Grant. Because of it, I was there badge around my neck, notebook in hand, learning , making new connections and friendships.</p><p>In this blog, I’ll be sharing my day to day journey in the most detailed way I can.</p><h3>Day 0</h3><p>Day 0 started with the <strong>ABW Welcome Party at Area 15</strong>.</p><p>This welcome party was exclusively for the ABW Grantee and the alumni advisors. Music. Food. Conversations flowing. It was overall break the ice before starting the re:Invent. ABW Grant Team welcomed us , sharing their personal experience and insights, we were assured that our selection was not by chance, it’s because we deserved to be there and it was meant for us. As someone coming from underrepresented background, being valued for who I’m , to feel seen, I was in an awe!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*ufzpcg_Nyo-o_eLLLssi1A.jpeg" /></figure><h3>Day 1- Sessions, Swags</h3><p>I started my day 1 by grabbing my ABW swags (yes, I was super excited ). Then, I attended a Portfolio Workshop for ABW grantee where I learned to build my own portfolio using Amazon Q. My second session was actually a leadership panel where I got to hear some amazing inspiring stories of great people which really motivated me and got some genuine advices from the panelists . I participated in a challenge near ceasur forum where I got the cutest dog toy and later in the evening I attended AMD cocktail party where I got to network with AWSome people.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*N8qlaLMTWDUCvntXAapULg.jpeg" /></figure><h3>Day 2 — Expo Hall , Fragrance Lab, APJC Mixer &amp; Cloud Captain Dinner</h3><p>I didn’t have my sessions on day 2 so I decided to explore and try out all the cool stuff in the Venetian and later go to the Expo. I went to House of Kiro- it was giving all spooky vibes and super cool, received a swag there.</p><p>Just beside House of Kiro, there was a Fragrance Lab. The queue was crazyyy long , but I was so curious about the lab and overall experience so I joined the queue. You could create a custom fragrance generated by Amazon Nova in Amazon Bedrock. I was handed a headphone and had a short conversation, 3 keywords were selected. Mine was Bold, attractive and countryside. Once I smelt the fragrance, I totally loved it. It was definitely my taste.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0yU6CVaWkZWxYPphc54Ccg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R-FQXxiVbzQ0iSOU_HJETQ.jpeg" /></figure><p>I was quite tired with all the walking and exploring so I decided to rest for a while and wait for the expo hall to open. So, I sat on the floor outside the expo and was having my coffee and chilling, and someone joined me there, we ended up talking for 45 minutes straight , it was such a deep and fun conversation, as we talked I got to know he was a sales manager at IBM. Those 45 minutes were one of the most knowledgable and inspiring, I learned so much about life, career and everything. And that’s one of the best thing about re:Invent , you get to meet and vibe with one of the coolest people out there so randomly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eECo2u0pN8bbdWzvHw4adg.jpeg" /></figure><p>And finally expo hall was open, I bid goodbye to my friend and went ahead to explore. The expo was sooo HUGE (I almost got lost, Literally). The products, idea, conversations, swags and networking that I got to experience is something that I only saw it from far before. One of my personal favourite stalls were GitHub, MongoDB and, Databricks and komodor. They not only had the best swags but the conversations I had with them was also one of the best. People were best. I made such a genuine connection with people from komodor that we still talk about things even after the months of re:Invent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DE_eLVjx4xeLGZO1zl8DEA.jpeg" /></figure><p>Since I had to attend APJC Mixer and cloud club captain mixer dinner so I decided to leave. As I reached at Buddy’s V: CB’s, AWS Heroes, User Group Leaders and Cloud Captains from across the world were there representing their country. It was soo cool most of them dressed up in a way that represented their country and community. Well, I couldn’t prepare myself for the dress so I just went there with the Flag of my country Nepal. And then comes a pinch me moment, I got to meet Jeff Barr, VP of AWS. I was absolutely thrilled.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D4NBSqZ-hyIr-1QNmYishw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*-OujmlGbA-wMIIKZSD8ZmA.jpeg" /></figure><p>Since I had cloud club captain dinner at the same place so I just strolled around and rest for a while. After an hour, I went back and there my beloved community managers and my fellow captains welcomed me warmly. Being surrounded by such passion, love and talent reminded me why I chose tech, community and this path in the first place. It felt like I truly belonged there, It felt like a family.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zX76B5KdTEdmRHb-omWwoQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jid9JWdZbmftrRKOIV5r1w.jpeg" /></figure><h3>Day 3 — J.P.Morgan Presents ACQUIRED, Leaders Fireside ,Certification lounge, Cloud Captain challenge</h3><p>I went to certification lounge because I was AWS Certified. One of the best coffee at re:Invent was certainly at the Certification lounge , I got my Certified t-shirt as well. After that, I attended the <strong>Leaders Fireside Panel where Jenn Bergstrom, Prabhu, and Koki shared </strong>how <strong>community can shape your certification journey</strong>, your confidence, and your career. Since I’m an AWS Cloud Club Captain actively working on a community so it was really beneficial for me. I had some major questions and was curious to learn more. And I got all my answers and I was so fascinated by the ‘Golden Jacket’ that I got my new goal and motivation as well. Then I went to attend my another session which was J.P.Morgan Presents ACQUIRED where Max Neukirchen, Global Co-Head, J.P. Morgan Payments, Greg Peters, Co-CEO, Netflix, Aravind Srinivas, Co-founder and CEO, Perplexity, and Matt Garman, AWS CEO was present for a special session focused on AI and the future of cloud computing, digital transformation in the era of agentic AI, how to maintain innovation at scale, and the role AWS plays in helping companies emerge as AI leaders and innovators. It was one of the thought provoking session . Later, we captains were assigned to do a fun challenge to win some cool swags. So, I was just excited to do it and won the voucher to get the swags.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*ld4jIwqNBOeK7ORig_DL2A.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6zP2XBgQdlEDN3dw3ELm8Q.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*K0TbEISd6h3TpD-9ejOOxQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f5HR9jA2lqg_2Vjk_r37nA.jpeg" /></figure><h3>Day 4 — Dr.Werner Vogels , AWS Research Session , re:Play</h3><p>It was the last day and what a way to end a day with werner Vogel’s keynote. I’ve been a fan girl of Werner Vogel’s for a long time and being able to attend his keynote in person was a dream come true for me. He talked about Renaissance Developer and guess what?? This was his last re:Invent, which made me realize how lucky I have been to attend his last keynote at re:Invent. After the keynote, I went to Ceasur’s Forum for my AWS Research Session. There was a team of AWS waiting for me, I was asked a lot of interesting questions about AWS and my experience and seeing how deeply AWS cares bout customers, quality and its core values was truly amazing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yNcWecG2bqaQUokYzWBpKg.jpeg" /></figure><p>Now that all the learning was done for the final day, it was time for the re:Play, official after party of re:Invent. It’s been organized as a high-energy networking event with lots of fun games and food where attendees could celebrate their achievements and network with each other. It is considered as “the ultimate party in tech”, which is absolutely true. With all the games, fun and concert it gives festival vibe.</p><p>I returned home with new ideas, new friendships, new memories and whole new level of confidence and motivation.</p><p>AWS re:Invent wasn’t just a conference for me. Thanks to <strong>All Builders Welcome (ABW) Grant</strong> it shaped my journey and gave me new direction in my career. So here are my top 5 key points:</p><ul><li>re:Invent is HUGE with 60,000 attendees from across the world and you have to know that you won’t be able to take 100% out of it, so being mindful about your time, prioritizing accordingly and knowing what you actually looking for would allow you to take most out of it.</li><li>Your network will increase rapidly, I have met so many people through the ABW program, during sessions, in the queue, expo, lunch and dinner and even just when walking around. These organic connections are incredibly valuable to me especially at such an early stage of career.</li><li>You’ll learn to get out of your comfort zone and learn a lot about yourself as well.</li><li>By truly being yourself will allow you to make true genuine connections , I’ve always had hard time finding my people and having genuine connection before but not trying to fit in and just by being myself , I found the people I truly belong with.</li><li>Last but not the least, ‘The SWAG’. You get the most AWSome swags at re:Invent. There’s so many I literally had to leave some of the swags in my hotel room because there was no place to keep them all, all my bags and suitcase were full. So, if you really love Swags then I would suggest you to take an extra bag for the swags.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m8k_vRS9LQ3ZzS64eyrYGA.jpeg" /></figure><p>I didn’t intend to write this much but got carried away. Thank you so much for taking the time to read my first re:Invent experience as an ABW recipient! If you have any questions, suggestions or just wanna say hi, my DM’s are always open: <a href="https://www.linkedin.com/in/yojanaghimire/">https://www.linkedin.com/in/yojanaghimire/</a></p><p>Until next time!</p><p>~Yojana Ghimire</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ae475ab0a973" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Becoming an AWS Cloud Club Captain: My Journey and Experience]]></title>
            <link>https://medium.com/@yojanaghimire06/building-a-serverless-marvel-data-ingestion-pipeline-2ee0bccfa2d5?source=rss-038b64eb2970------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ee0bccfa2d5</guid>
            <category><![CDATA[cloud-club]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[community]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-cloud-club]]></category>
            <dc:creator><![CDATA[Yojana Ghimire]]></dc:creator>
            <pubDate>Mon, 29 Dec 2025 16:00:00 GMT</pubDate>
            <atom:updated>2026-01-17T10:47:47.295Z</atom:updated>
            <content:encoded><![CDATA[<p>Becoming an AWS Cloud Club Captain: My Journey and Experience</p><p>Before I share my experience, let me be very clear about one thing: AWS Cloud Clubs are all about community, not promotion or profit. The title of Captain gives you responsibility and ‘With great power comes great responsibility’ and this responsibility taught me more about leadership than anything else in my life.</p><p>I first got to know about AWS Cloud Clubs through a local community event . It caught my eyes instantly , I was curious to know about it more but hesitated at the same time. Could someone like me really make an impact on a global platform? And despite the doubt, I applied!<br><br>The application had two main parts:<br> 1. Essay Questions — I wrote genuinely about my leadership experiences, AWS knowledge, and my intent to help students. My tip: be genuine. Don’t overthink it. Just write your story.<br> 2. Video Submission — I had to make a video explaining an AWS service and I made a video of explaining AWS Lambda. Here’s the link of the video: <a href="https://youtu.be/nlNzyme1rtc?si=nZpz5UZg0IxuxAWA">https://youtu.be/nlNzyme1rtc?si=nZpz5UZg0IxuxAWA</a></p><p>I wasn’t sure if I would get selected, but I knew I had to try. Late one night, I opened my email . There it is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yi41QUw-cLkNw0h7dFeBLQ.png" /><figcaption>AWS Cloud Club Captain Yojana Ghimire</figcaption></figure><p>It felt surreal!<br>And my journey began.<br>Being an AWS Cloud Captain is not just about leading a club but also taking on the responsibility of making AWS accessible to students and helping them. Captains organize events, sessions, webinars, bootcamps, and so much more to support students! It helps you become a professional leader as well as sharpen your technical skills , networking with so many AWSome people, and you also get direct guidance from the AWS Advocacy Team. Not only that, you get AWS perks as well like certification vouchers, Swags, invitations to monthly calls, and more!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1017/1*frrNWObyvGV4XaHz6mUgbQ.jpeg" /></figure><p>I built first AWS Cloud Club in our college- AWS Cloud Club IOE Pashchimanchal Campus. If you haven’t joined our community on meetup<br>Throughout the journey, my Community Managers and fellow AWS Cloud Clubs were always there to guide me and that’s the best thing about this community.</p><p>And then comes the historic moment, because of my dedication and continuous work, I got the opportunity to organize Pokhara’s first-ever AWS Student Community Day. Seeing students walk in with excitement, breaking the barriers and making cloud accessible to students and seeing students genuinely being curious about it , It was AWSome feeling. It was also the biggest cloud event Pokhara has ever seen. From this program, I learned that despite all the odds, stress, and difficulties, if you have people who support and help you, you can achieve great things.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JZLY2ImQEJU7vLPsw5T0Bg.jpeg" /><figcaption>AWS Student Community Day Pokhara 2025</figcaption></figure><p>What I didn’t expect was how far this journey would go.<br> Through consistent effort and dedication, I landed a global opportunity to attend AWS re:Invent 2025 in Las Vegas fully funded by AWS representing Nepal becoming the youngest person from Nepal to represent my country at AWS re:Invent 2025.<br>It was overwhelming to think about the version of me who once doubted if she belonged in cloud at all and now standing among global innovators. I couldn’t be more greatful.</p><p>Here’s me with the Jeff Bar, VP of AWS at re:invent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NCmCJ3KzuvvefkfM" /></figure><p>Here’s me with my community managers and fellow captains at re:invent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RDbU8m3FgeacrHLe" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SDspKnq1ZaoCFh1X" /></figure><p>If you ask me, becoming an AWS Cloud Club Captain was certainly one of the best things that happened to me in 2025.</p><p>Many students ask me, “ How can I become an AWS Cloud Captain?”. My only advice to them is to <strong>be honest</strong> and <strong>show the genuine interest to help the community</strong>.</p><p>And If this story resonates with you, then you don’t need to wait until you feel “ready”.This program is not about being a perfect leader , it values people who cares about community and if you have that, this role is for you!</p><p>Join our AWS Cloud Club to learn about AWS and get some AWSome swags: <a href="https://www.meetup.com/aws-cloud-club-at-institute-of-engineering-pashchimanchal/">https://www.meetup.com/aws-cloud-club-at-institute-of-engineering-pashchimanchal/</a></p><p>If you have any questions, ideas or just want to connect , here’s my LinkedIn: <a href="https://www.linkedin.com/in/yojanaghimire/">https://www.linkedin.com/in/yojanaghimire/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ee0bccfa2d5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Serverless Marvel Data Ingestion Pipeline]]></title>
            <link>https://medium.com/@yojanaghimire06/protect-your-data-and-privacy-tips-and-tricks-2025-6db6074837ba?source=rss-038b64eb2970------2</link>
            <guid isPermaLink="false">https://medium.com/p/6db6074837ba</guid>
            <category><![CDATA[data-and-ai]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-privacy-and-security]]></category>
            <category><![CDATA[data-piracy]]></category>
            <dc:creator><![CDATA[Yojana Ghimire]]></dc:creator>
            <pubDate>Tue, 04 Mar 2025 14:18:39 GMT</pubDate>
            <atom:updated>2026-01-19T07:20:02.453Z</atom:updated>
            <content:encoded><![CDATA[<p>Being someone that is actively transitioning deeper into data engineering, I wanted a project that is Realistic, Cloud-native and slightly fun.</p><p>Hence, my Marvel Data Ingestion Pipeline project was built which is a fully serverless pipeline that extracts Marvel movie data from Wikipedia, processes it through AWS Lambda, and then stores it in Amazon S3 for further analytics.</p><p>No need to manage servers. No manual runs. Only clean data flowing through cloud.</p><p>Why Marvel (and Not Another CSV File)?</p><p>Let’s be real we’ve all analyzed Titanic, Netflix, or some random Kaggle dataset.</p><p>Marvel movies include:<br>-Publicly documented<br>-Rich in structured + semi-structured data</p><p>-Familiar enough to focus on engineering, not guessing column meanings</p><p>Moreover, Wikipedia data is messy and real data is messy.</p><p>The Architecture<br>Here’s what the pipeline does:</p><p>1.Pull Marvel movie data from Wikipedia API</p><p>2.Does cleaning and processing of data by using AWS Lambda.</p><p>3.Stores structured clean data in Amazon S3</p><p>That’s all.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tz4swjxPejhPTvAhcCahIw.png" /></figure><p>Step 1: EXTRACTING DATA FROM Wikipedia</p><p>By using Python, I:</p><p>-Made API requests to Wikipedia APIs<br>-Retrieved raw JSON data<br>-Filter data exclusively related to Marvel movies</p><p>At this level, the data was:</p><p>-Messy<br>-Inconsistent<br>-Unformatted</p><p>-Clearly not ready for further analysis.</p><p>That is exactly what you want when you are a data engineer.</p><p>Step 2: Enter AWS Lambda — My Favorite Type of Server</p><p>Rather than performing the scripts on the local machine or dealing with the infrastructure, I chose to use AWS Lambda.<br>Why?</p><ul><li>No need to manage servers because AWS manages it for us.</li><li>Auto scaling</li><li>Only pay for running the code</li><li>Ideal for ETL tasks</li></ul><p>The Lambda function dealt with:</p><ul><li>Fetching data from Wikipedia</li><li>Cleaning and structuring it</li><li>Preparing it for storage</li></ul><p>This process helped me to deeply understand event-driven computing and a core concept in modern data platforms.</p><p>Step 3: Transforming data<br>I transformed raw messy data into clean data.</p><p>Within the lambda function, I:</p><ul><li>Eliminated unnecessary fields</li><li>Normalized movie titles</li><li>Standardized box office values</li><li>Structured everything into clean JSON format</li></ul><p>It is here that you stop to be someone who just collects data and start becoming someone who <em>builds data systems</em>.</p><p>Step 4: Loading into data lake, Amazon S3.</p><p>And once the data was fully cleaned , I loaded the data into Amazon S3.</p><p>I chose to use S3 because it is:</p><ul><li>Scalable</li><li>Durable</li><li>Industry standard for data lakes</li><li>Integrates seamlessly with analytics tools</li></ul><p>By this time, data was:</p><ul><li>Pipeline-ready</li><li>Query-ready</li><li>Dashboard-ready</li></ul><pre>from bs4 import BeautifulSoup<br>from io import StringIO<br>import pandas as pd<br>import requests<br>import re<br>import boto3<br>from bs4 import BeautifulSoup<br>from io import StringIO<br><br># Define your OMDB API Key and S3 bucket name directly in the code<br>OMDB_API_KEY = &quot;&quot;  # Replace with your actual OMDB API key<br>S3_BUCKET_NAME = &quot;&quot;  # Replace with your actual S3 bucket name<br><br># Step 1: Scrape Marvel Movie Titles<br>def scrape_marvel_movies():<br>    url = &quot;https://en.wikipedia.org/wiki/List_of_Marvel_Cinematic_Universe_films&quot;<br>    response = requests.get(url)<br>    soup = BeautifulSoup(response.content, &#39;html.parser&#39;)<br>    tables = soup.find_all(&#39;table&#39;, class_=&#39;wikitable&#39;)<br>    <br>    all_data = []<br>    for i, table in enumerate(tables[:7]):  # Phases 1-6 tables<br>        headers = [header.text.strip() for header in table.find_all(&#39;th&#39;)]<br>        rows = table.find_all(&#39;tr&#39;)<br>        data = []<br>        columns = [&#39;film&#39;, &#39;release_date&#39;, &#39;director&#39;, &#39;writer&#39;, &#39;producer&#39;, &quot;status&quot;]<br>        <br>        for row in rows:<br>            cols = row.find_all([&#39;td&#39;, &#39;th&#39;])<br>            cols = [ele.text.strip() for ele in cols]<br>            if len(cols) &lt; len(columns):<br>                cols.extend([None] * (len(columns) - len(cols)))<br>            elif len(cols) &gt; len(columns):<br>                cols = cols[:len(columns)]<br>            data.append(cols)<br>        <br>        df = pd.DataFrame(data[1:], columns=columns)  <br>        df[&#39;phase&#39;] = f&quot;Phase {i+1}&quot;<br>        all_data.append(df)<br>    <br>    return pd.concat(all_data, ignore_index=True)<br><br><br># Step 2: Clean Movie Data<br>def clean_movie_data(movies_df):<br>    movies_df[&#39;producer&#39;] = movies_df[&#39;producer&#39;].fillna(method=&#39;ffill&#39;)<br>    movies_df[&#39;status&#39;] = movies_df[&#39;status&#39;].fillna(method=&#39;ffill&#39;)<br>    <br>    def remove_references(text):<br>        return re.sub(r&#39;\s*\[\s*\d+\s*\]&#39;, &#39;&#39;, text)<br>    <br>    movies_df_cleaned = movies_df.applymap(lambda cell: remove_references(cell) if isinstance(cell, str) else cell)<br>    <br>    movies_df_cleaned[&#39;release_date&#39;] = pd.to_datetime(<br>        movies_df_cleaned[&#39;release_date&#39;].str.extract(r&#39;\((.*?)\)&#39;)[0], errors=&#39;coerce&#39;<br>    )<br>    movies_df_cleaned[&#39;release_date&#39;] = movies_df_cleaned[&#39;release_date&#39;].dt.strftime(&#39;%Y-%m-%d&#39;)<br>    <br>    return movies_df_cleaned<br><br><br># Step 3 : Scrape Character Data<br>def fetch_omdb_data(film_name):<br>    url = f&#39;http://www.omdbapi.com/?t={film_name}&amp;apikey={OMDB_API_KEY}&#39;<br>    response = requests.get(url)<br>    if response.status_code == 200:<br>        return response.json()<br>    else:<br>        return {&quot;Title&quot;: film_name, &quot;Error&quot;: &quot;Data not found&quot;}<br><br># Step 4: Scrape Characters Data<br>def scrape_characters_data():<br>    url = &quot;https://en.wikipedia.org/wiki/List_of_Marvel_Cinematic_Universe_films&quot;<br>    response = requests.get(url)<br>    soup = BeautifulSoup(response.content, &#39;html.parser&#39;)<br>    table = None<br>    <br>    for tbl in soup.find_all(&#39;table&#39;, class_=&#39;wikitable&#39;):<br>        caption = tbl.find(&#39;caption&#39;)<br>        if caption and &quot;Recurring cast and characters of Marvel Cinematic Universe films&quot; in caption.get_text():<br>            table = tbl<br>            break<br><br>    if table:<br>        headers = [th.get_text(strip=True) for th in table.find_all(&#39;tr&#39;)[0].find_all(&#39;th&#39;)]<br>        rows = []<br>        <br>        for tr in table.find_all(&#39;tr&#39;)[1:]:<br>            row = []<br>            for td in tr.find_all([&#39;th&#39;, &#39;td&#39;]):<br>                colspan = td.get(&#39;colspan&#39;)<br>                if colspan:<br>                    colspan = int(colspan)<br>                    row.extend([td.get_text(separator=&quot; &quot;, strip=True)] * colspan)<br>                else:<br>                    row.append(td.get_text(separator=&quot; &quot;, strip=True))<br><br>            while len(row) &lt; len(headers):<br>                row.append(None)<br><br>            if len(row) &gt; len(headers):<br>                row = row[:len(headers)] <br>            <br>            rows.append(row)<br><br>        return pd.DataFrame(rows, columns=headers)<br>    else:<br>        print(&quot;Table with specified caption not found.&quot;)<br>        return pd.DataFrame()<br><br><br><br><br><br>def fetch_omdb_data(film_name):<br>    url = f&#39;http://www.omdbapi.com/?t={film_name}&amp;apikey={OMDB_API_KEY}&#39;<br>    response = requests.get(url)<br>    if response.status_code == 200:<br>        return response.json()<br>    else:<br>        return {&quot;Title&quot;: film_name, &quot;Error&quot;: &quot;Data not found&quot;}<br><br><br># Step 5: Upload DataFrames to S3 with specific folder paths<br>def upload_to_s3(df, file_name):<br>    s3 = boto3.client(&#39;s3&#39;)<br>    csv_buffer = StringIO()<br>    df.to_csv(csv_buffer, index=False)<br>    s3.put_object(Bucket=S3_BUCKET_NAME, Key=f&quot;{file_name}&quot;, Body=csv_buffer.getvalue())<br><br># Lambda handler function to call each step<br>def lambda_handler(event, context):<br>    # Step 1: Scrape movie titles from Wikipedia<br>    movies_df = scrape_marvel_movies()<br>    <br>    # Step 2: Clean the scraped movie data<br>    movies_df_cleaned = clean_movie_data(movies_df)<br>    <br>    # Step 3: Fetch additional movie data from OMDB API<br>    omdb_results = []<br>    for film in movies_df_cleaned[&#39;film&#39;]:<br>        omdb_data = fetch_omdb_data(film)<br>        omdb_results.append(omdb_data)<br>    <br>    # Convert OMDB results to DataFrame<br>    omdb_df = pd.DataFrame(omdb_results)<br><br>    # Step 4: Scrape characters data<br>    characters_df = scrape_characters_data()<br><br>    # Upload DataFrames to S3 with specified folder paths<br>    upload_to_s3(movies_df_cleaned, &#39;movies.csv&#39;)<br>    upload_to_s3(omdb_df, &#39;omdb.csv&#39;)<br>    upload_to_s3(characters_df, &#39;characters.csv&#39;)<br>    <br>    return {<br>        &#39;statusCode&#39;: 200,<br>        &#39;body&#39;: &#39;Data uploaded successfully!&#39;<br>    }</pre><p>Further, I used PowerBI to analyze data and got the meaningful insights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wCrt1OxTBhlLfgf9zV3o4Q.png" /></figure><p>Tools That Powered This Pipeline</p><ul><li><strong>Python</strong> — Data extraction &amp; transformation</li><li><strong>Wikipedia API</strong> — Data source</li><li><strong>AWS Lambda</strong> — Serverless compute</li><li><strong>Amazon S3</strong> — Cloud storage</li><li><strong>AWS IAM</strong> — Secure</li></ul><p>From this project , I learned :</p><ul><li>How real world data pipelines are designed</li><li>Why serverless architecture changes the game</li><li>How cloud services work with each other</li><li>Why clean data matters more than big data</li></ul><p>To build impressive systems, you don’t always need complex tools, you need to have <strong>clarity, vision and a good architecture</strong>.</p><p>So, if you’re learning data engineering then you should start building projects like this. From real projects like this, you’ll be able to learn more .</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6db6074837ba" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Weather Data Pipeline for Nepal Using Python]]></title>
            <link>https://medium.com/@yojanaghimire06/data-pipeline-for-nepals-weather-data-using-python-70a1e76e9fd3?source=rss-038b64eb2970------2</link>
            <guid isPermaLink="false">https://medium.com/p/70a1e76e9fd3</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-warehouse]]></category>
            <category><![CDATA[data-pipeline]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Yojana Ghimire]]></dc:creator>
            <pubDate>Sat, 18 Jan 2025 03:41:59 GMT</pubDate>
            <atom:updated>2026-01-19T07:13:01.451Z</atom:updated>
            <content:encoded><![CDATA[<h3>Serverless Weather Data Pipeline for Nepal Using AWS Lambda &amp; S3 (Step-by-Step)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9Qox_Qdb9ifHffVUgHgYaQ.png" /></figure><p>Let’s build a pipeline, No, I’m not talking about the one that carries water to your basin, I am talking about a data pipeline that carries your data, hehe!</p><p>So, What is a Data Pipeline?</p><p>A data pipeline is a process to prepare enterprise data for analysis . It has 3 steps: Extract, Transform , Load</p><p>Actually , data pipelines are quite similar to your water pipeline!, But HOW?</p><p>Just like your water pipeline moves water from the reservoir to the taps, data pipeline moves data from the data points to the storage. A data pipeline extracts data from a sources(APIs, databases, data warehouses, file systems and many more), transforms the data and then load it in a specific destination.</p><p>Now, let’s build a fully serverless weather data pipeline that pulls weather data of Nepal from an API and store it in a data lake. And for this project, I’m using S3.</p><p>To complete this project , we require:</p><ul><li>Python</li><li>AWS lambda</li><li>An account with a weather API website</li><li>Amazon S3 to store data</li></ul><p>Project Overview:</p><p>Writing Python code for the ETL process and Amazon S3to store the Nepal’s weather data. If you haven’t installed python yet, follow these steps:</p><ul><li>pip install python</li></ul><p>Step 1: import the necessary modules:</p><pre>import json<br>import requests<br>import pandas as pd<br>import boto3<br>import os<br>from datetime import datetime</pre><p>Step 2: Extract Data from WeatherAPI</p><pre>def extract() -&gt; dict:<br>    api_url = f&quot;https://api.weatherapi.com/v1/current.json?key={API_KEY}&amp;q=Nepal&quot;<br>    try:<br>        response = requests.get(api_url)<br>        response.raise_for_status()<br>        return response.json()<br>    except Exception as e:<br>        print(f&quot;Extraction failed: {e}&quot;)<br>        return {}</pre><p>Step 3: Transform the raw data into data frame</p><p>APIs usually return <strong>nested JSON</strong>, which isn’t ideal for analytics.<br> In this step, AWS Lambda <strong>transforms raw data into a structured Pandas DataFrame</strong>.</p><pre>def transform(data: dict) -&gt; pd.DataFrame:<br>    extracted_data = {<br>        &quot;location&quot;: data[&quot;location&quot;][&quot;name&quot;],<br>        &quot;region&quot;: data[&quot;location&quot;][&quot;region&quot;],<br>        &quot;country&quot;: data[&quot;location&quot;][&quot;country&quot;],<br>        &quot;latitude&quot;: data[&quot;location&quot;][&quot;lat&quot;],<br>        &quot;longitude&quot;: data[&quot;location&quot;][&quot;lon&quot;],<br>        &quot;temperature_c&quot;: data[&quot;current&quot;][&quot;temp_c&quot;],<br>        &quot;temperature_f&quot;: data[&quot;current&quot;][&quot;temp_f&quot;],<br>        &quot;wind_kph&quot;: data[&quot;current&quot;][&quot;wind_kph&quot;],<br>        &quot;wind_mph&quot;: data[&quot;current&quot;][&quot;wind_mph&quot;],<br>        &quot;local_time&quot;: data[&quot;location&quot;][&quot;localtime&quot;]<br>    }<br><br>    return pd.DataFrame([extracted_data])<br></pre><p>Step 4: Load the data into Amazon s3</p><pre><br>def load(raw_data: dict, df: pd.DataFrame) -&gt; None:<br>    timestamp = datetime.utcnow()<br>    date_path = timestamp.strftime(&quot;%Y/%m/%d/%H%M%S&quot;)<br><br>    # Store raw json<br>    raw_key = f&quot;raw/weather_{date_path}.json&quot;<br>    s3.put_object(<br>        Bucket=BUCKET_NAME,<br>        Key=raw_key,<br>        Body=json.dumps(raw_data),<br>        ContentType=&quot;application/json&quot;<br>    )<br><br>    # Store transformed CSV <br>    processed_key = f&quot;processed/weather_{date_path}.csv&quot;<br>    csv_data = df.to_csv(index=False)<br><br>    s3.put_object(<br>        Bucket=BUCKET_NAME,<br>        Key=processed_key,<br>        Body=csv_data,<br>        ContentType=&quot;text/csv&quot;<br>    )<br><br>    print(&quot;Raw and transformed weather data successfully stored in S3&quot;)<br>        <br></pre><h3>Pipeline Execution</h3><p>This is where AWS Lambda <strong>executes the entire pipeline</strong>.</p><pre>def lambda_handler(event, context):<br>    data = extract()<br><br>    if not data:<br>        return {<br>            &quot;statusCode&quot;: 500,<br>            &quot;body&quot;: &quot;Weather data extraction failed&quot;<br>        }<br><br>    transformed_df = transform(data)<br>    load(data, transformed_df)<br><br>    return {<br>        &quot;statusCode&quot;: 200,<br>        &quot;body&quot;: &quot;Serverless weather data pipeline executed successfully&quot;<br>    }</pre><p>Conclusion:</p><p>Using python to extract the data from a weather API and transformed the raw data and loaded into Amazon S3. If you’re excited about this project and want to try it by yourself, I have uploaded the whole code to my GitHub repo so feel free to check it out, clone it and experiment with the data pipeline. Remember to create an account on the Weather API to be able to get those API parameters to use the API. And account on AWS to use S3 bucket and AWS Lambda, So that you can transform the data and load it into S3 bucket and create a fully serverless data pipeline. Moreover, I will be doing more such projects . So, stay tuned!</p><p>Linkdln: <a href="https://www.linkedin.com/in/yojanaghimire/">https://www.linkedin.com/in/yojanaghimire/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70a1e76e9fd3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Importance of Data in the Digital Age]]></title>
            <link>https://medium.com/@yojanaghimire06/the-importance-of-data-in-the-digital-age-d0713c8a09ac?source=rss-038b64eb2970------2</link>
            <guid isPermaLink="false">https://medium.com/p/d0713c8a09ac</guid>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Yojana Ghimire]]></dc:creator>
            <pubDate>Mon, 08 Jul 2024 14:30:18 GMT</pubDate>
            <atom:updated>2024-07-08T15:56:30.311Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Importance_of_data_digital_age" src="https://cdn-images-1.medium.com/max/1024/1*XWIgTk4VFD1b08DsuypMzw.jpeg" /></figure><p>Welcome to the digital age, where data is as important as oil . Such that data is also called a ‘<strong>new oil</strong>’. Whether you’re using your smartphone, watching TV, or shopping online, data is being created and used. From social media platforms to education and businesses and even scientific research, data helps make things smarter and more efficient. But what exactly is data, why is it so important, Let’s dive in and find out!</p><h3><strong>What is Data?</strong></h3><p>Data is simply information. It can be numbers, text, images, videos, or any other type of content that can be stored and analyzed. For example, the number of likes on an Instagram post, the comments on a Facebook status, or the videos you watch on YouTube are all types of data.</p><h3><strong>Importance of Data in Different fields</strong></h3><p>Data is used everywhere in every aspects and in every field. Here are some examples of how data is important or used in different fields:</p><figure><img alt="importance_of_data_business" src="https://cdn-images-1.medium.com/max/564/1*YTiD2P7Ph7_B5KLWQsGnUA.jpeg" /></figure><h4><strong>Healthcare</strong></h4><p>Data is a game-changer in healthcare, making patient care better and smarter. For example, Electronic Health Records (EHRs) give doctors a full picture of a patient’s medical history, allowing for personalized treatments and early warnings of health issues. Wearable devices, like fitness trackers, let people monitor their health in real-time and help doctors keep an eye on patients remotely. AI and Machine Learning (ML) are especially powerful in diagnosing brain tumors, using advanced image analysis to spot problems quickly and accurately. They also predict health risks and recommend the best treatments by learning from vast amounts of data. This blend of data and technology is transforming healthcare, leading to more precise and proactive health management.</p><figure><img alt="Importance_of_data_Healthcare" src="https://cdn-images-1.medium.com/max/1024/1*7Cc3JBC451et-vA4J6qa4g.jpeg" /></figure><h4><strong>Smart Cities</strong></h4><p>Data is revolutionizing how cities operate, making them smarter and more efficient. By analyzing data from traffic cameras and sensors, cities can optimize traffic light timings, reducing congestion and improving traffic flow. Smart grids use data to balance energy loads and reduce consumption, leading to cost savings and a smaller carbon footprint. Public services, such as waste management, benefit from data analytics by predicting optimal collection times and routes, enhancing efficiency and reducing operational costs. Additionally, data from various sources helps city planners design more sustainable and livable urban environments, addressing issues such as air quality and noise pollution.</p><h4>Education</h4><p>In the field of education, data plays a crucial role in enhancing student learning and teacher effectiveness. Educational data analytics allow schools to tailor teaching methods to individual students’ needs by tracking their progress, learning styles, and performance. Teachers can identify students who need extra help and provide targeted interventions to ensure no one falls behind. Data-driven insights help in creating more effective lesson plans and curricula, focusing on areas where students struggle the most. Additionally, predictive analytics can identify at-risk students early, enabling timely support and improving overall educational outcomes.</p><h4>Government</h4><p>Governments utilize data to improve policy-making, public services, and security. By analyzing data on crime rates, governments can allocate police resources more effectively, focusing on high-crime areas and improving public safety. Traffic data helps urban planners design better transportation systems, reducing congestion and improving mobility. Data on public health trends allows for quicker responses to epidemics and better resource allocation in healthcare. Moreover, data-driven governance increases transparency and accountability, as governments can track the effectiveness of policies and programs, making adjustments based on measurable outcomes. This approach ensures more responsive and efficient governance, benefiting citizens and communities.</p><h4>Business and Marketing</h4><p><strong>Ever wondered Instagram, Facebook, YouTube, and Netflix seem to know exactly what content you like?</strong> It’s all thanks to data! These platforms collect and analyze data on your interactions to recommend content that matches your interests.</p><figure><img alt="Importance_of_data_business_marketing" src="https://cdn-images-1.medium.com/max/563/1*K8lFZ_tgG0otRYLEre19VQ.jpeg" /></figure><p>When you use platforms like Instagram, Facebook, YouTube, and Netflix, they collect data on your interactions, like what you like, share, comment on, and watch. Think of it like your every click, tap, and swipe is stored in massive databases like a giant digital library. These platforms then use complex algorithms — like super-smart digital detectives — to analyze your habits and uncover your favorite trends, helping the platforms understand your preferences. Based on this analysis, they recommend content tailored to your interests; if you’re a fan of Mr. Beast , you’ll soon find your feed filled with more Mr. Beast videos. It’s like having a personal content curator who knows exactly what you love. So , that’s how data is important in businesses like Instagram, Facebook, YouTube and Netflix.</p><h3><strong>Big Data</strong></h3><p>Big Data is extremely large datasets that cannot be handled by traditional data-processing tools. These vast amounts of data come from various sources, including social media, sensors, and transactions, and play a crucial role in today’s data-driven world.</p><figure><img alt="Importance_of_big_data_how_bigdata_works" src="https://cdn-images-1.medium.com/max/1024/1*0moO-93uCoOQoYZEh17AIg.jpeg" /></figure><h4>How Big Data Works</h4><p>Big Data operates through a combination of technologies and processes designed to collect, store, process, and analyze vast amounts of information. Here’s a simplified breakdown:</p><p>1.<strong> Data Collection:</strong><br>-Sources: Data is gathered from various sources like social media platforms, sensors in IoT devices, online transactions, and more.<br> -Methods: Tools like web scraping, API integration, and data streams are used to collect this data in real-time or batch processes.</p><p>2. <strong>Data Storage:</strong><br>-Technologies: Big Data requires robust storage solutions like distributed databases (e.g., Hadoop, NoSQL databases) that can handle large volumes of unstructured data.<br>-Data Lakes: These are centralized repositories that allow storage of structured and unstructured data at any scale.</p><p>3. <strong>Data Processing</strong>:<br>-Frameworks: Big Data processing frameworks such as Apache Hadoop and Apache Spark enable efficient handling of large datasets.<br>-Techniques: Data is processed through methods like batch processing (handling large volumes of data at once) and stream processing (handling data in real-time).</p><p>4. <strong>Data Analysis:</strong><br>-Analytics Tools: Advanced analytics tools and techniques, including machine learning algorithms, data mining, and predictive analytics, are used to extract insights from Big Data.<br> -Visualization: Tools like Tableau and Power BI help visualize data, making complex insights understandable and actionable.</p><p>Big Data enables businesses to gain valuable insights into customer behavior, market trends, and operational efficiency. For instance, analyzing social media data helps companies understand customer preferences and tailor their marketing strategies accordingly. Sensor data, collected from IoT devices, allows industries to monitor equipment in real-time, predicting maintenance needs and preventing costly breakdowns.</p><p>In healthcare, Big Data improves patient care by analyzing data from electronic health records, wearable devices, and medical imaging. This helps in early diagnosis, personalized treatments, and efficient management of chronic diseases.</p><p>Governments use Big Data to enhance public services and security. By analyzing data on traffic patterns, crime rates, and public health, they can make informed decisions that improve urban planning, law enforcement, and healthcare services.</p><p>In education, Big Data personalizes learning experiences by tracking student performance and engagement. Schools can identify at-risk students and provide targeted support, ensuring better educational outcomes.</p><p>Overall, Big Data is essential for extracting meaningful insights from vast and diverse data sources, driving innovation, efficiency, and improved decision-making across various sectors.</p><h4><strong>Ethical Considerations</strong></h4><blockquote><em>With great power comes great responsibility. This responsibility extends across several critical dimensions: privacy, security, transparency, and bias. Each of these aspects plays a crucial role in maintaining trust and ensuring that data is used for the benefit of all, without causing harm or perpetuating inequalities. Handling data ethically and protecting privacy is crucial. This involves following steps:</em></blockquote><h4>Privacy: Respecting User Privacy and Protecting Personal Information</h4><p>Respecting user privacy is foundational to ethical data handling. It involves more than just complying with regulations; it’s about building trust with users by ensuring their personal information is collected, stored, and used in a manner that they are comfortable with. Key principles include:</p><ul><li><strong>Data Minimization</strong>: Collect only the data that is necessary for the intended purpose.</li><li><strong>Consent</strong>: Ensure that users are fully informed about what data is being collected and why, and obtain their explicit consent.</li><li><strong>Anonymization</strong>: Where possible, anonymize data to protect individuals’ identities.</li><li><strong>User Rights</strong>: Provide users with control over their data, including the right to access, correct, and delete their information.</li></ul><h4>Security: Ensuring Data is Stored and Processed Securely</h4><p>Data security is critical to protecting user privacy and maintaining the integrity of data. Security measures should be robust and continuously updated to guard against breaches and unauthorized access. This includes:</p><ul><li><strong>Encryption</strong>: Encrypt data both in transit and at rest to protect it from unauthorized access.</li><li><strong>Access Controls</strong>: Implement strict access controls to ensure that only authorized individuals can access sensitive data.</li><li><strong>Regular Audits</strong>: Conduct regular security audits to identify and address vulnerabilities.</li><li><strong>Incident Response</strong>: Have a clear incident response plan in place to quickly address any data breaches or security incidents.</li></ul><h4>Transparency: Being Clear About How Data is Collected and Used</h4><p>Transparency is essential for building trust and ensuring users are aware of how their data is being handled. This involves:</p><ul><li><strong>Clear Policies</strong>: Provide clear and concise privacy policies that explain what data is collected, how it is used, and with whom it is shared.</li><li><strong>Open Communication</strong>: Communicate openly with users about any changes to data practices or policies.</li><li><strong>Data Access</strong>: Allow users to easily access their data and understand how it is being used.</li><li><strong>Third-party Sharing</strong>: Be transparent about any third-party sharing of data and ensure those parties also adhere to strict privacy standards.</li></ul><h4>Bias: Recognizing and Addressing Biases in Data</h4><p>Data can reflect the biases present in society, which can lead to unfair outcomes if not addressed. For example, if a dataset used for hiring decisions contains more data about men than women, the system might unfairly favor male candidates. Addressing bias involves:</p><ul><li><strong>Diverse Datasets</strong>: Ensure datasets are representative of the diverse population they aim to serve.</li><li><strong>Bias Audits</strong>: Regularly audit algorithms and data processes for biases and take steps to mitigate them.</li><li><strong>Inclusive Practices</strong>: Implement inclusive data practices that consider the impact on all demographic groups.</li><li><strong>Continuous Monitoring</strong>: Continuously monitor and adjust practices to address emerging biases and ensure fair outcomes.</li></ul><h4><strong>Preparing for a Data-Driven Future</strong></h4><p>The future is data-driven. As we move further into the digital age, data is becoming an increasingly crucial asset. The future will be defined by how effectively we can harness and utilize data.</p><p>Artificial Intelligence (AI) is transforming how we analyze and use data. AI technologies will continue to improve predictions by analyzing vast amounts of data across various fields such as healthcare, finance, and marketing. It will also automate routine tasks, increasing efficiency and reducing human error, from customer service chatbots to complex data analysis.</p><p>The Internet of Things (IoT) is expanding rapidly, connecting more devices to the internet. This expansion will generate more data as IoT devices, like smart home appliances and wearable technology, produce large amounts of information, providing insights into user behavior and system performance. Connected devices will lead to smarter homes and cities, enhancing energy efficiency, safety, and convenience.</p><p>With growing concerns about privacy, new technologies and regulations will emerge to protect personal data. These measures will aim to ensure that as our world becomes more connected and data-driven, individual privacy is not compromised, fostering a balance between innovation and security.</p><p><strong>Conclusion</strong></p><p>Data isn’t just numbers and charts ,it is the lifeblood of our digital age, influencing everything from social media recommendations to scientific breakthroughs. Understanding how data works and its importance can help us navigate this data-driven world more effectively. So next time you use your smartphone or log into your favorite app, remember that data is at the heart of it all!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d0713c8a09ac" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Where is binary search tree used in real life?]]></title>
            <link>https://medium.com/@yojanaghimire06/a-binary-search-tree-is-tree-where-the-nodes-are-arranged-in-a-specific-order-for-each-node-all-f59061fcb957?source=rss-038b64eb2970------2</link>
            <guid isPermaLink="false">https://medium.com/p/f59061fcb957</guid>
            <category><![CDATA[preorder-traversal]]></category>
            <category><![CDATA[level-order-traversal]]></category>
            <category><![CDATA[post-order-traversal]]></category>
            <category><![CDATA[in-order-traversal]]></category>
            <category><![CDATA[binary-search-tree]]></category>
            <dc:creator><![CDATA[Yojana Ghimire]]></dc:creator>
            <pubDate>Sat, 29 Jun 2024 12:31:55 GMT</pubDate>
            <atom:updated>2024-07-05T14:51:32.351Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Where do we use binary search tree in real life?" src="https://cdn-images-1.medium.com/max/1024/1*P_X5ncVtYUbO_lxYyJO-0A.jpeg" /></figure><p><strong>Where is binary search tree used in real life?</strong></p><p><strong>A binary search tree</strong> is tree where the nodes are arranged in a specific order: for each node, all values in the left subtree are less than the node’s value, and all values in the right subtree are greater than the node’s value. Binary trees are useful for storing data in an organized way, which allows for it to be fetched, inserted, updated, and deleted quickly.</p><p>Binary search trees provide an average Big-O complexity of O(log(n)) for retrieval, insertion, update, and delete operations. Log(n) is much faster than the linear O(n) time required to find items in an unsorted array.</p><p>But many of us wonder how binary search trees (BSTs) are used and where they find practical applications in our daily lives. Here are some real-life examples of binary search trees (BSTs):</p><ol><li><strong>Dictionary</strong>: Imagine looking up a word in a dictionary. Words are in alphabetical order. If you’re looking for “plan” you know it’s after the word that starts from “o” but before “q” , so you know where to look. In such cases, we use <strong>Inorder traversal </strong>of BST, as nodes are visited in ascending order.</li><li><strong>Phone contacts</strong>: when you search for a contact on your phone, typing “D” shows names starting with D.As you type more letters, the search narrows down. This is similar to how a BST helps find information quickly. Here, we use <strong>preorder traversal</strong> as we would be going through the entire list in alphabetical order, contacting each person whose name appears.</li><li><strong>Online Store Inventory</strong>: An online store manages its inventory of products using a BST where each product is store based on its unique identifier. This helps in efficiently searching for products, managing stock levels, and handling orders. For generating reports on product sales or inventory updates, we might use <strong>level order traversal(Breadth-first search). Level order traversal</strong> explores nodes level by level, which can be useful for operations requiring a breadth-first approach.</li><li><strong>Computer file system: </strong>In a computer file system, directories and files can be organized using a hierarchical structure, which can be represented using a BST. Directories (nodes) can have subdirectories and files (children nodes).When performing tasks like calculating the total size of files within a directory tree, we might use <strong>postorder traversal</strong>. Postorder traversal can efficiently accumulate results from leaf nodes (files) up to the root (top-level directory).</li></ol><p><strong>Building a binary tree:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/792/1*K8uMmsbvmVD_itwmnITZ2w.png" /></figure><p>-&gt; Level order traversal:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/495/1*qhAU0-5f1E7n1bYkiZ65rg.png" /></figure><p>-&gt; Inorder and preorder traversal:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/323/1*cL1GMrkRSl-3M49A3B4KcA.png" /></figure><p>-&gt; postorder traversal:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/339/1*eBiamWiiNHvxD06yL7VX2g.png" /></figure><p><strong>Binary search tree</strong> are versatile data structures used in various real-life applications where efficient searching, sorting, and hierarchical are required. The choice of traversal algorithm (preorder, inorder, postorder, or level order) depends on the specific task and how you need to interact with or analyze the data stored in the BST. Each algorithm offers unique benefits and is chosen based on the specific requirements of the application.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f59061fcb957" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>