<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Rashid on Medium]]></title>
        <description><![CDATA[Stories by Rashid on Medium]]></description>
        <link>https://medium.com/@MuhammedRashid_?source=rss-034ff0c468c0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*iyNrmXvCfExxrZ_tObPnnw.jpeg</url>
            <title>Stories by Rashid on Medium</title>
            <link>https://medium.com/@MuhammedRashid_?source=rss-034ff0c468c0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 20:37:34 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@MuhammedRashid_/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Reading Like Humans: The Secret Behind Modern AI]]></title>
            <link>https://medium.com/@MuhammedRashid_/reading-like-humans-the-secret-behind-modern-ai-ed2234acbf50?source=rss-034ff0c468c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/ed2234acbf50</guid>
            <category><![CDATA[attention-is-all-you-need]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Rashid]]></dc:creator>
            <pubDate>Sat, 26 Jul 2025 04:16:56 GMT</pubDate>
            <atom:updated>2025-07-26T04:16:56.690Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*4WNreopURVM1YFI6Re6iTA.jpeg" /></figure><p>Have you ever tried translating an entire paragraph in one go?</p><p>It’s not easy, right?</p><p>That’s because we — humans — don’t process language all at once.<br> Instead, we move <strong>word by word</strong>, <strong>phrase by phrase</strong>, understanding meaning step-by-step and accumulating context along the way.</p><p>We don’t just see words — we <strong>focus</strong> on the ones that matter most, depending on what came before and what we expect next.</p><p>Interestingly, that’s exactly how early sequence models like <strong>RNNs</strong> and <strong>LSTMs</strong> worked.<br> They translated or generated language <strong>one token at a time</strong>, using the hidden state built from all the previous tokens.</p><p>But these models had a major limitation — they compressed all previous context into a <strong>single fixed-length vector</strong>.<br> And just like trying to cram an entire paragraph into short-term memory, this often led to information loss, especially with longer sequences.</p><h3>Enter Attention — Inspired by Human Reading</h3><p>In 2015, researchers introduced something revolutionary: the <strong>Attention Mechanism</strong>.</p><p>This concept was directly inspired by <strong>how we humans read</strong>.</p><p>When we read a sentence, we don’t treat every word equally.<br> We instinctively <strong>pay more attention to certain words</strong> — based on what we’ve read so far and what we expect next.</p><p>The attention mechanism allows models to do the same.<br> Instead of relying on a single compressed memory, the model can <strong>“look back” at all the input tokens</strong> and assign different weights to them — dynamically deciding which parts of the input are most relevant <strong>right now</strong>.</p><p>This small shift brought a massive leap in performance.</p><h3>The Foundation of Transformers and Modern LLMs</h3><p>Attention didn’t just improve translation models.<br> It laid the foundation for the <strong>Transformer architecture</strong>, introduced in the 2017 paper <em>“Attention Is All You Need”</em>.</p><p>Transformers abandoned recurrence altogether and relied entirely on <strong>self-attention</strong>, allowing models to process entire sequences in parallel while still maintaining contextual understanding.</p><p>This architecture powers today’s most powerful models — like <strong>BERT</strong>, <strong>GPT</strong>, <strong>T5</strong>, and beyond.</p><h3>🍳 What’s Cookin’?</h3><p>So, what’s been cooking in the world of AI?</p><p>Turns out, machines are learning language the way we do — step by step, word by word, with a focused mind.<br> The <strong>attention mechanism</strong> is the not-so-secret sauce behind this evolution.</p><p>It’s the bridge between human-like understanding and machine learning.<br> It’s how models <strong>pay attention</strong> — just like we do — and the reason why modern AI feels so <strong>fluent</strong>, <strong>contextual</strong>, and almost <strong>human</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ed2234acbf50" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ The 5 Evolutionary Stages of LLMs — A Journey from Seq2Seq to GPT]]></title>
            <link>https://medium.com/@MuhammedRashid_/the-5-evolutionary-stages-of-llms-a-journey-from-seq2seq-to-gpt-806f33560162?source=rss-034ff0c468c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/806f33560162</guid>
            <category><![CDATA[gpt]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <dc:creator><![CDATA[Rashid]]></dc:creator>
            <pubDate>Thu, 24 Jul 2025 06:15:50 GMT</pubDate>
            <atom:updated>2025-07-24T06:15:50.188Z</atom:updated>
            <content:encoded><![CDATA[<h3>🌱 The 5 Evolutionary Stages of LLMs — A Journey from Seq2Seq to GPT</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4-ScfdqFcF6DwlH6rZ_kGw.jpeg" /></figure><blockquote><em>“It all started with a seed…”<br> On July 24, 2002, I was born into a world where Artificial Intelligence was just beginning to bloom. Fast forward to today, I’m not just witnessing its evolution — I’m part of it, guiding future Data Analysts and Scientists through this incredible journey.<br> And like every great journey, this one started with roots…</em></blockquote><h3>🌱 Stage 1: The Root — Sequence to Sequence Learning (2014)</h3><p>In 2014, a groundbreaking paper titled <strong>“Sequence to Sequence Learning with Neural Networks”</strong> was published by <a href="https://en.wikipedia.org/wiki/Ilya_Sutskever">Ilya Sutskever</a> and his team at Google. This architecture — commonly called <strong>Seq2Seq</strong> — was a game-changer for processing tasks like machine translation, where both input and output are sequences.</p><p>Previously, we had Recurrent Neural Networks (RNNs), particularly LSTMs, handling sequential data decently. But when it came to tasks like translating long sentences, they stumbled. The reason? Traditional RNNs processed inputs into a <strong>fixed-length context vector</strong>, no matter how long the input was. This caused crucial information to be lost — especially in longer sequences.</p><p>Sutskever’s innovation was the <strong>Encoder-Decoder architecture</strong>, where the encoder converts the input into a compressed vector, and the decoder reconstructs the output. It worked well for shorter sentences and sparked a wave of research into neural translation. But there was one big problem…</p><blockquote><em>⚠️ </em><strong><em>Bottleneck Alert</em></strong><em>: One single context vector can’t carry all the meaning from longer sentences. The model’s performance dropped significantly on longer inputs, as reflected in lower </em><a href="https://en.wikipedia.org/wiki/BLEU"><em>BLEU</em></a><em> scores.</em></blockquote><h3>🔍 Stage 2: Focus Arrives — Attention Mechanism (2015)</h3><p>The fix came a year later, in 2015, when <strong>Bahdanau, Cho, and </strong><a href="https://en.wikipedia.org/wiki/Yoshua_Bengio"><strong>Bengio</strong></a> published:<br> <strong>“Neural Machine Translation by Jointly Learning to Align and Translate.”</strong></p><p>Enter the <a href="https://en.wikipedia.org/wiki/Attention_(machine_learning)"><strong>Attention Mechanism</strong></a>.</p><p>Instead of squashing everything into a single context vector, attention allowed the model to look at all encoder outputs. For every word generated by the decoder, it could “attend” to specific parts of the input, dynamically.</p><blockquote><em>Imagine reading a sentence and focusing on each word as you translate — this is exactly how attention works.</em></blockquote><p>This changed everything. It preserved meaning, improved accuracy on long sentences, and inspired further innovations. But attention added <strong>computational complexity</strong>, especially as sequence length grew — since similarity scores had to be computed at each step.</p><h3>⚙️ Stage 3: Parallelism Revolution — The Transformer (2017)</h3><p>The real turning point came in 2017 with the legendary paper:<br> <strong>“</strong><a href="https://en.wikipedia.org/wiki/Attention_Is_All_You_Need"><strong>Attention is All You Need</strong></a><strong>”</strong> by Vaswani et al.</p><p>This paper scrapped RNNs altogether and introduced a brand-new architecture: the <a href="https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)"><strong>Transformer</strong></a>.</p><blockquote><em>No more recurrence. No more sequential limitations. Enter: </em><strong><em>Self-Attention</em></strong><em> and </em><strong><em>Parallel Processing</em></strong><em>.</em></blockquote><p>The Transformer allowed all input tokens to be processed <strong>simultaneously</strong> using self-attention, vastly reducing training time and improving scalability.</p><p>The recipe?<br> <strong>Transformer = Self-Attention + Positional Encoding + Feedforward Neural Networks</strong></p><p>This architecture became the <strong>foundation of all modern LLMs</strong>, setting the stage for models like <a href="https://en.wikipedia.org/wiki/BERT_(language_model)">BERT</a> and <a href="https://en.wikipedia.org/wiki/Generative_pre-trained_transformer">GPT</a>.</p><h3>⛓️‍💥Stage 4: The Missing Link — Transfer Learning for NLP (2018, Early)</h3><p>While the vision of Transformers was taking shape, another challenge loomed: <strong>How do we make a general-purpose model for multiple NLP tasks?</strong></p><p>In computer vision, <a href="https://en.wikipedia.org/wiki/Transfer_learning"><strong>transfer learning</strong></a> was already a norm — train a model on ImageNet, then fine-tune for tasks like classification or detection. But in NLP, models trained on one task (e.g., translation) couldn’t easily be used for another (e.g., sentiment analysis).</p><p>This is where <a href="https://en.wikipedia.org/wiki/Jeremy_Howard_(entrepreneur)#ULMFiT"><strong>Jeremy Howard</strong></a><strong> and Sebastian Ruder</strong> stepped in with their 2018 paper:<br> <strong>“Universal Language Model Fine-tuning for Text Classification (ULMFiT)”</strong>.</p><p>Instead of training on task-specific data, they <strong>pretrained</strong> a model on general language modeling (i.e., next-word prediction) using any large text corpus, then fine-tuned it for downstream tasks.</p><p>Why this worked:</p><ul><li><strong>Unsupervised Pretraining</strong> using raw text</li><li><a href="https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)"><strong>Fine-tuning</strong></a> enabled model reuse across tasks</li><li>Dramatic reduction in labeled data required</li></ul><blockquote><em>They still used LSTMs, not Transformers. But ULMFiT proved that pretraining + fine-tuning could achieve </em><strong><em>state-of-the-art results</em></strong><em> with much less data.</em></blockquote><h3>🌳Stage 5: The Bloom — Large Language Models (2018–Present)</h3><p>And then came the final fusion — what if we combine <strong>Transformers</strong> with <strong>Transfer Learning</strong>?</p><p>That’s exactly what happened in late 2018.</p><p>In October, two massive transformer-based models were unveiled:</p><ul><li><strong>BERT (Bidirectional Encoder Representations from Transformers)</strong> — by Google</li><li><strong>GPT (Generative Pre-trained Transformer)</strong> — by OpenAI</li></ul><p>BERT focused on deep bidirectional understanding (encoder-only), while GPT focused on generative capabilities (decoder-only).</p><blockquote><em>These were the </em><strong><em>first true LLMs</em></strong><em>, trained on massive datasets and fine-tuned across a wide range of tasks.</em></blockquote><p>Since then:</p><ul><li><a href="https://en.wikipedia.org/wiki/GPT-2"><strong>GPT-2</strong></a><strong> (2019)</strong> scaled up parameters dramatically.</li><li><a href="https://en.wikipedia.org/wiki/GPT-3"><strong>GPT-3</strong></a><strong> (2020)</strong> brought few-shot learning and became a household name.</li><li><strong>BERT variants</strong> like RoBERTa, DistilBERT, and others pushed the boundaries of NLP understanding.</li><li><strong>ChatGPT and </strong><a href="https://en.wikipedia.org/wiki/GPT-4"><strong>GPT-4</strong></a> brought real-time applications and conversational fluency.</li></ul><h3>📜 Final Thoughts</h3><p>From the early seeds of Seq2Seq to the blooming tree of LLMs, this journey reflects not just the evolution of AI — but our evolving understanding of <strong>how language works</strong>.</p><p>It’s a tale of bottlenecks and breakthroughs, attention and acceleration.</p><p>As someone who grew up alongside this revolution and now trains the next generation of data professionals, I feel both awe and responsibility — to pass down the roots, the branches, and the blossoms of this incredible tree.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=806f33560162" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Python to Excel: A Surprising Journey in Coffee Sales Analysis]]></title>
            <link>https://medium.com/@MuhammedRashid_/from-python-to-excel-a-surprising-journey-in-coffee-sales-analysis-5d5778c702bb?source=rss-034ff0c468c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/5d5778c702bb</guid>
            <dc:creator><![CDATA[Rashid]]></dc:creator>
            <pubDate>Wed, 25 Sep 2024 09:34:42 GMT</pubDate>
            <atom:updated>2024-09-25T09:38:08.932Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zKsvvCe9ZNaB2_eC5xBwJg.jpeg" /></figure><p>Hi all! Recently, I took a break from my usual go-to for data analysis, Python, and decided to explore Excel for a change — and I was genuinely impressed! While Python has always been my tool of choice for handling data, Excel really surprised me with its powerful features for cleaning, transforming, and visualizing data. I decided to put it to the test by working on a Coffee Sales dataset, and the results were beyond my expectations.</p><p>I used Excel’s versatile functions like VLOOKUP to connect columns and bring in relevant data from different sheets. You can have the data from <a href="https://www.kaggle.com/datasets/rashid7274/data-analysis-on-coffee-sales-data">here</a>. Finally, I created an interactive dashboard that provides a comprehensive overview of the coffee sales data. The dashboard includes key metrics like total sales, total profit, and total customers. It also displays insights on sales by roast type, coffee types, loyalty card usage, monthly walk-ins, and a breakdown of sales by country and city.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BBc1roTfRo2xGI-nqBkuhQ.png" /><figcaption>sales overview of united states</figcaption></figure><p>You can check out the full dashboard and its creation process on my GitHub: <a href="https://github.com/Rashid9226/Excel_Data_Analysis/blob/main/Day-2_coffeeData.xlsx">here</a>. The process not only refreshed my skills but also demonstrated the powerful, yet often underrated, capabilities of Excel in data analysis and visualization. I highly recommend giving Excel a shot if you haven’t already — it might just surprise you!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5d5778c702bb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Attrition-Alchemy: A Machine Learning Approach Part 2]]></title>
            <link>https://medium.com/@MuhammedRashid_/attrition-alchemy-a-machine-learning-approach-part-2-bfb90e858246?source=rss-034ff0c468c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/bfb90e858246</guid>
            <dc:creator><![CDATA[Rashid]]></dc:creator>
            <pubDate>Sun, 18 Aug 2024 07:55:47 GMT</pubDate>
            <atom:updated>2024-08-18T07:58:29.381Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*osHeHCSxVpspoQvCWxHWwQ.jpeg" /></figure><blockquote>“Hello, Ready to take the fun part of Attrition-Alchemy, no more waiting just dive in…”</blockquote><p>Domain expertise on our data is a must for a data to be tortured, because when we know the weakness we can make the data suffer and confess more . As we have discussed about the Problem and the Domain in the previous story</p><p>The project get divided to the following parts:</p><ol><li>Data Collection</li><li>EDA</li><li>Feature Engineering</li><li>Data preparation</li><li>Model Building</li><li>Saving the Model</li></ol><p>Data Collection:</p><p>Every great analysis begins with gathering data. In this project, we start by importing the necessary libraries and loading our dataset.</p><pre>import pandas as pd<br>import numpy as np<br>import matplotlib.pyplot as plt<br>from sklearn.model_selection import train_test_split<br>from sklearn.metrics import confusion_matrix<br>from sklearn.metrics import accuracy_score<br><br>attrdata = pd.read_csv(&quot;attrition.csv&quot;)</pre><p>you can get the data from the following <a href="https://github.com/Rashid9226/Attrition-Alchemy/blob/main/attrition.csv">here</a></p><p>In this snippet, we’re using the pandas library to read our dataset, which is a CSV file containing information about employees. Think of it as our treasure map, guiding us through the data landscape.</p><p>2.Exploratory Data Analysis (EDA): The Torturing Begins</p><p>“if you Torture the data more enough it will confess” — Ronald H. Coase</p><p>we need to understand our data better. This is where Exploratory Data Analysis (EDA) comes into play. We’ll look at the dataset’s structure, identify any missing values, and visualize some key metrics. Here’s a glimpse of the code:</p><pre># Display first few Rows of the dataset<br>attrdata.head()<br><br># Check the structure of the data<br>attrdata.shape<br><br># Check for Missing Values and removing those with missing values<br>attrdata.isnull().sum()<br>attrdata.dropna(axis=0,inplace=True)<br><br># Summary statistics<br>attrdata.describe()</pre><p>By displaying the first few rows and summary statistics, we get a feel for the data’s overall structure and key metrics. Checking for missing values ensures we don’t have any hidden pitfalls that could trip us up later. Handling missing values doesn’t just mean dropping them. The approach should be tailored to the specific domain.</p><p>3.Feature Engineering: Know the Weakness Of the Data</p><p>Feature engineering involves creating new features or modifying existing ones to improve the model’s performance. I’ve gone through the hard way in which used function and created my own functions too</p><pre>gender_dict = attrdata[&quot;Gender &quot;].value_counts()<br>Hiring_dict = attrdata[&quot;Hiring Source&quot;].value_counts()<br>promoted_dict = attrdata[&quot;Promoted/Non Promoted&quot;].value_counts()</pre><p>The code is to explore the features in which we are creating variables which include the categorical values and its count in dataset.</p><pre>location_dict = attrdata[&quot;Location&quot;].value_counts()<br>print(location_dict)<br><br>location_dict_new = {<br>    &#39;Chennai&#39;:       7,<br>    &#39;Noida&#39;:         6,<br>    &#39;Bangalore&#39;:     5,<br>    &#39;Hyderabad&#39;:     4,<br>    &#39;Pune&#39;:          3,<br>    &#39;Madurai&#39;:       2,<br>    &#39;Lucknow&#39;:       1,<br>    &#39;other place&#39;:   0,<br>}</pre><p>We’re simplifying the ‘Location’ feature by creating a new dictionary that maps specific locations to values based on how often they appear. Locations with fewer counts than those listed are grouped as “other place” and given a value of 0. This reduces data complexity by focusing on the most common locations, which can help improve the model’s performance by minimizing noise(unwanted values) from less frequent locations.</p><pre>gen = pd.get_dummies(attrdata[&quot;Function&quot;]).astype(int)<br>hr = pd.get_dummies(attrdata[&quot;Hiring Source&quot;]).astype(int)</pre><p>The code converts the categorical features “Function” ,“Hiring Source” and others into binary (0 or 1) columns using one-hot encoding. This is how one-hot encoding works: it makes the data numerical and ready for machine learning models.</p><pre>def Mar(x):<br>    if str(x) in Marital_dict.keys() and Marital_dict[str(x)] &gt; 100:<br>        return str(x)<br>    else:<br>        return &#39;other status&#39;<br>    <br>data_l = attrdata[&quot;Marital Status&quot;].apply(Mar)<br>attrdata[&#39;New Marital&#39;] = data_l<br>attrdata.head()</pre><p>The function Mar(x) checks if a marital status is in the Marital_dict and has more than 100 occurrences. If it does, it returns the status; otherwise, it returns &#39;other status&#39;. This is applied to the &quot;Marital Status&quot; column, creating a new column &#39;New Marital&#39; with the simplified categories. This process helps group less common statuses under &#39;other status&#39; to reduce data complexity.</p><p>Explore more functions or similar data processing techniques, feel free to check out my <a href="https://github.com/Rashid9226/Attrition-Alchemy/blob/main/Attrition%20Rate.ipynb">GitHub </a>for additional examples and resources!</p><p>4. Data Preparation: Grooming the data</p><p>Data preparation is like getting all your ingredients ready before cooking a meal. In data analysis, it involves organizing and cleaning the data so it’s ready for analysis. This process ensures that the data is accurate, complete, and in a format that can be easily understood and used.</p><p>Why Data Preparation? without proper data preparation, the results of your analysis might be misleading or incorrect. By grooming the data, we make sure it’s reliable and focused on what we want to achieve.</p><p>Now that we’ve finished creating and refining our features (or variables), we need to decide which ones to include in our model.</p><pre>dataset = pd.concat([attrdata, hr, Mr, emp, tengrp, gen, gend], axis = 1)<br><br>dataset.drop([&quot;table id&quot;, &quot;name&quot;, &quot;Marital Status&quot;,&quot;Promoted/Non Promoted&quot;,&quot;Function&quot;,&quot;Emp. Group&quot;,&quot;Job Role Match&quot;,&quot;Location&quot;<br>              ,&quot;Hiring Source&quot;,&quot;Gender &quot;, &quot;Tenure&quot;, &quot;New Gender&quot;, &#39;New Marital&#39;, &#39;New EMP&#39;],axis=1,inplace=True)</pre><p>Once we have performed one-hot encoding on categorical variables, we are adding them with the data so that the number of features in our dataset increases. One-hot encoding converts categorical values into a binary format that machine learning models can interpret, making this step essential.</p><p>After encoding, we no longer need the original categorical columns or any other unnecessary features. Removing these ensures that our dataset remains lean and focused, containing only the features required for effective model training. This step not only simplifies the data but also enhances the model’s performance.</p><pre>dataset.columns<br>Index([&#39;table id&#39;, &#39;name&#39;, &#39;phone number&#39;, &#39;Location&#39;, &#39;Emp. Group&#39;,<br>       &#39;Function&#39;, &#39;Gender &#39;, &#39;Tenure&#39;, &#39;Tenure Grp.&#39;, &#39;Experience (YY.MM)&#39;,<br>       &#39;Marital Status&#39;, &#39;Age in YY.&#39;, &#39;Hiring Source&#39;,<br>       &#39;Promoted/Non Promoted&#39;, &#39;Job Role Match&#39;, &#39;Stay/Left&#39;, &#39;New Location&#39;,<br>       &#39;New Marital&#39;, &#39;New Promotion&#39;, &#39;New EMP&#39;, &#39;New Job Role Match&#39;,<br>       &#39;New Gender&#39;, &#39;Agency&#39;, &#39;Direct&#39;, &#39;Employee Referral&#39;, &#39;Marr.&#39;,<br>       &#39;Single&#39;, &#39;other status&#39;, &#39;B1&#39;, &#39;B2&#39;, &#39;B3&#39;, &#39;other group&#39;, &#39;&lt; =1&#39;,<br>       &#39;&gt; 1 &amp; &lt; =3&#39;, &#39;Operation&#39;, &#39;Sales&#39;, &#39;Support&#39;, &#39;Female&#39;, &#39;Male&#39;,<br>       &#39;other&#39;],<br>      dtype=&#39;object&#39;)<br><br>dataset1.columns<br>Index([&#39;Experience (YY.MM)&#39;, &#39;Age in YY.&#39;, &#39;Stay/Left&#39;, &#39;New Location&#39;,<br>       &#39;New Promotion&#39;, &#39;New Job Role Match&#39;, &#39;Agency&#39;, &#39;Direct&#39;,<br>       &#39;Employee Referral&#39;, &#39;Marr.&#39;, &#39;Single&#39;, &#39;other status&#39;, &#39;B1&#39;, &#39;B2&#39;,<br>       &#39;B3&#39;, &#39;other group&#39;, &#39;&lt; =1&#39;, &#39;&gt; 1 &amp; &lt; =3&#39;, &#39;Operation&#39;, &#39;Sales&#39;,<br>       &#39;Support&#39;, &#39;Female&#39;, &#39;Male&#39;, &#39;other&#39;],<br>      dtype=&#39;object&#39;)</pre><p>As part of our data preparation, we focused on refining the dataset by applying feature selection and encoding techniques. Initially, the dataset (dataset.columns) included a variety of columns, some of which were categorical variables that needed to be encoded for machine learning.</p><p>After performing one-hot encoding, we generated new features such as Agency, Direct, Female, Male, and others. These new features replaced the original categorical columns. Consequently, we removed the original and other unnecessary features, leaving us with a streamlined dataset (dataset1.columns).</p><p>This refined dataset now includes only the essential features required for model training, ensuring that our data is both concise and relevant to the predictive task.</p><p>5. Building Model: Shaping the Solution</p><p>In the Building Model phase, we focus on transforming our prepared data into a functional machine learning model that can make predictions or classifications.</p><p><em>5.1 Load and Spiting the data</em>:</p><pre>dataset = pd.read_csv(&quot;processed table.csv&quot;)<br>dataset = pd.DataFrame(dataset)<br>y = dataset[&quot;Stay/Left&quot;]<br>X = dataset.drop(&quot;Stay/Left&quot;, axis=1)</pre><p>The Prepared Data is to be separated into features (X) and target variable (y). The target variable here is “Stay/Left”, which we want to predict.</p><pre>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=4)</pre><p><strong>Training and Test Sets</strong>: We split the data into training and test sets. 80% of the data is used for training the model, and 20% is reserved for testing its performance.</p><p><em>5.2 Import and Initialize Models:</em></p><pre>from sklearn.linear_model import LogisticRegression<br>from sklearn.tree import DecisionTreeClassifier<br>from sklearn.ensemble import RandomForestClassifier<br>from sklearn.naive_bayes import GaussianNB<br><br>lr = LogisticRegression(C=0.1, random_state=42, solver=&#39;liblinear&#39;)<br>dt = DecisionTreeClassifier()<br>rm = RandomForestClassifier()<br>gnb = GaussianNB()</pre><ul><li><strong>Importing Models</strong>: We import several machine learning models: Logistic Regression, Decision Tree, Random Forest, and Naive Bayes.</li><li><strong>Initializing Models</strong>: We create instances of these models, with specific settings for Logistic Regression.</li></ul><p>5.3 <em>Train and Evaluate the Model:</em></p><pre>for a, b in zip([lr, dt, rm, gnb], [&quot;Logistic Regression&quot;, &quot;Decision Tree&quot;, &quot;Random Forest&quot;, &quot;Naive Bayes&quot;]):<br>    a.fit(X_train, y_train)<br>    prediction = a.predict(X_train)<br>    y_pred = a.predict(X_test)<br>    score1 = accuracy_score(y_train, prediction)<br>    score = accuracy_score(y_test, y_pred)<br>    msg1 = &quot;[%s] training data accuracy is : %f&quot; % (b, score1)<br>    msg2 = &quot;[%s] test data accuracy is : %f&quot; % (b, score)<br>    print(msg1)<br>    print(msg2)</pre><p><strong>Use of </strong><strong>zip</strong>: The zip function combines two lists into pairs. In this case, it pairs each model instance (lr, dt, rm, gnb) with its corresponding name (&quot;Logistic Regression&quot;, &quot;Decision Tree&quot;, &quot;Random Forest&quot;, &quot;Naive Bayes&quot;).</p><p><strong>Training and Testing</strong>:</p><ul><li><strong>Fit the Model</strong>: For each model, we train it using the training data (a.fit(X_train, y_train)).</li><li><strong>Predict</strong>: We make predictions on both the training data (a.predict(X_train)) and the test data (a.predict(X_test)).</li><li><strong>Calculate Accuracy</strong>: We measure the accuracy of the model on both the training data (accuracy_score(y_train, prediction)) and the test data (accuracy_score(y_test, y_pred)).</li><li><strong>Print Results</strong>: We display the accuracy of each model on both datasets using formatted messages.</li></ul><p>The resulting output will be:</p><pre>[Logistic Regression] training data accuracy is : 0.891061<br>[Logistic Regression] test data accuracy is : 0.877095<br>[Decision Tree] training data accuracy is : 0.998603<br>[Decision Tree] test data accuracy is : 0.860335<br>[Random Forest] training data accuracy is : 0.998603<br>[Random Forest] test data accuracy is : 0.882682<br>[Naive Bayes] training data accuracy is : 0.868715<br>[Naive Bayes] test data accuracy is : 0.826816</pre><blockquote>Logistic Regression is likely the preferred model in this case. It demonstrates strong generalization ability and avoids the overfitting issues(Stay tuned for my upcoming stories about generalization and overfitting) observed with more complex models like Decision Trees and Random Forests.</blockquote><p>6. Saving the Model:</p><p>Once you have trained and evaluated your machine learning model, you may want to save it for future use. This allows you to load the model later without needing to retrain it, which can save time and computational resources. You can use libraries like joblib or pickle to save and load your model.</p><pre>import pickle<br># save the model to disk<br>filename = &#39;Attrition_Alchemy.pickle&#39;<br>pickle.dump(model, open(filename, &#39;wb&#39;))</pre><p>In the final phase of our project, we transitioned from model development to deployment. We utilized Django to create a robust web application that integrates our machine learning model seamlessly. By leveraging the pickle library, we efficiently saved our trained model and ensured its easy integration with the Django app.</p><p>The <a href="https://github.com/Rashid9226/Attrition-Alchemy/tree/main/final_app">app</a> allows users to input data and receive predictions based on the model, providing a user-friendly interface and a practical application of our work. This approach not only demonstrates the effectiveness of our model but also showcases the potential for real-world applications.</p><p>Through this project, we’ve transformed a sophisticated machine learning model into a functional tool that delivers valuable insights, proving that with the right technologies, complex models can be made accessible and useful in everyday scenarios.</p><blockquote>“Thank you for joining me on this journey through Attrition-Alchemy. Stay tuned for more insights into generalization, overfitting, and further enhancements to our predictive models.”</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bfb90e858246" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Attrition-Alchemy: A Machine Learning Approach part 1]]></title>
            <link>https://medium.com/@MuhammedRashid_/attrition-alchemy-a-machine-learning-approach-7b77b300f45b?source=rss-034ff0c468c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b77b300f45b</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Rashid]]></dc:creator>
            <pubDate>Thu, 01 Aug 2024 15:17:08 GMT</pubDate>
            <atom:updated>2024-08-01T17:07:35.485Z</atom:updated>
            <content:encoded><![CDATA[<h3>Attrition-Alchemy: A Machine Learning Approach part 1</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KBfAwtJxbgdBc0A9g3OS9g.png" /></figure><blockquote>“Hi everyone, Is the title Attrition-Alchemy confusing you? well once done with the article you will get to know why it’s the perfect fit…”</blockquote><p>Attrition-Alchemy is a cutting-edge project dedicated to predicting employee attrition rates. Simply put, it helps organizations determine whether an employee is likely to stay with the company or leave. Using advanced data analysis techniques, this tool provides valuable insights that can help companies make better decisions about their workforce.</p><p>Why It’s Important:</p><ol><li><strong><em>Stay </em></strong><em>Ahead</em><strong><em> of </em></strong><em>Turnover</em>: By predicting which employees might leave, companies can proactively address potential issues and work on strategies to retain their top talent.</li><li><strong>Save Time and Money</strong>: Accurate predictions enable companies to plan ahead, reducing the time and costs associated with hiring and training new employees.</li><li>Tailored Solutions: The tool helps identify specific factors that could influence an employee’s decision to leave, allowing companies to implement targeted improvements and solutions.</li><li>Better Workforce Planning: With clear insights into potential attrition, organizations can plan their workforce more effectively, ensuring they maintain a stable and productive team.</li></ol><p>Attrition-Alchemy provides actionable predictions about employee retention, empowering companies to enhance their workforce stability and make informed decisions that drive long-term success.</p><blockquote>“Thanks for joining me on this journey! Now that you know why &#39;Attrition-Alchemy&#39; is such a fitting name, get ready for the fun stuff. In the next post, I’ll dive into the code and give you a step-by-step look at how Attrition-Alchemy works. Stay tuned! ”</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b77b300f45b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>