<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by KDataInc on Medium]]></title>
        <description><![CDATA[Stories by KDataInc on Medium]]></description>
        <link>https://medium.com/@kdatainc?source=rss-e9c6562f470c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*0oGm53hRwq2SoivAGqPasA@2x.jpeg</url>
            <title>Stories by KDataInc on Medium</title>
            <link>https://medium.com/@kdatainc?source=rss-e9c6562f470c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 19:28:12 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kdatainc/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Leveraging Market Basket Analysis to Boost Retail/E-Commerce Sales]]></title>
            <link>https://medium.com/@kdatainc/leveraging-market-basket-analysis-to-boost-retail-e-commerce-sales-49fd1682761b?source=rss-e9c6562f470c------2</link>
            <guid isPermaLink="false">https://medium.com/p/49fd1682761b</guid>
            <category><![CDATA[ecommerce]]></category>
            <category><![CDATA[retail]]></category>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[business]]></category>
            <dc:creator><![CDATA[KDataInc]]></dc:creator>
            <pubDate>Sat, 23 Aug 2025 15:30:55 GMT</pubDate>
            <atom:updated>2025-08-23T15:30:55.849Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/530/1*VZE0o_XoNnjykrIyT6Ocig.jpeg" /><figcaption>Picture by <a href="https://clipart-library.com/clipart/retail-cliparts-18.htm">Clipart</a></figcaption></figure><p>For many of us, the tangible experience of a physical store — the ability to touch a product, browse aisles, and discover items in person — creates a unique connection that online shopping can’t replicate. Yet, in the fierce competition of modern retail, this experience isn’t enough to guarantee profitability.</p><p>Whether you operate a storefront or a successful e-commerce platform, one shared challenge remains: <strong>maximizing the value of every single customer transaction.</strong> It’s about moving beyond basic product recommendations and truly understanding what customers want and need.</p><p>Today, we’ll dive into a data-driven solution that can help both brick-and-mortar and online retailers significantly increase the value of every single customer.</p><h3>Market Basket Algorithm: Apriori</h3><p>Now, before we get to the “how,” let’s understand the core logic of market basket analysis.</p><p>The <strong>Apriori</strong> algorithm is a foundational tool for uncovering patterns in large datasets. By design, its purpose is to <strong>find groups of items</strong> that <strong>frequently appear together</strong> in a collection of data. While it’s a general-purpose algorithm, its logic is a perfect match for the business problem of identifying product relationships in retail and e-commerce.</p><p>This powerful correlation is precisely why Apriori has become a staple for businesses looking to optimize their cross-selling and recommendation strategies. It helps you move beyond intuition and discover surprising connections in customer behavior.</p><p>In Apriori, there are 3 important KPIs to look at:</p><ol><li><strong>Support:</strong> This is the proportion of all transactions that contain a specific item or set of items. It measures the <strong>general popularity</strong> of an item set within your entire dataset.</li><li><strong>Confidence:</strong> This measures the likelihood of a person buying one item <strong>given that they’ve already purchased another.</strong> It’s a measure of the reliability of the “if-then” relationship between two items. For example, “People will likely purchase milk <strong>if they purchase bread</strong>.”</li><li><strong>Lift:</strong> This metric measures how much more likely a person is to buy one item in a set <strong>compared to the overall probability of that item being purchased.</strong> It tells you if there’s a truly meaningful relationship (a <strong>positive association</strong>) between the items or if their co-occurrence is just a coincidence.</li></ol><p>Now that we’ve grasped the core concepts of Apriori, let’s see how these metrics are used in a practical business scenario.</p><h3>The “How”: A Practical Example with a Local E-Commerce Store</h3><p>To see these metrics in action, let’s look at a small, hypothetical dataset from a local e-commerce platform called “Something Smell Good(s)”. Our objective is <strong>to find meaningful connections</strong> in their <strong>customer purchases</strong> that can <strong>lead to smarter business decisions</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/345/1*gzMxhRjnKGBUkbJu88vvCw.png" /><figcaption>Sample Datasets</figcaption></figure><p>Using this data, we’ll analyze the relationship between <strong>Bread</strong> and <strong>Jam</strong>.</p><h4>Step 1: Understanding Popularity (Support)</h4><p>The first step is to determine how popular the combination of Bread and Jam is across all of our customer transactions. This is measured by <strong>Support</strong>.</p><ul><li>Out of 6 total transactions, the combination of Bread and Jam appears in 2 of them.</li><li>This gives us a Support of <strong>33%</strong>.</li></ul><p>This tells us that <strong>the combination of Bread and Jam is a relatively common pairing</strong> among our customers. Because it’s a frequent occurrence, it’s worth our time to investigate further.</p><h4>Step 2: Measuring Reliability (Confidence)</h4><p>Next, we want to know how reliable this relationship is. This is where <strong>Confidence</strong> comes in. We’ll ask a crucial question: “Of all the customers who bought bread, what percentage also bought jam?”</p><ul><li>We found that 3 transactions included Bread.</li><li>Of those 3, 2 also included Jam.</li><li>This results in a Confidence of <strong>67%</strong>.</li></ul><p>This is a powerful insight on its own. It shows that there is a <strong>strong, predictable pattern</strong>: <strong>67% of the time, when a customer adds bread to their cart, jam is not far behind.</strong></p><h4>Step 3: Validating with Meaning (Lift)</h4><p>While a high confidence score is promising, we still need to confirm that this isn’t just a coincidence. This is the ultimate purpose of <strong>Lift</strong>. Lift helps us see if the relationship is truly meaningful, or if both items are simply popular on their own.</p><ul><li>By comparing our Confidence (67%) with the overall popularity of Jam (33% of all transactions), we arrive at a Lift score of <strong>2.03</strong>.</li></ul><p>A Lift score of over 1 is our green light. The result of <strong>2.03</strong> means that customers are <strong>2.03 times more likely to purchase jam when they buy bread, compared to their general likelihood of buying jam.</strong> This tells us that the relationship between these two products is significant and a perfect candidate for a strategic business decision.</p><h3>Beyond the Obvious: Turning Data into Actionable Insights</h3><p>With our calculations complete, we’ve moved beyond simple intuition. We’ve proven with data that there’s a strong, meaningful relationship between <strong>Bread and Jam</strong>. A traditional e-commerce platform might only suggest “popular items,” but with the power of the Apriori algorithm, we can find hidden opportunities.</p><p>This is where you can present the true value of your findings and translate them into a real-world strategy for “Something Smell Good(s)”.</p><h4>1. Strategic Product Bundling</h4><p>Instead of just listing both products separately, the company could create a “Breakfast Starter Pack” bundle that includes both the artisan bread and a jar of jam. By offering this as a single, convenient package, it makes it easier for customers to purchase both items, potentially increasing the average order value. This is particularly effective with a high-lift item combination, as customers are already inclined to buy them together.</p><h4>2. Targeted Cross-Selling</h4><p>The data tells us that a customer buying Bread is highly likely to also want Jam. This insight can be used to create personalized cross-selling prompts. As soon as a customer adds bread to their cart, a pop-up or a small notification could appear with the message: “Customers who bought this bread also loved our homemade jam!” This goes beyond generic recommendations and uses a data-backed prediction to guide the customer.</p><h4>3. Optimizing Website Layout</h4><p>The insights from this analysis can also inform the design of the e-commerce store itself. On the product page for Bread, it could strategically place Jam in a prominent “You might also like” section. This placement is not random; it’s a data-driven decision designed to maximize sales by putting related products right where the customer is most likely to see them.</p><p>By leveraging Support, Confidence, and especially Lift, businesses can move beyond guesswork and create a smarter, more profitable retail experience — both online and in physical stores. The Apriori algorithm isn’t just a statistical tool; it’s a powerful way to understand customer behavior and build a more intelligent business.</p><h3>The Road Ahead: Evolving Beyond Apriori</h3><p>The Apriori algorithm provides a powerful foundation for market basket analysis, but it’s just the beginning. While it excels at uncovering relationships based on co-occurrence, it doesn’t account for other crucial business factors like <strong>profit margins</strong> or the <strong>revenue</strong> each item drives. For instance, a rule might have high Lift, but if the items involved are low-margin products, the business impact may be minimal.</p><p>Moving forward, the evolution of market basket analysis involves integrating these key business metrics. Modern data science techniques and proprietary algorithms often combine the principles of Apriori with a product’s profitability or its ability to act as a “gateway” item that leads to higher-value purchases.</p><p>By building on these foundational insights, businesses can create even more sophisticated strategies — not just to sell more items, but to increase overall profitability and customer lifetime value.</p><p>If you found this article helpful, please give it a clap 👏 and follow me for more insights on using data to grow your business.</p><p><strong>Ready to talk strategy?</strong></p><p>For a complete, hands-on guide to implementing the Apriori algorithm and other powerful data science techniques, reach out to me directly at <strong>kdatainc@gmail.com</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49fd1682761b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pricing Optimization]]></title>
            <link>https://medium.com/@kdatainc/pricing-optimization-c51030f4b17d?source=rss-e9c6562f470c------2</link>
            <guid isPermaLink="false">https://medium.com/p/c51030f4b17d</guid>
            <category><![CDATA[pricing]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[KDataInc]]></dc:creator>
            <pubDate>Mon, 08 Aug 2022 06:34:02 GMT</pubDate>
            <atom:updated>2022-08-08T06:34:02.391Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RPlF1iXu7Bq3mPZD4pap0A.jpeg" /><figcaption>Image by <a href="https://pixabay.com/illustrations/price-tag-price-warranty-follower-374404/">pixabay</a></figcaption></figure><p>Well, I guess the headline of this article already gives all of you a great hint on what I am going to discuss here.</p><p>So, in every business settings, I personally think that pricing is the top 3 most important factors in order to maximize our revenue.</p><p>If a product is priced too high then most likely customers will switch to competitors, and on the other hand If the price is too low then the company is losing Its revenue over time.</p><p>The above scenario is just one of the many problems available, hence the main question is</p><blockquote><strong><em>“How much price should we give on any given product ?”</em></strong></blockquote><p>And I am going to show you how we can use python and Its library statsmodel to develop and optimize our pricing model.</p><p>Guidelines :</p><ol><li>Data Preparation</li><li>Developing Price Optimization</li><li>Summary</li></ol><h4><strong>1. Data Preparation</strong></h4><pre>#Load CSV Data<br>revenue = pd.read_csv(&#39;/kaggle/input/netflix2020/NetflixsRevenue2018toQ2_2020.csv&#39;)<br>subscriber = pd.read_csv(&#39;/kaggle/input/netflix2020/NetflixSubscribersbyCountryfrom2018toQ2_2020.csv&#39;)</pre><pre>#Convert Columns into Rows<br>revenue = pd.melt(revenue, id_vars = [&#39;Area&#39;], value_vars = revenue.columns[1:])<br>subscriber = pd.melt(subscriber, id_vars = [&#39;Area&#39;], value_vars = subscriber.columns[1:])</pre><pre>#Rename Columns for Readability<br>subscriber = subscriber.rename(columns = {&#39;value&#39; : &#39;customers&#39;})<br>revenue = revenue.rename(columns = {&#39;value&#39; : &#39;revenue&#39;})</pre><pre>df_merge = pd.merge(subscriber, revenue, on = [&#39;Area&#39;,&#39;variable&#39;], how = &#39;left&#39;)<br>df_merge</pre><pre>#Generate Price Columns per Quarter per Area<br>df_merge[&#39;avg_price&#39;] = round(df_merge[&#39;revenue&#39;]/df_merge[&#39;customers&#39;],2)<br>df_merge.sort_values(by = &#39;avg_price&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jp_FqErJoFpUVXy1VHY3HQ.png" /></figure><p>Here are the summary from the above steps :</p><ol><li>Load the CSV Files</li><li>Using pandas.melt to convert the Dataframes Columns into Rows</li><li>Changing our columns name</li><li>Merge both of the Revenue and Subscribers Data to generate a final summary table</li><li>Finally, since we don’t have the pricing data, we can generate It by dividing revenue with total customers</li></ol><p>After all those steps have been fulfilled, we can directly implement our optimization model to determine which price values provide the most optimal revenue per Area.</p><h4>2. Developing Price Optimization</h4><pre>#Developing our Pricing Model<br>from statsmodels.formula.api import ols</pre><pre>list_df = []<br>list_df_max = []<br>for a in df_merge[&#39;Area&#39;].unique() :<br>    df = df_merge[df_merge[&#39;Area&#39;] == a]</pre><pre>model = ols(&quot;customers~avg_price&quot;, data = df).fit()<br>    <br>    price = [x for x in df[&#39;avg_price&#39;]]<br>    revenue = []<br>    <br>    <br>    for b in price :<br>        demand = model.params[0] + (model.params[1]*b)<br>        revenue.append(demand*b)<br>        <br>    df_price = pd.DataFrame({&#39;area&#39; : a, &#39;price&#39; : price, &#39;revenue&#39; : revenue})<br>    df_price_max = df_price[df_price[&#39;revenue&#39;] == df_price[&#39;revenue&#39;].max()]<br>    list_df.append(df_price)<br>    list_df_max.append(df_price_max)<br>    <br>df_optimize = pd.concat(list_df_max, ignore_index = True)<br>df_optimize</pre><p>Here are the description for the above processes :</p><ol><li>From statsmodels.formula.api, we import ols ( Ordinary Least Squares ) as our main model</li><li>Then, we generate two lists for storing our generated dataframe ( one for the complete pricing list, and the other one for storing the maximum revenue )</li><li>Since every region/area has different economic factors, we would want to train our model separately based on the area ( Using For Loops based on Area )</li><li>In every Iterations, we will generate a dataframe and store It into the empty list we generated above.</li><li>For Maximum Revenue, we need to filter the dataframe based on MAX revenue per Area before storing It into the dataframe</li><li>Finally, combine all dataframes from the list to generate our final summary table.</li></ol><h4>Summary</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/714/1*AdqboT1X-bWDiEhript1jg.png" /></figure><p>This is the final result from the above codes, and as we can see, each Area has different sets of price with different revenue generated.</p><p>For pricing, this can be expanded into more factors such as category, cost, or other factors that matters to your business.</p><p>And that’s the end of It, I hope you find this article helpful to kick off your journey in developing your own pricing model.</p><p>Also, I am very interested to hear from all of you If you have any inputs or suggestions on how to make a better pricing model.</p><p>As usual, thank you and see you guys on the next article !</p><p><a href="https://www.kaggle.com/kelvinprawtama/pricing-optimization-for-maximizing-revenue">Pricing Optimization for Maximizing Revenue</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c51030f4b17d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RFM Model for Customer Scoring]]></title>
            <link>https://medium.com/@kdatainc/rfm-model-for-customer-scoring-92e66d7d4efb?source=rss-e9c6562f470c------2</link>
            <guid isPermaLink="false">https://medium.com/p/92e66d7d4efb</guid>
            <category><![CDATA[digital-marketing]]></category>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[customer]]></category>
            <category><![CDATA[data-analyst]]></category>
            <dc:creator><![CDATA[KDataInc]]></dc:creator>
            <pubDate>Sun, 19 Jun 2022 07:33:12 GMT</pubDate>
            <atom:updated>2022-06-19T07:33:12.699Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kWRFYWsHLCX13Nkan0PxGQ.jpeg" /><figcaption>Image by <a href="https://pixabay.com/photos/lego-star-wars-toys-stormtroopers-2539844/">pixabay</a></figcaption></figure><p>Recently, I have been looking for more analytical use cases where most digital companies might find It insightful and actionable.</p><p>I am sure some of you have heard RFM before, either from your colleagues or other sources that you might’ve read. If so, then I think It’s fine for you to still read the below guidance as a refresher.</p><p>RFM stands for R ( Recency ), F ( Frequency ), and M ( Monetary ). The Main Goal of this model is to group/rank our customers quantitatively based on the total of recency, frequency, and monetary value from the customers recent transaction data.</p><p>The RFM Model gives a 1 to 5 Scores from the main 3 Factors above and then the customers will be labelled based on Its final score ( e.g. High Value Customers, Low Value Customers, etc ).</p><p>Now, Let’s deep dive into Its Implementation so that you have a better idea on how It looks in python.</p><p>Guidelines :</p><ol><li>Data Preparation</li><li>Recency, Frequency, and Monetary Calculation</li><li>RFM Scoring</li></ol><h4>Data Preparation</h4><p>This is already a mandatory process in every analytics work, data preparation always involves, data cleaning, checking, and manipulation.</p><p>So, I have done several processes in the data which I won’t display here ( you can refer to the full code I attached at the end of this article ).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2rT23_rEqxG5RKOU0e1lXw.png" /></figure><p>So from the above data we have the following features we will use :</p><ol><li>Invoice : This is for calculating how many transactions a customer has made</li><li>Quantity and Price : The Two will be multiplied then sum to get the final revenue</li><li>Customer ID : This the Unique Customer ID to assign our final RFM Score</li><li>Invoice Date : The Date feature is crucial for calculating how recent the customer made the transaction.</li></ol><h4>Recency, Frequency, and Monetary</h4><p>After Preparing our dataset, we will generate the three main RFM Metrics.</p><h4>1. Recency</h4><pre>#Generate Recency Table</pre><pre>recency_table = new_df.groupby(&#39;Customer ID&#39;, as_index = False)[&#39;InvoiceDate&#39;].max()<br>recent_date = rfm_table[&#39;InvoiceDate&#39;].max()</pre><pre>recency_table[&#39;Recency&#39;] = rfm_table[&#39;InvoiceDate&#39;].apply(lambda x : (recent_date - x).days)<br>recency_table = recency_table.drop(recency_table.index[0])<br>recency_table.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/542/1*JB_uqY97rrgAY5zQtvpDVQ.png" /></figure><p>Here are the breakdowns from the above code :</p><ol><li>At First, we need to group our customer ID and get the latest date for each customer</li><li>Then, in order to get the <strong>Recency</strong> Value, get the maximum date from the grouped data and find the differences between each customer ID most recent date with the Max Date.</li><li>Finally, convert the final numbers into days value.</li></ol><p>After Recency has been generated, we’ll move on to the next RFM metrics, Frequency.</p><h4>2. Frequency</h4><pre>#Generate Frequency Table</pre><pre>frequency_table = new_df.groupby(&#39;Customer ID&#39;, as_index = False)[&#39;Invoice&#39;].count()<br>frequency_table.columns = [&#39;Customer ID&#39;,&#39;Frequency&#39;]<br>frequency_table = frequency_table.drop(frequency_table.index[0])<br>frequency_table.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*4Fp1buIO-9yjVRwYtCmbXQ.png" /></figure><p>The Below is the breakdown of the above code :</p><ol><li>Group the Customer ID like the above code</li><li>Count the Invoice per Customer ID</li></ol><p>These two steps will show how many total transaction that a single customer generated over a period of time.</p><p>Finally, we will jump into the monetary value on the RFM Model.</p><h4>3. Monetary</h4><pre>#Generate Monetary Table</pre><pre>new_df[&#39;Revenue&#39;] = new_df[&#39;Quantity&#39;]*new_df[&#39;Price&#39;]</pre><pre>monetary_table = new_df.groupby(&#39;Customer ID&#39;,as_index = False)[&#39;Revenue&#39;].sum()<br>monetary_table.columns = [&#39;Customer ID&#39;,&#39;Monetary&#39;]<br>monetary_table = monetary_table.drop(monetary_table.index[0])<br>monetary_table.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/1*BoyQPsLmtWNvkjGgjfaaTw.png" /></figure><p>The Monetary Value steps are similar to the above two metrics, the only differences are in the value used.</p><p>Since we want to know the Monetary Value per Customer ID, the we sum the Revenue grouped by customer ID to get the total revenue generated over a period of time.</p><p>After Preparing our RFM Value, we can move on to the final step of this model development which is scoring our customer.</p><h4>RFM Scoring</h4><pre>#Generate RFM DataFrame before we score our customer</pre><pre>rfm_df = recency_table.merge(frequency_table, on = &#39;Customer ID&#39;)<br>rfm_df = rfm_df.merge(monetary_table, on = &#39;Customer ID&#39;).drop(&#39;InvoiceDate&#39;, axis = 1)</pre><pre>#Ranking our Recency, Frequency, and Monetary Value</pre><pre>rfm_df[&#39;R_rank&#39;] = rfm_df[&#39;Recency&#39;].rank(ascending=False)<br>rfm_df[&#39;F_rank&#39;] = rfm_df[&#39;Frequency&#39;].rank(ascending=True)<br>rfm_df[&#39;M_rank&#39;] = rfm_df[&#39;Monetary&#39;].rank(ascending=True)</pre><pre>#Normalizing the Rank Value for RFM</pre><pre>rfm_df[&#39;R_rank_norm&#39;] = 100*(rfm_df[&#39;R_rank&#39;]/rfm_df[&#39;R_rank&#39;].max())<br>rfm_df[&#39;F_rank_norm&#39;] = 100*(rfm_df[&#39;F_rank&#39;]/rfm_df[&#39;F_rank&#39;].max())<br>rfm_df[&#39;M_rank_norm&#39;] = 100*(rfm_df[&#39;M_rank&#39;]/rfm_df[&#39;M_rank&#39;].max())</pre><pre>rfm_df = rfm_df.drop(columns = [&#39;R_rank&#39;,&#39;F_rank&#39;,&#39;M_rank&#39;])</pre><pre>#Finalizing our RFM Model by setting scoring for each Normalized Rank<br>#We will be capturing 20% from Recency, 30% from Frequency, and 50% from Monetary</pre><pre>rfm_df[&#39;RFM_Score&#39;] = 0.2*rfm_df[&#39;R_rank_norm&#39;] + 0.3*rfm_df[&#39;F_rank_norm&#39;] + 0.5*rfm_df[&#39;M_rank_norm&#39;]<br>rfm_df[&#39;RFM_Score&#39;] *= 0.05<br>rfm_df[&#39;RFM_Score&#39;] = round(rfm_df[&#39;RFM_Score&#39;],1)<br>rfm_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RxJ_I0aSIYXC8Olqdqvnmw.png" /></figure><p>Here are the step by step summary for understanding the above code :</p><ol><li>Merge our separated RFM data frame into a single data frame</li><li>Then We generate ranking per each RFM Metrics with higher ranking on lower Recency Value, higher ranking on higher Frequency Value, and higher ranking on higher Monetary Value.</li><li>Normalize per RFM Metrics Value by dividing the Value with the Maximum Value per RFM Metrics, and the result is multiplied by 100 to get the percentage value.</li><li>Finally, generate new columns for calculating the RFM Score using this three scoring rules 20% Recency, 30% Frequency, 50% Monetary.</li></ol><p>This is the distribution plot for the above scoring results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sT7QVQO7q6fGNHHXL159AQ.png" /></figure><p>We can see that the median and average for our customers are in the 2–3 range.</p><p>As for the labelling, there are no specific rules on how you want to do It, because since each department from different companies have different naming or profiling method, so I’ll put a stop here and let the audience be the judge of It.</p><p>That’s the end of this article, I hope you guys find this article helpful for developing your RFM Model.</p><p>Also, moving forward I am going to be posting more on Analytic Use Cases instead of Machine Learning Model since I have seen that many of you are more interested in one of my article, <a href="https://medium.com/mlearning-ai/cohort-analysis-in-python-abcc103917f0">Cohort Analysis</a>.</p><p>But If you have any suggestions or feedbacks, please feel free to comment down below and kindly support my article by subscribing to my newsletter.</p><p>Thank you again and all the best for your Analytic journey !</p><p>References :</p><ul><li><a href="https://www.techtarget.com/searchdatamanagement/definition/RFM-analysis">What is RFM analysis (recency, frequency, monetary)? | Definition from TechTarget</a></li><li><a href="https://www.kaggle.com/kelvinprawtama/rfm-model-for-retail-data">RFM Model for Retail Data</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=92e66d7d4efb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cohort Analysis for Customer Transaction Data]]></title>
            <link>https://medium.com/@kdatainc/cohort-analysis-in-python-abcc103917f0?source=rss-e9c6562f470c------2</link>
            <guid isPermaLink="false">https://medium.com/p/abcc103917f0</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[customer-retention]]></category>
            <category><![CDATA[digital-marketing]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <dc:creator><![CDATA[KDataInc]]></dc:creator>
            <pubDate>Sat, 23 Apr 2022 04:35:19 GMT</pubDate>
            <atom:updated>2022-05-01T06:09:04.454Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gtUc-J2WS8N-WI6JI1s9IA.jpeg" /><figcaption>Image by <a href="https://pixabay.com/photos/ecommerce-selling-online-2140604/">pixabay</a></figcaption></figure><p>Digital Marketing is the most sought after role in today’s online ecosystem, and I believe in each of you who work in the field have User Retentions as one of the main KPI.</p><p>But with abundant data, Excel files will be less effective in analyzing campaign results.</p><p>So, in this article, I’ll show how your Data/BI Analyst or maybe people from the marketing team can utilize Python for measuring user retention rate.</p><p>Guidelines :</p><ol><li>Data Cleaning and Data Manipulation</li><li>Create Cohort Month and Index</li><li>Generate User Retention Summary Table</li><li>Visualize the Result</li><li><strong>Data Cleaning and Manipulation</strong></li></ol><pre>!pip install openpyxl</pre><pre>df = pd.read_excel(&#39;/kaggle/input/customer-transaction-dataset/KPMG_VI_New_raw_data_update_final.xlsx&#39;, sheet_name = &#39;Transactions&#39;)</pre><pre>df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GFPkmvmr22zpy5pMgenPqQ.png" /><figcaption>Snapshot of Dataframe</figcaption></figure><p>Well, as you guys can see, from the data above, the first row at index 0 is the column name and we want to replace the current column with the first row.</p><pre>df.columns = df.iloc[0]<br>df.drop(df.index[0],inplace=True, axis = 0)</pre><pre>df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jzyfqtSBbUj0HzdX8totRg.png" /><figcaption>Snapshot After Replacing Columns</figcaption></figure><p>Now, the table looks like what we are familiar with, then moving on, we’ll select a few columns from the data above for our Cohort Analysis purposes.</p><pre>#Get the necessary columns for Cohort Analysis</pre><pre>df_final = df[[&#39;customer_id&#39;,&#39;transaction_date&#39;,&#39;online_order&#39;,&#39;order_status&#39;]]<br>df_final.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*0REgbsIlGMdqv8INjdktUw.png" /></figure><p>Since we want to measure user retention from users who have successfully completed a transaction, we’ll filter the data with ‘Approved’ status and remove duplicate values If any.</p><pre>df_final = df_final[df_final[&#39;order_status&#39;] == &#39;Approved&#39;]<br>df_final = df_final[~df_final.duplicated()]<br>df_final.info()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*YNj0NccJjf7AvCZqZYAR2w.png" /></figure><p>All set, and now we’ll create our Cohort Month and Index before calculating our user retention.</p><p><strong>2. Create Cohort Month and Index</strong></p><pre>#Get Transaction Month for the dataframe<br>import datetime as dt</pre><pre>def get_month(x) :<br>    return dt.datetime(x.year, x.month,1)</pre><pre>df_final[&#39;transaction_date&#39;] = pd.to_datetime(df[&#39;transaction_date&#39;])<br>df_final[&#39;transaction_month&#39;] = df[&#39;transaction_date&#39;].apply(get_month)</pre><pre>#Create Cohort Month per Rows</pre><pre>group = df_final.groupby(&#39;customer_id&#39;)[&#39;transaction_month&#39;]<br>df_final[&#39;cohort_month&#39;] = group.transform(&#39;min&#39;)</pre><pre>#Calculate Cohort Index for Each Rows</pre><pre>def get_date_int(df, column) :<br>    year = df[column].dt.year<br>    month = df[column].dt.month<br>    day = df[column].dt.day<br>    <br>    return year, month, day</pre><pre>transaction_year, transaction_month, transaction_day = get_date_int(df_final, &#39;transaction_month&#39;)<br>cohort_year, cohort_month, cohort_day = get_date_int(df_final,&#39;cohort_month&#39;)</pre><pre>#Calculate Year Differences<br>years_diff = transaction_year - cohort_year</pre><pre>#Calculate Month Differences<br>months_diff = transaction_month - cohort_month</pre><pre>df_final[&#39;cohort_index&#39;] = years_diff*12 + months_diff + 1</pre><pre>df_final.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y2ubb28NoiNuQr2HqlTfSw.png" /></figure><p>So, here are the step by step approach from the above code :</p><ol><li>Getting the ‘Month’ Values from each transaction_date data and Create a new column named ‘transaction_month’</li><li>Generate ‘cohort_month’ column based on each customer id first transaction month</li><li>Calculating the Differences between ‘transaction_month’ and ‘cohort_month’ and store the values into ‘cohort_index’ ( and the +1 is used for indicating the first month of the transaction instead of using 0 )</li></ol><p>All of the necessary data points for cohort analysis have been made and the next step is to calculate the retention rate per cohort index.</p><p><strong>3. Generate User Retention Summary Table</strong></p><pre>#Final Grouping to Calculate Total Unique Users in Each Cohort<br>cohort_group = df_final.groupby([&#39;cohort_month&#39;,&#39;cohort_index&#39;])</pre><pre>cohort_data = cohort_group[&#39;customer_id&#39;].apply(pd.Series.nunique)<br>cohort_data = cohort_data.reset_index()</pre><pre>cohort_counts = cohort_data.pivot_table(index = &#39;cohort_month&#39;,<br>                                        columns = &#39;cohort_index&#39;,<br>                                        values = &#39;customer_id&#39;   <br>                                       )</pre><pre>cohort_data.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/1*gaB8kARypFyMGptW-ejBHA.png" /></figure><p><strong>4. Visualize the Result</strong></p><pre>#Calculate Retention rate per Month Index</pre><pre>cohort_size = cohort_counts.iloc[:,0]</pre><pre>retention = cohort_counts.divide(cohort_size, axis = 0)</pre><pre>retention = retention.round(3)*100</pre><pre>retention.index = retention.index.strftime(&#39;%Y-%m&#39;)</pre><pre>#Plotting Heatmap for Retention Table</pre><pre>import matplotlib.pyplot as plt<br>import seaborn as sns</pre><pre>plt.figure(figsize = (16,10))</pre><pre>plt.title(&#39;MoM Retention Rate for Customer Transaction Data&#39;)</pre><pre>sns.heatmap(retention, annot = True, cmap=&quot;YlGnBu&quot;, fmt=&#39;g&#39;)</pre><pre>plt.xlabel(&#39;Cohort Index&#39;)<br>plt.ylabel(&#39;Cohort Month&#39;)<br>plt.yticks(rotation = &#39;360&#39;)</pre><pre>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hKixf_x_cbRNV4F8JVvx7w.png" /></figure><p>The Followings are the processes based on the above code :</p><ol><li>Calculate Total Unique User based on Its First Transaction Date and Index per Month</li><li>Generating Pivot Table from the calculated table and divided all the values based on total users in the first cohort month</li><li>Finally, Visualize the final table using heat map to better understand the data.</li></ol><p>That’s the end of this article, I hope that you guys can truly utilize the above code for your marketing campaign purposes.</p><p>Cohort Analysis has been used by many tech companies to determine whether the campaign is successful or not.</p><p>Again, thank you for giving me your support by clapping and saving my stories to your personal read.</p><p>Feel free to reach me out or giving comments down below so I can better understand what you guys want to read moving forward.</p><p>Thank you !</p><ul><li><a href="https://www.kaggle.com/code/kelvinprawtama/customer-transaction-cohort-analysis/notebook">Customer Transaction - Cohort Analysis</a></li><li><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=abcc103917f0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deep Learning Model for Lung Cancer Detection]]></title>
            <link>https://medium.com/@kdatainc/deep-learning-model-for-lung-cancer-detection-5219b27a0947?source=rss-e9c6562f470c------2</link>
            <guid isPermaLink="false">https://medium.com/p/5219b27a0947</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[cancer]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[keras]]></category>
            <dc:creator><![CDATA[KDataInc]]></dc:creator>
            <pubDate>Thu, 17 Mar 2022 15:27:41 GMT</pubDate>
            <atom:updated>2022-03-17T15:33:12.912Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/752/1*cJCGNRXhdhFYZnc2guOIMw.png" /><figcaption>Image by Canva</figcaption></figure><p>Cancer has been part of many people lives, and It always a heartfelt news when people from family, friends, and lovers tell us they have It.</p><p>There are many reasons on why someone can get this life threatening disease from genetics, habits, lifestyle, and many more. When people who are diagnosed with cancer, the most common option for them is to undergo a therapy or surgery.</p><p>It’s obvious that It’s better to prevent than curing the cancer, with prevention, many people can have a better and fulfilling life. But the problem is that not many people have the awareness of going to have their regular checkup and If there are small symptoms such as coughing or fatigue, most will just ignore them.</p><p>So, In this article, I am developing a deep learning model using Keras to detect If a particular person with sets of symptoms has cancer or not.</p><p>The dataset for this article can be downloaded <a href="https://www.kaggle.com/mysarahmadbhat/lung-cancer">here</a>. It contains the status of Lung Cancer for each individuals with their symptoms.</p><h4>Deep Learning Model for Lung Cancer Detection</h4><p>Objective : To Determine Whether an Individual has Cancer or Not using Keras Model</p><p>Guidelines :</p><ol><li>Data Preparation</li><li>Data Analysis and Visualization</li><li>Data Preprocessing</li><li>Train and Evaluate Model</li></ol><p>Column Description :</p><ol><li>Gender: M(male), F(female)</li><li>Age: Age of the patient</li><li>Smoking: YES=2 , NO=1.</li><li>Yellow fingers: YES=2 , NO=1.</li><li>Anxiety: YES=2 , NO=1.</li><li>Peer_pressure: YES=2 , NO=1.</li><li>Chronic Disease: YES=2 , NO=1.</li><li>Fatigue: YES=2 , NO=1.</li><li>Allergy: YES=2 , NO=1.</li><li>Wheezing: YES=2 , NO=1.</li><li>Alcohol: YES=2 , NO=1.</li><li>Coughing: YES=2 , NO=1.</li><li>Shortness of Breath: YES=2 , NO=1.</li><li>Swallowing Difficulty: YES=2 , NO=1.</li><li>Chest pain: YES=2 , NO=1.</li><li>Lung Cancer: YES , NO.</li></ol><h4><strong>1. Data Preparation</strong></h4><pre>df = pd.read_csv(&#39;/kaggle/input/lung-cancer/survey lung cancer.csv&#39;)<br>df.info()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*OKBnLcPtzWlVxmMsuIoM8g.png" /></figure><p>The dataset contains 309 rows with 16 columns and there are no null values from the dataset.</p><p>Let’s see If there are any duplicate rows from the data above.</p><pre>#Duplicate Checking</pre><pre>print(f&#39;Total Duplicated Rows : {df.duplicated().sum()}&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/1*xheDXhBnjL8d9vNcgVE7vg.png" /></figure><p>In the next code, we will remove the duplicate rows from our dataset to maintain accuracy for our model</p><pre>#Remove Duplicated Rows</pre><pre>df = df[~df.duplicated()]<br>print(f&#39;Total Duplicated Rows : {df.duplicated().sum()}&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*hrTewwtXMQoT6ByuUwgeeg.png" /></figure><p>After removing duplicate, we will substitute the columns containing string values into class ( 1 and 0 ).</p><pre>#Changing Value for Gender Column Male : 1, Female : 0</pre><pre>df[&#39;GENDER&#39;] = df[&#39;GENDER&#39;].replace({&#39;M&#39; : 1, &#39;F&#39; : 0})<br>df[&#39;GENDER&#39;].value_counts()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*nEExFw2j-uD_l-N16Xb-Xg.png" /></figure><pre>lung_key = {&#39;YES&#39; : 1, &#39;NO&#39; : 0}</pre><pre>df[&#39;LUNG_CANCER&#39;] = df[&#39;LUNG_CANCER&#39;].replace(lung_key)<br>df[&#39;LUNG_CANCER&#39;].unique()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/222/1*mYJwM1DD8_E4zBCTmWYyBA.png" /></figure><p>Now, our dataset is ready for analysis and model development.</p><h4>2. Data Analysis and Visualization</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*duokz5qvbSz5KXmQsJPo4A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZUewQC5nkvO0ATjJfTzTAw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yy8FsqMsIyI-LiPJd3Hl3A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3pq4zk6R4R1oBApn1HMNig.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-xnCx78aDFdrmiyfbNmMoQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lVayl-XL7caA_shBwmAk5Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JFefk4l3d06tLspRsumTRA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lPwHrSQBhjNTEx4O9-myCQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bLHdNZA1jFmMtpHaGe8MfA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KiRrRkPY96k_PtVUO9x9Fg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SpUYmGIKVhbi6dZ0wRsCmg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zeX6ZO6tWJ6BOF5DGExJZA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cyWm5ta-04MYihrXlFfwJg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5gCm0OWWy_7pzB28kzcIDg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZIKE4-ad1TyGmOGNoXtqmA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aNs5zX5lRYSWJDWIIy-bOg.png" /></figure><p>Key Insights :</p><ol><li>In total, Men has slightly higher chance of getting Lung Cancer compare to Female ( 125 vs 113 )</li><li>Based on Age distribution, people who get lung cancer are in their 60s</li><li>For people w/ Lung Cancer, the most common symptoms are Chest Pain, Shortness of Breath, Coughing, Fatigue, Allergy, and Wheezing</li><li>In addition to that, people who are alcoholic has higher risk in getting cancer compare to people who smoke.</li></ol><p>Now after we understand what the correlations between Lung Cancer and Its symptoms, the next step is to preprocess our data for training the model.</p><h4>3. Data Preprocessing</h4><pre>from keras.models import Sequential<br>from keras.layers import Dense<br>from sklearn.preprocessing import StandardScaler<br>from sklearn.model_selection import train_test_split<br>from sklearn.pipeline import Pipeline<br>from sklearn.compose import ColumnTransformer</pre><pre>#Splitting dataset</pre><pre>X,y = df[[&#39;CHEST PAIN&#39;,&#39;SHORTNESS OF BREATH&#39;,&#39;COUGHING&#39;,&#39;FATIGUE &#39;,&#39;ALLERGY &#39;,&#39;WHEEZING&#39;]], df[&#39;LUNG_CANCER&#39;]</pre><pre>X_train,X_test, y_train, y_test = train_test_split(X,y,test_size = 0.2, random_state= 42)</pre><pre>#Scale Features Pipeline<br>num_features = [x for x in range(len(X.columns))]<br>num_transformer = Pipeline(steps = [<br>    (&#39;scale&#39;, StandardScaler())<br>])</pre><pre>preprocessor = ColumnTransformer(transformers = [<br>    (&#39;num&#39;, num_transformer, num_features)<br>])</pre><pre>X_train = preprocessor.fit_transform(X_train)<br>print(X_train)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ggE_39CQX4QvPE_5-FFz3A.png" /></figure><p>After splitting our data and standardize the values from the features. It’s time to train the model.</p><h4>4. Train and Evaluate Model</h4><pre>model = Sequential()<br>    <br>model.add(Dense(6, input_shape = (6,), activation = &#39;relu&#39;))<br>model.add(Dense(6, activation = &#39;relu&#39;))<br>model.add(Dense(1, activation = &#39;sigmoid&#39;))<br>    <br>model.compile(loss = &#39;binary_crossentropy&#39;, optimizer = &#39;adam&#39;, metrics = [&#39;accuracy&#39;] )<br>    <br>model.fit(X_train,y_train, epochs= 100, batch_size = 5)<br>_,scores = model.evaluate(X_test,y_test, verbose = 0)<br>print(f&#39;Model Accuracy : {round(scores*100)}%&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/346/1*1BaESWz7f5IrZQA1Um_A4Q.png" /></figure><p>The above code is the steps needed to develop a deep learning model. So starting from our inputs, hidden layers, and outputs.</p><p>Then, the model is compiled based on Its loss, optimizer, and metrics. As for the loss argument, we put ‘binary_crossentropy’ since the target variable is a class label. The argument ‘adam’ is a replacement optimization algorithm for stochastic gradient descent for training deep learning models.</p><p>Finally, the model above showed 79% Accuracy which means It’s good enough for determining whether a single person with several symptoms and habits outlined above have cancer or not.</p><p>I hope that this article can encourage aspiring data scientist to tackle more on healthcare, by developing a robust model for determining a certain disease, I believe many can be saved.</p><p>Also, thank you so much for all the supports and claps, If you guys have any recommendations or comments or suggestions, feel free to give my any comments down below.</p><p>Stay Healthy and Stay Safe !</p><p>References :</p><ul><li><a href="https://www.kaggle.com/mysarahmadbhat/lung-cancer">Lung Cancer</a></li><li><a href="https://www.kaggle.com/kelvinprawtama/lung-cancer-detection-keras-dlm/notebook">Lung Cancer Detection - Keras DLM</a></li><li><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5219b27a0947" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Customers Profiling Using K-Means Clustering]]></title>
            <link>https://medium.com/@kdatainc/customers-profiling-using-k-means-clustering-2264c3c30775?source=rss-e9c6562f470c------2</link>
            <guid isPermaLink="false">https://medium.com/p/2264c3c30775</guid>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[customer-segmentation]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <dc:creator><![CDATA[KDataInc]]></dc:creator>
            <pubDate>Tue, 08 Mar 2022 15:49:31 GMT</pubDate>
            <atom:updated>2022-03-08T16:37:19.839Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*xpw7cvCpncQdG3fRyFR55A.jpeg" /><figcaption>Image By <a href="https://www.flickr.com/photos/arena_provietnam/4326152763/in/photolist-7AhEuK-bNRTYr-2jLjK8-D3gSyj-bn5rmB-CPwyF3-aDNXVy-aDNXVC-2kPu6ts-2mmjRCH-2mKkMDD-2n6Vwir-2jwVZeU-Co7hdS-aDP24N-aEALVo-aEwY8t-DMryM-asBUQc-aEADRG-czo48Q-3ewzM7-6ueRfu-614BRn-9jZaMs-267C5gP-bycueF-2jdmMVp-9bk9um-eZE1qS-9dzmwB-8sHnrT-8sHns6-95ULAc-68DFTd-9iVEUY-cLhvDN-2dzhv9g-nQDVQ4-2e8RPRW-2e8Se97-9dhMPt-8sHnpT-8sLq1m-8sLq17-7SYArZ-bZ76Mj-8sHnqc-bxKBCt-QJQyka">Link</a></figcaption></figure><p>Marketing team has been trying Its best to know more of who their customers are. By understanding more of their users, the team will have a better understanding on how to create a marketing campaign, promotion, special offers just to name a few based on the customer behavior.</p><p>In this article, I will demonstrate how to use K-Means Clustering to segment our customers based on their income and spending score from a Mall Dataset ( <a href="https://www.kaggle.com/kandij/mall-customers">Data Link</a> ).</p><h4>Clustering Model for Mall Customers Segmentation</h4><p>Objective : Create a Customer Profile based on Customers Income and Spending Score</p><p>Guidelines :</p><ol><li>Data Preparation, Cleaning, and Wrangling</li><li>Exploratory Data Analysis</li><li>Develop Cluster Models</li></ol><p>File Description :</p><ol><li>CustomerID : Unique ID for each customers</li><li>Genre : Sex of the users</li><li>Age : The users current length of life</li><li>Annual Income (k$) : Users Yearly Income in thousand dollars</li><li>Spending Score (1–100) : Users spending habit ( Higher Score means more spending and vice versa )</li></ol><h4>1. Data Preparation, Cleaning, and Wrangling</h4><pre>#Import Library and Load File<br>import pandas as pd<br>import numpy as np</pre><pre>df = pd.read_csv(&#39;/kaggle/input/mall-customers/Mall_Customers.csv&#39;)<br>df.info() <em>#checking data types and total null values</em></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*sD6k_luTJEk1xp6e4K1xFA.png" /><figcaption>Dataframe Summary</figcaption></figure><p>There are 5 columns and 200 rows in our data frame and no null values inside the data.</p><p>Let’s check If there are any duplicate rows inside the data frame.</p><pre><em>#Checking If any duplicated values </em><br><br>print(f&#39;Total Duplicated Rows : <strong>{</strong>df.duplicated().sum()<strong>}</strong>&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*5O6BOc5Q62BTsMo7CeKgUg.png" /></figure><p>Moving on, we will check each numerical columns percentile summary from 0–100th.</p><pre><em>#Let&#39;s see the percentile from each numerical columns from the dataset</em><br><br>def percentile(df, column) :<br>    print(f&#39;<strong>{</strong>column<strong>}</strong> Percentile Summary :&#39;)<br>    for a <strong>in</strong> range(0,101,10) :<br>        print(f&#39;- <strong>{</strong>a<strong>}</strong>th Percentile : <strong>{</strong>round(np.percentile(df[column],a),2)<strong>}</strong>&#39;)<br>        <br><em>#Percentile for Age</em><br>percentile(df, &#39;Age&#39;)</pre><pre><em>#Annual Income Percentile</em><br>percentile(df,&#39;Annual Income (k$)&#39;)</pre><pre><em>#Spending Score Percentile</em><br>percentile(df,&#39;Spending Score (1-100)&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/456/1*de4K2LuIBgBe9WQ-t3C81w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*tTYDU_vtGUNkelmvvK-u7w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*uKRN2XjzRnEqF2oa7yXPZg.png" /><figcaption>Numeric Columns Percentile Summary</figcaption></figure><pre><em>#Count Each Gender total </em><br><br>gender_total = df[&#39;Genre&#39;].value_counts().reset_index()<br>gender_total[&#39;perc_genre&#39;] = round(gender_total[&#39;Genre&#39;]/sum(gender_total[&#39;Genre&#39;]),2)*100<br>gender_total</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/440/1*kvjitfnkgxEncAvR15IBTA.png" /></figure><p>We have checked null, duplicated and showing the percentile of the numerical columns and total values for each unique values in categorical column.</p><p>Then, we’ll start to explore some of the data above to get better insights of our dataset.</p><h4>2. Exploratory Data Analysis</h4><pre>import matplotlib.pyplot as plt<br>import seaborn as sns<br>import plotly.express as px<br><br>num_cols = [&#39;Age&#39;,&#39;Annual Income (k$)&#39;,&#39;Spending Score (1-100)&#39;]<br>def plot_stats(df, col_list) :<br>    for a <strong>in</strong> num_cols :<br>        fig,ax = plt.subplots(1,2, figsize = (9,6))<br>    <br>        sns.distplot(df[a], ax = ax[0])<br>        sns.boxplot(df[a], ax = ax[1])<br>    <br>        ax[0].axvline(df[a].mean(), linestyle = &#39;--&#39;, linewidth = 2, color = &#39;green&#39;)<br>        ax[0].axvline(df[a].median(), linestyle = &#39;--&#39;, linewidth = 2 , color = &#39;red&#39;)<br>    <br>        ax[0].set_ylabel(&#39;Frequency&#39;)<br>        ax[0].set_title(&#39;Distribution Plot&#39;)<br>    <br>        ax[1].set_title(&#39;Box Plot&#39;)<br>    <br>        plt.suptitle(a)<br>        plt.show()<br>        <br>plot_stats(df, num_cols)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TRj8y3XpHJ9-xBA7HL2W-Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6GHJVozS9ju6YE_e4eB1SQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AN3MkGYwoh7Y2zS8LUzqYQ.png" /><figcaption>Distribution and Box Plot for Numerical Columns</figcaption></figure><p>Column ‘Age’ and ‘Annual Income (k$)’ are positively skewed and we want to replace the outliers with 10th and 90th Percentile to normalize the data.</p><pre><em>#Flooring and Capping by replacing outliers with 10th and 90th Percentile</em><br><br><em>#Age 10th Percentile and 90th Percentile </em><br>tenth_percentile_age = np.percentile(df[&#39;Age&#39;], 10)<br>ninetieth_percentile_age = np.percentile(df[&#39;Age&#39;], 90)<br><br>df[&#39;Age&#39;] = np.where(df[&#39;Age&#39;] &lt; tenth_percentile_age, tenth_percentile_age, df[&#39;Age&#39;])<br>df[&#39;Age&#39;] = np.where(df[&#39;Age&#39;] &gt; ninetieth_percentile_age, ninetieth_percentile_age, df[&#39;Age&#39;])<br><br><em>#Annual Income 10th Percentile and 90th Percentile</em><br>tenth_percentile_annualincome = np.percentile(df[&#39;Annual Income (k$)&#39;], 10)<br>ninetieth_percentile_annualincome = np.percentile(df[&#39;Annual Income (k$)&#39;], 90)<br>df[&#39;Annual Income (k$)&#39;] = np.where(df[&#39;Annual Income (k$)&#39;] &lt; tenth_percentile_annualincome, tenth_percentile_annualincome, df[&#39;Annual Income (k$)&#39;])<br>df[&#39;Annual Income (k$)&#39;] = np.where(df[&#39;Annual Income (k$)&#39;] &gt; ninetieth_percentile_annualincome, ninetieth_percentile_annualincome, df[&#39;Annual Income (k$)&#39;])</pre><pre>plot_stats(df, num_cols) <em>#Checking Distribution after replacing outliers with 10th and 90th Percentile</em></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pGAQLwWO5AoPIwlxPpwbnw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q4Sns1J5CM76zvAssNQWXg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cmYVy-SRo675j8SQV05Aeg.png" /><figcaption>Distribution and Box Plot After Replacing Outliers</figcaption></figure><p>The data has been normalized and as we can see from the graph above there were no outliers detected on the columns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LVADYpMup4Ja-mOE-9pPLw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*gj-KdOi_7l5GnUUL6NvGVQ.png" /></figure><p>The Majority of the customers are Female ( 56%) and based on the age group, people who go to mall are in the young adult (20–35 age group).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KdURNS4OGXo1wksmt2blKg.png" /></figure><p>Based on scatter plot above, we can see the majority of the customers are in the average income and average spending score. In addition to that, there are other 4 separate groups in our dataset based on income and spending score.</p><p>The Scatter Plot can be separated into the following category :</p><ol><li>High Income Low Spending</li><li>High Income High Spending</li><li>Average Income Average Spending</li><li>Low Income Low Spending</li><li>Low Income High Spending</li></ol><p>Moving on, we are going to label our data using the 5 Categories above.</p><h4>3. Develop Cluster Models</h4><pre>from sklearn.preprocessing import MinMaxScaler<br>from sklearn.decomposition import PCA<br>from sklearn.cluster import KMeans</pre><pre><em>#Normalize Numeric Features</em><br>scaled_features = MinMaxScaler().fit_transform(df.iloc[:,3:5])<br><br><em>#Get 2 Principal Components</em><br>pca = PCA(n_components = 2).fit(scaled_features)<br>features_2d = pca.transform(scaled_features)</pre><pre><em>#5 Centroids Model </em><br>model = KMeans(n_clusters = 5, init= &#39;k-means++&#39;, n_init = 100, max_iter = 1000, random_state=16)<br><br><em>#Fit to the data and predict the cluster assignments to each data points</em><br>feature = df.iloc[:,3:5]<br>km_clusters = model.fit_predict(feature.values)<br>km_clusters</pre><p>For developing our clustering models using KMeans, we need to scale/normalize the numeric features from our dataset.</p><p>In the above code, I am using MinMaxScaler() to transform features by scaling each feature to a given range. Followed by PCA, which basically is used for reducing dimensionality of large datasets.</p><p>I am using PCA in the current dataset just to set an example on how to use It in real application.</p><p>So, the data has been normalized and reduced then we can start train our clustering models to label the dataset.</p><p>Here are the generated results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*77WxzdM25G8kIg9mZpknOg.png" /><figcaption>Array Result After Model Training</figcaption></figure><p>Then, finally we’ll visualize the outcome using scatter plot to see our generated labels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/808/1*jIYcHm_k8yO8fAjKaiowrA.png" /></figure><p>For the code in developing above plot, I’ll post the link at the end of the article. But, for now, we can see that each group has been labelled based on the similarity in the dataset.</p><p>Here is the final outcome for the customer profiling.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WwUtLzL2qUnyoMcvOZh6pw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wb5nREMW9Qej8fKcha2AGQ.png" /></figure><p>And there you go, this is one of the most popular method in developing customer segmentation.</p><p>I hope that you find this article helpful in guiding you to create your own customer profile.</p><p>Thank you for reading this article and If you guys want to keep up to date on my post, kindly follow my medium profile !</p><p><strong>References :</strong></p><ul><li><a href="https://www.kaggle.com/kelvinprawtama/mall-customers-segmentation?scriptVersionId=89515113">Mall Customers Segmentation</a></li><li><a href="https://www.kaggle.com/kandij/mall-customers">Mall customers</a></li><li><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2264c3c30775" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>