<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anshuman on Medium]]></title>
        <description><![CDATA[Stories by Anshuman on Medium]]></description>
        <link>https://medium.com/@anshumand_46847?source=rss-4c791febdaf0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*dmbNkD5D-u45r44go_cf0g.png</url>
            <title>Stories by Anshuman on Medium</title>
            <link>https://medium.com/@anshumand_46847?source=rss-4c791febdaf0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 23:46:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@anshumand_46847/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Gaining Control Over Loan Servicing Health]]></title>
            <link>https://medium.com/@anshumand_46847/gaining-control-over-loan-servicing-health-f595bedc153d?source=rss-4c791febdaf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/f595bedc153d</guid>
            <category><![CDATA[tableau]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <category><![CDATA[case-study]]></category>
            <category><![CDATA[power-bi]]></category>
            <dc:creator><![CDATA[Anshuman]]></dc:creator>
            <pubDate>Wed, 13 Aug 2025 03:09:38 GMT</pubDate>
            <atom:updated>2025-08-13T03:09:38.872Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/941/1*3l3v2dGXWRqvkrkY7vPbkg.png" /></figure><p>In private lending, visibility into loan performance is critical for managing credit risk and maintaining financial health. Without centralized oversight, gaps in collections, delinquency issues, and escrow imbalances can quickly escalate.</p><p>We helped a private lending company take control of these challenges by implementing a Loan Servicing Dashboard that brought clarity and control to their operations.</p><p>With this solution, the company gained:</p><ul><li>Real-time monitoring of overall loan performance</li><li>Better delinquency management and early detection of issues</li><li>Escrow balance tracking to ensure sufficient funds</li></ul><p>These insights enabled the company to mitigate credit risks, improve collection efficiency, and strengthen overall financial stability — enhancing both operational effectiveness and strategic decision-making.</p><blockquote>📘 Explore the Full Case Study: <a href="https://www.perceptive-analytics.com/loan-servicing-dashboard/"><strong>Loan Servicing Dashboard Case Study</strong></a></blockquote><blockquote>📥 Download the Executive Summary PDF: <a href="https://www.perceptive-analytics.com/wp-content/uploads/2025/02/Loan-Servicing-Dashboard.pdf"><strong>Download the Loan Servicing Dashboard PDF</strong></a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f595bedc153d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How a Hospital Chain Improved Staff Scheduling, Resource Utilization, and Patient Care]]></title>
            <link>https://medium.com/@anshumand_46847/how-a-hospital-chain-improved-staff-scheduling-resource-utilization-and-patient-care-8506cdd40a21?source=rss-4c791febdaf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/8506cdd40a21</guid>
            <category><![CDATA[power-bi]]></category>
            <category><![CDATA[case-study]]></category>
            <category><![CDATA[tableau]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <dc:creator><![CDATA[Anshuman]]></dc:creator>
            <pubDate>Sun, 10 Aug 2025 12:10:29 GMT</pubDate>
            <atom:updated>2025-08-10T12:10:29.946Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-dCgx2waxC7WmPFY3k5o1A.png" /></figure><p>Efficient staff utilization is crucial for hospitals to deliver quality patient care while keeping operations cost-effective. Our client, a leading hospital chain, needed a better way to manage nurse distribution, identify critical staffing gaps, and ensure the right resources were available at the right time.</p><p>We helped them address these challenges with a custom utilization analytics dashboard that transformed their workforce management process.</p><h4>The Challenge</h4><p>The hospital chain faced inefficiencies in nurse allocation and scheduling. Without a unified view of staff availability and shift requirements, some departments were overstaffed while others experienced shortages. These imbalances led to operational bottlenecks, increased workload stress for certain teams, and inconsistent patient care.</p><h4>The Solution</h4><p>We developed a tailored dashboard that provided real-time visibility into staffing needs and availability across departments and shifts. The solution:</p><ul><li>Analyzed nurse requirements versus actual availability</li><li>Highlighted critical shifts and departments needing attention</li><li>Identified recurring scheduling inefficiencies</li><li>Enabled proactive adjustments to optimize distribution</li></ul><p>With these insights, hospital administrators could quickly resolve staffing gaps, ensure balanced workloads, and maintain consistent care delivery.</p><h3>The Outcome</h3><p>The dashboard empowered the hospital chain to improve operational efficiency and nurse allocation. They achieved better coverage for critical shifts, reduced workload imbalances, and enhanced patient care through consistent staffing. Leadership could now make data-driven workforce decisions faster and with greater confidence.</p><blockquote>📥 <strong>Download the full case study to explore the dashboard and results:</strong><br> <a href="https://www.perceptive-analytics.com/wp-content/uploads/2025/01/Resource-Utilisation-Dashboard-Perceptive-Analytics.pdf">Workforce Efficiency with Utilization Analytics — PDF</a></blockquote><blockquote>🔗 <strong>Read more on our website:</strong><br> <a href="https://www.perceptive-analytics.com/workforce-efficiency-with-utilization-analytics/">Workforce Efficiency with Utilization Analytics — Case Study</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8506cdd40a21" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How a Toy Manufacturer Improved Forecast Accuracy and Cross-Department Collaboration]]></title>
            <link>https://medium.com/@anshumand_46847/how-a-toy-manufacturer-improved-forecast-accuracy-and-cross-department-collaboration-6ddf90a3c3d4?source=rss-4c791febdaf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/6ddf90a3c3d4</guid>
            <category><![CDATA[case-study]]></category>
            <category><![CDATA[power-bi]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <category><![CDATA[tableau]]></category>
            <dc:creator><![CDATA[Anshuman]]></dc:creator>
            <pubDate>Fri, 08 Aug 2025 04:08:51 GMT</pubDate>
            <atom:updated>2025-08-08T04:08:51.776Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/979/1*NOQOJNzDMCaXPQz7iC3sEg.png" /></figure><p>For a toy manufacturer, sales forecasting isn’t just a numbers game — it’s the foundation of their entire business strategy. Toys must be designed, manufactured, and distributed months in advance to hit store shelves at the right time. In this industry, predicting demand <strong>at the item level</strong> a year ahead can make or break seasonal success.</p><h4>The Challenge</h4><p>The client’s forecasting process was <strong>email-driven and fragmented</strong>, making it hard to track updates and verify accuracy. This created several issues:</p><ul><li><strong>Low confidence in forecasts</strong> due to scattered communication</li><li><strong>Delayed updates</strong> that hindered timely decision-making</li><li><strong>Difficulty coordinating</strong> between multiple departments involved in production and sales</li></ul><p>They needed a solution that would centralize forecasting and make collaboration seamless.</p><h4>The Solution</h4><p>Perceptive Analytics developed a <strong>cloud-based collaborative web application</strong> designed for real-time, item-level forecasting. Key features included:</p><ul><li><strong>Centralized platform</strong> where all departments could share and update forecasts instantly</li><li><strong>Template upload functionality</strong> for standardized data collection</li><li><strong>Automated quality checks</strong> to ensure accuracy before data is finalized</li><li><strong>Role-based data access</strong> for secure, controlled information sharing</li></ul><p>This meant every stakeholder — from production to distribution — was working from the same, accurate, and up-to-date forecast.</p><h4>The Results</h4><p>The new forecasting tool delivered:</p><ul><li><strong>Greater forecast accuracy</strong>, enabling better production planning</li><li><strong>Faster updates</strong> through real-time collaboration</li><li><strong>Improved accountability</strong> with clear role-based access</li><li><strong>Higher confidence</strong> in data, reducing costly overproduction or stockouts</li></ul><p>By moving from email-based chaos to a streamlined cloud platform, the toy manufacturer gained the agility to plan ahead with confidence, ensuring their products reached the right stores at the right time.</p><blockquote>📥 <strong>Download the full case study:</strong><br> <a href="https://www.perceptive-analytics.com/wp-content/uploads/2018/07/Collaborative-Sales-Forecasting.pdf">Collaborative Sales Forecasting — PDF</a></blockquote><blockquote>🔗 <strong>Read it on our website:</strong><br> <a href="https://www.perceptive-analytics.com/collaborative-sales-forecasting/">Collaborative Sales Forecasting</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6ddf90a3c3d4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Financial Modeling to Raise Capital]]></title>
            <link>https://medium.com/@anshumand_46847/financial-modeling-to-raise-capital-5add8355553d?source=rss-4c791febdaf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/5add8355553d</guid>
            <category><![CDATA[use-cases]]></category>
            <category><![CDATA[case-study]]></category>
            <dc:creator><![CDATA[Anshuman]]></dc:creator>
            <pubDate>Tue, 29 Jul 2025 02:54:25 GMT</pubDate>
            <atom:updated>2025-07-29T02:54:25.405Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*G5JTRAvvmyLzV7zK5z7HMg.png" /></figure><h4>📈 <em>Plan Smart. Pitch Strong.</em></h4><p>We helped <strong>Insta Variety</strong>, a cloud-based identity and access management firm, secure investor funding with a custom financial model built for clarity and confidence.</p><p>The model offered:</p><ul><li>Integrated view of past performance and future forecasts</li><li>Scenario-based cash flow projections</li><li>Powerful what-if analysis to guide investor conversations</li></ul><p>With data in hand, the team negotiated better terms and built a solid foundation for growth.</p><p><strong>📘 Read the Full Case Study:</strong><br> <a href="https://www.perceptive-analytics.com/financial-modeling-raise-capital/">Financial Modeling to Raise Capital</a></p><p><strong>📥 Download the PDF Report:</strong><br> <a href="https://www.perceptive-analytics.com/wp-content/uploads/2017/03/Financial-Modeling-to-Raise-Capital.pdf">Download the Financial Modeling Case Study</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5add8355553d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Overview of Analytics in Marketing and Sales]]></title>
            <link>https://medium.com/@anshumand_46847/overview-of-analytics-in-marketing-and-sales-4980b3fcd668?source=rss-4c791febdaf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/4980b3fcd668</guid>
            <category><![CDATA[advanced-analytics]]></category>
            <category><![CDATA[power-bi]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[tableau]]></category>
            <dc:creator><![CDATA[Anshuman]]></dc:creator>
            <pubDate>Wed, 23 Jul 2025 18:25:50 GMT</pubDate>
            <atom:updated>2025-07-23T18:25:50.949Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZyKMBmlPd3eoa9_kpiDUPg.jpeg" /></figure><p>Every business would have silos of business data in its marketing/sales department. This data has hidden treasures. It contains information that can help target right audience more effectively, bring in more efficiency in the sales process and also forecast the future of business. To generate these insights from the large unorganized databases we use business analytics. These are some of the top applications of analytics in Marketing/Sales:</p><h3>Analytics on Consumer Behavior</h3><p>Analyzing consumer behavior would help us understand when, why, how, and where people do or do not buy a product. By understanding this we can bring in changes to our product and marketing strategy helping us attract more consumers.</p><h3>Marketing Mix Analytics</h3><p>Analyzing returns on marketing expenditure across various channels would help us evaluate the effectiveness of each of the marketing activities. These insights will help us to reallocate resources from a less effective to a more effective channel.</p><h3>Sales Force Analytics</h3><p>Analyzing sales process and team will help us diagnose the barriers to sales force performance. It will enable us bring more efficiency in sales force by providing us insights on issues like optimum sales territory size, optimum product bag size, the quality of leads and monthly target forecasts.</p><h3>Sales Pipeline Analytics</h3><p>Analyzing the flow of sale through several stages would help us find any loopholes in the sales cycle. It will help us evaluate an optimal time period for each stage to occur so that you can make sure your customers experience a quick and efficient transaction. It also helps determine the capacity of your sales process.</p><h3>Analytics on Communication Content</h3><p>Analyzing consumer behavior on the communication content helps us observe how consumers react to our marketing material. This gives us insights on how to draw consumer’s attention towards our products and will enable us convey our message clearly.</p><h3>Web Analytics</h3><p>Web analytics helps us understand user behavior on the web and consequently generate more leads and sales. It will provide insights to enhance the look and layout of your website and make it more user-friendly. With these insights we can also asses and strategize the effectiveness of marketing campaigns in order to get more valuable customers.</p><p>Hope this gave you a brief idea on how analytics is applied in marketing and sales. In the next few posts I will write about the above applications in detail. See you soon! :)</p><p>This article was originally published at Perceptive Analytics</p><p>At Perceptive Analytics, we help businesses convert data into strategic advantage. For over 20 years, we’ve served Fortune 500 companies and growing firms alike.<br> We’re trusted by leaders seeking <a href="https://www.perceptive-analytics.com/power-bi-consulting/"><strong>Power BI consulting</strong></a>, <a href="https://www.perceptive-analytics.com/microsoft-power-bi-developer-consultant/"><strong>Power BI implementation services</strong></a>, and <a href="https://www.perceptive-analytics.com/ai-consulting/"><strong>AI Consulting</strong></a> to solve critical data challenges and drive smarter decisions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4980b3fcd668" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How a Leading AI Finance Company Boosted Forecast Accuracy and Sales Performance]]></title>
            <link>https://medium.com/@anshumand_46847/how-a-leading-ai-finance-company-boosted-forecast-accuracy-and-sales-performance-f7351f3246de?source=rss-4c791febdaf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/f7351f3246de</guid>
            <category><![CDATA[real-time-data]]></category>
            <category><![CDATA[ai-case-studies]]></category>
            <category><![CDATA[case-study-help]]></category>
            <category><![CDATA[case-study]]></category>
            <dc:creator><![CDATA[Anshuman]]></dc:creator>
            <pubDate>Sat, 19 Jul 2025 19:58:44 GMT</pubDate>
            <atom:updated>2025-07-19T19:58:44.106Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*TgpetYbfnHWqmbNweN3Pzg.png" /></figure><p>In the fast-paced world of AI-powered financial solutions, sales and revenue teams need more than just data — they need clarity, speed, and precision.</p><p>We recently partnered with a high-growth AI finance solutions provider to help them <strong>optimize revenue forecasting and elevate sales performance</strong> through advanced analytics.</p><p>Our solution enabled:</p><ul><li>Deep analysis of <strong>bookings and revenue trends</strong></li><li>Granular visibility into <strong>pipeline opportunities</strong></li><li>Real-time <strong>sales benchmarking</strong> against targets</li></ul><p>These insights empowered their leadership to <strong>accurately forecast future revenue</strong>, <strong>prioritize high-value deals</strong>, and <strong>drive consistent progress toward sales goals</strong> — ultimately supporting sustained business growth.</p><p><strong>📘 Explore the Full Case Study:</strong><br> <a href="https://www.perceptive-analytics.com/sales-analysis-dashboard/">Sales Analysis Dashboard Case Study</a></p><p><strong>📥 Download the Executive Summary PDF:</strong><br> <a href="https://www.perceptive-analytics.com/wp-content/uploads/2025/02/Perceptive-Analytics-Sales-Analysis-Dashboard-Executive-Summary.pdf">Download the Sales Dashboard PDF</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f7351f3246de" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[10 Smart R Programming Tips to become Better R Programmer]]></title>
            <link>https://medium.com/@anshumand_46847/10-smart-r-programming-tips-to-become-better-r-programmer-8b2a0acab169?source=rss-4c791febdaf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/8b2a0acab169</guid>
            <category><![CDATA[r-programming]]></category>
            <dc:creator><![CDATA[Anshuman]]></dc:creator>
            <pubDate>Thu, 17 Jul 2025 03:12:56 GMT</pubDate>
            <atom:updated>2025-07-17T03:12:56.712Z</atom:updated>
            <content:encoded><![CDATA[<p>Coding is the process by which a programmer converts tasks from human-readable logic to machine-readable language. The reason behind coding being so popular is that there are so many ways to do the same thing that programmers don’t know the right choice anymore.</p><p>As a result, each programmer has his/her own style in writing implementations to the same part of an algorithm.</p><p>Writing code can sometimes be the most difficult and time-consuming part of any project. If the code is written in such a way that it is hard to change or requires a lot of work for every small update, then the investments will keep on piling up and more and more issues will crop up as the project progresses.</p><p>A good and well-written code is reusable, efficient and written cleverly by a smart programmer. This is what differentiates programmers from each other.</p><p>So, here are some tips to becoming a <strong>SMART coder</strong>:</p><h4>Table of contents:</h4><ul><li>Writing codes for Programmer, Developer, and Even for A Layman</li><li>Knowing how to improve the code</li><li>Writing robust code</li><li>When to use shortcuts and when not to use</li><li>Reduce effort through code reuse</li><li>Write planned out code</li><li>Active memory management</li><li>Remove redundant tasks</li><li>Learn to adapt</li><li>Peer review</li></ul><h4>1. Writing Codes for Programmer, Developer, and Even for A Layman</h4><p>Though codes are primarily written for the machine to understand. They should be structured and well organized for other developers or for any layman to understand. In reality, codes should be written for all the three.</p><p><strong>Those who keep this fact in mind are one step ahead of other coders</strong> while those who are able to make sure everyone can understand their code are miles ahead than their struggling friends.</p><p>Good programmers always document their codes and make use of <strong>IDE</strong>. I will use R language to explain the concept. Using IDE such as <strong>Rstudio</strong> makes it easier to write code quickly.</p><p>The main advantage available in almost all IDE is the <strong>auto-completion feature</strong> which suggests the function or command when part of it is written.</p><p>IDE is also known to suggest the syntax of the selected functions which saves time. Rstudio IDE environment also displays environment variables alongside with some basic details of each variable.</p><p>Documentation is another ability which differentiates good programmers from the rest.</p><p><strong>Let’s look at this viewpoint using an example.</strong> Say you read the following code:</p><h4>Code snippet 1</h4><p># Code snippet 1</p><p>a=16</p><p>b=a/2</p><p>c=(a+b)/2</p><h4>Code snippet 2</h4><p># Code snippet 2</p><p># store the max memory size</p><p>a=16</p><p># taking half of the maximum memory as the minimum memory</p><p>b=a/2</p><p># taking mean of maximum and minimum memory as the recommended memory</p><p>c=(a+b)/2</p><h4>Code snippet 3</h4><p># Code snippet 3</p><p># store the max memory size</p><p>max_mem=16</p><p># taking half of the maximum memory as the minimum memory</p><p>min_mem=max_mem/2</p><p># taking mean of maximum and minimum memory as the recommended memory</p><p>mean_mem=(max_mem+min_mem)/2</p><p>The difference in documentation is highlighted in these three code snippets and this is just a simple demonstration of code understandability.</p><p>The first code is difficult to understand. It just sets the values of three variables. There are no comments and the variable names do not explain anything.</p><p>The second code snippet explains that <strong>‘a’</strong> is the maximum memory, <strong>‘b’</strong> is the minimum memory and <strong>‘c’ </strong>is the mean of the two.</p><p>Without the comments in code snippet 2, no one can understand whether the calculation for ‘c’ is correct or not.</p><p>The third code is a step further with the variables representing what is stored in them.</p><p>The third code is the easiest to understand even though all the three codes perform similar tasks. Moreover, when the variables are used elsewhere, the variables used in the third snippet are self-explanatory and will not require a programmer to search in the code for what they store until an error occurs in the code.</p><h4>2. Knowing how to Improve</h4><p>R has multiple ways to achieve a task. Each of the possibilities comes from using more memory, faster execution or different algorithm/logic.</p><p>Whenever possible, good programmers make this choice wisely.</p><p>R has the feature to execute code in parallel. Lengthy tasks such as fitting models can be executed in parallel, resulting in time-saving. Other tasks can also be executed faster based on the logic and packages used.</p><p>As an illustration, the following code snippets reflects the same task, one with <strong>sqldf</strong> package and another with <strong>dplyr</strong> package.</p><p>These practices are foundational not only for efficient programming, but also for building scalable <a href="https://www.perceptive-analytics.com/ai-consulting/">AI and machine learning solutions</a>.</p><h4>Using sqldf version</h4><p>1</p><p>2</p><p>3</p><p>4</p><p>5</p><p>6</p><p>7</p><p># Using sqldf version</p><p>install.packages(“sqldf”)</p><p>library(sqldf)</p><p>Out_df=sqldf(“select * from table_a left outer join table_b on table_a.var_x=table_b.var_x”)</p><h4>Using dplyr version</h4><p># Using dplyr version</p><p>install.packages(“dplyr”)</p><p>library(dplyr)</p><p>Out_df=left_join(table_a,table_b)</p><p>I personally prefer the <strong>dplyr</strong> version whenever possible. However, there are some differences between the outputs.</p><p>The <strong>dplyr version</strong> will look at all variables with the same name and join using them. If there is more than one such variable, I need to use them by field. Moreover, left join using dplyr will not keep both copies of the variable used to join tables whereas sqldf does.</p><p>One advantage of <strong>sqldf</strong> is that sqldf is not case sensitive and can easily join tables even if the variable names in the two tables are completely different. <strong>However, it is slower than dplyr.</strong></p><h4>3. Writing Robust Code</h4><p>While writing code, you can make the code simple but situation specific or write a generic code. One such way in which programmers write simple but situation-specific code is by <strong>‘Hard Coding’.</strong></p><p>It is the term given to fixing values of variables and is never recommended.</p><p>For example, dividing the sum of all salaries in a 50,000-row salary data by 50,000 rather than dividing the sum of that sum with the number of rows may seem to make the same sense but have a different meaning in programming.</p><p>If the data changes with the change in the number of rows, the number 50,000 needs to be searched and updated. If the programmer misses making the small change, all the work goes down the drain. On the other hand, the latter approach automatically does the task and is a robust method.</p><p>Another popular programming issue quite specific to languages such as R is Code Portability. Codes running on one computer may not work on another because the other computer does not have some packages installed or has outdated packages.</p><p>Such cases can be handled by checking for installed packages first and then installing them. These tasks can be collectively called as robust programming and make the code error free.</p><p>Using an illustration for checking and installing/updating <strong>h2o</strong> package.</p><p># If h2o package is already loaded, unload it and uninstall</p><p>if (“package:h2o” %in% search()) { detach(“package:h2o”, unload=TRUE) }</p><p># Checking</p><p>if (“h2o” %in% rownames(installed.packages())) { remove.packages(“h2o”) }</p><p># Next, we download packages that H2O depends on.</p><p># methods</p><p>if (! (“methods” %in% rownames(installed.packages()))) { install.packages(“methods”) }</p><p># statmod</p><p>if (! (“statmod” %in% rownames(installed.packages()))) { install.packages(“statmod”) }</p><p># stats</p><p>if (! (“stats” %in% rownames(installed.packages()))) { install.packages(“stats”) }</p><p># graphics</p><p>if (! (“graphics” %in% rownames(installed.packages()))) { install.packages(“graphics”) }</p><p># Rcurl</p><p>if (! (“RCurl” %in% rownames(installed.packages()))) { install.packages(“RCurl”) }</p><p># jsonlite</p><p>if (! (“jsonlite” %in% rownames(installed.packages()))) { install.packages(“jsonlite”) }</p><p># tools</p><p>if (! (“tools” %in% rownames(installed.packages()))) { install.packages(“tools”) }</p><p># utils</p><p>if (! (“utils” %in% rownames(installed.packages()))) { install.packages(“utils”) }</p><p># Finally install and load h2o package</p><p>install.packages(“h20”)</p><p>library(h2o)</p><h4>4. When to Use Shortcuts and When Not to</h4><p>Using shortcuts may be tempting in the pursuit of writing code swiftly but the right practice is to know when to use them.</p><p>For instance, shortcut keys are something which is really helpful and can always be used. Using <strong>Ctrl+L</strong> in windows clears the console output screen, Using <strong>Ctrl+Shift+C</strong> in windows comments and un-comments all selected lines of code in one go are my favorite shortcuts in Rstudio.</p><p>Another shortcut is writing code for fixing code temporarily or writing faulty fixes which are not desired.</p><p>Here are some of the examples of faulty fixes.</p><h4>This code changes a particular column name without checking its existing name</h4><p>1</p><p>2</p><p>3</p><p># This code changes a particular column name without checking its existing name</p><p>colnames(data_f)[5]=”new_name”</p><h4>This removes certain columns using a number. This may remove important ones and code may give the error if the number of columns less than 10 in this case.</h4><p>1</p><p>2</p><p>3</p><p># This removes certain columns using a number. This may remove important ones and code may give error if the number of columns are less than 10 in this case</p><p>data_f=data_f[,1:4,6:10]</p><h4>This converts a value to numeric without checking if it actually has all numbers. If the value does not contain numbers, it may produce NAs by coercion</h4><p>1</p><p>2</p><p>3</p><p># This converts a value to numeric without checking if it actually has all numbers. If the value does not contain numbers, it may produce NAs by coercion</p><p>Num_val=”123″</p><h4>The following converts Num_val to 123 correctly</h4><p>1</p><p>2</p><p>3</p><p>4</p><p>5</p><p># The following converts Num_val to 123 correctly</p><p>Num_val=as.numeric(Num_val)</p><p>char_val=”A_Name”</p><h4>The following issues a warning and converts Num_val to NA as it is not a number</h4><p>1</p><p>2</p><p>3</p><p># The following issues a warning and converts Num_val to NA as it is not a number</p><p>char_val=as.numeric(char_val)</p><h4>5. Reduce Effort Through Code Reuse</h4><p>When you start writing a code, you don’t need to waste time if a particular piece of logic has already been written for you. Better known as <strong>“Code Re-use”,</strong> you can always use your own code you previously wrote or even google to reach out the large R community.</p><p>Don’t be afraid to search. Looking up for already implemented solutions online is very helpful in learning the methods prevalent for similar situations and the pros and cons associated with them.</p><p>Even when it becomes necessary to reinvent the wheel, the existing solutions can serve as a benchmark to test your new solution. An equally important part of writing code is to make your own code reusable.</p><p>Here are two snippets which highlight reusability.</p><h4>Code which needs to be edited before resuing it</h4><p>1</p><p>2</p><p>3</p><p>4</p><p>5</p><p>6</p><p>7</p><p># Code which needs to be edited before reusing it</p><p>for(i in 1:501) {</p><p>df[,i]=as.numeric(df[,i])</p><p>}</p><h4>Code which can be reused with lesser editing</h4><p>1</p><p>2</p><p>3</p><p>4</p><p>5</p><p>6</p><p>7</p><p># Code which can be reused with lesser editing</p><p>for(i in 1:ncol(df)) {</p><p>df[,i]=as.numeric(df[,i])</p><p>}</p><h4>6. Write Planned Out Code</h4><p>Writing code on the fly may be a cool-to-have skill but not helpful for writing efficient codes. Coding is most efficient when you know what you are writing.</p><p>Always plan and write your logic on a piece of paper before implementing it. Inculcating the habit of adding tabs and spaces and basic formatting as you code is another time-saving skill for a good programmer.</p><p>For instance, every new ‘if’, ‘for’ or ‘while’ statement can be followed by tabs so that indentation is clearly visible. Although optional, such actions separate out blocks of code and helpful in identifying breakpoints as well as debugging.</p><p>A more rigorous but helpful approach is to write code using functions and modules and explaining every section with examples in comments or printing progress inside loops and conditions. Ultimately it all depends on the programmer how he/she chooses to document and log in the code.</p><h4>7. Active Memory Management</h4><p>Adding memory handling code is like handling a double-edged sword. It may not be useful for small-scale programs due to a slowdown in execution speed but nevertheless a great skill to have for writing scalable code.</p><p>In Rstudio, removing variables and frames when they are no longer required with the <strong>rm()</strong> function, garbage collection using <strong>gc()</strong> command and selecting the relevant features and data for proceeding are ways to manage memory.</p><p>Adjusting RAM usage with <strong>memory.limit()</strong> and setting parallel processing are also tasks for managing your memory usage. Remember! Memory management goes hand in hand with data backup.</p><p>It only takes a few seconds create and store copies of data. It should be done to ensure that data loss does not occur if backtracking is required.</p><p>Have a look at this example snippet which stores the master data and then frees up memory.</p><p># dividing master dataset into train and test with ratio 7:3</p><p>library(dplyr)</p><p>train&lt;-sample_frac(master_data, 0.7</p><p>train_ind&lt;-as.numeric(rownames(train))</p><p>test&lt;-master_data[-train_ind,]</p><p># saving backup of master_data and removing unneeded data</p><p>write.csv(master_data,”master_data.csv”)</p><p>rm(master_data)</p><p>rm(train_ind)</p><p>gc()&lt;span style=”font-family: -apple-system, BlinkMacSystemFont, ‘Segoe UI’, Roboto, Oxygen-Sans, Ubuntu, Cantarell, ‘Helvetica Neue’, sans-serif; font-size: 16px; background-color: #ffffff;”&gt; &lt;/span&gt;</p><h4>8. Remove Redundant Tasks</h4><p>Sometimes programmers do some tasks repeatedly or forget to remove program code without knowing it.</p><p>Writing separate iterations for each data manipulation step, leaving libraries loaded even after they are no longer required, not removing features until the last moment, multiple joins and queries,etc. are some examples of <strong>redundancy lurking</strong> in your code.</p><p>While these happen somewhat naturally as more and more changes are made and new logic is added. It is a good practice to look at existing code and adjust your new lines to save runtime.</p><p>Redundancy can slow your code so much that removing it can do wonders in execution speed.</p><p># Redundant code</p><p># Takes about 0.5 seconds for iris data</p><p>for(i in 1:ncol(df)) {</p><p>df[,i]=as.numeric(df[,i])</p><p>}</p><p>for(i in 1:ncol(df)) {</p><p>#storing missing values per column in mis vector</p><p>mis[i]=length(which(is.na(df[,i])))</p><p>}</p><p>#Better implementation (implementations faster than the one below also exist)</p><p>#Gives a similar output but takes about 0.3 seconds for iris data — 35% improvement</p><p>for(i in 1:ncol(df)) {</p><p>df[,i]=as.numeric(df[,i])</p><p>#storing missing values per column in mis vector</p><p>mis[i]=length(which(is.na(df[,i])))</p><p>}</p><h4>9. Learn to Adapt</h4><p>No matter how good a programmer you are, you can always be better! This tip is not related to typical coding practices but teamwork. Sharing and understanding codes from peers, Reading codes online (such as from repositories).</p><p>setting yourself up to date with books and blogs and learning about new technologies and packages which are released for R are some ways to learn.</p><p>Being flexible and adaptive to new methods and keeping yourself up to date with what’s happening in the analytics industry today can help you in avoiding becoming obsolete with old practices.</p><h4>10. Peer Review</h4><p>The code you write may be straightforward for you but very complex for everyone else. How will you know that? The only way is to know what others think about it.</p><p>Code review is thus the last but not the least in terms of importance for better coding. Ask people to go through your code and be open to suggested edits. You may come across situations when some code you thought is written beautifully can be replaced with more efficient code.</p><p>Code review is a process which helps both the coder and reviewer as it is a way of helping each other to improve and move forward.</p><h4>The Path is Not So Difficult: Conclusion</h4><p>Becoming a good programmer is no easy feat but becoming better at programming as you progress is possible. Though it will take time, persevering to add strong programming habits will make you a strong member in every team’s arsenal.</p><p>These tips are just the beginning and there may be more ways to improve. The knowledge to always keep improving will take you forward and let you taste the sweet results of being a hi-tech programmer.</p><p>In the rapidly changing analytics world, staying with the latest tools and techniques is a priority and being good at R programming can be a prime factor towards your progress in your analytics career.</p><p>So go out there and make yourself acquainted with the techniques of becoming better at R programming.</p><p>At Perceptive Analytics, our mission is simple: “to enable businesses unlock value in data.” With 20+ years of experience, we’ve served over 100 clients — including Fortune 500 firms — by delivering actionable insights and scalable analytics solutions.</p><p>Recognized among top <a href="https://www.perceptive-analytics.com/tableau-consulting/"><strong>Tableau Consulting Companies</strong></a>, and also we offer expert <a href="https://www.perceptive-analytics.com/power-bi-consulting/"><strong>Power BI consulting</strong></a> and <a href="https://www.perceptive-analytics.com/microsoft-power-bi-developer-consultant/"><strong>Power BI consultant</strong></a> services to help businesses build powerful dashboards and make confident, data-driven decisions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8b2a0acab169" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Assumptions of K-means Clustering using R]]></title>
            <link>https://medium.com/@anshumand_46847/exploring-assumptions-of-k-means-clustering-using-r-6b0f504d62a2?source=rss-4c791febdaf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b0f504d62a2</guid>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[rlang]]></category>
            <category><![CDATA[r-language]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Anshuman]]></dc:creator>
            <pubDate>Fri, 04 Jul 2025 08:41:59 GMT</pubDate>
            <atom:updated>2025-07-04T08:41:59.619Z</atom:updated>
            <content:encoded><![CDATA[<p>K-Means Clustering is a well known technique based on unsupervised learning. As the name mentions, it forms ‘K’ clusters over the data using mean of the data. Unsupervised algorithms are a class of algorithms one should tread on carefully. Using the wrong algorithm will give completely botched up results and all the effort will go down the drain. Unlike supervised learning algorithms where one can still get around keeping some parts as an unknown black box, knowing the technique inside out starting from the assumptions made to the process, methods of optimization and uses is essential. So let us begin step by step starting from the assumptions. I will then explain the process and a hands-on illustration using R</p><h4>Assumptions and Process</h4><p>Why do we assume in the first place? The answer is that making assumptions helps simplify problems and simplified problems can then be solved accurately. To divide your dataset into clusters, one must define the criteria of a cluster and those make the assumptions for the technique. K-Means clustering method considers two assumptions regarding the clusters — first that the clusters are spherical and second that the clusters are of similar size. Spherical assumption helps in separating the clusters when the algorithm works on the data and forms clusters. If this assumption is violated, the clusters formed may not be what one expects. On the other hand, assumption over the size of clusters helps in deciding the boundaries of the cluster. This assumption helps in calculating the number of data points each cluster should have. This assumption also gives an advantage. Clusters in K-means are defined by taking the mean of all the data points in the cluster. With this assumption, one can start with the centers of clusters anywhere. Keeping the starting points of the clusters anywhere will still make the algorithm converge with the same final clusters as keeping the centers as far apart as possible.</p><p>Now let’s understand how the algorithm works. The first step is to assign initial clusters. You can specify any K clusters or let the algorithm assign them randomly. The algorithm works in iterations and in every iteration, all the data points are then assigned to one of the clusters based on the nearest distance from the centers. After all points are assigned to one of the cluster, the cluster centers are now updated. The new centers are decided based on the centroid mean of all the points within the cluster. This is repeated, iteration after iteration until the there is no change in the cluster assignment of any of the data points but there are a lot of calculations which are not fixed in this algorithm. For example, one can decide how the distance for each data point from the cluster center is defined. All of us are familiar with the Euclidean distance. The algorithm is straightforward and easy to understand but using the technique is not as easy as it looks. Let’s try out some examples in R.</p><h4>K-Means Starter</h4><p>To understand how K-Means works, we start with an example where all our assumptions hold. R includes a dataset about waiting time between eruptions and the duration of the eruption for the Old Faithful geyser in Yellowstone National Park known as ‘faithful’. The dataset consists of 272 observations of 2 features.</p><p>#Viewing the Faithful dataset</p><p>plot(faithful)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*A_FieSkrl75VUpd0" /></figure><p>Looking at the dataset, we can notice two clusters. <br> I will now use the kmeans() function in R to form clusters. <br> Let’s see how K-Means clustering works on the data</p><p>#Specify 2 centers</p><p>k_clust_start=kmeans(faithful, centers=2)</p><p>#Plot the data using clusters</p><p>plot(faithful, col=k_clust_start$cluster,pch=2)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*qRARExIasCZqpcQm" /></figure><p>Being a small dataset, clusters are formed almost instantaneously but how do we see the clusters, their centers or sizes? The k_clust_start variable I used contains information on both centers and the size of clusters. Let’s check them out</p><p>#Use the centers to find the cluster centers</p><p>k_clust_start$centers</p><p>eruptions waiting</p><p>1 4.29793 80.28488</p><p>2 2.09433 54.75000</p><p>#Use the size to find the cluster sizes</p><p>k_clust_start$size</p><p>[1] 172 100</p><p>This means the first cluster consists of 172 members and is centered at 4.29793 value of eruptions and 80.28488 value of waiting. Similarly the second cluster consists of 100 members with 2.09433 value of eruptions and 54.75 value of waiting. Now this information is golden! We know that these centers are the cluster means. So, the eruptions typically happen for either ~2 mins or ~4.3 mins. For longer eruptions, the waiting time is also longer.</p><p>Getting into the depths</p><p>Imagine a dataset which has clusters which one can clearly identify but k-means cannot. I’m talking about a dataset which does not satisfy the assumptions. A common example is a dataset which represents two concentric circles. Let’s generate it and see how it looks like</p><p>#The following code will generate different plots for you but they will be similar</p><p>library(plyr)</p><p>library(dplyr)</p><p>#Generate random data which will be first cluster</p><p>clust1 = data_frame(x = rnorm(200), y = rnorm(200))</p><p>#Generate the second cluster which will ‘surround’ the first cluster</p><p>clust2 =data_frame(r = rnorm(200, 15, .5), theta = runif(200, 0, 2 * pi),</p><p>x = r * cos(theta), y = r * sin(theta)) %&gt;%</p><p>dplyr::select(x, y)</p><p>#Combine the data</p><p>dataset_cir= rbind(clust1, clust2)</p><p>#see the plot</p><p>plot(dataset_cir)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*9ny0zVYvRc04nntE" /></figure><p>Simple, isn’t it? There are two clusters — one in the middle and the other circling the first. However, this violates the assumption that the clusters are spherical. The inner data is spherical while the outer circle is not. Even though the clustering will not be good, let’s see how does k-means perform on this data</p><p>1</p><p>2</p><p>3</p><p>4</p><p>#Fit the k-means model</p><p>k_clust_spher1=kmeans(dataset_cir, centers=2)</p><p>#Plot the data and clusters</p><p>plot(dataset_cir, col=k_clust_spher1$cluster,pch=2)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*4FzPG-834RCJekcQ" /></figure><p>How do we solve this problem? There are clearly 2 clusters but k-means is not working well. A simple way in this case is to transform our data into polar format. Let’s convert it and plot it.</p><p>#Using a function for transformation</p><p>cart2pol=function(x,y){</p><p>#This is r</p><p>newx=sqrt(x^2 + y^2)</p><p>#This is theta</p><p>newy=atan(y/x)</p><p>x_y=cbind(newx,newy)</p><p>return(x_y)</p><p>}</p><p>dataset_cir2=cart2pol(dataset_cir$x,dataset_cir$y)</p><p>plot(dataset_cir2)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*IHchrkZ1JlI7a2Un" /></figure><p>Now we run the k-means model on this data</p><p>k_clust_spher2=kmeans(dataset_cir2, centers=2)</p><p>#Plot the data and clusters</p><p>plot(dataset_cir2, col=k_clust_spher2$cluster,pch=2)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*1GdY0gkoLqcDBSdZ" /></figure><p>This time k-means algorithm works well and correctly transform the data. We can also view the clusters on the original data to double-check this.</p><p>1</p><p>plot(dataset_cir, col=k_clust_spher2$cluster,pch=2)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*8PEw4QsKheJWfH8F" /></figure><p>By transforming our data into polar coordinates and fitting k-means model on the transformed data, we fulfil the spherical data assumption and data is accurately clustered. Transforming data before clustering is key. We use similar preprocessing techniques in our <a href="https://www.perceptive-analytics.com/talend-consultants/">Talend consulting</a> to prepare high-quality data for analysis.<br>Now let’s look at a data where the clusters are not of similar sizes. Similar size does not mean that the clusters have to be exactly equal. It simply means that no cluster should have ‘too few’ members.</p><p>#Make the first cluster with 1000 random values</p><p>clust1 = data_frame(x = rnorm(1000), y = rnorm(1000))</p><p>#Keep 10 values together to make the second cluster</p><p>clust2=data_frame(x=c(5,5.1,5.2,5.3,5.4,5,5.1,5.2,5.3,5.4),y=c(5,5,5,5,5,4.9,4.9,4.9,4.9,4.9))</p><p>#Combine the data</p><p>dataset_uneven=rbind(clust1,clust2)</p><p>plot(dataset_uneven)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*HhhcmT5cEw0ms8ZH" /></figure><p>Here again, we have two clear clusters but they do not satisfy similar size requirement of k-means algorithm.</p><p>1</p><p>2</p><p>k_clust_spher3=kmeans(dataset_uneven, centers=2)</p><p>plot(dataset_uneven, col=k_clust_spher3$cluster,pch=2)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*LCAsA1uVe1IQFtoh" /></figure><p>Why did this happen? K-means tries to minimize inter cluster and intracluster distance and create ‘tight’ clusters. In this process, it assigns some data points in the first cluster to the second cluster incorrectly. This makes the clustering inaccurate.</p><h3>How to decide the value of K in data</h3><p>The datasets I worked on in this article are all simple and it is easily to identify clusters by plotting them. However, complicated datasets do not have this luxury. The Elbow method is popular for finding a suitable value of ‘K’ for k-means clustering. This method uses SSE within groups for different values of k and plots them. Using this plot, we can choose the ‘k’ which shows an abrupt change in SSE, creating an ‘elbow effect’. I will show an illustration on iris dataset using petal width and petal length.</p><p>#Create a vector for storing the sse</p><p>sse=vector(&#39;numeric&#39;)</p><p>for(i in 2:15){</p><p>#k-means function in R has a feature withinss which stores sse for each cluster group</p><p>sse[i-1]=sum(kmeans(iris[,3:4],centers=i)$withinss)</p><p>}</p><p>#Converting the sse to a data frame and storing corresponding value of k</p><p>sse=as.data.frame(sse)</p><p>sse$k=seq.int(2,15)</p><p>#Making the plot. This plot is also known as screeplot</p><p>plot(sse$k,sse$sse,type=&quot;b&quot;)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*k62mycUlFfLfCtfo" /></figure><p>In this plot, the first elbow is formed at k=3. This method suggest that we should have 3 clusters for this data.</p><h3>Conclusion</h3><p>K-means clustering is one of the first and most basic clustering techniques whenever one thinks of unsupervised clustering. However, this technique is not just powerful, but also teaches the importance of understanding the data in unsupervised learning. If any of the assumptions are violated then the clusters are not formed properly. Similarly, while determining the value of ‘k’, using improper or arbitrary values may lead to improper clusters. In a way, k-means clustering also relies on the correct value of k for clustering the data accurately. Unsupervised learning is fun but not a wild attempt. One must be clear of all aspects of the algorithm and its assumptions before implementing it rather than treating it as a black box and shooting in the dark. This article illustrates the various shortcomings of improperly clustering data on simple datasets which do not satisfy the assumptions of the algorithm. The full code used in this article is given below.</p><p>#Viewing the Faithful dataset</p><p>plot(faithful)</p><p>#Specify 2 centers</p><p>k_clust_start=kmeans(faithful, centers=2)</p><p>#Plot the data using clusters</p><p>plot(faithful, col=k_clust_start$cluster,pch=2)</p><p>#Use the centers to find the cluster centers</p><p>k_clust_start$centers</p><p>#Use the size to find the cluster sizes</p><p>k_clust_start$size</p><p>#The following code will generate different plots for you but they will be similar</p><p>library(plyr)</p><p>library(dplyr)</p><p>#Generate random data which will be first cluster</p><p>clust1 = data_frame(x = rnorm(200), y = rnorm(200))</p><p>#Generate the second cluster which will ‘surround’ the first cluster</p><p>clust2 =data_frame(r = rnorm(200, 15, .5), theta = runif(200, 0, 2 * pi),</p><p>x = r * cos(theta), y = r * sin(theta)) %&gt;%</p><p>dplyr::select(x, y)</p><p>#Combine the data</p><p>dataset_cir= rbind(clust1, clust2)</p><p>#see the plot</p><p>plot(dataset_cir)</p><p>#Fit the k-means model</p><p>k_clust_spher1=kmeans(dataset_cir, centers=2)</p><p>#Plot the data and clusters</p><p>plot(dataset_cir, col=k_clust_spher1$cluster,pch=2)</p><p>#Using a function for transformation</p><p>cart2pol=function(x,y){</p><p>#This is r</p><p>newx=sqrt(x^2 + y^2)</p><p>#This is theta</p><p>newy=atan(y/x)</p><p>x_y=cbind(newx,newy)</p><p>return(x_y)</p><p>}</p><p>dataset_cir2=cart2pol(dataset_cir$x,dataset_cir$y)</p><p>plot(dataset_cir2)</p><p>k_clust_spher2=kmeans(dataset_cir2, centers=2)</p><p>#Plot the data and clusters</p><p>plot(dataset_cir2, col=k_clust_spher2$cluster,pch=2)</p><p>plot(dataset_cir, col=k_clust_spher2$cluster,pch=2)</p><p>#Make the first cluster with 1000 random values</p><p>clust1 = data_frame(x = rnorm(1000), y = rnorm(1000))</p><p>#Keep 10 values together to make the second cluster</p><p>clust2=data_frame(x=c(5,5.1,5.2,5.3,5.4,5,5.1,5.2,5.3,5.4),y=c(5,5,5,5,5,4.9,4.9,4.9,4.9,4.9))</p><p>#Combine the data</p><p>dataset_uneven=rbind(clust1,clust2)</p><p>plot(dataset_uneven)</p><p>k_clust_spher3=kmeans(dataset_uneven, centers=2)</p><p>plot(dataset_uneven, col=k_clust_spher3$cluster,pch=2)</p><p>#Create a vector for storing the sse</p><p>sse=vector(&#39;numeric&#39;)</p><p>for(i in 2:15){</p><p>#k-means function in R has a feature withinss which stores sse for each cluster group</p><p>sse[i-1]=sum(kmeans(iris[,3:4],centers=i)$withinss)</p><p>}</p><p>#Converting the sse to a data frame and storing corresponding value of k</p><p>sse=as.data.frame(sse)</p><p>sse$k=seq.int(2,15)</p><p>#Making the plot. This plot is also known as scree-plot</p><p>plot(sse$k,sse$sse,type=&quot;b&quot;)</p><p>We use this in our <a href="https://www.perceptive-analytics.com/ai-consulting/">AI consulting</a> to help businesses uncover patterns in unlabeled data.</p><p>Looking to do more with your data? Check out our other services :<a href="https://www.perceptive-analytics.com/microsoft-power-bi-developer-consultant/">Microsoft Power BI Consultant</a> | <a href="https://www.perceptive-analytics.com/power-bi-consulting/">Power BI Consulting</a> | <a href="https://www.perceptive-analytics.com/power-bi-expert/">Power BI Expert</a> | <a href="https://www.perceptive-analytics.com/power-bi-consultant/">Power BI Consultant</a> | <a href="https://www.perceptive-analytics.com/tableau-consultants/">Tableau Consultants</a> | <a href="https://www.perceptive-analytics.com/tableau-consulting/">Tableau Consulting</a> | <a href="https://www.perceptive-analytics.com/tableau-expert/">Tableau Expert</a> | <a href="https://www.perceptive-analytics.com/tableau-contractor/">Tableau Contractor</a> | <a href="https://www.perceptive-analytics.com/tableau-freelance-developer/">Tableau Freelance Developer</a> | <a href="https://www.perceptive-analytics.com/tableau-developer/">Tableau Developer</a> | <a href="https://www.perceptive-analytics.com/talend-consultants/">Talend Consultant</a> | <a href="https://perceptive-analytics.com/looker-consultant/">Looker Consulting</a> | <a href="https://www.perceptive-analytics.com/chatbot-consulting-services/">Chatbot Consulting</a> | <a href="https://www.perceptive-analytics.com/snowflake-consultants/">Snowflake Consulting</a></p><p>This article was originally published on <a href="https://www.perceptive-analytics.com/">Perceptive Analytics</a><br><a href="https://www.perceptive-analytics.com/exploring-assumptions-k-means-clustering-using-r/">https://www.perceptive-analytics.com/exploring-assumptions-k-means-clustering-using-r/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b0f504d62a2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimization Using R]]></title>
            <link>https://medium.com/@anshumand_46847/optimization-using-r-df81f3155522?source=rss-4c791febdaf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/df81f3155522</guid>
            <category><![CDATA[r-language]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[rlang]]></category>
            <category><![CDATA[data-consulting]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Anshuman]]></dc:creator>
            <pubDate>Thu, 26 Jun 2025 19:12:09 GMT</pubDate>
            <atom:updated>2025-06-26T19:12:09.726Z</atom:updated>
            <content:encoded><![CDATA[<p>Optimization is a technique for finding out the best possible solution for a given problem for all the possible solutions. Optimization uses a rigorous mathematical model to find out the most efficient solution to the given problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/528/0*B19wRCAC9cEoL-LB.png" /></figure><h4>What is optimization?</h4><p>Optimization is a technique for finding out the best possible solution for a given problem for all the possible solutions. Optimization uses a rigorous mathematical model to find out the most efficient solution to the given problem. To start with an optimization problem, it is important to first identify an objective. An objective is a quantitative measure of performance. For example: to maximize profits, minimize time, minimize costs, maximize sales.</p><h4>There are a variety of optimization techniques -</h4><p><strong>Unconstrained optimization</strong></p><p>In certain cases the variable can be freely selected within it’s full range. The optim() function in R can be used for 1- dimensional or n-dimensional problems. The general format for the optim() function is -</p><p>optim(objective, constraints, bounds = NULL, types= NULL, maximum = FALSE)</p><p>We start off with an example, let’s define the objective function what we are looking to solve -</p><p>&gt; f &lt;- function(x) 4 * (x[1]-1)² + 7 * (x[2]-3)² + 30<br>&gt; f<br>function(x) 4 * (x[1]-1)² + 7 * (x[2]-3)² + 30</p><p>Setting up the constraints next</p><p>&gt; c &lt;- c(1, 1)<br>&gt; c<br>[1] 1 1</p><p>The optimization function is invoked</p><p>&gt; r &lt;- optim(c, f)<br>&gt; r<br>$par<br>[1] 0.9999207 3.0001660</p><p>$value<br>[1] 30</p><p>$counts<br>function gradient <br> 69 NA</p><p>$convergence<br>[1] 0</p><p>$message<br>NULL</p><p>Next we check if the optimization converged to a minimum or not. The easy way to do this is to check if</p><p>&gt; r$convergence == 0<br>[1] TRUE</p><p>The optimization has converged to minimum. Finding out the input arguments for the optimization function can be obtained by</p><p>&gt; r$par<br>[1] 0.9999207 3.0001660</p><p>The value of objective function at the minimum is obtained by</p><p>&gt; r$value<br>[1] 30</p><h4>Linear programming</h4><p>Here is a good definition from <a href="https://www.techopedia.com/definition/20403/linear-programming-lp">technopedia</a><em> — “Linear programming is a mathematical method that is used to determine the best possible outcome or solution from a given set of parameters or list of requirements, which are represented in the form of linear relationships. It is most often used in computer modeling or simulation in order to find the best solution in allocating finite resources such as money, energy, manpower, machine resources, time, space and many other variables. In most cases, the “best outcome” needed from linear programming is maximum profit or lowest cost.</em>”</p><p>An example of a LP problem is -</p><p>Maximize or Minimize objective function: <strong><em>f(y1, y2) = g1.y1 + g2.y2</em></strong></p><p>Subjected to inequality constraints:</p><ul><li><strong><em>g11.y1 + g12.y2 &lt;= p1</em></strong></li><li><strong><em>g21.y1 + g22.y2 &lt;= p2</em></strong></li><li><strong><em>g31.y1 + g32.y2 &lt;= p3</em></strong></li><li><strong><em>y1 &gt;= 0, y2 &gt;=0</em></strong></li></ul><h4>Example 1</h4><p>A company wants to maximize the profit for two products A and B which are sold at $ 25 and $ 20 respectively. There are 1800 resource units available every day and product A requires 20 units while B requires 12 units. Both of these products require a production time of 4 minutes and total available working hours are 8 in a day. What should be the production quantity for each of the products to maximize profits?</p><p>A LP problem can either be a maximization problem or a minimization problem. The Above problem is a maximization problem. Some of the steps that should be followed while defining a LP problem are -</p><ul><li>Identify the decision variables</li><li>Write the objective function</li><li>Mention the constraints</li><li>Explicitly state the non-negativity restriction</li></ul><p>Lets walk through the above example -</p><p>As already defined this is a maximization problem, first we define the objective function.</p><p><strong><em>max</em></strong><em>(Sales) = </em><strong><em>max</em></strong><em>(25 y1 + 20 y2)</em></p><p><strong><em>where,</em></strong></p><ul><li>y1 is the units of Product A produced</li><li>y2 is the units of Product B produced</li><li>y1 and y2 are called the decision variables</li><li>25 and 20 are the selling price of the products</li></ul><p>We are trying to maximize the sales while finding out the optimum number of products to manufacture. Now we set the constraints for this particular LP problem. We are dealing with both resource and time constraints.</p><p><em>20y1 + 12 y2 &lt;= 1800 </em><strong><em>(Resource Constraint)</em></strong><br><em>4y1 + 4y2 &lt;= 8*60 </em><strong><em>(Time constraint)</em></strong></p><h4>There are two ways to solve a LP problem</h4><ul><li>Graphical Method</li><li>Simplex Method</li></ul><p>We will be solving this problem using the simplex method but in R. We shall also explain another example with excel’s solver. There are a couple of packages in R to solve LP problems. Some of the popular ones are -</p><ul><li>lpsolve</li><li>lpsolveAPI</li></ul><h4>Implementation in R using Lpsolve</h4><p>Let’s use lpsolve for this problem. First we need to set the objective function, this has already been defined.</p><p>&gt; require(lpSolve)<br>Loading required package: lpSolve<br>&gt; objective.in &lt;- c(25, 20)<br>&gt; objective.in<br>[1] 25 20</p><p>Creating a matrix for the constraints, we create a 2 by 2 matrix for this. And then setting constraints.</p><p>&gt; const &lt;- matrix(c(20, 12, 4, 4), nrow=2, byrow=TRUE)<br>&gt; const<br> [,1] [,2]<br>[1,] 20 12<br>[2,] 4 4<br>&gt; time_constraints &lt;- (8*60)<br>&gt; resource_constraints &lt;- 1800<br>&gt; time_constraints<br>[1] 480<br>&gt; resource_constraints<br>[1] 1800</p><p>Now we are basically creating the equations that we have already defined by setting the rhs and the direction of the constraints.</p><p>&gt; rhs &lt;- c(resource_constraints, time_constraints)<br>&gt; rhs<br>[1] 1800 480<br>&gt; direction &lt;- c(“&lt;=”, “&lt;=”)<br>&gt; direction<br>[1] “&lt;=” “&lt;=”</p><p>The final step is to find the optimal solution. The syntax for the lpsolve package is -</p><p><strong><em>lp</em></strong><em>(direction , objective, const.mat, const.dir, const.rhs)</em></p><p>&gt; optimum &lt;- lp(direction=”max”, objective.in, const, direction, rhs)<br>&gt; optimum<br>Success: the objective function is 2625<br>&gt; summary(optimum)<br> Length Class Mode <br>direction 1 -none- numeric <br>x.count 1 -none- numeric <br>objective 2 -none- numeric <br>const.count 1 -none- numeric <br>constraints 8 -none- numeric <br>int.count 1 -none- numeric <br>int.vec 1 -none- numeric <br>bin.count 1 -none- numeric <br>binary.vec 1 -none- numeric <br>num.bin.solns 1 -none- numeric <br>objval 1 -none- numeric <br>solution 2 -none- numeric <br>presolve 1 -none- numeric <br>compute.sens 1 -none- numeric <br>sens.coef.from 1 -none- numeric <br>sens.coef.to 1 -none- numeric <br>duals 1 -none- numeric <br>duals.from 1 -none- numeric <br>duals.to 1 -none- numeric <br>scale 1 -none- numeric <br>use.dense 1 -none- numeric <br>dense.col 1 -none- numeric <br>dense.val 1 -none- numeric <br>dense.const.nrow 1 -none- numeric <br>dense.ctr 1 -none- numeric <br>use.rw 1 -none- numeric <br>tmp 1 -none- character<br>status 1 -none- numeric</p><p>Now we get the optimum values for y1 and y2, i.e the number of product A and product B that should be manufactured.</p><p>&gt; optimum$solution<br>[1] 45 75</p><p>The maximum sales figure is -</p><p>&gt; optimum$objval<br>[1] 2625</p><h4>Example 2</h4><p>Below is good example taken from<a href="http://lpsolve.sourceforge.net/5.1/formulate.htm">lpsolve’s sourceforge</a>website -</p><blockquote><em>“Suppose a farmer has 75 acres on which to plant two crops: wheat and barley. To produce these crops, it costs the farmer (for seed, fertilizer, etc.) $120 per acre for the wheat and $210 per acre for the barley. The farmer has $15000 available for expenses. But after the harvest, the farmer must store the crops while awaiting avourable market conditions. The farmer has storage space for 4000 bushels. Each acre yields an average of 110 bushels of wheat or 30 bushels of barley. If the net profit per bushel of wheat (after all expenses have been subtracted) is $1.30 and for barley is $2.00, how should the farmer plant the 75 acres to maximize profit?”</em></blockquote><p>As we did in the previous example, let’s define the optimization function and the constraints.</p><p><em>maximize</em><br><em> g = (110)(1.30)x + (30)(2.00)y = 143x + 60y</em></p><p><em>subject to</em><br><em> 120x + 210y &lt;= 15000</em><br><em> 110x + 30y &lt;= 4000</em><br><em> x + y &lt;= 75</em><br><em> x &gt;= 0</em><br><em> y &gt;= 0</em></p><p>We will be solving this problem in both excel and R.</p><h4>Excel Solver</h4><p>There is a way to solve LP problems using Excel. Firstly we have to define the number of acres to plant for both these crops.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/298/0*SMWC1Y5RD9jN-bAR.png" /></figure><p>Initially they are zero, the optimum values can be obtained by using the solver plugin in excel. Next we need to define the constraints, the equations for the constraints have already been defined.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/496/0*oR4Qw2w_fKnaXnpk.png" /></figure><p>Subject to,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/207/0*wAC8Kkuh3lqcvFmF.png" /></figure><p>The initial values for the number of used resources is set at zero initially. The main workhorse behind these LP problems is to use the sumproduct equation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/669/0*WtgTeOZ5mE-H9zWY.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/643/0*e1m0ciS25DzCa8RY.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/645/0*fkkJVAS74ayHUR0i.png" /></figure><p>The sumproduct equation is basically a product of cost multiplied by the number of acres.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/539/0*iFxT_whaAWVeZWzm.png" /></figure><p>Now we define the maximization equation. In this case the profit needs to be maximized. The total profit cell that is seen in the below image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/525/0*_TsQKrDRjXvmb1re.png" /></figure><p>The same sumproduct formula that was previously used is repeated again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/565/0*13Mh1vktVwx8keaW.png" /></figure><p>Now try changing the values for the wheat and Barley cells. Just put some random number for both the cells. Observe the total profit cell and the used cell for all the constraints.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/524/0*BCTRv_BVvjZQqtuv.png" /></figure><p>As seen we get a total profit figure, however have a look at the used resources. Clearly the values in the cell are more than the values in the available cells. We can try to decrease the number of acres and see what the resulting values will be.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/528/0*wD6eQAclMxABsDjD.png" /></figure><p>The values are low indeed, but it is not the optimum solution. To get the optimum values for total profit. We use the solver plugin to solve this. Navigate to the data and see if there is the solver option under the analysis tab.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8oRUL3N0swGKaZC_.png" /></figure><p>If it is not present, follow these steps to install it -</p><ol><li>Navigate for the file tab and then click on options</li><li>Click on the Add-ins button, and then go next to the manage</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/805/0*gD0WCGcm3WFI60JK.png" /></figure><p>Once this is done, a popup should appear.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/289/0*KUTDgpTk5NnyEvUa.png" /></figure><p>Select the Solver add-in and press OK.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/488/0*GGg1vdR44fkJaFDP.png" /></figure><p>There you go now the solver option should show up. Before we proceed further, let’s set the number of acres to zero.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/513/0*AIBo2rXoaIE-pM-L.png" /></figure><p>Now, set the objective to total profit</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vWzYLG1MGiGxmjfM.png" /></figure><p>Set the -</p><ul><li>Solving method to simplex LP.</li><li>The objective to max</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Sg-2qFeY1ds3oRcK.png" /></figure><p>Next add the three constraints,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/0*A6kPWuREIuaVKGJX.png" /></figure><p>Once done press solve, the following popup should show up along with the optimized values for the profit cell.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*q8OPWpgydva2VxaN.png" /></figure><p>Press OK, now we have the number of acres to plant for Wheat and Barley. Along with the profit value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/507/0*OLf2Ke4QYAiCC9qr.png" /></figure><p>Implementation in R using LpsolveAPI<br>We previously used the Lpsolve package, in this example we illustrate how to use LpsolveAPI package. We start off by calling the lpsolveapi package.</p><p>&gt; require(lpSolveAPI)<br>Loading required package: lpSolveAPI</p><p>&gt; lprec &lt;- make.lp(0, 2)<br>&gt; lprec<br>Model name: <br> C1 C2 <br>Minimize 0 0 <br>Kind Std Std <br>Type Real Real <br>Upper Inf Inf <br>Lower 0 0</p><p>Next we set the objective function to maximization.</p><p>&gt; lp.control(lprec, sense=”max”)<br>$anti.degen<br>[1] “none”</p><p>$basis.crash<br>[1] “none”</p><p>$bb.depthlimit<br>[1] -50</p><p>$bb.floorfirst<br>[1] “automatic”</p><p>$bb.rule<br>[1] “pseudononint” “greedy” “dynamic” “rcostfixing”</p><p>$break.at.first<br>[1] FALSE</p><p>$break.at.value<br>[1] 1e+30</p><p>$epsilon<br> epsb epsd epsel epsint epsperturb epspivot <br> 1e-10 1e-09 1e-12 1e-07 1e-05 2e-07</p><p>$improve<br>[1] “dualfeas” “thetagap”</p><p>$infinite<br>[1] 1e+30</p><p>$maxpivot<br>[1] 250</p><p>$mip.gap<br>absolute relative <br> 1e-11 1e-11</p><p>$negrange<br>[1] -1e+06</p><p>$obj.in.basis<br>[1] TRUE</p><p>$pivoting<br>[1] “devex” “adaptive”</p><p>$presolve<br>[1] “none”</p><p>$scalelimit<br>[1] 5</p><p>$scaling<br>[1] “geometric” “equilibrate” “integers”</p><p>$sense<br>[1] “maximize”</p><p>$simplextype<br>[1] “dual” “primal”</p><p>$timeout<br>[1] 0</p><p>$verbose<br>[1] “neutral”</p><p>We define the objective function and the constraints</p><p>&gt; set.objfn(lprec, c(143, 60))<br>&gt; add.constraint(lprec, c(120, 210), “&lt;=”, 15000)<br>&gt; add.constraint(lprec, c(110, 30), “&lt;=”, 4000)<br>&gt; add.constraint(lprec, c(1, 1), “&lt;=”, 75)</p><p>Let’s have a look at the LP problem that we have defined.</p><p>&gt; lprec<br>Model name: <br> C1 C2 <br>Maximize 143 60 <br>R1 120 210 &lt;= 15000<br>R2 110 30 &lt;= 4000<br>R3 1 1 &lt;= 75<br>Kind Std Std <br>Type Real Real <br>Upper Inf Inf <br>Lower 0 0</p><p>Solving</p><p>&gt; solve(lprec)<br>[1] 0</p><p>Getting the optimized profit value</p><p>&gt; get.objective(lprec)<br>[1] 6315.625</p><p>Let’s finally get the values for how many acres of Wheat and Barley to be planted.</p><p>&gt; get.variables(lprec)<br>[1] 21.875 53.125</p><p>Thus, to achieve the maximum profit ($6315.625), the farmer should plant 21.875 acres of wheat and 53.125 acres of barley.</p><h4>Conclusion</h4><p>Now that we have figured out how to solve LP problems using excel solver as well as the packages in R, namely Lpsolve and LpsolveAPI. Go ahead and start solving your own Linear Programming problems.</p><p>LP Problem sources -</p><ul><li><a href="http://lpsolve.sourceforge.net/5.1/formulate.htm">http://lpsolve.sourceforge.net/5.1/formulate.htm</a></li></ul><p>Our <a href="https://www.perceptive-analytics.com/ai-consulting/">AI consulting company</a> frequently leverages these methods in supply chain optimization and pricing models.</p><p>Looking to do more with your data? Check out our other services : <a href="https://www.perceptive-analytics.com/power-bi-consulting/">Power BI Consulting</a> | <a href="https://www.perceptive-analytics.com/power-bi-expert/">Power BI Expert</a> | <a href="https://www.perceptive-analytics.com/power-bi-consultant/">Power BI Consultant</a> | <a href="https://www.perceptive-analytics.com/tableau-consultants/">Tableau Consultants</a> | <a href="https://www.perceptive-analytics.com/tableau-consulting/">Tableau Consulting</a> | <a href="https://www.perceptive-analytics.com/tableau-expert/">Tableau Expert</a> | <a href="https://www.perceptive-analytics.com/tableau-contractor/">Tableau Contractor</a> | <a href="https://www.perceptive-analytics.com/tableau-freelance-developer/">Tableau Freelance Developer</a> | <a href="https://www.perceptive-analytics.com/tableau-developer/">Tableau Developer</a></p><p>This article was originally published on <a href="https://www.perceptive-analytics.com/">Perceptive Analytics</a></p><p><a href="https://www.perceptive-analytics.com/optimization-using-r/"><strong>Optimization Using R</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df81f3155522" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reinforcement Learning with R]]></title>
            <link>https://medium.com/@anshumand_46847/reinforcement-learning-with-r-83ef2d9a7b18?source=rss-4c791febdaf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/83ef2d9a7b18</guid>
            <category><![CDATA[r-language]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[rlang]]></category>
            <category><![CDATA[r-training]]></category>
            <category><![CDATA[r-tutorial]]></category>
            <dc:creator><![CDATA[Anshuman]]></dc:creator>
            <pubDate>Wed, 25 Jun 2025 06:20:20 GMT</pubDate>
            <atom:updated>2025-06-25T06:20:20.382Z</atom:updated>
            <content:encoded><![CDATA[<h4>Reinforcement Learning with R</h4><p>Machine learning algorithms were mainly divided into <strong>three</strong> main categories.</p><ul><li>Supervised learning algorithms</li><li>Classification and regression algorithms</li><li>Unsupervised learning algorithms</li><li>Clustering algorithms</li><li>Reinforcement learning algorithms</li></ul><p>We have covered supervised learning and unsupervised learning algorithms couple of times in our blog articles. In this article, you are going to learn about the third category of machine learning algorithms. Which are reinforcement learning algorithms.</p><p>Before we drive further let quickly look at the table of contents.</p><h4>Table of contents:</h4><ul><li>Reinforcement learning real-life example</li><li>Typical reinforcement process</li><li>Reinforcement learning process</li><li>Divide and Rule</li><li>Reinforcement learning implementation in R</li><li>Preimplementation background</li><li>MDP toolbox package</li><li>Using Github reinforcement learning package</li><li>How to change environment</li><li>Complete code</li><li>Conclusion</li><li>Related courses</li><li>Practical Reinforcement learning</li></ul><h4>Reinforcement learning real-life example</h4><p>The modern education system follows a standard pattern of teaching students. The teacher goes over the concepts need to be covered and reinforces them through some example questions. After explaining the topic and the process with a few solved examples, students are expected to solve similar questions from their exercise book themselves.</p><p>This mode of learning is also adopted in machine learning algorithms as a separate class known as reinforcement learning. Though it is easy to know and understand how reinforcement works, the concept is hard to implement.</p><h4>Typical reinforcement process</h4><p>In a typical reinforcement process, the machine acts as the ‘student’ trying to learn the concept.</p><p>To learn, the machine interacts with a ‘teacher’ to know the classes of specific data points and learns it. This learning is guided by assigning rewards and penalties to correct and incorrect decisions respectively. Along the way, the machine makes mistakes and corrects itself so as to maximize the reward and minimize the penalty.</p><p>As it learns through trial and error and continuous interaction, a framework is built by the algorithm. Since it is so human-like, it has used in specific facets in the industry where a predefined training data is not available. Some examples include puzzle navigation and tic-tac-toe games.</p><h4>Reinforcement Learning process</h4><p>Before developing Reinforcement learning algorithm using R, one needs to break down the process into smaller tasks. In programming terminology Divide and Rule.</p><h4>Divide and Rule: Breaking down reinforcement learning process</h4><p>Following a step-wise approach, one needs a set of ‘policies’ laid down for the machine to follow. A set of reward and penalty rules for the machine to assess how it is performing. The training limit specifying the trial and error experiences which the machine uses to train itself.</p><p>Now let’s start with a toy example: Navigating to the exit in a <strong>3 by 3</strong> matrix. Let’s say we have the following matrix.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/690/0*RsY3c1OhTwkvwqYd" /></figure><p>Reinforcement Learning: Image01</p><p>In this example, the machine can navigate in <strong>4 directions</strong>.</p><ul><li>UP</li><li>DOWN</li><li>LEFT</li><li>RIGHT</li></ul><p>From the <strong>‘Start’,</strong> the aim is to reach the <strong>‘Exit’</strong> without going through the ‘Pit’. The only path to reach <strong>Exit</strong> from <strong>Start</strong> is the below sequence.</p><ol><li>UP</li><li>UP</li><li>LEFT</li><li>LEFT</li></ol><h4>But how does the machine learn it?</h4><p>Here the policies are the set of actions ( UP, DOWN, LEFT, RIGHT) with rules that an action is not available if choosing it takes you out of the boundary or to the block named ‘Wall’.</p><p>Then we have the reward matrix where taking each step is a small penalty, falling into the pit is a big penalty and reaching the exit has a reward. The final piece is the way experience is calculated.</p><p>In this case, the sum of all the actions. Assigning a small penalty to each step will be instrumental for the machine to minimize the number of steps. Assigning a big penalty to the pit should make the machine avoid it and the reward to the goal will attract the machine towards it. This is how the machine trains.</p><p>Let’s now understand the same from a coding perspective before we try it using R!</p><h4>Reinforcement learning implementation in R</h4><p>Before we straightway implementing the reinforcement learning in R programming language, Let’s understand about some background implementation concepts.</p><h4>Reinforcing yourself — Learning the background before the actual implementation</h4><p>To make the navigation possible, the machine will continuously interact with the puzzle and try to learn the optimal path. Over time, it will start seeking the reward and avoiding the pit. When the optimal path is obtained, the output is provided in the form of a set of actions performed and the rewards associated with each of them.</p><p>While learning, the machine iterates by taking each of the possible actions and the change in reward after each action. This is usually followed using the <strong>‘Markov Process’</strong> which implies that the decision the machine makes at any given state is independent of the decisions the machine has made at the previous states.</p><p>As a result, the machine arrives with the following five elements of reinforcement learning.</p><ul><li>Possible set of states, <strong>s</strong></li><li>Set of possible actions, <strong>A</strong> — Defined for the algorithm</li><li>Rewards and Penalties — <strong>R</strong></li><li>Policy, <strong>?</strong>; and</li><li>Value, <strong>v</strong></li></ul><p>In defined terms, we want to explore the set of possible states,<strong>s</strong>, by taking actions, <strong>A</strong> and come up with an optimal policy<strong> ?</strong>* which maximizes the value, <strong>v</strong> based on rewards and penalties, <strong>R</strong>.</p><p>Now that we have understood the concept, let’s try a few examples using R.</p><h4>Teaching the child to walk — MDP toolbox package</h4><p>The ‘MDPtoolbox’ package in R is a simple Markov decision process package which uses the Markov process to learn reinforcement. It is a good package for solving problems such as the toy example demonstrated in this article earlier.</p><p>Let’s load the package first.</p><p># Teaching the child to walk — MDPtoolbox package</p><p># Installing and loading the package</p><p># install.packages(“MDPtoolbox”)</p><p>library(MDPtoolbox)</p><p>To define the elements of reinforced learning. We need to assign a label to each of the states in the navigation matrix. For the sake of simplicity, we will take a shot-down 2*2 version of the navigation matrix which looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/690/0*dys0X1Qhg8NWrb4S" /></figure><p>Reinforcement Learning: Image 02</p><p>I have labeled each block as a state from S1 to S4. S1 is the start point and S4 is the endpoint. One cannot go directly from S1 to S4 due to the wall. In S1, we see that there is no way to reach S4. One can only move to S2 or remain in S1.</p><p>Hence, the down matrix will have the probabilities only for S1 and S2 in the first row. We can similarly define the probabilities for every action in each state.</p><h4>Let’s define the actions now.</h4><p># 1. Defining the Set of Actions — Left, Right, Up and Down for 2*2 matrix</p><p># Remember! This will be a probability matrix, so we will use the matrix() function such that the sum of probabilities in each row is 1</p><p>#Up Action</p><p>up=matrix(c( 1, 0, 0, 0,</p><p>0.7, 0.2, 0.1, 0,</p><p>0, 0.1, 0.2, 0.7,</p><p>0, 0, 0, 1),</p><p>nrow=4,ncol=4,byrow=TRUE)</p><p>#Down Action</p><p>down=matrix(c(0.3, 0.7, 0, 0,</p><p>0, 0.9, 0.1, 0,</p><p>0, 0.1, 0.9, 0,</p><p>0, 0, 0.7, 0.3),</p><p>nrow=4,ncol=4,byrow=TRUE)</p><p>#Left Action</p><p>left=matrix(c( 0.9, 0.1, 0, 0,</p><p>0.1, 0.9, 0, 0,</p><p>0, 0.7, 0.2, 0.1,</p><p>0, 0, 0.1, 0.9),</p><p>nrow=4,ncol=4,byrow=TRUE)</p><p>#Right Action</p><p>right=matrix(c( 0.9, 0.1, 0, 0,</p><p>0.1, 0.2, 0.7, 0,</p><p>0, 0, 0.9, 0.1,</p><p>0, 0, 0.1, 0.9),</p><p>nrow=4,ncol=4,byrow=TRUE)</p><p>#Combined Actions matrix</p><p>Actions=list(up=up, down=down, left=left, right=right)</p><p>The second element is the rewards and penalties function. The only penalty is the small penalty on every additional step. Let’s keep it -1.</p><p>The reward is obtained on reaching state S4. Let’s keep the weight to be +10. Hence our Rewards matrix R can be obtained</p><p>#2. Defining the rewards and penalties</p><p>Rewards=matrix(c( -1, -1, -1, -1,</p><p>-1, -1, -1, -1,</p><p>-1, -1, -1, -1,</p><p>10, 10, 10, 10),</p><p>nrow=4,ncol=4,byrow=TRUE)</p><p>That’s it! Now it is up to the algorithm to come up with the optimal policy and its value.</p><p>The <strong>mdp_policy_iteration()</strong> function is used to solve the problem in R. The function requires actions, rewards, and discount as inputs to calculate the results.</p><p>Discount is used to decrease the value of the current reward or penalty as each of the steps are taken.</p><p>Let’s see if the defined problem can be solved correctly by the package.</p><p>1</p><p>2</p><p>#3. Solving the navigation</p><p>solver=mdp_policy_iteration(P=Actions, R=Rewards, discount = 0.1)</p><p>The result gives us the policy, the value at each step and additionally, the number of iterations and time taken. As we know, the policy should dictate the correct path to reach the final state S4. We use the policy function to know the matrices used for defining the policy and then the names from the actions list.</p><p>1</p><p>2</p><p>3</p><p>#4. Getting the policy</p><p>solver$policy #2 4 1 1</p><p>names(Actions)[solver$policy] #”down” “right” “up” “up”</p><p>The values are contained in V and show the reward at each step.</p><p>1</p><p>2</p><p>#5. Getting the Values at each step. These values can be different in each run</p><p>solver$V #58.25663 69.09102 83.19292 100.00000</p><p>iter and time can be used to know the iterations and time to keep track of the complexity.</p><p>1</p><p>2</p><p>#6. Additional information: Number of iterations</p><p>solver$iter #2</p><p>1</p><p>2</p><p>#7. Additional information: Time taken. This time can be different in each run</p><p>solver$time #Time difference of 0.009523869 secs</p><h4>Using Github reinforcement learning package</h4><p>Cran provides documentation to ‘ReinforcementLearning’ package which can partly perform reinforcement learning and solve a few simple problems.</p><p>However, since the package is experimental, it has to be installed after installing <strong>‘devtools’</strong> package first and then installing from GitHub as it is not available in cran repository.</p><h4>Getting into rough games (Reinforcement learning GitHub package)</h4><p># Getting into rough games — ReinforcementLearning github package</p><p># install.packages(“devtools”)</p><p>library(devtools)</p><p># Option 1: download and install latest version from GitHub</p><p>install_github(“nproellochs/ReinforcementLearning”)</p><p>library(ReinforcementLearning)</p><p>If we attempt the same problem using this package, we have to first define a function of actions and states to indicate the possible actions in each state. We also define the reward associated in each state.</p><p>This package has this toy example pre-built hence, we just look at the function which should have otherwise been defined.</p><p># Viewing the pre-built function for each state, action and reward</p><p>print(gridworldEnvironment)</p><p>function (state, action)</p><p>{</p><p>next_state &lt;- state</p><p>if (state == state(“s1”) &amp;&amp; action == “down”)</p><p>next_state &lt;- state(“s2”)</p><p>if (state == state(“s2”) &amp;&amp; action == “up”)</p><p>next_state &lt;- state(“s1”)</p><p>if (state == state(“s2”) &amp;&amp; action == “right”)</p><p>next_state &lt;- state(“s3”)</p><p>if (state == state(“s3”) &amp;&amp; action == “left”)</p><p>next_state &lt;- state(“s2”)</p><p>if (state == state(“s3”) &amp;&amp; action == “up”)</p><p>next_state &lt;- state(“s4”)</p><p>if (next_state == state(“s4”) &amp;&amp; state != state(“s4”)) {</p><p>reward &lt;- 10</p><p>}</p><p>else {</p><p>reward &lt;- -1</p><p>}</p><p>out &lt;- list(NextState = next_state, Reward = reward)</p><p>return(out)</p><p>}</p><p>&lt;environment: namespace:ReinforcementLearning&gt;</p><p>We now define the names of the states and actions and start solving using the <strong>sampleExperience()</strong> function right away.</p><p># Define names for state and action</p><p>states &lt;- c(“s1”, “s2”, “s3”, “s4”)</p><p>actions &lt;- c(“up”, “down”, “left”, “right”)</p><p># Generate 1000 iterations</p><p>sequences &lt;- sampleExperience(N = 1000, env = gridworldEnvironment, states = states, actions = actions)</p><p>#Solve the problem</p><p>solver_rl &lt;- ReinforcementLearning(sequences, s = “State”, a = “Action”, r = “Reward”, s_new = “NextState”)</p><p>#Getting the policy; this may be different for each run</p><p>solver_rl$Policy</p><p>s1 s2 s3 s4</p><p>“down” “right” “up” “down”</p><p>#Getting the Reward; this may be different for each run</p><p>solver_rl$Reward #-351</p><p>Here we see that the first three steps are always the same and correct to reach s4. The fourth action is random and can be different for each run</p><h4>Adapting to the changing environment</h4><p>The package also has the tic-tac-toe game data generated in it’s pre-built library. The data contains about 4 lac rows of steps for tic-tac-toe.</p><p>We can directly load the data and perform reinforcement learning on the data.</p><p># Conclusion: Adapting to the changing environment</p><p># Load dataset</p><p>data(“tictactoe”)</p><p># Perform reinforcement learning on tictactoe data</p><p>model_tic_tac &lt;- ReinforcementLearning(tictactoe, s = “State”, a = “Action”, r = “Reward”, s_new = “NextState”, iter = 1)</p><p>Since the data is very large, it will take some time to learn. We can then see the model policy and reward.</p><p># Optimal policy; this may be different for each run</p><p>model_tic_tac$Policy #This will print a very large matrix of the possible step in each state</p><p># Reward; this may be different for each run</p><p>model_tic_tac$Reward #5449</p><h4>Complete code used in this article</h4><p># Teaching the child to walk — MDPtoolbox package</p><p># Installing and loading the package</p><p># install.packages(“MDPtoolbox”)</p><p>library(MDPtoolbox)</p><p># 1. Defining the Set of Actions — Left, Right, Up and Down for 2*2 matrix</p><p># Remember! This will be a probability matrix, so we will use the matrix() function such that the sum of probabilities in each row is 1</p><p># Up Action</p><p>up=matrix(c( 1, 0, 0, 0,</p><p>0.7, 0.2, 0.1, 0,</p><p>0, 0.1, 0.2, 0.7,</p><p>0, 0, 0, 1),</p><p>nrow=4,ncol=4,byrow=TRUE)</p><p># Down Action</p><p>down=matrix(c(0.3, 0.7, 0, 0,</p><p>0, 0.9, 0.1, 0,</p><p>0, 0.1, 0.9, 0,</p><p>0, 0, 0.7, 0.3),</p><p>nrow=4,ncol=4,byrow=TRUE)</p><p># Left Action</p><p>left=matrix(c( 0.9, 0.1, 0, 0,</p><p>0.1, 0.9, 0, 0,</p><p>0, 0.7, 0.2, 0.1,</p><p>0, 0, 0.1, 0.9),</p><p>nrow=4,ncol=4,byrow=TRUE)</p><p># Right Action</p><p>right=matrix(c( 0.9, 0.1, 0, 0,</p><p>0.1, 0.2, 0.7, 0,</p><p>0, 0, 0.9, 0.1,</p><p>0, 0, 0.1, 0.9),</p><p>nrow=4,ncol=4,byrow=TRUE)</p><p># Combined Actions matrix</p><p>Actions=list(up=up, down=down, left=left, right=right)</p><p># 2. Defining the rewards and penalties</p><p>Rewards=matrix(c( -1, -1, -1, -1,</p><p>-1, -1, -1, -1,</p><p>-1, -1, -1, -1,</p><p>10, 10, 10, 10),</p><p>nrow=4,ncol=4,byrow=TRUE)</p><p># 3. Solving the navigation</p><p>solver=mdp_policy_iteration(P=Actions, R=Rewards, discount = 0.1)</p><p># 4. Getting the policy</p><p>solver$policy #2 4 1 1</p><p>names(Actions)[solver$policy] #”down” “right” “up” “up”</p><p># 5. Getting the Values at each step. These values can be different in each run</p><p>solver$V #58.25663 69.09102 83.19292 100.00000</p><p># 6. Additional information: Number of iterations</p><p>solver$iter #2</p><p># 7. Additional information: Time taken. This time can be different in each run</p><p>solver$time #Time difference of 0.009523869 secs</p><p># Getting into rough games — ReinforcementLearning github package</p><p># install.packages(“devtools”)</p><p>library(devtools)</p><p># Option 1: download and install latest version from GitHub</p><p>install_github(“nproellochs/ReinforcementLearning”)</p><p>library(ReinforcementLearning)</p><p># Viewing the pre-built function for each state, action and reward</p><p>print(gridworldEnvironment)</p><p># Define names for state and action</p><p>states &lt;- c(“s1”, “s2”, “s3”, “s4”)</p><p>actions &lt;- c(“up”, “down”, “left”, “right”)</p><p># Generate 1000 iterations</p><p>sequences &lt;- sampleExperience(N = 1000, env = gridworldEnvironment, states = states, actions = actions)</p><p># Solve the problem</p><p>solver_rl &lt;- ReinforcementLearning(sequences, s = “State”, a = “Action”, r = “Reward”, s_new = “NextState”)</p><p># Getting the policy; this may be different for each run</p><p>solver_rl$Policy</p><p># Getting the Reward; this may be different for each run</p><p>solver_rl$Reward #-351</p><p># Conclusion: Adapting to the changing environment</p><p># Load dataset</p><p>data(“tictactoe”)</p><p># Perform reinforcement learning on tictactoe data</p><p>model_tic_tac &lt;- ReinforcementLearning(tictactoe, s = “State”, a = “Action”, r = “Reward”, s_new = “NextState”, iter = 1)</p><p># Optimal policy; this may be different for each run</p><p>model_tic_tac$Policy #This will print a very large matrix of the possible step in each state</p><p># Reward; this may be different for each run</p><p>model_tic_tac$Reward #5449</p><p>You can clone this article code in our GitHub.</p><p>Reinforcement learning has picked up the pace in the recent times due to its ability to solve problems in interesting human-like situations such as games. Recently, Google’s Alpha-Go program beat the best Go players by learning the game and iterating the rewards and penalties in the possible states of the board.</p><p>Being human-like makes it associated with behavioral psychology and thus, it gives the opportunity to add human behavior and artificial intelligence to machine learning and include it in one’s arsenal of newest technologies.</p><h3>Conclusion</h3><p>The field of data science is changing rapidly with so many new methods and algorithms being developed in every field for all purposes. Reinforcement learning is one such technique, though experimental and incomplete, it can solve the problem of completing simple tasks easily.</p><p>At present, machines are adept at performing repetitive tasks and solve complex problems easily but cannot solve easy tasks without getting into complexity. This is why, making machines perform simple tasks such as walking, moving hands or even playing tic-tac-toe is very difficult though we, as humans, perform this every day without much effort. With reinforcement learning, these tasks can be trained with an order of complexity — a direction that many <a href="https://www.perceptive-analytics.com/ai-consulting/">AI consulting projects</a> are starting to explore for automation and adaptive system design.</p><p>This article is aimed at explaining the same process of reinforcement learning to data science enthusiasts and open the gates of a new set of learning opportunities with reinforcement.</p><p>Looking to do more with your data? Check out our other services : <a href="https://www.perceptive-analytics.com/power-bi-consulting/">Power BI Consulting</a> | <a href="https://www.perceptive-analytics.com/power-bi-expert/">Power BI Expert</a> | <a href="https://www.perceptive-analytics.com/power-bi-consultant/">Power BI Consultant</a> | <a href="https://www.perceptive-analytics.com/microsoft-power-bi-developer-consultant/">Power BI Developer Consultant</a> | <a href="https://www.perceptive-analytics.com/tableau-consultants/">Tableau Consultants</a> | <a href="https://www.perceptive-analytics.com/tableau-consulting/">Tableau Consulting</a> | <a href="https://www.perceptive-analytics.com/tableau-expert/">Tableau Expert</a> | <a href="https://www.perceptive-analytics.com/tableau-contractor/">Tableau Contractor</a> | Tableau Freelance Developer | <a href="https://www.perceptive-analytics.com/tableau-developer/">Tableau Developer</a> | <a href="https://www.perceptive-analytics.com/ai-consulting/">AI Consultin</a>g | <a href="https://www.perceptive-analytics.com/snowflake-consultants/">Snowflake Consulting</a> | <a href="https://perceptive-analytics.com/looker-consultant/">Looker Consulting</a> | <a href="https://www.perceptive-analytics.com/talend-consultants/">Talend Consulting</a> | <a href="https://www.perceptive-analytics.com/chatbot-consulting-services/">Chatbot Consulting</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=83ef2d9a7b18" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>