<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yash Kanoongo on Medium]]></title>
        <description><![CDATA[Stories by Yash Kanoongo on Medium]]></description>
        <link>https://medium.com/@yashkanoongo?source=rss-88f57cfbdbd6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*22eahic7Pd1Xv1sH.</url>
            <title>Stories by Yash Kanoongo on Medium</title>
            <link>https://medium.com/@yashkanoongo?source=rss-88f57cfbdbd6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 04:00:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@yashkanoongo/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[ADDRESSING BIAS IN HR ALGORITHMS]]></title>
            <link>https://medium.com/@yashkanoongo/addressing-bias-in-hr-algorithms-2b0f9003ed64?source=rss-88f57cfbdbd6------2</link>
            <guid isPermaLink="false">https://medium.com/p/2b0f9003ed64</guid>
            <category><![CDATA[people-analytics]]></category>
            <category><![CDATA[hr]]></category>
            <category><![CDATA[data-ethics]]></category>
            <category><![CDATA[algorithm-bias]]></category>
            <dc:creator><![CDATA[Yash Kanoongo]]></dc:creator>
            <pubDate>Wed, 18 Mar 2020 22:28:16 GMT</pubDate>
            <atom:updated>2020-03-18T22:28:16.783Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/679/1*4atNsUrwPFQ3Z_G03RY29Q.png" /></figure><h3>INTRODUCTION</h3><p>There exists a widespread notion that AI is beneficial to society because it removes human bias in decision making. Contrary to popular belief, if we are not careful, AI can simply replicate the bias that already persists in society. With big data analytics gaining traction in recent years, HR analytics is viewed as a revolutionary and effective approach to increase hiring objectivity, processing efficiency all while reducing cost. Even though it is in businesses’ and HR professionals’ vested interest to increase the fairness and objectivity of HR analytical algorithms, biases and discriminations still persist within the system of automations processing.</p><p>In this white paper, we will reference Solon Barocas’ paper on “Big Data’s Disparate Impact” to build our framework, in which we layout two common themes of biases: historical and selection bias. We will then layout potential solutions to address historical biases and selection biases. We address both technical solutions that concern the data itself and general solutions which are structural changes. This white paper will help guide data scientists and HR personnel recognize the biases and heuristics that can be introduced or reinforced through algorithms and lay out a framework to mitigate some of these potential problems while building an algorithm.</p><h3>BACKGROUND</h3><p>Before we address the various methods biases can seep into algorithms it is important to understand where algorithms are currently being utilized in the HR world. Over the last few years, there has been an increasing demand to use algorithms in companies for HR purposes in order to reduce costs, increase efficiency and objectivity. According to LinkedIn’s Global Recruiting Trends Report, 76% of HR respondents believe that AI’s impact on recruiting will be at least somewhat significant.i From applying to jobs to predicting employee attrition, AI has taken a more prominent role in attempting to streamline the HR process. The following graphic represents the steps of the HR process that have incorporated AI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*4LEZlh4ZtQIYp8Q-XFfTUw.png" /><figcaption><em>Application of AI in the Hiring Process</em></figcaption></figure><h3>PROBLEM STATEMENT</h3><p>Bias is defined as “producing outcomes which are systematically less favorable to individuals within a particular group and where there is no relevant difference between groups that justifies such harm. Title VII was passed to protect the certain classes from discrimination particularly in an HR context, the problem is these laws have not been updated to reflect the widespread use of technology used in HR today. Legislation in the US has failed to address the fact that certain AI tools used in HR can perpetuate discrimination against protected classes. For the purposes of this paper we will use the terms HR and hiring interchangeably. We are using Barocas’ “Big Data’s disparate impact” to define each step of the data mining process and highlight the biases each step can produce. The five processes that can introduce bias include: defining the target variable, data collection and labelling, feature selection, proxies, and masking.iii Based on the observed characteristics we have then categorized each step of the data mining process into two broader themes of biases: historical bias and selection bias.</p><h3>Historical Bias</h3><blockquote><em>“When we blithely train algorithms on historical data, to a large extent we are setting ourselves up to merely repeat the past. … We’ll need to do more, which means examining the bias embedded in the data.” — Cathy O’Neill, Weapons of Math Destruction</em></blockquote><p>For the context of this paper we define historical bias as biases that emulate human like biases based on a historical or social context. Historically biases have always clouded decision making and these biases were both prejudiced and implicit. When AI emerged on the scene, it was looked at as a solution to eliminate such biases. However, machine learning algorithms are developed and trained using historical data, which can be biased given historical discriminatory practices and decision making. The data mining phases which are most susceptible to historical biases are data collection, defining the target variable, and masking.</p><h4>Defining the Target Variable and Class Labels</h4><blockquote><em>“In the case of systems meant to automate candidate search and hiring, we need to ask ourselves: What assumptions about worth, ability and potential do these systems reflect and reproduce? Who was at the table when these assumptions were encoded?” — Meredith Whittaker, Executive Director, AI Now Institute</em></blockquote><p>The first stage of the data mining process is defining the target variable and class labels. Bias can be introduced at this phase if the target variable is subjective. For example, if HR wants to use an AI prediction tool to evaluate employee performance, the definition of what dictates a “good’ employee may be an implicitly biased criterion. Furthermore, if a good employee is defined as never being late to work, then this definition will negatively impact those in a lower socioeconomic class because some employees might choose to live farther away from work where the rent is cheaper so they can pay off loan obligations or save more money. Living farther away from work increases the probability of being tardy and would resultingly be viewed negatively by the algorithm. This phase of data mining would be categorized as a historical bias because certain subjective target variable definitions can be problematic to certain classes of people.</p><h4>Labelling and Collecting the Training Data</h4><p>The labelling and data collection phase of the data mining process can be classified as a historical bias because often the training data that is used to build the algorithm can contain a sample which is reflective of biases that exist in society. For instance, in 2018 Amazon came under fire for a recruiting algorithm that rejected female candidates in favor of men for technical positions. The algorithm was based on historical data from resumes submitted, the problem was the resumes largely came from men, reflecting the male dominance in tech. In hiring, minority groups tend to be underrepresented in the workplace, thus the training data is likely to reflect this societal bias. This scenario would be classified as a data collection problem since the data collected failed to represent female applicants.</p><p>As mentioned earlier, companies have been increasingly relying on AI powered platforms like LinkedIn and Zip recruiter to advertise job listings. This presents a bias in terms of data collection because only people who have access to these platforms are at an advantage since they end up making up most of the applicant pool. Essentially, people who do not have access to these platforms due to lack of internet or other socioeconomic reasons are at a disadvantage. To summarize, a biased sample will produce a biased output.</p><h4>Masking</h4><blockquote><em>“Data mining significantly enhances the ability to conceal acts of intentional discrimination by finding ever more remote and complex proxies for proscribed criteria.” — Barocas</em></blockquote><p>Masking is an example of a historical bias because it is when individuals intentionally utilize technology to discriminate. An example of masking can be when individuals use algorithms to perpetuate society’s bias and then claim that since algorithms are fair and objective the results of the algorithm were valid. We assume that HR professionals and data scientists are not in the business to intentionally discriminate against individuals.</p><h4>Selection Bias</h4><p>In the context of this white paper we define selection bias as those data mining processes which pertains to the interpretation of data which influences variable selection. The variables selected to be featured in a data set play a huge part of the conclusions reached by the algorithms. Thus, it is pertinent that the variables included in the data set are carefully assessed for objectivity and value.</p><p>The data mining phases which we have categorized as a selection bias are feature selection and proxies.</p><h4>Feature Selection</h4><p>Feature selection is an example of selection bias because the features you select to include in your data set can influence the outcome of the algorithm. Feature selection can pose a problem in the screening process of hiring. For instance, if an organization includes the feature of having attended an Ivy League university in their hiring algorithm, then by including this feature, you have inadvertently introduced socio-economic bias against those candidates who might be equally smart but did not have the means to attend an elite private institution.</p><h4>Proxies</h4><blockquote><em>“However, when you create a model from proxies, it is far simpler for people to game it. This is because proxies are easier to manipulate than the complicated reality they represent.” — Cathy O’Neil, Weapons of Math Destruction</em></blockquote><p>In resume screening algorithms, potential issues of bias may occur since the names of schools attended, or zip codes of applicants can be proxies for race, gender, or socioeconomic status. In certain circumstances, zip code can be highly indicative of race or ethnicity. This is an example of selection bias because by leaving out race, the proxy variable of zip code would be treated the same as race.</p><h4>Importance of Debiasing</h4><p>Since the algorithms that are used by HR, influence both employer and the employees, it is in any business’s vested interest to increase the fairness and objectivity of these algorithms. Moreover, there are PR and legal incentives for companies to avoid potential discrimination lawsuits. Since the use of AI is only set to increase in the future, society should strive to mitigate any bias that could arise from these algorithms. In hiring, there has been an increasing emphasis on needing diversity in the workforce as research demonstrates organizations that embrace diversity yield higher innovation revenue. With this in mind, companies should seek to eliminate any bias that could hinder diversity in hiring. With careful oversight, more transparency and better processes AI technologies can not only streamline the hiring process but also make it more equitable for everyone.</p><h3>SOLUTION</h3><p>Before the use of algorithms in HR becomes more mainstream, it is crucial that HR personnel and data scientists take a proactive approach to addressing biases involved. In this section, we will be outlining potential solutions for both historical and selection biases and highlighting how they address the problems created in the data mining process. We will do so by providing technical solutions to improve the objectivity of the data mining process as well as more general solutions requiring structural changes that the industry can advocate for. The following solutions can help guide organizations to build a data mining process that accounts for any potential bias.</p><h3>Addressing Historical Bias</h3><h4>Technical Solutions</h4><p>In order to resolve historical bias, we recommend several technical solutions. First, the most obvious solutions to prevent bias in training data is to “train the model on a bigger, more diverse set of data and monitor training and results in order to quickly respond to issues [while testing] early and often.” Referencing the Amazon case mentioned earlier, Amazon’s mistake was that the training data contained a gender imbalance. If Amazon had ensured that the resumes collected contained an equal proportion of men and women, the output of the algorithm may have been fairer.</p><p>Second, we can achieve fairness by shifting the decision boundary for protected groups. Put differently, we can leverage the decision-making threshold for fairness by minimizing the effect of negative labels for data subjects that belong to a protected class with a small conﬁdence value. In essence, a way to maximize the performance of machine learning algorithms and to minimize bias is to “shift the boundary threshold to make an algorithmic or statistical decision accommodating for protected groups.” This solves the bias created by historical data and addresses the concerns using the target variable.</p><p>Third, we encourage companies to consider investing in explainable tools for tracking biases in the algorithm building process. In order to ensure models are properly monitored, technological giants and smaller startups are developing algorithms within the boundaries of explainable AI. An example of a company doing work in this field is Fiddler Labs. Fiddler Labs aims to provide the following:</p><blockquote>“· AI Awareness: understand, explain, analyze, and validate the why and how behind AI predictions to deliver explainable decisions to the business and its end users</blockquote><blockquote>· AI Performance: continuously monitor production, test, or training AI models to ensure high-performance while iterating and improving models based on explanations</blockquote><blockquote>· AI Compliance: with AI regulations becoming more common, ensure industry compliance with the ability to audit AI predictions and track and remove inherent bias.”</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*VeDQkq9A21SR5uAQwin75w.jpeg" /><figcaption>Fiddler Lab’s Explainable AI</figcaption></figure><p>This product would help bring transparency to algorithms and as a result make more holistic models. By being more conscious and aware of what and how an algorithm is working could help reduce the biases and discrimination caused by the labelling and data collection phase.</p><h4>General Solutions</h4><p>In the interest of addressing historical bias, we also recommend general solutions such as increasing diversity and legislation. When we talk about an increase in diversity, we are not just referring to diversity of race and gender but also diversity in academic background and specialty. An interdisciplinary approach to data science is necessary in order to provide historical and social context to the data. For instance, it is essential that social scientists and ethicists are made a permanent fixture in teams that produce algorithms. Reason being “that providing individuals with different expertise, viewpoints and interests will assess the ethical significance and implications of their work in different and potentially complementary ways.” Maintaining diverse perceptive in an algorithm producing team can help contextualize the data and provide recognition of biases or undue advantages when defining a target variable or class label. According to “Everyday Ethics for Artificial Intelligence”, “diverse teams [of data scientists and programmers can also] help to represent a wider variation of experiences to minimize bias [within algorithms].”The diversity of gender, ethnicity, educational background, and cultural perspectives within the team will likely reflect an improved algorithm development process.</p><p>In 1964, Title VII was passed to protect certain classes of people from discrimination in the context of employment. Title VII was a landmark piece of legislation as it acknowledged the historical biases at play in the context of employment. As companies seek to replace human decision makers with algorithms in the field of HR; it is imperative that this piece of legislation is updated to reflect the technology of AI tools. An update to Title VII would prevent masking, which represents a deliberate form of discrimination. We need to advocate for civil anti-discrimination legislation to be updated to accommodate the innovations in technology in particular the use of AI. The HR industry needs to be proactive in advocating for legislation that acknowledges that algorithms can be used to perpetuate discrimination.</p><h3>Addressing Selection Bias</h3><h4>Technical Solutions</h4><p>The first technical solution to addressing selection bias is a concept known as RRB, Resilience to Random Bias. This method requires the introduction of a new random instrumental variable, which has no correlation with the target variable, to be assigned to each data subject. Next, we randomly select a set of subjects using this new instrumental variable and introduce a known bias based on selected features to these subjects. If we can recover the original unbiased subjects without using the instrumental variable, we can consider the algorithm fair for the features we use. In addition, we can use various empirical methods like t-test to understand this form of statistical disparity. However, this solution accounts for biases that carry statistical evidence and could fail in the non-ideal scenario of an already present historical bias. Hence, this method should be used “as a minimally necessary condition but not necessarily a sufficient condition for individual fairness.” This method serves as a very good solution to tackling unconscious/hidden statistical biases.</p><p>The second solution to addressing selection bias is adversarial debiasing. Adversarial debiasing is the technique of minimizing discrimination in machine learning by evaluating the fairness of the algorithm. In “Generative Adversarial Nets”, Ian J. Goodfellow explains the mechanism of building a fairness metric and improving it in machine learning algorithms. If the fairness metric is not desirably high, it is debiased using a type of neural network called the adversarial network.xiv This is relevant because hiring algorithms use several sensitive attributes and features of data subjects to predict a target variable. The adversarial network calculates the target variable and calculates the accuracy with and without using each sensitive attribute. Features that point to historical discrimination can be used as sensitive attributes and for each sensitive attribute, the percentage ratio of accuracy (commonly referred to as p%) can be calculated as the ratio of accuracy with and without using the sensitive attribute in the model. Using this ratio, we get a fair sight of accuracy-costs vs debiasing-benefits of selecting a feature. In the article, the author shows an example where he uses race as a sensitive attribute to calculate accuracy and fairness. By using the adversarial network for a sensitive attribute, the author improves the fairness from 38% to 70% and sacrifices the accuracy by 6% (from 77% to 71%). However, this technique is still heavily dependent on the developer of the algorithm because the choice of sensitive attributes lies in the intuition of developer.</p><p>Both the methods mentioned above solve different problems of the Barocas’ framework in an ideal world assuming the bias is one-dimensional. However, real world scenarios are a lot more complex. Hence, the best way to mitigate such biases is by contextual use of both the solutions mentioned above to assist in minimizing bias caused by hidden data and unhealthy feature selection methods.</p><h4>General Solutions</h4><p>As a result of GDPR we can expect that some form of an algorithm accountability legislation will be implemented in states across the US if not nationally in the coming years. Given this fact it is in the best interest of HR professionals and data scientists to be at the forefront of these policy discussions. In fact, we can encourage members of congress to pass the Algorithm Accountability Act, the specifics of which are highlighted below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/711/1*u3SpdXEJ4OgyE3bIbbQEkA.png" /><figcaption><em>Specifics of the Algorithm Accountability Act</em></figcaption></figure><p>The assessments outlined by the Algorithm Accountability Act would ensure companies take a closer look at the inputs of their models and enable them to identify potential proxies and other forms of selection bias. Legislation is not a perfect solution as there are always issues of enforcement that can pose a problem. Nevertheless, legislation is a necessary first step to presiding over algorithms as they establish universal standards for companies to abide by.</p><p>Another proposed solution to selection bias would be implementing audits of algorithms. A roadblock to increasing transparency of algorithms is the fact that companies consider their algorithms trade secrets. In order to appease companies and maintain accountability the auditing process can look like companies disclosing their algorithms to “expert third parties who hold them in a safe escrow, permitting public interest scrutiny but not allowing the algorithm to become public.”Auditing can absolve feature selection issues because a non-biased third party can question why certain features were included in the data.</p><h3>CONCLUSION</h3><p>We can expect the use of AI to further revolutionize the HR industry. Thus, it is imperative that as HR professionals and data scientists we are proactive in addressing the risks and consequences algorithms can create. This paper has outlined historical and selections biases that the various steps of the data mining process can create. In order to mitigate historical and selection bias we need a combination of both technical solutions that fix problems with the data and more general solutions such as diversity, legislation and auditing. By acknowledging these problems, we can utilize AI tools in HR to benefit everyone in society not just a select few.</p><p><em>** This study was co-authored by Yash Kanoongo, Keena Desai, Jiawen Li, Peter Law and Sai Kiran Srikeddy of the Fuqua School of Business at Duke University</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b0f9003ed64" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting Factors for Employee Attrition in IBM]]></title>
            <link>https://medium.com/@yashkanoongo/ibm-attrition-81975347db14?source=rss-88f57cfbdbd6------2</link>
            <guid isPermaLink="false">https://medium.com/p/81975347db14</guid>
            <category><![CDATA[people-analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[analytics]]></category>
            <dc:creator><![CDATA[Yash Kanoongo]]></dc:creator>
            <pubDate>Fri, 21 Feb 2020 21:51:04 GMT</pubDate>
            <atom:updated>2020-03-15T02:31:49.597Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O9BvgU99-SM4Rs6bt8HT1w.jpeg" /></figure><h3><strong>Business Understanding</strong></h3><p>Attrition is defined as when an employee resigns or retires from a company. Training a new employee is a costly and long process, it is in a company’s best interest to decrease employee attrition. Through exploring the IBM dataset, we would like to analyze factors that would be highly suggestive/predictive of an employee leaving the company. The analysis can give the HR department and management some insights into some of the reasons that employees decided to leave the organization. They can then implement any changes that might incentivize employees to continue to work with the company. Moreover, this model can also help change hiring practices, by anticipating the qualities in individuals that would make them more likely to stay with the company for a longer period of time and hiring accordingly. We will be making use of unsupervised models such as PCA and K-means to gain an understanding of the variables in our dataset, and supervised models such as regression modelling, random forest, lasso and post lasso and neural network to establish which variables best explain attrition within IBM.</p><h3><strong>Data Understanding</strong></h3><p>The dataset was created by IBM employees and was downloaded from Kaggle. The dataset is fictional and that data does not actually represent any actual IBM employees. The dataset consists of 1470 observations and 35 variables. This is supervised learning as we have identified the target outcome as attrition. We anticipate the variables that relate to compensation (ex: percent salary hike, monthly income), overtime work, age, and tenure with the organization will be the most useful. One thing we noticed was that majority of the Attrition values were “No” in the dataset, however, we were not as concerned about it as the difference between the “No” and “Yes” values were not extreme.</p><h3><strong>Data Preparation</strong></h3><p>In terms of data preparation, there were no missing values in the dataset. There were, however, many categorical variables that were converted into dummy variables. Furthermore, there were three variables “Attrition”, “OverTime” and “Gender” that were coded as “yes” or “no” and had to be converted to a 0 and 1. There were also three variables “Over18”, “Standard Hours” and “Employee Count” that all had the same value for each observation, so we decided to remove these variables. Furthermore, we also had to rename certain variables due to issues with spacing and hyphens. In order to perform cross validation and get out of sample results we split up the data into a train and test set. We divided this data based on 80% sample data and 20% holdout data. In order to check for balance, we took the means of cases for where attrition was 1 in both the train and test set to see if they were close together. As shown below the means were relatively similar, so we can assume balance.</p><p>mean(IBMdata2$Attrition==1) : 0.1632653</p><p>mean(IBMdata2.holdout$Attrition==1) : 0.1530612</p><h3><strong>Data Exploration</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/671/1*v7esyq7Ijfp9va3Rl8xTzw.png" /></figure><p><strong>B.</strong> <strong>Data Visualization</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*-PnRlqRtdE_PrBTZUmL1ng.png" /></figure><p>In order to further explore the relationships between the various variables in our dataset, we identified certain variables that might intuitively explain some of the Attrition within IBM. The boxplot is broken into two parts in order to develop some intuitive reasoning behind the reasons and timing of Attrition in IBM. From the first boxplot, we can see that the median avg daily salary of an employee, peaks at year 7 when working under the same manager and is observed to be highly variable thereafter. From the second boxplot in the figure, we expectedly notice higher attrition within the first 5 years of working under the same manager, possibly in anticipation of declining or highly variable salaries in the future. Alternatively, if an employee is under the same manager for 7+ years, it could also possibly mean that he/she is not getting promoted, explaining the decline in median salaries. Therefore, in order to further confirm our intuition, we expanded the scope to all employees, not just the ones managed by the same manager over several years.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*_MYLiF7JMopWJMzk7ReRYA.png" /></figure><p>Observing the trend of the scatter plot, we can observe that Daily Wage of employees, on average, is expected to drop, the longer they stay at IBM. This explains the high Attrition Rate in the first 5–10 years (as shown by the purple color points in the figure). It appears that the employees, after spending some years within the organization, tend to realize that their salaries will decline and their value to the organization will decrease, which results in them leaving to pursue other opportunities.</p><p><strong>C. Unsupervised Modelling:</strong></p><p><strong>i) Principal Component Analysis</strong></p><p>We looked at PCA to find patterns and extract the latent features from our dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/509/1*vdh-5xKnncUE5JSZmVus_Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*3ckNi-tBBd5Xe9djNQkJmQ.png" /></figure><p>The histogram and the table above indicate that the drop off in variation between principal components is not very drastic, which hints at the interpretation that PCA might not be the most accurate unsupervised model. As seen above, our first four principal components explain only 26.78% variation in our model.</p><p><em>First Principal Component</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/447/1*u4LNFsytX6mJhyOCshbuYg.png" /></figure><p>The first principal component explains the relationship between Salary and Work Experience. There is a high negative value for Monthly Income, Total Working Years, and Years at Company. This is likely because of the observed negative relation between wages and years of work experience. We did not consider the other Principal Components as they do not explain significant variation in our data.</p><h4>ii) K-Means</h4><p>We also looked at k-means and found only 17% variation explained with four clusters. We then looked at how the clusters related to attrition and found the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/426/1*FZBsBbh2Re5PXIfTp9VkrA.png" /></figure><p>Both our unsupervised models point to the fact that no cluster of variables explain a large amount of variation in our dataset. Since the variation is evenly distributed across variables in our dataset, clustering methods are not very suitable for our dataset.</p><h4>Modeling</h4><p>As this is a classification problem, we chose to build several supervised models and test the out of sample R² for these models.</p><h4>A. Random Forest</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*nf5H-vRlK_JseCQcDrOkIQ.jpeg" /></figure><p>The Random Forest that we ran used 500 trees with 10 variables split at each iteration. The random forest performs better than the null model but is not as good as the logistic regression. However, the random forest helps explain the significance of the variables by using the percentage increase in mean squared errors. The comparison of all the variables is shown on the left with their Mean Square Error Percentage. We can see that older aged employees, employees working over-time and long serving employees are the ones likely to quit.</p><h4>B. Logistic Regression</h4><p>We chose to use logistic regression since it can provide us with the probability of attrition, and this is a classification problem. We used both forwards and backwards stepwise function in order to get a logistic regression model. The step model selected 30 variables and received an AIC of 737.35. Some notable variables that we saw that had a large coefficient and statistically significant p value include: OverTime, Job Role Healthcare Representative and Job Role Research Scientist.</p><h4>C. Decision Tree</h4><p>We built a decision tree using the variables from the step wise logistic regression. However, with all the statistically significant variables of the logistic regression, the performance of the decision tree is worse than the null model. The ROC curve for decision tree can be misleading and the residual sum of squares and the R-squared values show us that the performance of the decision tree is sub optimal.</p><h4>D. Logistic Regression with interactions</h4><p>We used second degree interactions with all the possible pairs and performed a logistic regression to predict the attrition rate. The model performs worse than the null model. The number of variables used is very high and hence, the model suffers from too many correlations without causation. The bad performance combined with the complication of too many variables to evaluate the model does not make this model a good one.</p><h4>E. Lasso</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/542/1*G022jXsS_8HFQSkSYpan8A.jpeg" /></figure><p>Lasso is a good way to help us select variables. We chose to run a lasso on an interaction model with minimum MSE optimization selection of 99 variables. The out of sample R² performance was 0.35 and was achieved using aggressive variable selection(feature.min). The downside of this model is that it selected too many variables, and apart from ‘JobInvolvement’ all the other 98 variables are interaction terms, which means the influence of the variables are confounded.</p><h4>F. Post Lasso</h4><p>The Post Lasso performance is better with variable selection than Lasso by theory, conservative and aggressive approaches. The aggressive approach under Post Lasso performs the best and has the highest R2. We calculated the lambda min and lambda 1se and created features for both the cases. Using these lambda thresholds, we could analyze the performance metric of interest which is the R2 along with all the other models. Post Lasso with aggressive variable selection has the best overall performance with an out of sample measure of 0.401.</p><h4>G. Neural Network</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/1*keE2xQRfXwVc8LbH-bLe4w.jpeg" /></figure><p>Apart from the basic data mining models above, we also chose to apply advanced Neural Network model. The biggest advantage of using this model is that it generates a high prediction accuracy of the employee attrition level. However, it is hard to know how the model weighs each feature, which is the essence of our business problem. In our case, the prediction accuracy is 85.7%, which is much higher than other basic models. After adding regularizer or the dropout, the accuracy doesn’t chance much. The graph above shows the training error after iteration and the bottom is showing the prediction accuracy for the holdout sample.</p><h4>Evaluation</h4><p>We are using R-Squared as a performance metric to compare different models and methods from variable selection among the models. The null model shows that the R-squared value is approximately equal to zero and we compare our models with the null model as the baseline. The ROC curve shows that Logistic Regression has the best True Positive and False Negative Rates. Furthermore, it also shows the best R- squared value before performing k-fold cross validation and Lasso. However, Lasso and Post Lasso help us with better variable selection and boosts the R-squared, especially aggressive variable selection in 10- fold cross validation. In order to understand statistical significance of variables on the attrition rate, the random forest performed with 500 tree iterations helps us evaluate the MSE percentages. However, the R-squared value is not as high as Logistic regression, so we can use the MSE percentages (%IncMSE) to evaluate statistical significances. The model suggests that Age, higher Number of working years, and OverTime are significant influencers of attrition rate.</p><p><em>R² as a performance evaluation metric</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/1*vrc0wfKPKRsm3TYgggXXTA.jpeg" /></figure><h4>ROC curve</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G8GA1ixNCiYoZ_aYgtpZdQ.jpeg" /></figure><h3><strong>Deployment</strong></h3><p>The resulting model would be the most useful to IBM’s human resource department. Given that the logistic regression model was the best predictor for attrition within IBM, we suggest IBM to pay close attention to two variables explaining a significant amount of attrition in the organization. The most important being Overtime. Through our model, we observed that employees that worked overtime have the highest likeliness to leave the organization. Alternatively, it was also observed that no business travel was associated with strong negative correlation with Attrition, explaining that the employees preferred not having to travel for work. Our models can thus give IBM some insight into the attributes of their employees and can assist them in building policies keeping these attributes in mind. Some concerns and ethical consideration need to be kept in mind when using our models as a basis for decision making in the organization. Our performance metric, R2 is observed to be low across models. Thus, while our models should help IBM understand some of the reasonings behind why employees leave their organization, there is potential for confounding factors that may be playing a large but unexplained role leading employees to leave the company. Finally, when it comes to data associated to personal information, there are always ethical and data privacy concerns to keep in mind.</p><p><em>** This study was co-authored by Yash Kanoongo, Keena Desai, Jiawen Li, Peter Law and Sai Kiran Srikeddy of the Fuqua School of Business at Duke University</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=81975347db14" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting Attributes Influencing User Engagement on Yelp]]></title>
            <link>https://medium.com/@yashkanoongo/yelp-engagement-bbd9cb1ce2c9?source=rss-88f57cfbdbd6------2</link>
            <guid isPermaLink="false">https://medium.com/p/bbd9cb1ce2c9</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[yelp]]></category>
            <dc:creator><![CDATA[Yash Kanoongo]]></dc:creator>
            <pubDate>Fri, 21 Feb 2020 21:48:17 GMT</pubDate>
            <atom:updated>2020-03-15T02:33:43.369Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*osqvYwMWshtq7DSiFuMp-Q.jpeg" /></figure><h3>Introduction</h3><p>Yelp is an online platform founded in 2004 with the goal to help consumers connect with other consumers and engage with local businesses. Yelp has an average of 37 million unique visitors per month and has nearly 200 million reviews in Q2 2019. Yelp makes money by selling ads and recommending sponsored results to Yelp users from their database. This Yelp dataset is a subset of Yelp’s businesses, reviews, and user data. It consists of 5 file data files and 52 items and they are uploaded to Kaggle by Yelp for students to analyze their Yelp’s data and seek for potential insight. For the purpose of this project we will be utilizing the business, check in and tips data file.</p><h3>Business Understanding</h3><p>Looking at the historical data on variables such as attributes, reviews and postal codes, we want to know what number of check-ins, in other words, engagement level among Yelpers<strong>, </strong>would a business expect to receive? We are attempting to use this dataset to predict the number of check-ins of a restaurant in Toronto, Canada segmented by zip code, the count of reviews, category of the business, days of operation, and other business attributes. According to a 2017 analysis conducted by the USDA, restaurants brought in sales revenue of $27 Billion annually in the Greater Toronto/Ontario region, and are expected to grow at 5.6% YOY for the next 5 years. This makes the Toronto/Ontario region as one of the most lucrative regions in North America to invest in the restaurant/hospitality business. Check-ins serve a twofold purpose — firstly, it acts as an important marketing tool for listed businesses to reach the friends, families and interested followers of a checking-in customer. Secondly, it serves as a crucial data point for Yelp to consider when understanding its users’ preferences for future restaurant recommendations. A data mining solution can help confirm or reject a business’s hypothesis of what attract customers to check in to their establishment. We will be using a linear model to understand which variables influence a customer’s decision to check in at a restaurant.</p><h3>Data Understanding</h3><p>The original business dataset available online consist of 14 items and 6 of them could either be used as categorical or numerical variables. The check in dataset consists of 2 items while the tips file consists of 5 items. The dataset can be downloaded from <a href="https://www.kaggle.com/yelp-dataset/yelp-dataset/version/9">Yelp Dataset Version 9</a>. Since we are conducting our analysis solely for the city of Toronto, we have excluded all geographical and categorical variables not associated</p><p>with our target location. We decided on the city of Toronto because it is the city with the second highest number of datapoints in the dataset. The highest is Las Vegas but we felt that because of the location and nature of the Las Vegas Strip, Las Vegas would not be reflective of a typical city. Furthermore, we recognize that using check-ins as a measure of Yelp engagement is not completely reflective of popularity since there are Yelp users who use the app but do not necessarily check-in or write reviews. We understand that users from each city might have various subjective understandings of the review grading system and the consuming habits within the city of Toronto could enormously differ from other cities. Hence, all quantitative analysis conducted based on the project dataset is solely applicable for the city of Toronto.</p><h3>Data Preparation</h3><p>The cleaned data consists a total of 71 variables and 2,153 observations. Among the 71 variables, 22 of the stored variables originated from the “category” variable, indicating the type of restaurants each entry belongs to. During the data cleaning process, we have eliminated categorical variables that are neither associated with types of cuisines nor food. We also added a new column called the ‘simplified zip code’ which is a collection of postal codes within an area — This was done using Wikipedia — Postal Codes in Canada. Within the existing cleaned dataset, we are using the combination of 71 variables to predict the specific check-ins (indicator for engagement) for an existing or an upcoming restaurant. For missing data cells within a data entry, we have replaced the null value with the mode of the variable for each simplified zip code i.e., the most popular/repeating restaurant feature in the area. 44 columns/features have been cleaned with this methodology. The variables in our dataset can be grouped in the following categories: Yelp engagement metrics (ex: check-ins, compliments, reviews), categories (ex: Bakeries, Coffee, Bars), Attributes ( ex: Free Wi-Fi, Delivery, Price), and general descriptors ( ex: Post Code).</p><h3>Exploratory Data Analysis</h3><p>One of the first aspects of the data that we noticed was the price range and pin codes for which there was a high frequency of check ins. From our exploratory analysis we noticed that restaurants with a price range of 2 on a scale of 1–4, had the most number check ins as well as the largest spread, as evidenced by the graph Check-ins vs Price Range boxplot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*6KeXp6g3Mu9WyoXfIHzSXA.png" /></figure><p>In order to get an idea of the postal code of restaurants with the highest concentration of check ins in Toronto, we created a Check-ins vs Postal Code histogram and through it we identified that postal code ‘V’ had the highest number of check ins. Postal code ‘V’ is located in the Scarborough/Downtown Toronto region. This is consistent with our spatial analysis, (found in the appendix) which maps the location of restaurants in Toronto — the highest density of them being in the downtown region.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*KcCiR3jkkCMt7JBS7CwDgQ.png" /></figure><p>Next, we explored the correlation between variables in our dataset using a matrix of scatter plots. This was done to get an idea of the variables that might significantly influence the likeliness of check-ins. We observed that review count had an extremely high correlation with check-ins. This was expected as those customers who are likely to take the time to leave a review of a restaurant will more often then not, also check-in to the restaurant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*KMBCm1ZXQjjePc2ieuv58Q.png" /></figure><p>Based on our intuition and the results of the scatter plot matrix, we built two scatter plots — User Experience vs Check-ins and Review Count vs Check-ins. We see a strong positive correlation in both these scatter plots, which was in line with our intuition and the data from the scatter plot matrix that we saw above.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*AN76_deXYUjv_XHM4PDQSg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*Q4KccwSPyuZN4OTlML4ZOQ.png" /></figure><h3>Interaction Exploration</h3><p>We investigated a few interaction eﬀects between variables that had a large correlation with check- ins.</p><h4>Compliment Count vs. Free Wi-Fi</h4><p>The plot below shows the interaction between the compliment count and whether a restaurant has a free Wi-Fi or not (0 indicating no, 1 indicating yes). From this plot, we can see that, for restaurants that have free Wi-Fi with the same compliment count, typically would have more check-ins compared to restaurants without Wi-Fi.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/506/1*OWOmnYknPW1As8ScD5YAiA.png" /></figure><h4>Review Count vs. Pricing Range</h4><p>We suspect that users are more likely to check into establishments with a lower price range because they are more affordable for the general population, therefore more accessible. This plot below agrees with the general intuition that the higher the pricing range the lower the check-in records. Price range = 1 and price range = 2 nearly overlaps with similar slopes, likely suggesting that the two variables have almost identical effects. This plot also shows that there appears to be a signiﬁcant disparity between average prices of restaurant vs. review counts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/520/1*CETd5Vb4d6lydnoaQ7r1xg.png" /></figure><h3>Modeling</h3><h4>Base Model</h4><p>To observe complete linearity without considering specific features that might matter for someone to check-in to a restaurant, we built a base model where check-ins are predicted purely on popularity of the restaurant (stars, review count) and Yelpers engagement with the restaurant(compliments). The rest of our regression models are built on top of this base model.</p><p><em>Predicted(Checkins) </em>= stars + compliment_count + review_count</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*8U8PDhwYCV6YsUHF-hAeNw.png" /><figcaption><strong>R-Square </strong>= 0.7123, <strong>AIC </strong>= 19359.94</figcaption></figure><p>We see that the ratings do not really indicate anything about the check-ins as much as number of people reviewing it or number of people complimenting the reviews does.</p><h4>Intuitive Variable Selection</h4><p>Based on our general understanding of Yelp user engagement we built incremental models based on what factors we thought would lead to a person checking in. We assumed that the price range, star rating, review count, and the compliment count would influence a user’s propensity to check in at an establishment. Compliment count, review count and star rating are examples of Yelp engagement metrics, therefore, we thought these variables would be a strong indicator of check ins, another Yelp engagement metric.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*ZbS8qZD4jg3KV256J83gqQ.png" /><figcaption><strong>R-Square = 0.7134, AIC = 19200.62</strong></figcaption></figure><p><em>Predicted(Checkins) </em>= as.factor(RestaurantsTableService) + as.factor(OutdoorSeating) + as.factor(Bars) + as.factor(PriceRange2) + compliment_count + as.factor(RestaurantsTakeOut) + as.factor(FreeWiFi) + as.factor(Desserts) + review_count + as.factor(RestaurantsDelivery) + as.factor(CoffeeTea) + as.factor(IceCream) + as.factor(Shopping)</p><p>During our iterations, we saw that UserExp (People who have tipped at a restaurant and reviewed the restaurant) is a stronger indicator of engagement (check-ins) than the overall review count. These two variables are highly interactive, which can also be confirmed by their correlation value. Due to the higher significance of user experience and high interaction with review count, we chose the model with only user experience which gave us the higher R square value too.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*L6Iz0vzwjgyNv1Eu97-AJQ.png" /><figcaption><strong>R-Square </strong>= 0.7885, <strong>AIC </strong>= 18703.91</figcaption></figure><p><em>Predicted(Checkins) </em>= as.factor(HasTV) + as.factor(Bars) + as.factor(PriceRange2) + compliment_count + as.factor(RestaurantsDelivery) + as.factor(FreeWiFi) + as.factor(Desserts) + UserExp + as.factor(IceCream)+ as.factor(Shopping)</p><p>We also strongly believed that a few features of restaurants are very good indicators of why a user would visit/engage with a restaurant. For example, free Wi-Fi is a very good method of generating users to check in, in fact some restaurants often require customers to check in on Yelp in order to receive the Wi-Fi password. Hence, we see high significance and positive correlation. Restaurants with the delivery option evidently will have less check-ins, which makes sense because if a person is receiving delivery from a restaurant they are not physically going to the restaurant. Hence, as shown in the above coefficients, Restaurants delivery has a significant negative correlation with check-ins. The Desserts cuisine however turned out to surprise us showing that, less people check-in at those restaurants (shown by the negative correlation).</p><h4>Manual Forward Selection</h4><p>After 19 incremental iterations, we arrive at the following model:</p><p><em>Predicted(Checkins) = </em>review_count + UserExp + compliment_count + as.factor(FreeWiFi) + as.factor(HasTV) + as.factor(Bars) + as.factor(PriceRange2) + as.factor(RestaurantsDelivery) + as.factor(Desserts) + as.factor(CoffeeTea) + as.factor(IceCream) + as.factor(Shopping)</p><p>Coefficients can be seen below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/1*__DGmWdLjVElBXLlnuZWAw.png" /><figcaption><strong>R-Square </strong>= 0.7947, <strong>AIC </strong>= 18642.53</figcaption></figure><p>In the manual forward selection, we start with the base model and incrementally build up to maximize R- square value and minimize the AIC. After 19 iterations, we arrived at 13 final variables. 7 of the 13 variables are positively correlated to check-ins. Among variables that positively affect the dependent variable, the number of compliments heavily influences the dependent variable with a coefficient of 46.99. Restaurants with delivery services and Ice Cream/Dessert sellers have the most significance negative impact on the dependent variable. However, the restaurants delivery feature is far more significant than the restaurant being an ice cream seller, which is evident from the p-values. Among all pricing levels, restaurant with price range = 2, which is $11-$30 per diner, is the only level that significantly affects number of check-ins, with its coefficient at 11.34.</p><h4>Automatic Backwards Selection</h4><p>After 20 iterations through the step() function, we arrive at the following model:</p><p><em>Predicted(Checkins) </em>= as.factor(Sandwiches) + as.factor(Chinese) + as.factor(Pubs) + as.factor(EthnicFood)+ as.factor(Japanese) + as.factor(Bakeries) + as.factor(HasTV) + as.factor(Bars) + as.factor(PriceRange2) + compliment_count + as.factor(RestaurantsDelivery) + as.factor(FreeWiFi) + as.factor(Desserts) + review_count + as.factor(CoffeeTea) + as.factor(IceCream) + as.factor(Shopping) + UserExp</p><p>Coeﬃcients can be seen below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*OKAKTC8Hdbs32w8jgFGjcg.png" /><figcaption><strong>R-Square </strong>= 0.797, <strong>AIC </strong>= 18624.26</figcaption></figure><p>The backward selection method automatically selects variables in the data by eliminating the least significant variables until it has reached the highest possible R-square. Through the built-in function in R, step(), we have trimmed our final model to one that consist of 18 predictor variables. Among all variables present in the model, 10 of these variables positively affect the dependent variable, while the other 8 negatively affect the dependent variable. Among variables that positively affect the dependent variable, the number of compliment count and review count (Overall review count) and UserExp (Tipped review count) have the highest significance within the model; whether the restaurant has free Wi-Fi comes closely in third. Among all variables that positively affects the dependent variable, the number of compliment count has the highest positive effect, with its coefficient at 45.71. On the other hand, just like in the manual selection process, restaurants with delivery services and Ice Cream sellers have the most significance negative impact on the dependent variable. Among all pricing levels, restaurant with price range = 2, which is $11-$30 per diner, is the only level that significantly affects number of check-ins, with its coefficient at 11.33.</p><h3>Evaluation</h3><p>The models provided indicate that the influencers are the same in each of the models, except for in the base model which uses only the numerical variables. The variables that are most significant are the following:</p><p>1. Number of Tipped Reviews for each restaurant.</p><p>2. Number of Compliments to the reviews on each restaurant.</p><p>3. If the restaurants provide Free Wi-Fi.</p><p>4. If the restaurants provide Delivery option — Less people visit/check-in.</p><p>5. If the restaurants have a full bar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*J3s7UGxoJT-8h5gS_kubVA.png" /></figure><p>· The base model does not use a lot of variables and uses only the numerical variables. Hence, it is not really an accurate model to give us enough insight into engagement of Yelpers.</p><p>· The first intuitive model uses the overall review count of the reviews along with other business features that are relevant to engagement of Yelpers.</p><p>· The second intuitive model uses the same set of business features but uses tipped review counts. The tipped review counts can be a substitute for the review counts, and they provided a much more accurate result than overall review counts.</p><p>· The manual forward selection is much more accurate than the intuitive models. However, the intuitive model gives a better insight into the business relevance and isn’t as time consuming as the intuitive model.</p><p>· The automatic backward selection is the most accurate model and takes less time. However, this gives results only based on what is in the data and does not carry full business context.</p><h3>Deployment</h3><p>In order to fully utilize the results of the model, current and future restaurant owners must be associated with Yelp; with that in mind, restaurant owners will be able to determine the popularity of the restaurant, measured by the number of check-ins to the restaurant. From our evaluations above, future restaurant owners should be aware that geographical location within the city of Toronto does not have a significant effect over the popularity of the business. Based on our models we can inform business owners possible actions that they can take to try to increase their Yelp check-ins. For instance, from the business operations perspective, we believe the key variables that enhance popularity are the following: Wi-Fi availability, being at a pricing level of $11-$30 and whether the restaurant has a bar. From the marketing perspective, number of reviews on Yelp and the compliments to the reviews on Yelp are some of the main determining variables for popularity and engagement. As far as we are concerned, there would not be any ethical considerations since all data would be provided by Yelp based on data collected from users after they have agreed to the data policies on Yelp. However, restaurant owners must understand merely focusing on enhancing the marketing effort on Yelp and other social media platforms might not enhance the popularity of the restaurant as the results of the model is only correlational, not causational. Therefore, for the model to function properly, the data entries of the restaurant on Yelp should not be purposefully bolstered by marketing efforts.</p><p><em>** This study was co-authored by Yash Kanoongo, Keena Desai, Jiawen Li, Peter Law and Sai Kiran Srikeddy of the Fuqua School of Business at Duke University</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bbd9cb1ce2c9" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>