<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Surmayi on Medium]]></title>
        <description><![CDATA[Stories by Surmayi on Medium]]></description>
        <link>https://medium.com/@surmayi7?source=rss-7de3ddde7f7b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*uT0wWlZ_w3v4zn8nv6zwzA.png</url>
            <title>Stories by Surmayi on Medium</title>
            <link>https://medium.com/@surmayi7?source=rss-7de3ddde7f7b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 02:57:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@surmayi7/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Are men overrated by women?]]></title>
            <link>https://medium.com/analytics-vidhya/are-men-overrated-by-women-9b3b926e0a8b?source=rss-7de3ddde7f7b------2</link>
            <guid isPermaLink="false">https://medium.com/p/9b3b926e0a8b</guid>
            <category><![CDATA[anova]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <dc:creator><![CDATA[Surmayi]]></dc:creator>
            <pubDate>Sun, 13 Dec 2020 13:07:05 GMT</pubDate>
            <atom:updated>2020-12-13T16:18:05.557Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Abstract</strong></p><p>The research is a part of an observational study that records the ratings of several personality attributes by people who volunteered in a Speed Dating Experiment. The focus of this paper is to find if, for the same level of various personality traits, do men enjoy more affection compared to what they impart on women. The result of this research shows that a male is rated about 0.23 points (on a scale of 0–10) higher than a female of a similar persona.</p><p><strong>Introduction</strong></p><p>Some researches have shown that men overestimate their attractiveness while another research says evolution has made women more attractive. The objective of this paper is to see if gender has a difference in how people perceive their partners.</p><p><strong>Problem Statement</strong></p><p>The focus is to find if gender influences how one rates one’s partner. Also, to find the most significant traits of a personality that contribute to the overall likeness towards one’s partner.</p><p><strong>Research Methodology</strong></p><p>The primary research was done by Professor Ray Fisman and Professor Sheena Iyengar from Columbia Business School. Around 552 heterosexuals from various professions have taken part in this experiment where they each have a four-minute date with various other participants of the opposite sex. They are rated before, during, and after the short date on the parameters of Attractiveness, Sincerity, Intelligence, being Fun, and Ambitious. Then a likeness rating was awarded which denotes the overall affection towards the partner. The experiment also includes interests and preferences of date, how the persons rate themselves on these parameters, etc. The results are available on Kaggle and can be used to answer a wide variety of questions regarding the mating behavior. This paper is focused on the five personality attributes and the overall likeness rating awarded to the participants. All the ratings are recorded on a scale of 0 to 10.</p><p><strong>Hypotheses</strong></p><p>The null hypothesis is that a male and a female with similar personalities, i.e., who received the same rating on the five attributes, also receive the same overall likeness rating from their respective partners. The alternate hypothesis is that an equally rated male and female, receive different likeness ratings from the partners.</p><p>Ho: Males and females with similar personalities standards receive similar likeness rating.</p><p>Ha: Males and females with similar personalities standards receive different likeness rating.</p><p><strong>Solution</strong></p><p>The analysis is broken down into four steps. The first step is to analyze the data set and perform data pre-processing. It includes inspecting the observations to ensure a random spread from the true population so that the sample is not biased. The second step is to visualize, and, if required, perform the transformations, on the independent and dependent variables. The third step is to shortlist the most significant variables that help in explaining the overall likeness rating. The model obtained with the most significant variables is termed as Reduced model. Then the indicator variable of gender, which denotes 1 for males and 0 for females is added to create a Full model. Finally, ANOVA is conducted to compare the reduced and full models and find the significance of gender in determining the overall likeness rating.</p><p><strong>Analysis</strong></p><p><strong>Data Selection</strong></p><p>The raw data set has about 2000 observations. After removing the rows that were missing at random, the data set is found to be disproportionately based on profession as almost half of the data set contains participants who either belonged to Law or to Business fields. An approx. same ratio and sample size of men and women are randomly chosen from the various professions like healthcare, politics, etc. to create a balanced data set of 706 observations.</p><p><strong>Analysis, Visualization, and Transformation</strong></p><p>There are approx. 45% of females and 55% of males. The proportion of couples belonging to the same race is 51% and to different races is 49% (<a href="#Table1Open">Table 1</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/913/1*MGzpGMPNHeGC9KIlzaFkXw.png" /><figcaption>Table 1</figcaption></figure><p>Participants from 6 professions each contribute within the range of 14–19% to the total sample size (Table 2).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/931/1*5G04boE6Q28M4st6jQ-MSw.png" /><figcaption>Table 2</figcaption></figure><p>On visualization, dependent (Display1), as well as the independent (Display2) variables, look approximately normal with skewness lying within the range of -0.4 to -0.6. The boxplot shows a few outliers towards the lower tail but no skewness. Hence, no transformation is required for the data set.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/410/1*In0F_t-DXKzdnmLXQSX3Yw.png" /></figure><p>The Interquartile range and median of ambitious-sincerity; and attractiveness-fun are approx. same. Distribution of Intelligence shows, in general, it is rated higher than other attributes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/536/1*b1F870TlvfaucHGxLPzniQ.png" /></figure><p>A gender-wise F-test to compare the variances on each independent variable signifies unequal variances of Sincerity, being Fun and ambitious(<a href="#Table3Open">Table3</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/934/1*RuWNXER3Y5WoEuOpDlL--g.png" /><figcaption>Table 3</figcaption></figure><p>Interestingly, the two-sample t-test, at a 5% significance level, suggests that the mean rating of females for Attractiveness and being fun is higher than that of males, whereas, on average, males are rated higher for Intelligence and being ambitious(<a href="#Table4Open">Table4</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1018/1*GrMomVCE6kClj6x-Fcs6vg.png" /><figcaption>Table 4</figcaption></figure><p><strong>Variable Selection</strong></p><p>The initial model is fed with nine Independent variables, namely, Attractiveness, Intelligence, Ambitious, Sincerity, Fun, age, race, same race and an interaction term of race and the same race is also included, as people might prefer a partner from the same race and hence, it could be a significant variable for the model.</p><p>Stepwise model selection criteria have been used with both, AIC and BIC values to determine the most significant variables for the model. The minimum Cp statistic value for each of the AIC and BIC models created from Stepwise is selected. All the techniques, AIC, BIC, and Cp statistic suggest 4 explanatory variables(<a href="#Table5Open">Table5</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/934/1*ironpSazUTLgtPGXZP5bng.png" /><figcaption>Table 5</figcaption></figure><p>Hence, following Occum’s razor principle and the results from the model selection criteria, a model with four independent variables is chosen. The final variables selected are Fun, Attractiveness, Intelligence, and Sincerity. The R squared or the variation in overall likeness rating explained from the model is approx. 64%.</p><p><strong>Model Creation</strong></p><p>The correlation chart (<a href="#Display6Open">Display 6</a>) which shows the correlation of the overall ‘like’ rating with the independent variables also suggests the same four explanatory variables for the model. This is called the Reduced Model (<a href="#Display3Open">Reduced Model</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*BlN7phf-AGXt6u2VoSEOKw.png" /><figcaption>Display 6</figcaption></figure><p>The variable of interest ‘gender’, which indicates 1 for males and 0 for females, is then added to the Reduced model to create the Full model, resulting in 5 independent variables(<a href="#Display4Open">Full Model</a>). The significance of Intelligence is reduced after adding the gender variable.</p><p>Reduced Model : likeness = b0 + b1.fun + b2. attr + b3. sinc + b4. intel</p><p>Full Model : likeness = b0 + b1.fun + b2. attr + b3. sinc + b4. intel + b5. gender</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*QkyDjb62MJm4_xN4h-8NiQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/1*CGCYybaftXAgmaKQmJ1KLA.png" /></figure><p><strong>Model Comparison</strong></p><p>The lack of fit test, ANOVA, is conducted on the Full and Reduced model to find the significance of gender (<a href="#Display5Open">Display 5</a>).</p><p>The null hypotheses is that the coefficient on gender is insignificant, Ho : b5 =0</p><p>The alternate Hypotheses is that the coefficient on gender is significant, Ha : b5 !=0</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/739/1*SPwEQHFx8u0pXRe354G7AA.png" /><figcaption>ANOVA</figcaption></figure><p>At a 5% significance level, the F statistic of 6.83 and the two-tailed p value&lt;0.05, (being 0.0092), suggests that the gender variable is significant for the model.</p><p><strong>Conclusion</strong></p><p>The research provides enough evidence to reject the Null hypotheses that there is no difference in how males and females are rated by their partners, for the same personality standards. The estimated coefficient of slope on gender is 0.232 (on a scale of 0–10), which means, males are rates about 0.23 points higher as compared to females in the overall rating of likeness. Further, out of the five traits that were judged in this research, Fun, Attractiveness, Intelligence and Sincerity are the most significant in determining the overall likeness towards a person.</p><p><strong>Scope of Inference</strong></p><p>Since this is an observational study, that is conducted as an experiment with the people who volunteered, an inference to the true population can not be made. It consists of heterosexual males and females, which means the allocation is by nature and not random, hence, causal inference cannot be established with the results.</p><p><strong>References</strong></p><p>1. Fisman, Raymond; Iyengar, Sheena, 2006. ‘Gender Differences in Mate Selection: Evidence From a Speed Dating Experiment’. <em>The Quarterly Journal of Economics</em></p><p>2. Regan, Pamela C.,1998, ‘Minimum Mate Selection Standards as a Function of Perceived Mate Value, Relationship Context, and Gender’, <em>Journal of Psychology &amp; Human Sexuality</em>.</p><p>3. Speed Dating Experiment, ‘What attributes influence the selection of a romantic partner?’, <a href="https://www.kaggle.com/annavictoria/speed-dating-experiment">https://www.kaggle.com/annavictoria/speed-dating-experiment</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9b3b926e0a8b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/are-men-overrated-by-women-9b3b926e0a8b">Are men overrated by women?</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Logistic Regression in Python using Pandas and Seaborn(For Beginners in ML)]]></title>
            <link>https://medium.com/analytics-vidhya/logistic-regression-in-python-using-pandas-and-seaborn-for-beginners-in-ml-64eaf0f208d2?source=rss-7de3ddde7f7b------2</link>
            <guid isPermaLink="false">https://medium.com/p/64eaf0f208d2</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[logistic-regression]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Surmayi]]></dc:creator>
            <pubDate>Sat, 31 Oct 2020 13:03:28 GMT</pubDate>
            <atom:updated>2020-10-31T17:05:47.055Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Data Set and Problem Statement</strong></p><p>We will be working with an advertising data set, indicating whether or not a particular internet user clicked on an Advertisement. We will try to create a model that will predict whether or not they will click on an advertisement based on the features of that user. This data set contains the following features:</p><ul><li>‘Daily Time Spent on Site’: consumer time on site in minutes</li><li>‘Age’: cutomer age in years</li><li>‘Area Income’: Avg. Income of geographical area of consumer</li><li>‘Daily Internet Usage’: Avg. minutes a day consumer is on the internet</li><li>‘Ad Topic Line’: Headline of the advertisement</li><li>‘City’: City of consumer</li><li>‘Male’: Whether or not consumer was male</li><li>‘Country’: Country of consumer</li><li>‘Timestamp’: Time at which consumer clicked on Ad or closed window</li><li>‘Clicked on Ad’: 0 or 1 indicated clicking on Ad</li></ul><p>Lets go step by step in analysing, visualizing and modeling a Logistic Regression fit using Python</p><p>#First, let&#39;s import all the necessary libraries-</p><pre>import pandas as pd<br>import numpy as np<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>%matplotlib inline</pre><p>#Read the data in a data frame-</p><p>ad_data = pd.read_csv(‘advertising.csv’)</p><p>#Checkout the data using the head, describe, and info functions provided by pandas</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bdXGnJunHjsRVXq152KdNw.png" /><figcaption>ad_data.head()</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/644/1*XbOFUM1jD11Yo84VcAmz-g.png" /><figcaption>ad_data.info()</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E8ca7xi-WSm8XaTtF3vo7g.png" /><figcaption>ad_data.describe()</figcaption></figure><h3>Exploratory Data Analysis</h3><p>This is always helpful to understand the data, behavioral properties of various features, and dependencies if any.</p><p>We try to do as much visualization as possible. But here, lets do the common graphs, that would help us understand the key features.</p><p>Let&#39;s explore the data using Data Visualization libraries provided by Python, Seaborn</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/574/1*XgSNMKkz_Vb9raua8o9kaw.png" /></figure><p># Plot a histogram of age</p><p>plt.hist(ad_data[‘Age’],bins=40)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*Bh65Mxmj0j5Su8AhZ4GKTw.png" /></figure><p># Creating a jointplot showing Area Income versus Age.</p><p>sns.jointplot(‘Age’,’Area Income’ ,data=ad_data)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*gBNYq1L36cK6eO5NHg7P4Q.png" /></figure><p>#Creating a jointplot showing the kde distributions of Daily Time spent on site vs. Age</p><p>sns.jointplot(‘Age’,’Daily Time Spent on Site’ ,data=ad_data,kind=’kde’,color=’red’)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/1*FwI7A8v95k4rN8qIDSD66w.png" /></figure><p># Creating a jointplot of ‘Daily Time Spent on Site’ vs. ‘Daily Internet Usage’</p><p>sns.jointplot(‘Daily Time Spent on Site’,’Daily Internet Usage’ ,data=ad_data,color=’green’)</p><p># Finally, creating a pairplot with the hue defined by the ‘Clicked on Ad’ column feature to analyze the relationship between each and every variable</p><p>sns.pairplot(ad_data,hue=’Clicked on Ad’)</p><p># We can see that the data points of blue and orange are actually separated, which is a good indicator.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jLWLmKsgmEL_SsjAfwW7zA.png" /></figure><h3>Logistic Regression —</h3><h3>Split Data into Training and Test set</h3><p>from sklearn.model_selection import train_test_split</p><p>Variable X contains the explanatory columns, which we will use to train our model to fit whether the Ad is clicked or not</p><pre>X= ad_data.drop([‘Ad Topic Line’,’City’,’Timestamp’,’Clicked on Ad’,’Country’],axis=1)<br>y = ad_data[‘Clicked on Ad’]</pre><pre>X_train, X_test,y_train, y_test = train_test_split(X,y,test_size=0.33, random_state=42)</pre><h3>Training the model</h3><pre>from sklearn.linear_model import LogisticRegression<br>logreg = LogisticRegression()</pre><pre>logreg.fit(X_train,y_train)</pre><p>We get below, which shows the parameters which are set by default using the fit() method-</p><pre>LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,<br>          intercept_scaling=1, max_iter=100, multi_class=&#39;ovr&#39;, n_jobs=1,<br>          penalty=&#39;l2&#39;, random_state=None, solver=&#39;liblinear&#39;, tol=0.0001,<br>          verbose=0, warm_start=False)</pre><h3>Predictions and Evaluation of our model</h3><p>Lets now predict values from the test data -</p><p>prediction = logrig.predict(X_test)</p><p>#We create a classification report for the logistic regression model. This compares our actual and predicted values</p><p>from sklearn.metrics import classification_report</p><p>print(classification_report(y_test,prediction))</p><pre>precision    recall  f1-score   support<br><br>          0       0.87      0.96      0.91       162<br>          1       0.96      0.86      0.91       168<br><br>avg / total       0.91      0.91      0.91       330</pre><p>This shows our model has an accuracy of about 91%.</p><p>All Done!! We have just completed the logistic regression in python using sklearn.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=64eaf0f208d2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/logistic-regression-in-python-using-pandas-and-seaborn-for-beginners-in-ml-64eaf0f208d2">Logistic Regression in Python using Pandas and Seaborn(For Beginners in ML)</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Linear Regression Using Pandas & Numpy — For Beginners in Data Science]]></title>
            <link>https://medium.com/analytics-vidhya/linear-regression-using-pandas-numpy-for-beginners-in-data-science-fe57157ed93d?source=rss-7de3ddde7f7b------2</link>
            <guid isPermaLink="false">https://medium.com/p/fe57157ed93d</guid>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[numpy]]></category>
            <dc:creator><![CDATA[Surmayi]]></dc:creator>
            <pubDate>Sun, 25 Oct 2020 15:25:13 GMT</pubDate>
            <atom:updated>2024-02-26T10:03:59.825Z</atom:updated>
            <content:encoded><![CDATA[<h3>Linear Regression Using Pandas &amp; Numpy — For Beginners in Data Science</h3><p><strong>Problem Statement </strong>An eCommerce company based in New York City that sells clothing online also have in-store style and clothing advice sessions. Customers come into the store, have sessions/meetings with a personal stylist, then they can go home and order either on a mobile app or website for the clothes they want.</p><p>The company is trying to decide whether to focus their efforts on their mobile app experience or their website. Let’s figure it out.</p><p>#Import the required packages</p><p><em>import pandas as pd<br>import numpy as np<br>import matplotlib.pyplot as py<br>import seaborn as sns<br>%matplotlib inline</em></p><p># Read the file</p><p><em>customers = pd.read_csv(“Ecommerce Customers”)</em></p><p># Let’s checkout the data -</p><p><em>customers.head()</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fBbpUZA8ASW2ESL6joUrWA.png" /></figure><p><em>customers.describe()</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OyceNsrHUyuj67qqBkUFuw.png" /></figure><h3>Exploratory Data Analysis</h3><p># First step with data is to <a href="https://www.analyticsvidhya.com/blog/2022/07/step-by-step-exploratory-data-analysis-eda-using-python/?utm_source=Backlink&amp;utm_medium=SEO"><strong>analyze the data</strong></a>, explore what relationships exist and how those are correlated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*vPvj8zTTHuPjcdvTKBxahg.png" /></figure><p># Created a jointplot (using seaborn) to compare the Time on Website and Yearly Amount Spent columns. This is to check if the correlation makes sense?</p><p><em>sns.jointplot(x=’Time on Website’,y=’Yearly Amount Spent’, data=customers)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/1*MvFnzBAEWhSKt2Z5cABlfw.png" /></figure><p># The same for App data</p><p><em>sns.jointplot(x=’Time on App’,y=’Yearly Amount Spent’, data=customers)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/1*Vju_oScYEepfT8qBMoGpeg.png" /></figure><p># Created a jointplot of type hex for Time spent on App vs the Length of Membership</p><p><em>sns.jointplot(x=’Time on App’,y=’Length of Membership’, data=customers,kind=’hex’)</em></p><p># We can create a pairplot to explore the types of relationships across the entire data set. <strong>Notice</strong> the positive linear correlation between Yearly amount spent and length of membership.</p><p><em>sns.pairplot(data=customers)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0u_usYIL0GNXTh1EZCP_0A.png" /></figure><p># So we dig deep into this relationship by creating a <a href="https://www.analyticsvidhya.com/blog/2021/06/linear-regression-in-machine-learning/?utm_source=Backlink&amp;utm_medium=SEO"><strong>linear plot</strong></a> (using seaborn’s lmplot) of Yearly Amount Spent vs. Length of Membership</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/1*010i-0wrRwRXMpM3dUn8Ag.png" /></figure><p><em>sns.lmplot(x=’Length of Membership’,y=’Yearly Amount Spent’,data=customers)</em></p><h3>Training and Testing Data</h3><p>Now that we’ve explored the data a bit, let’s go ahead and split the data into training and testing sets. Set a <strong>variable X</strong> equal to the numerical features of the customers and a <strong>variable Y </strong>equal to the “Yearly Amount Spent” column.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*m6JCVEQ7535vyspaYOwucg.png" /></figure><p><em>X =customers[[‘Avg. Session Length’,’Time on App’,’Time on Website’,’Length of Membership’]]<br>X.head()</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/534/1*619oMKk4OBLzcmtzsDwvLQ.png" /></figure><p><em>Y=customers[‘Yearly Amount Spent’]<br>Y.head()</em></p><p><strong>Use model_selection.train_test_split from sklearn to split the data into training and testing sets. Set test_size=0.3 and random_state=101</strong></p><p><em>from sklearn.model_selection import train_test_split</em></p><p><em>X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size=0.3,random_state=101)</em></p><h3>Training the Model</h3><p>Now its time to train our model on our training data!</p><p><em>from sklearn.linear_model import LinearRegression</em></p><p><em>lm = LinearRegression() </em># Creating an Instance of LinearRegression model</p><p><em>lm.fit(X_train,Y_train) # </em>Train/fit on the trainingdata, this will give-</p><pre>LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)</pre><p># The coefficients/slopes of model -</p><p><em>print(lm.coef_)</em></p><pre>[25.98154972 38.59015875  0.19040528 61.27909654]</pre><h3>Predicting Test Data</h3><p>Now that we have fit our model, let’s evaluate its performance by predicting off the test values!</p><p><em>prediction = lm.predict(X_test)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/1*mHCqvfUoUEpT6sSbbGAirg.png" /></figure><p>#Let’s create a scatterplot of the real test values versus the predicted values to check the performance of our model</p><p>py.scatter(Y_test,prediction)</p><h3>Evaluating the Model</h3><p>Let’s evaluate our model performance by calculating the residual sum of squares and the explained variance score (R²)</p><p><em>from sklearn import metrics<br>print(‘MAE= ‘, metrics.mean_absolute_error(Y_test,prediction) )<br>print(‘MSE= ‘, metrics.mean_squared_error(Y_test,prediction))<br>print(‘RMSE:’, np.sqrt(metrics.mean_squared_error(Y_test, prediction)))</em></p><pre>MAE=  7.228148653430853<br>MSE=  79.81305165097487<br>RMSE: 8.933815066978656</pre><h3>Residuals</h3><p>Let’s quickly explore the residuals to make sure everything was okay with our data</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/562/1*fc1BTsaniI1E-8yVY1uVSA.png" /></figure><p># Plotting a histogram of the residuals and make sure it looks normally distributed using plt.hist().</p><p><em>py.hist(prediction-Y_test,bins=50)</em></p><h3>Conclusion</h3><p>We still want to figure out the answer to the original question, do we focus our efforts on the mobile App or website development? Or maybe that doesn’t even really matter, and Membership Time is what is really important. Let’s interpret the coefficients to get an idea</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/340/1*N7jp84gPjnSn_Z-VLXS7-w.png" /></figure><p><em>co=pd.DataFrame(lm.coef_,X.columns)<br>co.columns = [‘Coefficient’]<br>co</em></p><p>Interpreting the coefficients:</p><ul><li>Holding all other features fixed, a 1 unit increase in <strong>Avg. Session Length</strong> is associated with an <strong>increase of 25.98 total dollars spent</strong>.</li><li>Holding all other features fixed, a 1 unit increase in <strong>Time on App</strong> is associated with an <strong>increase of 38.59 total dollars spent</strong>.</li><li>Holding all other features fixed, a 1 unit increase in <strong>Time on Website</strong> is associated with an <strong>increase of 0.19 total dollars spent</strong>.</li><li>Holding all other features fixed, a 1 unit increase in <strong>Length of Membership</strong> is associated with an <strong>increase of 61.27 total dollars spent</strong>.</li></ul><p><strong>Final Conclusion —</strong></p><p>There are two ways to think about this: 1. Develop the Website to catch up to the performance of the mobile app, or 2. develop the App more since that is what is working better.</p><p>Being a data person, we can present both the options to the company with the numbers and help them make a decision.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fe57157ed93d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/linear-regression-using-pandas-numpy-for-beginners-in-data-science-fe57157ed93d">Linear Regression Using Pandas &amp; Numpy — For Beginners in Data Science</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>