<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pranjal Pandey on Medium]]></title>
        <description><![CDATA[Stories by Pranjal Pandey on Medium]]></description>
        <link>https://medium.com/@pranjalpandeysta1999?source=rss-4fa9dfe37370------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*WApRyKTX_OksnGmt</url>
            <title>Stories by Pranjal Pandey on Medium</title>
            <link>https://medium.com/@pranjalpandeysta1999?source=rss-4fa9dfe37370------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 12:02:09 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@pranjalpandeysta1999/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[R-commands for fitting Basic Regression Models]]></title>
            <link>https://medium.com/analytics-vidhya/r-commands-for-fitting-basic-regressions-4a1bc1962856?source=rss-4fa9dfe37370------2</link>
            <guid isPermaLink="false">https://medium.com/p/4a1bc1962856</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[r]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[regression]]></category>
            <dc:creator><![CDATA[Pranjal Pandey]]></dc:creator>
            <pubDate>Wed, 12 Aug 2020 08:32:24 GMT</pubDate>
            <atom:updated>2020-08-31T05:38:14.173Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*J5gngQnww0aEAH_sMuPRiA.png" /><figcaption><strong>Image from </strong><a href="https://twitter.com/rcodes_official"><strong>R.codes (@Rcodes_Official) | Twitter</strong></a></figcaption></figure><h4>Introduction</h4><p>In this Article, I am going to provide you all <strong>R-commands</strong> that helps us in <strong>fitting of Basic Regression Models</strong>. Basically, This article consists of following sections -</p><ol><li>R-Command for fitting <strong>Simple Linear Regression</strong></li><li>R-Command for fitting <strong>Multiple Linear Regression</strong></li><li>R-Command for fitting <strong>Polynomial Regression</strong></li><li>R-Command for fitting <strong>Orthogonal Polynomial Regression</strong></li><li>R-Command for fitting <strong>Ridge Regression with best value of lambda</strong></li><li>R-Command for fitting <strong>Lasso Regression with best value of lambda</strong></li><li>R-Command for fitting <strong>Elastic-Net Regression with best value of hyperparameters</strong></li></ol><p><strong>1. R-Command for fitting Simple Linear Regression</strong></p><p>Want to fit following model -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/1*CSo0DFHOVgwD06rTcWVhLg.png" /></figure><p>use <strong>lm()</strong> function already included in the <strong>base package of R</strong> as follows -</p><pre>Simple_Linear_Model &lt;- lm(formula = y ~ X  , data =              train_data_with_target_and_predictor)</pre><p>If you want to remove intercept term, just add <strong>-1</strong> in <strong>formula</strong> argument as follows -</p><pre>Simple_Linear_Model &lt;- lm(formula = y ~ X <strong>-1</strong>  ,  data = train_data_with_target_and_predictor)</pre><p><strong>2. R-Command for fitting Multiple Linear Regression</strong></p><p>Want to fit following model -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*dA2yJXxucxes1y42IVe_ng.png" /></figure><p>Again, use <strong>lm()</strong> function already included in the <strong>base package of R</strong> as follows -</p><pre>Multiple_Linear_Model &lt;- lm(formula = y ~ X_1 + X_2 + ... + X_k  ,  data = train_data_with_target_and_predictor)</pre><p>An alternative way is, Use a <strong>single dot</strong> <strong>(.)</strong> instead of writing all predictors separately as follows -</p><pre>Multiple_Linear_Model &lt;- lm(formula = Y ~ <strong>.</strong>   ,  data = train_data_with_target_and_predictor)</pre><p>If you want to remove a particular predictor, say X_2, then just add <strong>-X_2</strong> in <strong>formula</strong> argument as follows -</p><pre>Multiple_Linear_Model &lt;- lm(formula = Y ~ . - X_2 , data = train_data_with_target_and_predictor)</pre><p>Want to remove intercept term, just add <strong>-1</strong> in <strong>formula</strong> argument as follows -</p><pre>Multiple_Linear_Model &lt;- lm(formula = Y ~ . - 1 , data = train_data_with_target_and_predictor)</pre><p><strong>3. R-Command for fitting Polynomial Regression</strong></p><p>Want to fit following model -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*pKTqLvhEYxE4pTi9aqmU_g.png" /></figure><p>Again, use <strong>lm()</strong> function already included in the <strong>base package of R</strong>. Also use <strong>I()</strong> in <strong>formula</strong> argument to include higher order predictor terms as follows -</p><pre>Polynomial_Model &lt;- lm(formula = y ~ X + I(X^2) , data = train_data_with_target_and_predictor)</pre><p>Want to fit following model -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*IMFfeDz3-Ms_gwBO9MPMVA.png" /></figure><p>R-code for this purpose is as follows -</p><pre>Polynomial_Model &lt;- lm(formula = y ~ X_1 + X_2 + I((X_1)^2) + I((X_2)^2) + X_1:X_2 , data = train_data_with_target_and_predictor)</pre><p><strong>4. R-Command for fitting Orthogonal Polynomial Regression</strong></p><p>Want to fit following model -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/735/1*lsBxOlrktzXRMmFAIhh8rg.png" /></figure><p>Again, we will use <strong>lm()</strong> function. Also, use <strong>poly()</strong> function in <strong>formula</strong> argument as follows -</p><pre>Ortho_Polynomial_Model &lt;- lm(formula = Y ~ poly(X_1 , upto_which_order) + poly(X_2 , upto_which_order) +.....+ poly(X_k , upto_which_order) , data = train_data_with_target_and_predictor)</pre><p><strong>In the above code, I have fitted orthogonal polynomial for each predictor. But if there is a need for including any predictor in linear form then include that predictor accordingly as i have done in fitting of Multiple Regression or in Simple Linear Regression</strong>. See the following code -</p><pre>Ortho_Polynomial_Model &lt;- lm(formula = Y ~ X_1 + poly(X_2 , upto_which_order) +.....+ poly(X_k , upto_which_order) , <br>data =train_data_with_target_and_predictor)</pre><p>In the above code, <strong><em>I have included predictor X_1 in linear form, while all other predictors have been included in the form of Orthogonal Polynomial</em></strong><em>.</em></p><p><strong>5. R-Command for fitting Ridge Regression with best value of lambda</strong></p><p>As we know that, To fit Ridge Regression we have to find out the best value of hyperparameter lambda. For this purpose, the following R-code is used -</p><pre>cv_for_best_lambda &lt;-  cv.glmnet(x_train, y_train, family = &quot;gaussian&quot;, alpha = 0, type.measure = &quot;mse&quot;)</pre><p>In the above code, Notice that <strong>train data set is provided in a slightly different manner</strong>. We will have to separate predictors and target and provide them in <strong>matrix</strong> form as I have provided x_train and y_train</p><p>Now, To fit Ridge Regression, use the following R-code -</p><pre>ridge = glmnet(x_train, y_train, family = &quot;gaussian&quot;,alpha = 0, lambda = cv_for_best_lambda$lambda.min)</pre><p>In the above code, you must notice the value of argument lambda. <strong>cv_for_best_lambda</strong> is R-object created in first step of fitting Ridge Reression and you will have to write <strong>$lambda.min</strong> after <strong>cv_for_best_lambda</strong></p><p><strong>6. R-Command for fitting Lasso Regression with best value of lambda</strong></p><p>To fit Lasso Regression we have to find out the best value of hyperparameter lambda. For this purpose, the following R-code is used -</p><pre>cv_for_best_lambda &lt;-  cv.glmnet(x_train, y_train, family = &quot;gaussian&quot;, alpha = 1, type.measure = &quot;mse&quot;)</pre><p>Here again, Notice that <strong>train data set is provided in a slightly different manner</strong>. We will have to separate predictors and target and provide them in <strong>matrix</strong> form as I have provided x_train and y_train</p><p>Now, To fit Lasso Regression, use the following R-code -</p><pre>lasso = glmnet(x_train, y_train, family = &quot;gaussian&quot;, alpha = 1, lambda = cv_for_best_lambda$lambda.min)</pre><p>Again, In the above code, you must notice the value of argument lambda. <strong>cv_for_best_lambda</strong> is R-object created in first step of fitting Lasso Regression and you will have to write <strong>$lambda.min</strong> after <strong>cv_for_best_lambda</strong></p><p><strong>7. R-Command for fitting Elastic-Net Regression with best value of hyper parameters</strong></p><p>To fit Elastic-Net Regression we have to find out the best value of hyper parameters alpha and lambda. For this purpose, the following R-code is used -</p><pre><strong># Define Training Control</strong><br>set.seed(123)<br>train.control &lt;- trainControl(method = &quot;repeatedcv&quot;, number = 10, repeats = 3 , search = &quot;random&quot;)<br>                              <br><strong># Train the model</strong><br>cv_for_best_value &lt;- train(target_name ~ ., data = train_data, method=&quot;glmnet&quot;, trControl = train.control)</pre><p>In the above code, Notice that <strong>train_data</strong> consists of both <strong>target</strong> and <strong>predictors</strong> in a <strong>data frame</strong>.</p><p>Now, To fit Elastic-Net Regression, use the following R-code -</p><pre>enet &lt;- glmnet(x_train, y_train, alpha = cv_for_best_value$bestTune[1,1] , lambda = cv_for_best_value$bestTune[1,2] , family = &quot;gaussian&quot;</pre><p>In the above code, Notice the value of argument alpha and lambda. For more about Ridge, Lasso and Elastic-Net Regression, Read my article <a href="https://medium.com/analytics-vidhya/multicollinearity-ridge-lasso-elastic-net-regression-using-r-6582cbabf7f3?source=friends_link&amp;sk=9aefa18ff84a10e36d0ab2695172f638"><strong>Multicolinearity/Ridge/Lasso/Elasticnet Regression</strong></a></p><blockquote>Till now, That’s all in this notebook. But this notebook will be updated regularly. I want to include R-codes of different Regression techniques in this notebook. Keep in touch with this notebook for more R-codes.</blockquote><h4>Thanks for Reading my Article !</h4><p>My kaggle Notebook : <a href="https://www.kaggle.com/pranjalpandey12/r-commands-for-fitting-basic-regressions?scriptVersionId=40603664"><strong>Click</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4a1bc1962856" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/r-commands-for-fitting-basic-regressions-4a1bc1962856">R-commands for fitting Basic Regression Models</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multicollinearity / Ridge / Lasso / Elastic-Net Regression using R]]></title>
            <link>https://medium.com/analytics-vidhya/multicollinearity-ridge-lasso-elastic-net-regression-using-r-6582cbabf7f3?source=rss-4fa9dfe37370------2</link>
            <guid isPermaLink="false">https://medium.com/p/6582cbabf7f3</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[regression]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Pranjal Pandey]]></dc:creator>
            <pubDate>Thu, 30 Jul 2020 14:09:09 GMT</pubDate>
            <atom:updated>2021-02-21T15:26:38.317Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/794/1*MgB3eqwAlLvJKQTg0KcHoA.png" /></figure><h3>Introduction</h3><p>This Article is in the continuation of my <a href="https://medium.com/@pranjalpandeysta1999/how-to-proceed-from-simple-to-multiple-and-polynomial-regression-in-r-84b77f5673c5?source=friends_link&amp;sk=b15b49da90c1bece7fe935d6dc443135"><strong>Previous Article</strong></a> in which I have shown you How Multiple Linear Regression is prepared and using the information obtained from its diagnostic plot, how we proceed towards Orthogonal Polynomial Regression and obtain a better model for the given data set (I have used Advertising Data Set).</p><p>In the previous article, I have created Orthogonal Polynomial model to avoid the problem of multicollinearity. But now, In this article <strong>I will first create problem of multicollinearity by introducing polynomial features of predictors TV and Radio and then show you how to tackle this multicollinearity problem using Ridge, Lasso and Elastic-Net Regression techniques</strong>.</p><p>This Article consists of the following sections -</p><ol><li>Loading Required Libraries</li><li>Loading Outlier Free Data set</li><li>Recap (of Previous Article)</li><li>Creating Multicollinearity Problem</li><li>Fitting Polynomial Regression (Note : Not Orthogonal Polynomial)</li><li>Checking Assumptions</li><li>Making Predictions Using Polynomial Regression Model</li><li>Average Performance of Polynomial Regression Model</li><li>Comparison Between Polynomial and Orthogonal Polynomial Model</li><li>Data Preparation for further analysis</li><li>Tackling Multicollinearity by Ridge/Lasso/Elastic-Net Regression</li><li>Comparison of Different Models (Polynomial, Orthogonal Polynomial, Ridge, Lasso, Elastic-Net)</li><li>Obtaining Best Model</li><li>Conclusion</li></ol><p>I am going to use kaggle online platform for analysis work. You may use any software like R-studio or R-cran version.</p><h3>1. Loading Required Libraries</h3><p>It is not necessary to load all libraries in the beginning but I am doing it for simplicity. I am loading one more library <strong>glmnet</strong> for Ridge/Lasso/Elastic-Net Regression.</p><pre><strong># Loading Libraries</strong><br>library(tidyverse)<br>library(caret)<br>library(car)<br>library(lmtest)<br>library(olsrr)<br>library(glmnet)       <strong># For Ridge/Lasso/Elastic-Net Regression</strong></pre><h3>2. Loading Outlier Free Data set</h3><p>Link to download Outlier free data set already stored in R-objects-</p><ol><li><a href="https://www.kaggle.com/pranjalpandey12/outlier-free-advertising-data-set/download"><strong>data</strong></a></li><li><a href="https://www.kaggle.com/pranjalpandey12/traindata1-and-testdata-for-further-analysis/download"><strong>train.data1 and test.data</strong></a></li></ol><p>in my previous notebook.</p><pre><strong># Loading Outlier free data set</strong><br>data = read.csv(&quot;../input/outlier-free-advertising-data-set/outlier free advertising data.csv&quot; , header = T)<br><br><strong># Loading outlier free train and test data already splitted in previous notebook</strong><br>train.data1 &lt;- read.csv(&quot;../input/traindata1-and-testdata-for-further-analysis/train.data1.csv&quot;, header = T)<br>test.data &lt;- read.csv(&quot;../input/traindata1-and-testdata-for-further-analysis/test.data.csv&quot;, header = T)</pre><p><strong>Inspection of Data Sets -</strong></p><p>Just take a look of different data sets as follows -</p><pre><strong># Inspection of data sets</strong><br>head(data)<br>head(train.data1)<br>head(test.data)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*Zw8ciTNpbAbcqaZsoxyJFg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/1*p2LlwTvFb8o3u6yTEKjoBQ.png" /><figcaption><strong>Output — 1</strong></figcaption></figure><pre><strong># Structure of different data sets</strong><br>str(data)<br>str(train.data1)<br>str(test.data)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/1*Voun1ZB40QY0JqRrl8Tgrw.png" /><figcaption><strong>Output — 2</strong></figcaption></figure><p><strong>Note that Newspaper Variable is also present here. We will remove it in further steps as this feature is statistically non-significant already discussed in previous notebook</strong>.</p><h3>3. Recap (of previous Article)</h3><p>In Previous Article, The better model we had obtained is <strong>Orthogonal Polynomial Model</strong> which captures <strong>on an average 93.69%</strong> variability of target <strong>Sales</strong>. Train, Test and Cross-Validation Results for Orthogonal Polynomial Model are shown in the following table -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/1*VGvwlPv6CJrm1X3azB8J2A.png" /></figure><p>Now, In this article, I am going to show you the consequences of <strong>Polynomial Regression Model</strong> (Note : Not-Orthogonal) and methods to tackle this problem.</p><h3>4. Creating Multicollinearity Problem</h3><p>Let us now create polynomial features of Predictors TV and Radio in the data sets. In our previous notebook, We had seen that up to second order of predictor TV and third order of predictor Radio is statistically significant. Hence, Create the following polynomial features -</p><ol><li><strong>Second order of predictor TV</strong></li><li><strong>Second and Third order of predictor Radio</strong></li></ol><p>Also, Create a column of interaction effect of TV and Radio in data sets for simplicity. Otherwise, we will have to mention this interaction effect separately in the function <strong>lm()</strong>.</p><pre><em># Creating Polynomial Features in R-object data</em><br>data[&quot;TV2&quot;]&lt;- (data$TV)^2<br>data[&quot;Radio2&quot;]&lt;- (data$Radio)^2<br>data[&quot;Radio3&quot;]&lt;- (data$Radio)^3<br>data[&quot;TVRadio&quot;]&lt;- (data$TV)*(data$Radio)<br><em># Removing Newspaper Column as it is Statistically non-significant already discussed in previous notebook</em><br>data &lt;- data[-3]<br><br><br><em># Creating Polynomial Features in R-object train.data1</em><br>train.data1[&quot;TV2&quot;]&lt;- (train.data1$TV)^2<br>train.data1[&quot;Radio2&quot;]&lt;- (train.data1$Radio)^2<br>train.data1[&quot;Radio3&quot;]&lt;- (train.data1$Radio)^3<br>train.data1[&quot;TVRadio&quot;]&lt;- (train.data1$TV)*(train.data1$Radio)<br><em># Removing Newspaper Column </em><br>train.data1 &lt;- train.data1[-3]<br><br><br><em># Creating Polynomial Features in R-object test.data</em><br>test.data[&quot;TV2&quot;]&lt;- (test.data$TV)^2<br>test.data[&quot;Radio2&quot;]&lt;- (test.data$Radio)^2<br>test.data[&quot;Radio3&quot;]&lt;- (test.data$Radio)^3<br>test.data[&quot;TVRadio&quot;]&lt;- (test.data$TV)*(test.data$Radio)<br><em># Removing Newspaper Column </em><br>test.data &lt;- test.data[-3]</pre><p><strong>Again Inspection of Data Sets -</strong></p><p>Again take a look of different data sets as follows -</p><pre><strong># Inspection of data sets</strong><br>head(data)<br>head(train.data1)<br>head(test.data)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/808/1*n1S9VJhCXZGKTF6w7ke1Ug.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/806/1*dqTKI41WTQM9bYh8w_73UA.png" /><figcaption><strong>Output — 3</strong></figcaption></figure><p><strong>All polynomial features and interaction of TV and Radio has been successfully added in different data sets.</strong></p><h3>5. Fitting Polynomial Regression (Note : Not Orthogonal Polynomial)</h3><p>Since, we have already stored polynomial features and interaction term of TV and Radio in the data sets. Hence, we will just use the code of fitting multiple linear Regression to fit polynomial model using train data set stored in R-object <strong>train.data1</strong> as follows -</p><pre><strong># Fitting Polynomial model (Note : It is not Orthogonal Polynomial)</strong><br>polynomial_model &lt;- lm(Sales ~ ., data = train.data1)<br><br><strong># Take a look on summary of the model</strong><br>summary(polynomial_model)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/1*GORs7O_EAC5tOULMNKJh4A.png" /><figcaption><strong>Output — 4</strong></figcaption></figure><p><strong>The above output shows that -</strong></p><ol><li>Created polynomial model is statistically significant since p-value &lt;&lt;&lt; 0.05 (see in the last line of output)</li><li>From the coefficients section, it is clear that all coefficients are statistically significant since p-value &lt;&lt;&lt; 0.05</li><li>This polynomial model explains 93.21% variability of target (Sales).</li><li>Residual standard error for the model is 1.347</li></ol><h3>6. Checking Assumptions</h3><p>Let’s check whether all assumptions of linear regression are satisfied.</p><p><strong>Linearity Assumption -</strong></p><pre><strong># Residual Plot</strong><br>plot(polynomial_model,1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*phE5uv5uRBm4NRh3qQZLjA.png" /><figcaption><strong>Output — 5</strong></figcaption></figure><p><strong>Normality Assumption -</strong></p><pre><strong># Normality Test of Errors</strong><br>shapiro.test(polynomial_model$residuals)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/1*USIvfJ61F5238dfUJw2gaQ.png" /><figcaption><strong>Output — 6</strong></figcaption></figure><p><strong>Homoscedasticity Assumption -</strong></p><pre><strong># Hetroscedasticity Test</strong><br>ols_test_score(polynomial_model)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/811/1*mws16J4X41V1zyWWFiI9_A.png" /><figcaption><strong>Output — 7</strong></figcaption></figure><p><strong>Auto-correlation Assumption -</strong></p><pre><strong># Auto-correlation Test</strong><br>durbinWatsonTest(polynomial_model)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*G9ZM0GFZbFNdBdjImJG1zw.png" /><figcaption><strong>Output — 8</strong></figcaption></figure><p><strong>Multicollinearity Assumption -</strong></p><pre><strong># Detecting Multicollinearity </strong><br>vif(polynomial_model)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*PH4OlQcSprJk5njv_tAAwQ.png" /><figcaption><strong>Output — 9</strong></figcaption></figure><p><strong>From the above outputs, It is clear that all assumptions except multicollinearity have been satisfied.</strong> So we have to deal with Multicollinearity problem. We will deal with it after few steps.</p><blockquote>If you don’t know how to interpret the above results just read my <a href="https://medium.com/@pranjalpandeysta1999/how-to-proceed-from-simple-to-multiple-and-polynomial-regression-in-r-84b77f5673c5?source=friends_link&amp;sk=b15b49da90c1bece7fe935d6dc443135"><strong>previous article</strong></a>. I have discussed there all in detail.</blockquote><h3>7. Making Predictions Using Polynomial Regression Model</h3><p>Now, just make predictions with this model on test data set and see how well it is working on unseen data set.</p><pre><strong># Making predictions using Polynomial model</strong><br>prediction = polynomial_model %&gt;% predict(test.data)<br><br><strong># Checking performance by calculating R2, RMSE and MAE</strong><br>data.frame( R2 = R2(prediction, test.data$Sales),<br>            RMSE = RMSE(prediction, test.data$Sales),          <br>            MAE = MAE(prediction, test.data$Sales))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*2oxPHklWxNzK7ibELoiAcw.png" /><figcaption><strong>Output — 10</strong></figcaption></figure><p><strong>We obtain : R² = 0.9526385 , which indicates a better fit.</strong></p><h3>8. Average Performance of Polynomial Regression Model</h3><p>Since, the above result is based on only one test data set. Hence, we can not sure that the model will perform better on all unseen data. To be more confident in this respect, we will use the method of repeated 10-fold cross validation to test the performance of model on different test data set.</p><p>This will be done as follows -</p><pre><strong># Repeated 10-fold Cross validation using polynomial model</strong><br>set.seed(123)<br>train.control &lt;- trainControl(method = &quot;repeatedcv&quot;, number = 10, repeats = 3)<br>                              <br><strong># Train the model</strong><br>poly_model_cv &lt;- train(Sales ~ ., data = data, method = &quot;lm&quot;, trControl = train.control)<br>                  <br><strong># Summarize the results</strong><br>print(poly_model_cv)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*IsWRH4eJlLJiAgFm3SXGPg.png" /><figcaption><strong>Output — 11</strong></figcaption></figure><p><strong>Well, On an average, This Polynomial Regression Model captures 93.69% variability available in the target (Sales).</strong></p><h3>9. Comparison Between Polynomial and Orthogonal Polynomial Model</h3><p>Make comparison between both models on the basis of R² and RMSE metrics as follows -</p><p>The following table gives Train, Test, Cross-Validation Results for both Polynomial and Orthogonal Polynomial Regression Model -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/1*__2raM22qvb2u-noMO28Fw.png" /></figure><p>The above table shows that <strong>performance of both models are same</strong>.</p><p>But see the following table -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*AKjSt6jvux4XaeP7y_2gBw.png" /></figure><p>The above table shows that <strong>Multicollinearity Assumption is violated by Polynomial Regression Model</strong>.</p><p>So, there is a need for dealing with multicollinearity.</p><h3>10. Data Preparation for further analysis</h3><p>Now, we are going to deal with problem of multicollinearity but before that we have to prepare our data set for further analysis. We will use <strong>glmnet</strong> library that <strong>requires predictors and target separately in matrix form</strong>. So, just separate predictors and target and convert them into matrix form as follows -</p><pre><strong># Storing Predictors and Target in Matrix form (train data set)</strong><br>x_train &lt;- train.data1 %&gt;% select(-Sales) %&gt;% as.matrix()<br>y_train &lt;- train.data1  %&gt;% select(Sales) %&gt;%  as.matrix()<br><br><strong># Storing Predictors and Target in Matrix form (test data set)</strong><br>x_test &lt;- test.data %&gt;% select(-Sales) %&gt;% as.matrix()<br>y_test &lt;- test.data %&gt;% select(Sales) %&gt;% as.matrix()</pre><h3>11. Tackling Multicollinearity by Ridge / Lasso / Elastic-Net Regression</h3><p>Basically, there are three methods to deal with Multicollinearity Problem -</p><ol><li><strong>Ridge Regression</strong></li><li><strong>Lasso Regression</strong></li><li><strong>Elastic-Net Regression</strong></li></ol><p>Ridge and Lasso Regressions are particular cases of Elastic-Net Regression. There are two hyper parameters in Elastic-Net Regression named -</p><ol><li><strong>alpha</strong> — It is known as <strong>Mixing Parameter</strong>.</li><li><strong>lambda</strong> — It is known as <strong>Regularization Penalty</strong>.</li></ol><p>It is our duty to find out best value for these hyper parameters. For this purpose we use Cross-Validation method. We will see it in later stage.</p><p>Range of these hyper parameters -</p><ol><li>alpha lies in the closed interval 0 to 1</li><li>lambda lies between 0 to infinity.</li></ol><p>Three more points to be remembered -</p><ol><li>For <strong>Ridge</strong> Regression : <strong>alpha = 0</strong> , Lambda may take any value in its range</li><li>For <strong>Lasso</strong> Regression : <strong>alpha = 1</strong> , Lambda may take any value in its range</li><li>For <strong>Elastic-Net</strong> Regression : <strong>alpha lies between 0 and 1</strong> , Lambda may take any value in its range</li></ol><p>Now, I am going to find out which method out of these three methods will perform best for given Advertising Data Set.</p><h3>Fitting of Ridge Regression</h3><p>In case of Ridge Regression, Value of a hyper parameter, i.e., alpha is fixed. Our aim is to find out best value of lambda. For this purpose, we use cross validation method and evaluation criteria for cross validation will be Mean Square Error.</p><p>R-code for the above purpose is as follows -</p><pre><strong># Cross Validation for obtaining best value of lambda for ridge regression (alpha = 0)</strong><br>cv_for_best_lambda &lt;-  cv.glmnet(x_train, y_train, family = &quot;gaussian&quot;, alpha = 0, type.measure = &quot;mse&quot;)</pre><p>For alpha = 0, The above code performs 10-fold (by default) cross validation using train data set and gives best value for lambda.</p><p>Check the value of lambda as follows -</p><pre><strong># Checking best value of lambda</strong><br>print(cv_for_best_lambda)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/811/1*FuQSMl7o4BX1BOi1-Qa61g.png" /><figcaption><strong>Output — 12</strong></figcaption></figure><p><strong>Notice in the above output -</strong></p><ol><li>There are two values of lambda (<strong>0.4638</strong> and <strong>0.6729</strong>) in last two lines of output. I will take the value <strong>0.4638</strong> since <strong>MSE is minimum (2.573)</strong> for that value. <strong>Why these two values ?</strong> Both values are used in different situations. This will be more clear to you when we fit Lasso Regression. But the rule is <strong>select that value of lambda for which MSE is minimum</strong>.</li><li>Last column of above output shows that for both values of lambda there are 6 non-zero coefficients in the model and also we have 6 predictors in our data set. This means that all predictors are important for Ridge Regression.</li></ol><blockquote><em>Actually, These methods (Ridge/Lasso/Elastic-Net) are also used in selection of best predictors. In such situations, the last column of above output plays very important role. At that time, I am not going towards feature selection. Just focus on multicollinearity problem.</em></blockquote><p>Now, Store the best value of lambda, i.e., <strong>0.4638</strong> in R-object <strong>best_lambda</strong> as follows -</p><pre><strong># Storing best value of lambda in a R-object best_lambda</strong><br>best_lambda = cv_for_best_lambda$lambda.min</pre><p>Now, Fit Ridge Regression Model and check its performance on train data set as follows -</p><pre><strong># Fitting Ridge Regression Model</strong><br>ridge = glmnet(x_train, y_train, family = &quot;gaussian&quot;,alpha = 0, lambda = best_lambda)<br><br><strong># Checking its performance on train data set</strong><br>predict_train &lt;- ridge %&gt;% predict(x_train)<br>data.frame( R2 = R2(predict_train, y_train),<br>            RMSE = RMSE(predict_train, y_train),<br>            MAE = MAE(predict_train, y_train))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/1*ckpjqasQdRhEWh04EwtRPQ.png" /><figcaption><strong>Output — 13</strong></figcaption></figure><p><strong>From the above output, We get that -</strong></p><ol><li>Adjusted R² = 90.9% approx.</li><li>RMSE = 1.5585</li></ol><p>Now, Check its performance on unseen data as follows -</p><pre><strong># Checking performance of Ridge regression on test data set</strong><br>predict_test &lt;- ridge %&gt;% predict(x_test)<br>data.frame( R2 = R2(predict_test, y_test),<br>            RMSE = RMSE(predict_test, y_test),<br>            MAE = MAE(predict_test, y_test))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/804/1*KGT_XatcFHm1Y1XQkdcLHw.png" /><figcaption><strong>Output — 14</strong></figcaption></figure><p>Well, We obtain : <strong>R² = 0.9526385</strong> and <strong>RMSE = 1.4779</strong></p><p>To be more confident, about its performance on unseen data, calculate average performance of Ridge Regression using Repeated K-fold cross validation as follows -</p><pre><strong># Repeated 10-fold cross validation for average performance of Ridge Regression</strong><br>set.seed(123)<br>train.control &lt;- trainControl(method = &quot;repeatedcv&quot;, number = 10, repeats = 3)<br>                              <br><strong># Train the model</strong><br>Ridge_model_cv &lt;- train(Sales ~ ., data = data, method=&quot;glmnet&quot;, trControl = train.control, tuneGrid = expand.grid(alpha = 0, lambda = best_lambda))<br>                  <br><strong># Summarize the results</strong><br>print(Ridge_model_cv)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*50GOhTlbMmOsBavjxbDztQ.png" /><figcaption><strong>Output — 15</strong></figcaption></figure><p><strong>On an average, Ridge Regression Model captures 91.34% variability available in the target (Sales) and RMSE for the model is 1.538825</strong></p><p>This shows that <strong>Ridge Regression has not performed well than Polynomial / Orthogonal Polynomial Model</strong>.</p><p>Just stop it here and go for fitting of Lasso Regression.</p><h3>Fitting of Lasso Regression</h3><p>In case of Lasso Regression, Value of a hyper parameter, i.e., alpha is fixed. Our aim is to find out best value of lambda. For this purpose, we again use cross validation method and evaluation criteria for cross validation will be Mean Square Error.</p><p>R-code for the above purpose is as follows -</p><pre><strong># Cross Validation for obtaining best value of lambda for lasso regression (alpha = 1)</strong><br>cv_for_best_lambda &lt;-  cv.glmnet(x_train, y_train, family = &quot;gaussian&quot;, alpha = 1, type.measure = &quot;mse&quot;)</pre><p>For alpha = 1, The above code performs 10-fold (by default) cross validation using train data set and gives best value for lambda.</p><p>Check the value of lambda as follows -</p><pre><strong># Checking best value of lambda</strong><br>print(cv_for_best_lambda)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*CdctYS6eUbV6NoZH_8bObQ.png" /><figcaption><strong>Output — 16</strong></figcaption></figure><p><strong>Notice in the above output that -</strong></p><ol><li>There are two values of lambda (0.00187 and 0.04859) in last two lines of output. I will take the value 0.00187 since MSE is minimum (1.916) for that value. Also see the value of last column, i.e., <strong>6</strong> indicating that <strong>all predictors are important</strong>.</li><li>Value in the last line and in Last column of above output, i.e., <strong>4</strong> indicates that if we include only four predictors then the value of lambda = 0.04859 but we don’t select this value of lambda because MSE for this case is slightly higher than that of previous lambda value (0.00187).</li></ol><blockquote><em>Sometimes, It may be possible that including lesser number of predictors provide minimum MSE. In such a case, we select second value of lambda but that is not in our case. As i have earlier said that these methods (Ridge/Lasso/Elastic-Net) are also used in predictor selection.</em></blockquote><blockquote><em>But In our case, these methods only deal with multicollinearity problem as we see in the above output, if we include only four predictors then the value of lambda = 0.04859 but we don’t select this value of lambda because MSE for this case is slightly higher than that of previous lambda value (0.00187)</em></blockquote><p>Now, Store the best value of lambda, i.e., 0.00187 in R-object best_lambda as follows -</p><pre><strong># Storing best value of lambda in a R-object best_lambda</strong><br>best_lambda = cv_for_best_lambda$lambda.min</pre><p>Now, Fit Lasso Regression Model and check its performance on train data set as follows -</p><pre><strong># Fitting Lasso Regression Model</strong><br>lasso = glmnet(x_train, y_train, family = &quot;gaussian&quot;, alpha = 1, lambda = best_lambda)<br><br><strong># Checking its performance on train data set</strong><br>predict_train &lt;- lasso %&gt;% predict(x_train)<br>data.frame( R2 = R2(predict_train, y_train),<br>            RMSE = RMSE(predict_train, y_train),<br>            MAE = MAE(predict_train, y_train))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/809/1*OP1FylflKMPgu66R4XMLaQ.png" /><figcaption><strong>Output — 17</strong></figcaption></figure><p><strong>From the above output, We get that -</strong></p><ol><li>Adjusted R² = 93.43% approx.</li><li>RMSE = 1.319708</li></ol><p>Now, Check its performance on unseen data as follows -</p><pre><strong># Checking performance of Lasso regression on test data set</strong><br>predict_test &lt;- lasso %&gt;% predict(x_test)<br>data.frame( R2 = R2(predict_test, y_test),<br>            RMSE = RMSE(predict_test, y_test),<br>            MAE = MAE(predict_test, y_test))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/803/1*u-X126zTMxY5mU8KMfQXrQ.png" /><figcaption><strong>Output — 18</strong></figcaption></figure><p>Well, We obtain : <strong>R² = 0.954106</strong> and <strong>RMSE = 1.131341</strong></p><p>To be more confident, about its performance on unseen data, calculate average performance of Lasso Regression using Repeated K-fold cross validation as follows -</p><pre><strong># Repeated 10-fold cross validation for average performance of Lasso Regression</strong><br>set.seed(123)<br>train.control &lt;- trainControl(method = &quot;repeatedcv&quot;, number = 10, repeats = 3)<br>                              <br><strong># Train the model</strong><br>Lasso_model_cv &lt;- train(Sales ~ ., data = data, method=&quot;glmnet&quot;, trControl = train.control, tuneGrid = expand.grid(alpha = 1, lambda = best_lambda))<br>                  <br><strong># Summarize the results</strong><br>print(Lasso_model_cv)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*yu4IZ2HYt5AEEmh1fDkv4Q.png" /><figcaption><strong>Output — 19</strong></figcaption></figure><p><strong>On an average, Lasso Regression Model captures 93.58% variability available in the target (Sales) and RMSE for the model is 1.304869</strong></p><p>This shows that <strong>Lasso Regression has performed well than Ridge Regression Model (captures 91.34% variability)</strong>.</p><p>Just stop it here and go for fitting of Elastic-Net Regression.</p><h3>Fitting of Elastic-Net Regression</h3><p>In case of Elastic-Net Regression, Value of both hyper parameters are unknown. Our aim is to find out best value of alpha and lambda. For this purpose, we again use cross validation method. Here this method is used in a different way. So, See and notice the difference between the codes used in this case and in previous cases (Ridge/Lasso)</p><p>R-code for the above purpose is as follows -</p><pre><strong># Cross Validation for obtaining best value of alpha and lambda for elasticnet regression </strong><br><br><strong># Define Training Control</strong><br>set.seed(123)<br>train.control &lt;- trainControl(method = &quot;repeatedcv&quot;, number = 10, repeats = 3 , <strong>search = &quot;random&quot;</strong>)<br>                              <br><strong># Train the model</strong><br>cv_for_best_value &lt;- train(Sales ~ ., data = train.data1, method=&quot;glmnet&quot;, trControl = train.control)<br>                  <br><br><strong># Obtaining Best Value of alpha and lambda</strong><br>cv_for_best_value$bestTune</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/804/1*WtdIdJzTosoP4t4iLMQLnw.png" /><figcaption><strong>Output — 20</strong></figcaption></figure><p><strong>The above output gives the best value of alpha and lambda.</strong> We will use these values in fitting of Elastic-Net Regression.</p><p>Now, Fit Elastic-Net Regression Model and check its performance on train data set as follows -</p><pre><strong># Fitting Elasticnet Regression Model</strong><br>enet &lt;- glmnet(x_train, y_train, alpha = 0.4089769, lambda = 0.001472246 ,family = &quot;gaussian&quot;)<br><br><strong># Checking its performance on train data set</strong><br>predict_train &lt;- enet %&gt;% predict(x_train)<br>data.frame( R2 = R2(predict_train, y_train),<br>            RMSE = RMSE(predict_train, y_train),<br>            MAE = MAE(predict_train, y_train))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*EyXFVhgFe2hyA6S23H8D3Q.png" /><figcaption><strong>Output — 21</strong></figcaption></figure><p><strong>From the above output, We get that -</strong></p><ol><li>Adjusted R² = 93.46% approx.</li><li>RMSE = 1.317211</li></ol><p>Now, Check its performance on unseen data as follows -</p><pre><strong># Checking performance of Elasticnet regression on test data set</strong><br>predict_test &lt;- enet %&gt;% predict(x_test)<br>data.frame( R2 = R2(predict_test, y_test),<br>            RMSE = RMSE(predict_test, y_test),<br>            MAE = MAE(predict_test, y_test))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/803/1*FsH8ZAm5ATiLWtfNRgZ6fw.png" /><figcaption><strong>Output — 22</strong></figcaption></figure><p>Well, We obtain : <strong>R² = 0.9538105</strong> and <strong>RMSE = 1.134862</strong></p><p>To be more confident, about its performance on unseen data, calculate average performance of Elastic-Net Regression using Repeated K-fold cross validation as follows -</p><pre><strong># Repeated 10-fold cross validation for average performance of Elastic-Net Regression</strong><br><br><strong># Define Training control</strong><br>set.seed(123)<br>train.control &lt;- trainControl(method = &quot;repeatedcv&quot;, number = 10, repeats = 3)<br>                              <br><strong># Train the model</strong><br>Elasticnet_model_cv &lt;- train(Sales ~ ., data = data, method=&quot;glmnet&quot;, trControl = train.control, <strong>tuneGrid = expand.grid(alpha = 0.4089769, lambda = 0.001472246)</strong>)<br>                  <br><strong># Summarize the results</strong><br>print(Elasticnet_model_cv)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/811/1*qHcXCDK7drDNNPk7Geacsw.png" /><figcaption><strong>Output — 23</strong></figcaption></figure><h3>Great Result !</h3><p><strong>On an average, Elastic-Net Regression Model captures 93.66% variability available in the target (Sales) and RMSE for the model is 1.297969</strong></p><p>This shows that <strong>Elastic-Net Regression has performed well than both Ridge and Lasso Regression Model</strong>.</p><h3>12. Comparison of Different Models (Polynomial, Orthogonal Polynomial, Ridge, Lasso, Elastic-Net)</h3><p>Now, It’s time to compare different models and obtain the best one.</p><p>The following table contains Train, Test and Cross-validation Results for all previously fitted models -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/704/1*hjsqW_v5ag1QFAiggIMWFg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/707/1*uDmLxl_j-EUec0OJFU3RCw.png" /></figure><p><strong>Results from the above table -</strong></p><ol><li>Orthogonal Polynomial and Polynomial Model have same performance since value of R² and RMSE is same for both models.</li><li>Elastic-Net Regression performs better than Ridge/Lasso Regression since value of R² is maximum and RMSE is minimum for Elastic-Net Regression.</li><li>Elastic-Net Regression also performs better than Polynomial/Orthogonal Polynomial model since RMSE is minimum (1.297969) for Elastic-Net.</li><li>Also note that Lasso Regression performs well than other models on test data set but since this result is based on only one data set. Hence, we can not sure that it will also work well on other unseen data and that’s why we calculate average performance using cross-validation and obtain that Lasso is not better than Elastic-Net for the given data set.</li><li>Till now, <strong>Elastic-Net Regression model is the best model for the given Advertising Data Set that captures on an average 93.7% variability of target (Sales) with RMSE = 1.297969</strong></li></ol><h3>13. Obtaining Best Model</h3><p>Finally, We have got that <strong>Elastic-Net Regression performs well</strong> for the given advertising data set. So, <strong>What is the equation for this model ?</strong> For this purpose, we have to obtain values of Regression Coefficients.</p><p>R-code for the above purpose is as follows -</p><pre><strong># Obtaining Coefficients of Elastic-Net Regression Model</strong><br>enet$beta</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/811/1*Cs7EsjkC68r7zogL3vz-kA.png" /><figcaption><strong>Output — 24</strong></figcaption></figure><p><strong>Hence, The Elastic-Net Regression model is -</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*ISGZlxd1wQcvbdvyuPlrYg.png" /></figure><h3>14. Conclusion</h3><p>Finally, I want to conclude here is that in this notebook, <strong>I started with fitting of polynomial model and then shown you that multicollinearity assumption is violated by polynomial model and to get rid of this problem, we have three different methods that are nothing but Ridge, Lasso and Elastic-Net Regression. I have shown you in detail how to fit these models, how to get best values of hyper parameters of these models and at last I have mentioned the best Regression model that is Elastic-Net Regression for the given Advertising Data Set</strong>.</p><p>That’s all.</p><p><strong>Just do it yourself once for better understanding !</strong></p><p><a href="https://www.kaggle.com/pranjalpandey12/multicolinearity-ridge-lasso-elasticnet-regression"><strong>Link</strong></a> of my kaggle Notebook.</p><h3>Thanks for Reading my article.</h3><p><strong>If you find any mistake, just let me know.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6582cbabf7f3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/multicollinearity-ridge-lasso-elastic-net-regression-using-r-6582cbabf7f3">Multicollinearity / Ridge / Lasso / Elastic-Net Regression using R</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to proceed from Simple to Multiple and Polynomial Regression in R]]></title>
            <link>https://medium.com/analytics-vidhya/how-to-proceed-from-simple-to-multiple-and-polynomial-regression-in-r-84b77f5673c5?source=rss-4fa9dfe37370------2</link>
            <guid isPermaLink="false">https://medium.com/p/84b77f5673c5</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[polynomial-regression]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[linear-regression]]></category>
            <dc:creator><![CDATA[Pranjal Pandey]]></dc:creator>
            <pubDate>Wed, 22 Jul 2020 04:55:15 GMT</pubDate>
            <atom:updated>2021-02-21T15:24:33.563Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*LIFTcaAiITblpi1U6ZP26w.png" /></figure><h3>Introduction -</h3><p>This article is in the continuation of my <a href="https://medium.com/%E0%A4%B5%E0%A4%BF%E0%A4%9C%E0%A5%8D%E0%A4%9E%E0%A4%BE%E0%A4%A8%E0%A4%82-%E0%A4%AC%E0%A5%8D%E0%A4%B0%E0%A4%B9%E0%A5%8D%E0%A4%AE/step-by-step-procedure-to-perform-simple-linear-regression-with-statistical-analysis-in-r-340ef0a392d6"><strong>first article</strong></a> in which I have shown a complete procedure to perform Simple Linear Regression in detail. Now in this article, I am taking a little more complex data set (<a href="https://www.kaggle.com/ashydv/advertising-dataset"><strong>Advertising Data set</strong></a>) and going to show you <strong>How Multiple Linear Regression is prepared and using the information obtained from its diagnostic plot, how we proceed towards Orthogonal Polynomial Regression and obtain a better model for this data set.</strong></p><p>This article consists of the following sections -</p><ol><li>Loading Required Libraries</li><li>Loading Data set</li><li>Exploring Data set</li><li>Splitting Data set</li><li>Fitting Simple Linear Regression Model</li><li>Fitting Multiple Linear Regression Model with Diagnostic Plots and Statistical Tests</li><li>Fitting Orthogonal Polynomial Linear Regression Model with Diagnostic Plots and Statistical Tests</li><li>Making Predictions</li><li>Repeated 10-fold Cross Validation</li><li>Conclusion</li><li>Task for you (If interested)</li><li>Information about Next Article</li></ol><p>I am going to use kaggle online R-Notebook for analysis work. You may use any software like R-studio or R-cran version to work offline.</p><h3>1. Loading Required Libraries</h3><p>It is not mandatory to load libraries in the beginning but I am doing it for simplicity.</p><pre><strong># Loading required libraries</strong><br>library(tidyverse)     <strong>#</strong> Pipe operator (<strong>%&gt;%</strong>) and other commands<br>library(caret)         <strong>#</strong> Random split of data/cross validation<br>library(olsrr)         <strong>#</strong> Heteroscedasticity Testing (<strong>ols_test_score</strong>)<br>library(car)           <strong>#</strong> Muticolinearity detection (<strong>vif</strong>)<br>library(broom)         <strong># </strong>Diagnostic Metric Table (<strong>augment</strong>)</pre><h3>2. Loading Data set</h3><p>First of all, Load the data set in your R-Session. <a href="https://www.kaggle.com/ashydv/advertising-dataset/download"><strong>Link</strong></a> to download data set.</p><pre><strong># Loading Data set</strong><br>data = read.csv(&quot;../input/advertising-dataset/advertising.csv&quot; , header = T)</pre><p><strong>Advertising data set has been successfully loaded in the R-object “data”.</strong></p><h3>3. Exploring Data set</h3><p>Make some understanding about the given data set as follows -</p><pre><strong># Inspection of top 5-rows of data</strong><br>head(data)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*LjGv--Z0zcPDYhUafi3QPw.png" /><figcaption><strong>Output — 1</strong></figcaption></figure><p>The above output shows top 5-rows of given data set. At this stage, just see the data and make some understanding as — <strong>There are four variables (TV,Radio,Newspaper,Sales) in the data set and all are numeric variables</strong>.</p><pre><strong># Inspection of 5-bottom rows of data</strong><br>tail(data)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*MekiZ-5eD-N_dMeTMtTMHg.png" /><figcaption><strong>Output — 2</strong></figcaption></figure><p>From the above output, it is clear that there are <strong>200 rows</strong> in the data set and one more important point is that <strong>there are no rows that contains the information on something like “Totals”</strong>. It may be possible that in your data set there is a last row that contains the information of <strong>Totals of each column</strong>. Such rows are not useful in further analysis or during the model preparation. So, if there exists such row, just remove it from the data.</p><pre><strong># Getting Structure of whole data set</strong><br>str(data)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/835/1*f7E1tAh252RwB1Lo55Fhow.png" /><figcaption><strong>Output — 3</strong></figcaption></figure><p><strong>The above output gives information as-</strong></p><ol><li>There are 200 rows and 4 variables.</li><li>Variables are : TV,Radio,Newspaper,Sales</li><li>All are numeric variables.</li></ol><pre><strong># Checking Outliers</strong><br>boxplot(data)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/1*lW0m6M9BhDlqq4a2-rF6vw.png" /><figcaption><strong>Output — 4</strong></figcaption></figure><p><strong>The above plot shows that two outliers are present in the variable “Newspaper”</strong>. Just remove these outliers by the following command -</p><pre><strong># Removing Outliers</strong><br>data &lt;- data[-which(data$Newspaper %in%    boxplot.stats(data$Newspaper)$out), ]</pre><p>Again, See the boxplot -</p><pre><strong># Again Checking Outliers</strong><br>boxplot(data)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/834/1*qeAsuvry2A4VAg2cHUFQeA.png" /><figcaption><strong>Output — 5</strong></figcaption></figure><p><strong>Now, Outliers have been removed.</strong></p><pre><strong># Checking Missing Values</strong><br>table(is.na(data))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/835/1*tEIGCTJB5UTaGQp9aH2XEw.png" /><figcaption><strong>Output — 6</strong></figcaption></figure><p><strong>The above output shows that there is no missing value in the given data set.</strong></p><p><strong>Deciding the Target and Predictors </strong>— It is always known to us which variable must be taken as Target and which as Predictors. This depends on the problem what you want to predict. I am taking here <strong>Sales</strong> as <strong>Target</strong> and <strong>rest variables</strong> as <strong>Predictors</strong>.</p><p>We have four numeric variables. Just take a look on scatter plot of these Variables as follows -</p><pre><strong># Creating scatter plot matrix </strong><br>pairs(data , upper.panel = NULL)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/834/1*GyY3TGZxrO5AE1lh7l4peg.png" /><figcaption><strong>Output — 7</strong></figcaption></figure><p><strong>This output shows that -</strong></p><ol><li>No or very low linear relationship between TV and Radio variable.</li><li>Low linear relationship between TV and Newspaper variable.</li><li>Moderate linear relationship between Radio and Newspaper variable.</li><li>High linear relationship between TV and Sales , Radio and Sales , Newspaper and Sales.</li><li>A small curvilinear relationship is also present between TV and Sales as well as Radio and Sales.</li></ol><p>Let’s get a more closer view to be more confident about existing relationship by plotting separate scatter plots -</p><pre><strong># Scatter Plot between TV and Sales</strong><br>plot(data$TV , data$Sales)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/873/1*zaP2j03NFFQlOQhPZYCQWg.png" /><figcaption><strong>Output — 8</strong></figcaption></figure><p><strong>Notice, there is a small curvilinear relationship between TV and Sales.</strong></p><pre><strong># Scatter Plot between Radio and Sales</strong><br>plot(data$Radio , data$Sales)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/835/1*eqPMaJlis0QNkyvktwP3-g.png" /><figcaption><strong>Output — 9</strong></figcaption></figure><p><strong>Notice, there is a curvilinear relationship between Radio and Sales.</strong></p><pre><strong># Scatter Plot between Newspaper and Sales</strong><br>plot(data$Newspaper , data$Sales)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/839/1*1zCSdBHNm7yKP4G_LY4p3Q.png" /><figcaption><strong>Output — 10</strong></figcaption></figure><p><strong>Low linear relationship between Newspaper and Sales variable.</strong></p><pre><strong># Scatter Plot between TV and Radio</strong><br>plot(data$TV , data$Radio)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/845/1*OPXdoD3RvlfOKE4bpX7JqA.png" /><figcaption><strong>Output — 11</strong></figcaption></figure><p><strong>No linear relationship between TV and Radio variable.</strong></p><pre><strong># Scatter Plot between Newspaper and TV</strong><br>plot(data$TV , data$Newspaper)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/855/1*JTD8g7eKgznZzHILrTno_A.png" /><figcaption><strong>Output — 12</strong></figcaption></figure><p><strong>No linear relationship between TV and Newspaper variable.</strong></p><pre><strong># Scatter Plot between Newspaper and Radio</strong><br>plot(data$Radio , data$Newspaper)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*Y8yhkKD7gi7UPbpvi7IGLA.png" /><figcaption><strong>Output — 13</strong></figcaption></figure><p><strong>Moderate linear relationship between Radio and Newspaper variable.</strong></p><p><strong>Remember these points in your mind that will help you to prepare a better model.</strong></p><h3>4. Splitting Data Set</h3><p>Now, I am going to split the whole data set into two parts. One part is known as <strong>train data set</strong> and other is <strong>test data set</strong>. We do this because first we train/fit the model using train data set and then use the test data set to check the performance of the obtained model on new data set that has not been used during training period. Splitting is done by the following code -</p><pre><strong># Randomly Split the data into training and test set</strong><br>set.seed(123)<br>training.samples &lt;- data$Sales %&gt;%<br>  createDataPartition(p = 0.75, list = FALSE)<br>train.data  &lt;- data[training.samples, ]<br>test.data &lt;- data[-training.samples, ]</pre><p><strong>Train data set</strong> and <strong>Test data set</strong> has been <strong>stored in</strong> R-object <strong>train.data</strong> and <strong>test.data</strong> respectively. Note that :</p><ol><li>We will use the data available in <strong>train.data</strong> for <strong>fitting/training the model</strong>.</li><li>We will use the data available in <strong>test.data</strong> to check the <strong>performance of model</strong>.</li></ol><h3>5. Fitting Simple Linear Regression</h3><p>Since, we have only three predictors, we may fit three separate simple linear regression model one for each predictor, i.e.,</p><ol><li>Sales ~ TV</li><li>Sales ~ Radio</li><li>Sales ~ Newspaper</li></ol><p>Fit these three models and try to find the percentage variance explained by these models.This is achieved by <strong>Adjusted R² </strong>and in R using <strong>summary()</strong> function.</p><pre><strong># Fitting Sales ~ TV</strong><br>sm1 &lt;- lm(Sales ~ TV , data = train.data)<br><br><strong># Take a look on summary of the model</strong><br>summary(sm1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/1*I9jrIZE6F96MI-WGwqTrIw.png" /><figcaption><strong>Output — 14</strong></figcaption></figure><p><strong>From the above output, you must notice that -</strong></p><ol><li>Created model is statistically significant since p-value &lt;&lt;&lt; 0.05 (see in the last line of output)</li><li>From the coefficients section, it is clear that both coefficients (slope and intercept) are statistically significant since p-value &lt;&lt;&lt; 0.05</li><li>This model with TV as predictor explains approximately <strong>81%</strong> variability of target (Sales).</li><li><strong>Residual standard error</strong> for the model is <strong>2.29</strong></li></ol><pre><strong># Fitting Sales ~ Radio</strong><br>sm2 &lt;- lm(Sales ~ Radio , data = train.data)<br><br><strong># Take a look on summary of the model</strong><br>summary(sm2)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*0W3Ni-gfcbRP07KUCClouA.png" /><figcaption><strong>Output — 15</strong></figcaption></figure><p><strong>From the above output, you must notice that -</strong></p><ol><li>Created model is statistically significant since p-value &lt;&lt; 0.05 (see in the last line of output)</li><li>From the coefficients section, it is clear that both coefficients (slope and intercept) are statistically significant since p-value &lt;&lt; 0.05</li><li>This model with TV as predictor explains approximately <strong>13%</strong> variability of target (Sales).</li><li><strong>Residual standard error</strong> for the model is <strong>4.917</strong></li></ol><pre><strong># Fitting Sales ~ Newspaper</strong><br>sm3 &lt;- lm(Sales ~ Newspaper , data = train.data)<br><br><strong># Take a look on summary of the model</strong><br>summary(sm3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/839/1*L42lrBrzCjymTIx5RzYNLw.png" /><figcaption><strong>Output — 16</strong></figcaption></figure><p><strong>From the above output, you must notice that -</strong></p><ol><li>Created model is statistically significant since p-value &lt; 0.05 (see in the last line of output)</li><li>From the coefficients section, it is clear that both coefficients (slope and intercept) are statistically significant since p-value &lt; 0.05</li><li>This model with TV as predictor explains approximately <strong>2%</strong> variability of target (Sales).</li><li><strong>Residual standard error</strong> for the model is <strong>5.21</strong></li></ol><p><strong>Till now, we have obtained that Simple Linear Regression Model with TV as predictor is explaining more variability of target (Sales).</strong></p><p>Just <strong>draw Scatter plot between TV and Sales</strong> and also <strong>draw the Simple Linear Regression Line in the plot</strong> as follows -</p><pre><strong># Scatter plot with Simple Linear Regression Line</strong><br>plot(train.data$TV , train.data$Sales)<br><br><strong># Adding Regression Line</strong><br>abline(lm(train.data$Sales ~ train.data$TV) , col = &quot;blue&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/829/1*tjVldhMVw0I64ABmATlawg.png" /><figcaption><strong>Output — 17</strong></figcaption></figure><p><strong>One problem occurs here :</strong> The above plot shows that it is not feasible to predict Sales only on the basis of a single predictor due to more variability in the Sales. Also, if we use single predictor then we completely neglect the effect of rest two other predictors on Sales, that may not be the case in real. <strong>So, why not extend this model ?</strong></p><h3>6. Fitting Multiple Linear Regression with Diagnostic Plot</h3><p>There are many methods to extend the above simple linear regression model such as <strong>Forward Selection method</strong>, <strong>Backward Selection Method</strong>, <strong>Mixed Selection Method</strong> and many more. In this article, <strong>I want to go with Forward Selection method to explore some more concepts</strong>.</p><p><strong>Extending Simple Linear Regression Model using Forward Selection Method -</strong></p><p>In this context, <strong>What will i do ?</strong></p><p>I will simply <strong>include the predictor Radio</strong> in Simple Linear Regression model <strong>Sales ~ TV</strong> (Why in this model ? : <strong>Due to explaining a huge part of variability of Sales</strong>).</p><p><strong>Why we include Radio at this stage ?</strong></p><p>Because <strong>it explains more variability (13%)</strong> of Sales in comparison to <strong>Newspaper (2%)</strong> after <strong>TV (81%)</strong>. — {Results from Simple Linear Regression has been used here.}</p><p>So, Fit a <strong>Multiple Linear Regression model</strong> with <strong>two predictors TV and Radio</strong> and obtain summary of the model as follows -</p><pre><strong># Fitting MLR model with predictors TV and Radio</strong><em> </em><br>mm1 &lt;- lm(Sales ~ TV + Radio , data = train.data)<br><br><strong># Take a look on summary of the model</strong><br>summary(mm1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*2upUerzInMFmYPPmwJJk_Q.png" /><figcaption><strong>Output — 18</strong></figcaption></figure><p><strong>Well, From the above output, notice that -</strong></p><ol><li>Created model is statistically significant since p-value &lt;&lt;&lt; 0.05 (see in the last line of output)</li><li>From the coefficients section, it is clear that both coefficients (slopes and intercept) are statistically significant since p-value &lt;&lt;&lt; 0.05</li><li>This model with TV and Radio as predictors explains approximately <strong>89%</strong> variability of target (Sales) that is a better indication with respect to the model with TV alone as predictor.</li><li><strong>Residual standard error</strong> for the model is <strong>1.715</strong></li></ol><p>But <strong>we must test, whether the improvement in Adjusted R-squared is statistically significant ?</strong></p><p>i.e., we want to test the null hypothesis — <strong>H0 : The improvement in Adjusted R-squared is not statistically significant.</strong> Vs the alternative hypothesis — <strong>H1 : The improvement in Adjusted R-squared is statistically significant.</strong></p><p>For this testing, we use <strong>ANOVA (Analysis of Variance) technique</strong> and code for the same is as follows -</p><pre><strong># Performing ANOVA to test the above stated null hypothesis</strong><br>anova(sm1 , mm1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*Xru6ZFaBs567NGRDdkddCQ.png" /><figcaption><strong>Output — 19</strong></figcaption></figure><p>In the above output, <strong>Notice the value in the last column of second row</strong>. This value (<strong>2.051808e-20</strong>) indicates the <strong>p-value</strong> for testing null hypothesis. <strong>Since this value is extremely less than 0.05, hence we have sufficient evidence from the data to reject the null hypothesis and accept the alternative.</strong></p><p><strong>That’s why the improvement in Adjusted R-squared is statistically significant.</strong></p><p>Hence, <strong>Adopt</strong> the model <strong>Sales ~ 0.05462 TV + 0.10239 Radio</strong> at this stage.</p><p><strong>Why not extend this model further ?</strong></p><p>i.e., <strong>Include the third predictor Newspaper also in your multiple linear regression model and see what happens.</strong></p><pre><strong># Extending further the MLR including the predictor Newspaper</strong><br>mm2 &lt;- lm(Sales ~ TV + Radio + Newspaper , data = train.data)<br><br><strong># Take a look on summary of the model</strong><br>summary(mm2)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*VmsNDQKkE2IHSvlE11_3ww.png" /><figcaption><strong>Output — 20</strong></figcaption></figure><p><strong>From the above output and using the information from previously fitted model, Notice that -</strong></p><ol><li>From the coefficients section of the above output, It is clear that Newspaper predictor is not statistically significant for the model due to p-value (0.69) &gt; 0.05</li><li>Adjusted R-squared has been reduced 89.41 to 89.35</li><li>Residual standard error has been increased from 1.715 to 1.72</li><li>Although, the created model is statistically significant since p-value &lt;&lt;&lt; 0.05 (see in the last line of output)</li></ol><p>So, we have sufficient evidence from the data for <strong>not to include the Newspaper as predictor</strong> in the model.</p><p>Hence, Remove it from the model and we get the model as in previously fitted <strong>multiple linear regression model</strong> already stored in R-object <strong>mm1</strong> -</p><p>i.e., <strong>Sales ~ 0.05462 TV + 0.10239 Radio</strong></p><p><strong>We will consider this model for further discussion.</strong></p><h3>Diagnostic Plots -</h3><p>To check whether all the assumptions of Multiple Linear Regression is fulfilled, we use different diagnostic plots.</p><p><strong>Checking Linearity Assumption -</strong></p><p><strong>Residual plot</strong> is used to check the first assumption, i.e., <strong>Linearity</strong> assumption between target and predictors (Joint of TV , Radio) as follows -</p><pre><strong># Residual Plot</strong><br>plot(mm1 , 1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*dsL4csAH_x7bEU3e7wECmg.png" /><figcaption><strong>Output — 21</strong></figcaption></figure><p><strong>Notice from the above plot -</strong></p><ol><li>Red line is approximately horizontal and linear which indicates that <strong>Linearity assumption holds well</strong>.</li><li>Residual fluctuates in a random manner inside a band drawn between Residuals = -4 to +4 which indicates that the fitted model is good for prediction to some extent. <strong>Why to some extent ?</strong></li><li>Because the points <strong>131</strong> and <strong>151</strong> may be potential <strong>outliers</strong> since they are very far from other points. But we know that <strong>Large residuals (as in our case with 131 and 151 data points) could also indicate that either the variance is not constant (heteroscedasticity) or the true relationship between target and predictors is nonlinear</strong>. So, These possibilities should be investigated before the points are considered outliers.</li></ol><p><strong>Checking Homoscedasticity Assumption -</strong></p><p>I am going to use <strong>Score Test</strong>, but you may apply other tests also Breusch Pagan Test, Bartlett Test etc.</p><pre><strong># Score Test for Heteroscedasticity</strong><br>ols_test_score(mm1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/841/1*79Vb7WwaZjzZcD1Zhpo71w.png" /><figcaption><strong>Output — 22</strong></figcaption></figure><p><em>From the last line of the above output, It is clear that p-value is greater than the significance level 0.05. Hence, we may accept the null hypothesis and conclude that the variance is homogeneous. i.e., Homoscedasticity</em></p><p><strong>Checking Auto-correlation Assumption -</strong></p><p>Durbin Watson Test is used to detect the effect of Auto-correlation as follows -</p><pre><strong># Checking effect of</strong> <strong>Auto-correlation</strong><br>durbinWatsonTest(mm1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/830/1*v-y3aWMfSqEKuFR5iNvWHg.png" /><figcaption><strong>Output — 23</strong></figcaption></figure><p><em>From the above output, It is clear that p-value (0.166) &gt; 0.05 , Hence, we may accept the null hypothesis and conclude that there is no auto-correlation between errors. i.e., Errors are uncorrelated.</em></p><p><strong>Checking Multicolinearity -</strong></p><p>Generally, Variance Inflation Factor is used to detect Multicolinearity. As a rule of thumb, VIF greater than 5 or 10 represents Multicolinearity.</p><pre><strong># Detecting</strong> <strong>Multicolinearity</strong><br>vif(mm1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/1*Dd_jz5BvY8V15l7uI6SEAQ.png" /><figcaption><strong>Output — 24</strong></figcaption></figure><p><em>Note that Variance inflation factor for both predictors are less than 5 (as a rule of thumb) , Hence there is no multicollinearity between predictors.</em></p><p><strong>Checking Normality Assumption -</strong></p><p>Shapiro Wilk Test is generally used to check normality assumption.</p><pre><strong># Checking Normality of Errors</strong><br>shapiro.test(mm1$residuals)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*VMIKl0SsyC1t4OIa1HIusQ.png" /><figcaption><strong>Output — 25</strong></figcaption></figure><p><em>Normality does not hold since p-value &lt; 0.05</em></p><p>Just plot histogram for residuals to get an idea about the pattern of distribution -</p><pre><strong># Plotting Histogram for Residuals</strong><br>hist(mm1$residuals)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*Hg0g1y3V-KylLljOJJn4jg.png" /><figcaption><strong>Output — 26</strong></figcaption></figure><p><strong>We see that there is some problem with left tail. It may be due to the data points 131 and 151 as pointed out earlier.</strong></p><p><strong>Now, Finally what we have obtained is that Variance is constant and one last possibility is to check whether there is any nonlinear relationship between target and predictors before considering the data points 131 and 151 as outliers</strong>.</p><p><strong>Also from the previously plotted scatter plots between target and different predictors we have noticed that there exists some type of curvilinear relationship</strong>. There are so many algorithms that deals with curvilinear relationship but I am going to take a very basic algorithm to deal with the existing curvilinear relationship and that is nothing but Polynomial Regression. (Generally,<strong> we use orthogonal polynomial to avoid multicollinearity problem</strong>)</p><p>So, All these facts directly indicate us <strong>why not to use Orthogonal Polynomial Regression ?</strong></p><p>And now we move towards fitting of <strong>Orthogonal Polynomial Regression between Sales and predictors TV and Radio</strong>.</p><p><strong>Why these two predictors only ?</strong></p><p>Because we have seen that <strong>Newspaper variable is not statistically significant</strong> when we had fitted Multiple Linear Regression.</p><h3>7. Fitting Orthogonal Polynomial Regression with Diagnostic Plot</h3><p>Now, I am going to fit a second order orthogonal polynomial in two variables. R-code for fitting second order orthogonal polynomial model in two variables TV and Radio is as follows -</p><pre><strong># Fitting second order orthogonal polynomial model in two variables to avoid multicollinearity</strong><br>pm1 &lt;- lm(Sales ~ poly(TV , 2) + poly(Radio , 2) + TV:Radio  , data = train.data)<br><br><strong># Take a look on summary of the model</strong><br>summary(pm1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*IansgfdPHliTDgp0WuhwLQ.png" /><figcaption><strong>Output — 27</strong></figcaption></figure><p><strong>From the above output, Notice that -</strong></p><ol><li>Created model is statistically significant since p-value &lt;&lt;&lt; 0.05 (see in the last line of output)</li><li>From the coefficients section, it is clear that all coefficients are statistically significant since p-value &lt;&lt;&lt; 0.05</li><li>This second order orthogonal polynomial model explains <strong>92.58%</strong> variability of target (Sales) that is a better indication with respect to the multiple linear regression model with TV and Radio as predictor.</li><li><strong>Residual standard error</strong> for the model is <strong>1.436</strong></li></ol><p><strong>Checking Whether this improvement in Adjusted R-squared is statistically significant -</strong></p><p>i.e., we want to test the null hypothesis — <strong>H0 : The improvement in Adjusted R-squared is not statistically significant.</strong></p><p>Vs the alternative hypothesis — <strong>H1 : The improvement in Adjusted R-squared is statistically significant.</strong></p><pre><strong># Performing ANOVA to test the above stated null hypothesis</strong><br>anova(mm1 , pm1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/809/1*rQSOJzueucVlJF-gkKhHhg.png" /><figcaption><strong>Output — 28</strong></figcaption></figure><p>In the above output, <strong>Notice the value in the last column of second row</strong>. This value (<strong>9.441734e-12</strong>) indicates the <strong>p-value</strong> for testing null hypothesis. <strong>Since this value is extremely less than 0.05, hence we have sufficient evidence from the data to reject the null hypothesis and accept the alternative.</strong></p><p><strong>That’s why the improvement in Adjusted R-squared is statistically significant.</strong></p><p>Hence, <strong>Adopt the second order orthogonal polynomial model at this stage.</strong></p><p><strong>Why not to use third order (orthogonal) Polynomial Regression in two variable ?</strong></p><p><em>Since, We have noticed that Adjusted R-squared has been increased to a great extent from </em><strong><em>89%</em></strong><em> to </em><strong><em>92.58%</em></strong><em>. So, Move towards fitting of third order orthogonal Polynomial Regression in two variable and see what happens.</em></p><pre><strong># Fitting third order (orthogonal) polynomial model in two variables to avoid multicollinearity</strong><br>pm2 &lt;- lm(Sales ~ poly(TV , 3) + poly(Radio , 3) + TV:Radio  , data = train.data)<br><br><strong># Take a look on summary of the model</strong><br>summary(pm2)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/838/1*mIih_WLV8lgwwdMBsZys4g.png" /><figcaption><strong>Output — 29</strong></figcaption></figure><p>It is clear from the coefficients section of the above output that <strong>third order of TV predictor is not statistically significant (p-value &gt; 0.05). Hence, Don’t include this term in the model.</strong></p><p>Hence, Fit the model as follows -</p><pre><strong># Fitting third order (orthogonal) polynomial model in two variables to avoid multicolinearity but after removing third order of TV predictor</strong><br>pm3 &lt;- lm(Sales ~ poly(TV , 2) + poly(Radio , 3) + TV:Radio  , data = train.data)<br><br><strong># Take a look on summary of the model</strong><br>summary(pm3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*mW_n5KNnKS0MVOW-gVIpng.png" /><figcaption><strong>Output — 30</strong></figcaption></figure><p><strong>From the above output and using the information from second order orthogonal polynomial model stored in R-object pm2, Notice that -</strong></p><ol><li>Created model is statistically significant since p-value &lt;&lt;&lt; 0.05 (see in the last line of output)</li><li>From the coefficients section, it is clear that all coefficients are statistically significant since p-value &lt;&lt;&lt; 0.05</li><li>This third order orthogonal polynomial model in two variables after removing third order of TV predictor explains <strong>92.93%</strong> variability of target (Sales) that is a better indication with respect to the second order orthogonal polynomial regression model.</li><li><strong>Residual standard error</strong> for the model is <strong>1.401</strong></li></ol><p><strong>Again, Checking Whether this improvement in Adjusted R-squared is statistically significant -</strong></p><p>i.e., we want to test the null hypothesis — <strong>H0 : The improvement in Adjusted R-squared is not statistically significant.</strong></p><p>Vs the alternative hypothesis — <strong>H1 : The improvement in Adjusted R-squared is statistically significant.</strong></p><pre><strong># Performing ANOVA to test the above stated null hypothesis</strong><br>anova(pm1 , pm3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*PGS__2xLkgSNf1sB_uFvOA.png" /><figcaption><strong>Output — 31</strong></figcaption></figure><p>In the above output, <strong>Notice the value in the last column of second row</strong>. This value (<strong>0.004968654</strong>) indicates the <strong>p-value</strong> for testing null hypothesis. <strong>Since this value is extremely less than 0.05, hence we have sufficient evidence from the data to reject the null hypothesis and accept the alternative.</strong></p><p><strong>That’s why the improvement in Adjusted R-squared is statistically significant.</strong></p><p>Hence, <strong>Adopt the third order orthogonal polynomial model without third order of TV predictor at this stage.</strong></p><p><strong>We will consider this model for further discussion.</strong></p><blockquote><strong><em>Note :</em></strong><em> If you want to increase the order of predictor Radio from 3 to 4, you may increase but you will see that this new coefficient with fourth order of predictor Radio will not be statistically significant (Try it yourself). Hence, you have to remove it and </em><strong><em>go with second order of TV and third order of Radio only</em></strong><em>. After that you can not increase the order further.</em></blockquote><h3>Diagnostic Plots -</h3><p>Now, again check all the assumptions of Linear Regression are satisfied or not.</p><p><strong>Checking Linearity Assumption -</strong></p><pre><strong># Residual Plot</strong><br>plot(pm3 , 1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/806/1*pQ5vEFEHcIA7Q9Vj3hNBVA.png" /><figcaption><strong>Output — 32</strong></figcaption></figure><p><strong>Notice from the above plot -</strong></p><ol><li>Red line is approximately horizontal and linear at Residuals = 0 which indicates that <strong>Linearity assumption holds well</strong>.</li><li>Residual fluctuates in a random manner inside a band drawn between Residuals = -4 to +4 which indicates that the fitted model is good for prediction to some extent. <strong>Why to some extent ?</strong></li><li>Because again we see that the point <strong>131</strong> (Note : 151 is now with the other points) may be potential <strong>outlier</strong> since this is very far from other points.</li></ol><p><strong>Checking Homoscedasticity Assumption -</strong></p><pre><strong># Score Test for Heteroscedasticity</strong><br>ols_test_score(pm3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/819/1*jeTDKgS6GADHqOmSYUvdzA.png" /><figcaption><strong>Output — 33</strong></figcaption></figure><p><em>Errors have constant variance, p-value &gt; 0.05</em></p><p><strong>Checking Auto-correlation Assumption -</strong></p><pre><strong># Checking effect of Auto-correlation</strong><br>durbinWatsonTest(pm3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*83fO4YuhNrfODCOSg4xKFg.png" /><figcaption><strong>Output — 34</strong></figcaption></figure><p><em>Errors are uncorrelated.</em></p><p><strong>Checking Normality Assumption -</strong></p><pre><strong># Checking Normality of Errors</strong><br>shapiro.test(pm3$residuals)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/1*FmbxLBNAXHY01pAdG2FADw.png" /><figcaption><strong>Output — 35</strong></figcaption></figure><p><em>Errors are normally distributed.</em></p><p><strong>Checking Multicollinearity -</strong></p><pre><strong># Detecting Multicolinearity</strong><br>vif(pm3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*yr0BBkaSfBXdGZFuF3RiYA.png" /><figcaption><strong>Output — 36</strong></figcaption></figure><p><em>Note that all values in the last column of the above output are less than 5 (as a rule of thumb) , Hence there is no multicolinearity.</em></p><p><strong>Removing Observation number 131 from train data set -</strong></p><p>Now we have only a choice that Delete the observation number 131 from the train data set as it has large residual (See : Residual Plot for pm3 object) and check whether Adjusted R-squared improves significantly. If yes, then remove it otherwise include observation number 131 too.</p><pre><strong># Creating Diagnostic metrics Table for model pm3</strong><br>dm = augment(pm3)<br><br><strong># See the Table</strong><br>head(dm)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/767/1*wlhuzd2GGjGK73ykHgBRqA.png" /><figcaption><strong>Output — 37</strong></figcaption></figure><p><strong>Notice from the above output that -</strong></p><ol><li>This table consists of information on different diagnostic metrics such as Residuals (column — 9), cook’s distance (column — 12) and Studentized Residuals (column — 13) and many more.</li><li>I will use last column of the above table to delete observation number 131.</li></ol><pre><strong># Checking minimum value of last column (Studentized Residual)</strong><br>min(dm$.std.resid)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*rskvI88WQ4D0r_AsBxJTIQ.png" /><figcaption><strong>Output — 38</strong></figcaption></figure><p><strong>The above value of Studentized residual is less than -3 (Rule of thumb), Hence it indicates an outlier.</strong> So just remove that observation as follows -</p><pre><strong># Checking the index of that observation in train data</strong><br>which(dm$.std.resid  %in% &quot;-3.4452042988145&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*amicu8c4Ybz8mLyWBMZHfQ.png" /><figcaption><strong>Output — 39</strong></figcaption></figure><p><strong>The above output indicates that the outlier is at index 98 in train data set.</strong> Just check the complete information about that row as follows -</p><pre><strong># Info. about 98th row of train data set</strong><br>train.data[98,]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/809/1*-HBuSkoc6n310ws_UBXWkg.png" /><figcaption><strong>Output — 40</strong></figcaption></figure><p><strong>This is our target. We have to remove it from our train data set.</strong></p><pre><strong># Removing 98th row of outlier</strong><br>train.data1 = train.data %&gt;% filter(train.data$Sales !=  1.6)<br><br><strong># Checking number of rows in old train data set</strong><br>nrow(train.data)<br><br><strong># Checking number of rows in new train data set (train.data1)</strong><br>nrow(train.data1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/739/1*Dbk7qsdWxfvgwHeQdvj2uw.png" /><figcaption><strong>Output — 41</strong></figcaption></figure><p><strong>One observation has been successfully removed.</strong></p><p>Now, again fit the same polynomial model as is stored in pm3 but using the data stored in R-object<strong> train.data1 -</strong></p><pre><strong><em># Fitting third order (orthogonal) polynomial model in two variables to avoid multicolinearity but after removing third order of TV predictor using train.data1</em></strong><br>pm4 &lt;- lm(Sales ~ poly(TV , 2) + poly(Radio , 3) + TV:Radio  , data = train.data1)<br><br><strong># Take a look on summary of the model</strong><br>summary(pm4)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/830/1*_PWF1xMbsz0sfIsCD_iXVw.png" /><figcaption><strong>Output — 42</strong></figcaption></figure><p><strong>From the above output and using the information from second order orthogonal polynomial model stored in R-object pm3, Notice that -</strong></p><ol><li>Created model pm4 is statistically significant since p-value &lt;&lt;&lt; 0.05 (see in the last line of output)</li><li>From the coefficients section, it is clear that all coefficients are statistically significant since p-value &lt;&lt;&lt; 0.05</li><li>This polynomial model after removing the outlier explains <strong>93.21%</strong> variability of target (Sales) that is a better indication with respect to the polynomial regression model stored in R-object pm3.</li><li><strong>Residual standard error</strong> for the model is <strong>1.347</strong></li></ol><blockquote><strong><em>Note :</em></strong><em> At that time we can not perform ANOVA to test whether this improvement in Adjusted R-squared is significant, because model pm3 is based on 150 observations and pm4 is on 149 only. In such situation. since we have noticed that Adjusted R-squared has increased and Residual standard error has decreased, hence we may adopt this model stored in pm4.</em></blockquote><p><strong>Now, Check all other assumptions in a quick -</strong></p><pre><strong># Linearity Assumption</strong><br>plot(pm4 ,1)<br><br><strong># Homoscedasticity Assumption </strong><br>ols_test_score(pm4)<br><br><strong># Autocorrelation Assumption</strong><em> </em><br>durbinWatsonTest(pm4)<br><br><strong># Normality Assumption</strong><br>shapiro.test(pm4$residuals)<br><br><strong># Multicolinearity Assumption</strong><br>vif(pm4)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/819/1*xMus_BGmRIGPQ8yTaWnyYA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/803/1*HXX0a4SGjGxsCCRUUnJPXQ.png" /><figcaption><strong>Output — 43</strong></figcaption></figure><p><em>All assumptions have been satisfied now.</em></p><p><strong>Checking outliers again by creating Diagnostic metric table for model pm4 -</strong></p><pre><strong># Creating Diagnostic metric table for model pm4</strong><br>dm1 = augment(pm4)<br><br><strong># Checking minimum and maximum value of studentized residual</strong><br>min(dm1$.std.resid)<br>max(dm1$.std.resid)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*U6O5MlSJVNhWMoTYlxo96g.png" /><figcaption><strong>Output — 44</strong></figcaption></figure><p><em>The above output shows that studentized residuals are not greater than 3 (rule of thumb) in absolute value. Hence, there are no potential outliers.</em></p><p><strong>Finally, Adopt this model (Stored in R-object pm4) for making predictions.</strong></p><h3>8. Making Predictions -</h3><p>Now, It’s time to make prediction on test data set (unseen data) and check the performance of the model as follows -</p><pre><strong># Making Predictions</strong><br>prediction = pm4 %&gt;% predict(test.data)<br><br><strong># Checking performance by calculating R2 , RMSE and MAE</strong><br>data.frame( R2 = R2(prediction, test.data$Sales),<br>            RMSE = RMSE(prediction, test.data$Sales),<br>            MAE = MAE(prediction, test.data$Sales))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/805/1*f_diUU-84dshxmNUaTk9xQ.png" /><figcaption><strong>Output — 45</strong></figcaption></figure><p><strong>We obtain : R² = 0.9526385 , which indicates a best fit.</strong></p><p>Since, this result is based on only one test data set. Hence, we can not sure that the model will perform better on all unseen data. To be more confident in this respect, we will use the method of repeated K-fold cross validation to test the performance of model on different test data set.</p><p>This will be done as follows -</p><h3>9. Repeated 10-fold Cross Validation</h3><p>Before performing Cross validation, just remove that observation that is identified as outlier, i.e., the row that contains Sales = 1.6</p><pre><strong># Removing outlier, i.e., the row that contains Sales = 1.6</strong><br>data &lt;- data %&gt;% filter(Sales != 1.6)</pre><p>Now, Perform Cross Validation as follows -</p><pre><strong># Define training control</strong><br>set.seed(123)<br>train.control &lt;- trainControl(method = &quot;repeatedcv&quot;, <br>                              number = 10, repeats = 3)<br><strong># Train the model</strong><br>model_cv &lt;- train(Sales ~ poly(TV , 2) + poly(Radio , 3) + TV:Radio , data = data, method=&quot;lm&quot;,<br>                  trControl = train.control)<br><br><strong># Summarize the results</strong><br>print(model_cv)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/811/1*PnR6D_oeHMx9H6uqjoqcVQ.png" /></figure><h3>Great Result !</h3><p><strong>On an average, This Orthogonal Polynomial Regression Model (stored in R-object pm4) captures 93.69% variability available in the target (Sales). That is, 93.69% variability in Sales is due to the predictor “TV” and “Radio”. Rest variability is due to random causes or may be due to some other causes.</strong></p><h3>10. Conclusion -</h3><p>Finally, I want to conclude here. <strong>I started from fitting of Simple Linear Regression, then shown you what problem occurs with this model and then to get rid of that problem how you must proceed towards Multiple Linear Regression model. After that I have shown you how you will get an idea of how to proceed towards Orthogonal Polynomial Regression. I have also included different Statistical tests, Diagnostic plots, Diagnostic metrics to do the task of preparing a better basic model for predicting Sales on the basis of given Advertising budget for TV, Radio and Newspaper.</strong></p><p>Further, <strong>I want to mention here that this is not the end. It is just a starting and that’s why I have said earlier that I have prepared a better basic model for the given data set. There are several advanced algorithms such as fitting of splines (Parametric algorithm) and many non-parametric algorithms like Decision tree, Random forest, Support Vector Machine etc.. which may deal with curvilinear relationship and may give more accurate results. So you must learn and try these advanced algorithms further to improve and gain more accuracy as well as knowledge in this field</strong>.</p><p>Just do it once yourself for better understanding !</p><h3><strong>11. Task for you (If interested) -</strong></h3><p>If you want to do more practice, I recommend you to work on this <a href="https://www.kaggle.com/pranjalpandey12/advertising-tricky-data-set/download"><strong>Advertising Data Set</strong></a>. This data set requires some more analysis work related to Residual Plots. Before working on this data set just read the concepts of Residual Analysis from <a href="http://home.iitk.ac.in/~shalab/regression/Chapter4-Regression-ModelAdequacyChecking.pdf"><strong>here</strong></a><strong> </strong>(Page no :18 to 20) and then apply Simple, Multiple and polynomial Regression and analyze the Diagnostic plots.</p><h3>12. Information about Next Article -</h3><p>Just save the data that is completely outlier free obtained while performing cross validation and also save splitted data stored in <strong>train.data1</strong> and <strong>test.data</strong> to read my next article named <strong>Multicolinearity / Ridge / Lasso / Elastic-Net Regression using R</strong>. I will use the same outlier free data set in the next article to understand more concepts.</p><pre><em># Saving outlier free data set stored in R-object data</em><br>write.csv(data , &quot;Outlier Free Data Set.csv&quot; , row.names = F)<br><br><em># Saving outlier free train data set stored in R-object train.data1</em><br>write.csv(data , &quot;Outlier Free Data Set.csv&quot; , row.names = F)<br><br><em># Saving test data set stored in R-object test.data</em><br>write.csv(data , &quot;Outlier Free Data Set.csv&quot; , row.names = F)</pre><h3>Thanks for Reading my article.</h3><p>My kaggle Notebook — <a href="https://www.kaggle.com/pranjalpandey12/simple-to-multiple-and-polynomial-regression-in-r"><strong>click</strong></a></p><p><strong>If you find any mistake, just let me know.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=84b77f5673c5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/how-to-proceed-from-simple-to-multiple-and-polynomial-regression-in-r-84b77f5673c5">How to proceed from Simple to Multiple and Polynomial Regression in R</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating Informative and Decorative Simple Bar Chart in R (Detailed Explanation of Concepts and…]]></title>
            <link>https://medium.com/analytics-vidhya/creating-informative-and-decorative-simple-bar-chart-in-r-detailed-explanation-of-concepts-and-6cf45ef1fa75?source=rss-4fa9dfe37370------2</link>
            <guid isPermaLink="false">https://medium.com/p/6cf45ef1fa75</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[r]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Pranjal Pandey]]></dc:creator>
            <pubDate>Thu, 16 Jul 2020 13:40:10 GMT</pubDate>
            <atom:updated>2021-02-21T15:30:35.019Z</atom:updated>
            <content:encoded><![CDATA[<h3>Creating Informative and Decorative Simple Bar Chart in R</h3><h4>(Detailed <em>Explanation of Concepts and Codes)</em></h4><p><strong>Hey</strong>, Want to create such Simple Bar Chart? Read the Article (For Codes Go to the End of Article)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/449/1*XD_4fp70Y4P7o35M2wT-Yw.jpeg" /></figure><h4>Introduction -</h4><p>The main objective of this article is to create an <strong>Informative and Decorative Simple Bar Chart</strong>. This article consists of the following contents -</p><ol><li>A Problem Statement</li><li>Solution of Problem</li><li>What is Simple Bar Chart ?</li><li>When it is used ?</li><li>Parts of a Chart</li><li>How it is plotted in R ?</li><li>Save the chart for offline use</li><li>Some Important Points</li><li>All Useful Codes</li><li>References</li></ol><h4>1. Problem Statement -</h4><p>Suppose you have a data set in which there are a lot of features and one (or more) of them is (are) a categorical Feature. <strong>My question is :</strong> How such feature(s) is(are) analyzed visually one by one?</p><h4>2. Solution of Problem -</h4><p>Such feature(s) is(are) analyzed visually one by one with the help of <strong>Simple Bar Chart</strong>.</p><h4>3. Simple Bar Chart -</h4><p>Simple Bar Chart provide visual representation of a categorical feature at an instance. It generally have -</p><ol><li>Categories on X-axis</li><li>Numbers(frequency or relative frequency) of the respective category on Y-axis</li></ol><p>Such type of Simple Bar Chart is also known as <strong>Vertical Bar Chart</strong> or <strong>Column Chart</strong>.</p><h4>4. When it is used -</h4><p>Use this chart type when -</p><ol><li>You have to compare numbers(frequencies or relative frequencies) across a few categories of a categorical feature/variable.</li><li>The order of categories is not important.</li></ol><h4>5. Parts of a Simple Bar Chart -</h4><p>Before plotting a chart, one must understand the different parts of a chart. Mainly, there are six parts of a Simple Bar Chart -</p><ol><li>Main Title</li><li>Sub Title</li><li>Axes</li><li>Axis Titles</li><li>Axis labels</li><li>Data and Data Labels (Data Panel)</li></ol><p>All these parts are shown in the following figure -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*UQmeERt4UY80uIzaJyQ3Uw.jpeg" /></figure><h4>6. How it is plotted in R -</h4><p>It consists of following three steps -</p><ol><li>Create a Simple Bar Chart</li><li>Make it Informative</li><li>Make it Decorative</li></ol><p>Now, I will discuss each of these steps in detail one by one.</p><p><strong>Loading Data set -</strong></p><p>Before proceeding further, Let’s load the data set in R-session (<strong><em>I am using kaggle R-Notebook, You may use R-studio to work offline</em></strong>) by the following code -</p><blockquote>Want to Download the Data set for offline use, click <a href="https://www.kaggle.com/pranjalpandey12/data-for-bar-chart/download"><strong>here</strong></a></blockquote><pre><strong># Loading Data</strong><br>data = read.csv(&quot;../input/data-for-bar-chart/Data for Bar chart.csv&quot; , header = T)</pre><p><strong>Inspection of Data Set -</strong></p><p>See top 5-rows of the data set to make an idea about types of features as follows -</p><pre><strong># See top 5-rows of data</strong><br>head(data)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/833/1*xKlKiDy-bGbmuwNwWXwYfA.png" /><figcaption><strong>Output : 1</strong></figcaption></figure><p>From the above output, it is clear that there are two categorical features <strong>“Embarked”</strong> and <strong>“Survived”</strong>.</p><p>I will plot Simple Bar Chart for feature <strong>“Survived”</strong> and leave other feature <strong>“Embarked”</strong> for your practice.</p><p>Now, come back to the above discussed three steps, i.e.,</p><ol><li>Create a Simple Bar Chart</li><li>Make it Informative</li><li>Make it Decorative</li></ol><p><strong>1. Steps to create a Simple Bar Chart -</strong></p><ul><li><strong>Step — (i)</strong> : Create a frequency table (<strong>“table(column_name)” function is used</strong>) for Survived and assign this table in R-object “a”</li><li><strong>Step — (ii)</strong>: Create Simple Bar Chart by <strong>barplot(a)</strong> function, passing R-object already created in step — (i) as its argument and assign this plot in R-object “x” for further use.</li></ul><pre><strong># Step - (i)</strong><br><br><strong># Creating frequency table for Survived and assigning this table in R-object &quot;a&quot; </strong><br>a &lt;- table(data$Survived)<br><br><strong># Print object a</strong><br>a</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*vaXavqHq2iyJl3catLcpPw.png" /><figcaption><strong>Output : 2</strong></figcaption></figure><p><strong>The above output shows the frequency table for “Survived”. It means that in “Survived” column of “data” 0 occurs 549 times and 1 occurs 342 times.</strong></p><pre><strong># Step - (ii)</strong><br><br><strong># Creating Simple bar chart and assign it in R-object &quot;x&quot; for further use</strong><br>x &lt;- barplot(a)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*XNHjdVjPydOjE7XFtjh6DA.png" /><figcaption><strong>Output : 3</strong></figcaption></figure><p>Well, Simple Bar Chart is now ready but it is incomplete. <strong>Why ?</strong></p><blockquote>Let us think, you are the person who know about the data set and if you create such type of chart, you may be able to know what this chart is explaining. But if you show the same chart to a person who does not know about your data set, then he can not understand your chart. And that’s why there is a need to make it more informative.</blockquote><p><strong>2. Steps to make the above obtained Simple Bar Chart more Informative -</strong></p><ul><li><strong>Step-(i)</strong> : <strong>Adjust Limit of Y-axis</strong> : use <strong>ylim = c(lower_limit,upper_limit)</strong> argument in <strong>barplot()</strong> function.</li><li><strong>Step-(ii)</strong> : <strong>Rename Labels of X-axis : </strong>use <strong>names.arg = c(“label_for_0” , “label_for_1”)</strong> argument in <strong>barplot()</strong> function.</li><li><strong>Step-(iii)</strong> : <strong>Add all titles (Main title, Sub title , Axis titles)</strong> : use <strong>title()</strong> function after <strong>barplot()</strong> function.</li></ul><p><strong>Arguments for title() function -</strong></p><ol><li><strong>Add Main Title</strong> : use <strong>main = “Main_title”</strong> argument in <strong>title()</strong> function.</li><li><strong>Add Sub Title</strong> : use <strong>sub = “Sub_title”</strong> argument in <strong>title()</strong> function.</li><li><strong>Add X-axis Title</strong> : use <strong>xlab = “X-axis_title”</strong> argument in <strong>title()</strong> function.</li><li><strong>Add Y-axis Title</strong> : use <strong>ylab = “Y-axis_title”</strong> argument in <strong>title()</strong> function.</li></ol><ul><li><strong>Step-(iv)</strong> : <strong>Add data labels</strong> : use <strong>text()</strong> function after <strong>barplot()</strong> function.</li></ul><p><strong>Arguments for text(d1 , d2 , d3) function -</strong></p><ol><li><strong>d1</strong> : Represents the R-object containing the barplot. (In my case it is : x)</li><li><strong>d2</strong> : Represents the position of Labels, i.e., where we want to write the labels in plot. In my case it is : a + 16 , where a : R-object containing frequency table of the underlying categorical feature and i have added 16 to a , which gives a suitable position for labels for all categories.</li></ol><blockquote><strong>Note :</strong> The value : 16 varies from plot to plot and is obtained by hit and trial method, i.e., First try a + 1 and plot the graph. If it is well, then good , Otherwise Increase it until you get a suitable place for labels.</blockquote><p>3. <strong>d3</strong> : Represents labels what you want to write as a character. Use <strong>labels = as.character(a)</strong> in place of d3, where a : R-object containing frequency table.</p><pre><strong># Step-(i)</strong><br><br><strong># Increasing limit of y-axis</strong><br>x &lt;- barplot(a , ylim = c(0,600))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*mxKW6wyJbPn2t0lR6vz8lA.png" /><figcaption><strong>Output : 4</strong></figcaption></figure><pre><strong># Step-(ii)</strong><br><br><strong># Renaming X-axis Labels</strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*8NDVGydqz8RRorjAJN_c6Q.png" /><figcaption><strong>Output : 5</strong></figcaption></figure><pre><strong># Step-(iii)</strong><br><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;))<br><br><strong># Add all type of titles</strong><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; )</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*3IIjrGezRsW-dX_yxDd3Og.png" /><figcaption><strong>Output : 6</strong></figcaption></figure><pre><strong># Step-(iv)</strong><br><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;))<br><br><strong># Add all type of titles</strong><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; )<br>      <br><strong># Add data labels</strong><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/841/1*T3ZmYIrV2LiVK4lA-cUiTg.png" /><figcaption><strong>Output : 7</strong></figcaption></figure><p><strong>This is an Informative Simple Bar Chart. It explains all about the feature “Survived”.</strong></p><p>Now, Create an Informative simple bar chart in a single shot as follows -</p><pre><strong># Just make an informative simple bar chart in a single shot</strong><br><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;))<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; )    <br><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/841/1*T3ZmYIrV2LiVK4lA-cUiTg.png" /><figcaption><strong>Output : 8</strong></figcaption></figure><h4>Good !</h4><blockquote>There is something missed in the above chart. <strong>What ?</strong></blockquote><blockquote>Hmm…. I think “Decoration”. Hahahaha…..</blockquote><blockquote><strong>Why decorative chart ?</strong></blockquote><blockquote>Generally, People love with colors. A decorative chart as well as informative attracts the attention of people more than an informative chart. Hence, there is also a need of Decorative Chart.</blockquote><p><strong>3. Steps to make the above obtained informative Simple Bar Chart as a Decorative chart -</strong></p><ul><li><strong>Step-(i)</strong> : <strong>Rotate Y-axis labels for a better view</strong> : use <strong>las = 1</strong> as argument in <strong>barplot()</strong> function.</li><li><strong>Step-(ii)</strong> : <strong>Decorate Rectangular Bars</strong> : 3-more arguments are used in <strong>barplot()</strong> function.</li></ul><p><strong>Argument Details -</strong></p><ol><li><strong>col :</strong> To colour Rectangular Bars</li><li><strong>border</strong> : To colour border of Rectangular Bars</li><li><strong>density</strong> : To adjust the density of lines in each bars</li></ol><ul><li><strong>Step-(iii)</strong> : <strong>Decorate Axis Labels</strong> : 4-more arguments are used in <strong>barplot()</strong> function.</li></ul><p><strong>Argument Details -</strong></p><ol><li><strong>cex.axis</strong> : To adjust size of Y-axis Labels</li><li><strong>cex.names</strong> : To adjust size of X-axis Labels</li><li><strong>font.axis</strong> : To change font style of Axes (font = 1 for Normal, font = 2 for Bold, font = 3 for Italic, font = 4 for Bold and Italic)</li><li><strong>col.axis</strong> : To change the color of Axis Labels</li></ol><ul><li><strong>Step-(iv)</strong> : <strong>Decorate Titles</strong> : 9-more arguments are used in <strong>title()</strong> function.</li></ul><p><strong>Argument Details -</strong></p><ol><li><strong>cex.main</strong> : To adjust size of Main title</li><li><strong>cex.sub</strong> : To adjust size of Sub title</li><li><strong>cex.lab</strong> : To adjust size of Axis titles</li><li><strong>font.main</strong> : To change font style of Main title</li><li><strong>font.sub</strong> : To change font style of Sub title</li><li><strong>font.lab</strong> : To change font style of Axis titles</li><li><strong>col.main</strong> : To change the color of Main title</li><li><strong>col.sub</strong> : To change the color of Sub title</li><li><strong>col.lab</strong> : To change the color of Axis titles</li></ol><ul><li><strong>Step-(v)</strong> : <strong>Decorate Data Labels</strong> : 3-more arguments are used in <strong>text()</strong> function.</li></ul><p><strong>Argument Details -</strong></p><ol><li><strong>cex</strong> : To adjust size of Data Labels</li><li><strong>font</strong> : To change font style of Data Labels</li><li><strong>col</strong> : To change the color of Data Labels</li></ol><pre><strong># Step-(i)</strong><br><br><strong># Rotate Y-axis labels</strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; )    <br><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*YChOWGYLFwxqpp-cMOHKiw.png" /><figcaption><strong>Output : 9</strong></figcaption></figure><p><strong>Notice, Y-axis labels are now horizontal.</strong></p><pre><strong># Step-(ii)<br><br># Using &quot;col&quot; argument in barplot() function</strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; )    <br><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/839/1*FgylYqo6TnOqPJXHaI6m-w.png" /><figcaption><strong>Output : 10</strong></figcaption></figure><blockquote><strong>Notice -</strong></blockquote><blockquote><strong>1. Color of bars are spring green.</strong></blockquote><blockquote><strong>2. Border of Bars are black.</strong></blockquote><blockquote><strong>3. Density of lines in each Bar is very high.</strong></blockquote><pre><strong># Step-(ii)</strong><br><br><strong># Using &quot;border&quot; argument in barplot() function</strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; )    <br><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/834/1*9jyCBEqzb9Dsahr4Rwk_ig.png" /><figcaption><strong>Output : 11</strong></figcaption></figure><blockquote><strong>Notice -</strong></blockquote><blockquote><strong>1. Border of Bars are also spring green.</strong></blockquote><blockquote><strong>2. Density of lines in each Bar is very high.</strong></blockquote><pre><strong># Step-(ii)</strong><br><br><strong># Using &quot;density&quot; argument in barplot() function</strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; )    <br><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/845/1*8zC8O8tCFcJkUXrhmYxr7Q.png" /><figcaption><strong>Output : 12</strong></figcaption></figure><p><strong>Notice, Density of lines in each Bar is low.</strong></p><pre><strong># Step-(iii)</strong><br><br><strong># Using &quot;cex.axis&quot; argument in barplot() function</strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; )    <br><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*Dx_dpMVC0zQKFPq9lRII4A.png" /><figcaption><strong>Output : 13</strong></figcaption></figure><p><strong>Notice the difference in the size of labels on Y-axis.</strong></p><pre><strong># Step-(iii)</strong><br><br><strong># Using &quot;cex.names&quot; argument in barplot() function</strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; )    <br><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*NHHMIuv-uxUqZgf262CQDg.png" /><figcaption><strong>Output : 14</strong></figcaption></figure><p><strong>Notice the difference in the size of labels on X-axis.</strong></p><pre><strong># Step-(iii)</strong><br><br><strong># Using &quot;font.axis&quot; argument in barplot() function</strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3 , font.axis = 2)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; )    <br><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/1*g73Yb7-AG39cfImuo0EEiw.png" /><figcaption><strong>Output : 15</strong></figcaption></figure><p><strong>Notice Axis Labels are now in bold.</strong></p><pre><strong># Step-(iii)</strong><br><br><strong># Using &quot;col.axis&quot; argument in barplot() function</strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3 , font.axis = 2 , col.axis = &quot;dark blue&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; )    <br><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*gJZKkD6o5By-u0PaNyedww.png" /><figcaption><strong>Output : 16</strong></figcaption></figure><p><strong>Notice Axis Labels are now in dark blue.</strong></p><pre><strong># Step-(iv)</strong><br><br><strong># Using &quot;cex&quot; argument in title() function for all titles</strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3 , font.axis = 2 , col.axis = &quot;dark blue&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; , cex.main = 1.8 , cex.lab = 1.5 , cex.sub = 1.15)    <br><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*4Un56ehN4ArCClIrsVZU1Q.png" /><figcaption><strong>Output : 17</strong></figcaption></figure><p><strong>Notice the difference in the size of all titles.</strong></p><pre><strong># Step-(iv)</strong><br><br><strong># Using &quot;font&quot; argument in title() function for all titles</strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3 , font.axis = 2 , col.axis = &quot;dark blue&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; , cex.main = 1.8 , cex.lab = 1.5 , cex.sub = 1.15 , font.main = 2 , font.lab =2 , font.sub = 2)    <br><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/849/1*DhthbEpgeycLcV5v6wygfQ.png" /><figcaption><strong>Output : 18</strong></figcaption></figure><p><strong>Notice all titles are now in bold.</strong></p><pre><strong># Step-(iv)</strong><br><br><strong># Using &quot;col&quot; argument in title() function for all titles</strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3 , font.axis = 2 , col.axis = &quot;dark blue&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; , cex.main = 1.8 , cex.lab = 1.5 , cex.sub = 1.15 , font.main = 2 , font.lab =2 , font.sub = 2 , col.main = &quot;red&quot; , col.lab = &quot;darkgreen&quot; , col.sub = &quot;magenta&quot;)    <br><br>text(x , a+16, labels = as.character(a))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/841/1*7LEsoNE2V2u1BPWz_ug1jg.png" /><figcaption><strong>Output : 19</strong></figcaption></figure><p><strong>Notice the colors of titles.</strong></p><pre><strong># Step-(v)</strong><br><br><strong># Using &quot;cex&quot; argument in text() function </strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3 , font.axis = 2 , col.axis = &quot;dark blue&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; , cex.main = 1.8 , cex.lab = 1.5 , cex.sub = 1.15 , font.main = 2 , font.lab =2 , font.sub = 2 , col.main = &quot;red&quot; , col.lab = &quot;darkgreen&quot; , col.sub = &quot;magenta&quot;)    <br><br>text(x , a+16, labels = as.character(a) ,cex = 1.4 )</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/839/1*f9D8Ad8dd_JzIRAG_7kIdw.png" /><figcaption><strong>Output : 20</strong></figcaption></figure><p><strong>Notice the difference in the size of Data Labels.</strong></p><pre><strong># Step-(v)</strong><br><br><strong># Using &quot;font&quot; argument in text() function </strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3 , font.axis = 2 , col.axis = &quot;dark blue&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; , cex.main = 1.8 , cex.lab = 1.5 , cex.sub = 1.15 , font.main = 2 , font.lab =2 , font.sub = 2 , col.main = &quot;red&quot; , col.lab = &quot;darkgreen&quot; , col.sub = &quot;magenta&quot;)    <br><br>text(x , a+16, labels = as.character(a) ,cex = 1.4 , font = 2)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/833/1*mGub112MNpBvY5NXxy4aTw.png" /><figcaption><strong>Output : 21</strong></figcaption></figure><p><strong>Notice Data Labels are now in bold.</strong></p><pre><strong># Step-(v)</strong><br><br><strong># Using &quot;col&quot; argument in text() function </strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3 , font.axis = 2 , col.axis = &quot;dark blue&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; , cex.main = 1.8 , cex.lab = 1.5 , cex.sub = 1.15 , font.main = 2 , font.lab =2 , font.sub = 2 , col.main = &quot;red&quot; , col.lab = &quot;darkgreen&quot; , col.sub = &quot;magenta&quot;)    <br><br>text(x , a+16, labels = as.character(a) ,cex = 1.4 , font = 2 , col = &quot;orange&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*q3oMmcsHm9RlM6jsrWNcqw.png" /><figcaption><strong>Output : 22</strong></figcaption></figure><p><strong>Notice the color of Data Labels.</strong></p><p><strong>Just create an Informative and well-Decorative Simple Bar Chart in one shot as follows -</strong></p><pre><strong># Just create an Informative and well-Decorative Simple Bar Chart in one shot </strong><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3 , font.axis = 2 , col.axis = &quot;dark blue&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; , cex.main = 1.8 , cex.lab = 1.5 , cex.sub = 1.15 , font.main = 2 , font.lab =2 , font.sub = 2 , col.main = &quot;red&quot; , col.lab = &quot;darkgreen&quot; , col.sub = &quot;magenta&quot;)    <br><br>text(x , a+16, labels = as.character(a) , cex = 1.4 , font = 2 , col = &quot;orange&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*hxrmNqhtXdTttZ6-9n26pA.png" /><figcaption><strong>Output : 23</strong></figcaption></figure><h4>Great !</h4><p>This is an Informative and well-Decorative Simple Bar Chart.</p><h4>7. Save the chart for offline use -</h4><pre><strong># Saving the Chart for offline use</strong><br>jpeg(&quot;BarChart.jpg&quot;)<br><br><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3 , font.axis = 2 , col.axis = &quot;dark blue&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; , cex.main = 1.8 , cex.lab = 1.5 , cex.sub = 1.15 , font.main = 2 , font.lab =2 , font.sub = 2 , col.main = &quot;red&quot; , col.lab = &quot;darkgreen&quot; , col.sub = &quot;magenta&quot;)    <br><br>text(x , a+16, labels = as.character(a) , cex = 1.4 , font = 2 , col = &quot;orange&quot;)<br><br><br>dev.off()</pre><p>Download the file from output section in the data panel of your Notebook.</p><h4>8. Some important points -</h4><ol><li>Till now, I have made Simple Bar Chart using frequency. <strong>You may also use Relative Frequency</strong>. In this case, only one difference is in creating the R-object containing frequency table. <strong>Now, you have to create a R-object containing relative frequency table and this is achieved as follows -</strong></li></ol><pre><strong># Creating Relative frequency table and assigning it in a R-object</strong><em> </em><strong>&quot;a&quot;</strong></pre><pre>a &lt;- table(data$Survived)/length(data$Survived)<br><br><em># Print the object a</em><br>a</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/1*H8QybywBmaTpI8-ovAZu7A.png" /><figcaption><strong>Output : 24</strong></figcaption></figure><blockquote><strong>The above output shows the Relative Frequency Table for Survived feature.</strong></blockquote><blockquote><strong>All other steps of plotting remains same as in the above case.</strong></blockquote><p>2. X-axis and Y-axis can be interchanged, i.e., we may represent different categories on Y-axis and the numbers of the respective category on X-axis. Such chart is known as <strong>Horizontal Simple Bar Chart</strong>. To plot it, just pass one more argument <strong>horiz = TRUE</strong> in <strong>barplot()</strong> function and make some changes accordingly, such as X-axis and Y-axis title, position of data labels etc. (<strong>Do it yourself</strong>)</p><p>3. Horizontal Simple Bar Chart is prefered over Vertical Simple Bar Chart when there are a large number of categories to be compared.</p><p>4. Height(Length) of bars is proportional to the numbers (frequency/relative frequency) that they represent.</p><p>5. The width of bars are arbitrary.</p><p>6. Bar charts are usually scaled so that all the data can fit properly on the chart. (This concept is used while you are working on a graph-paper)</p><p>7. The values of arguments like cex, cex.main, cex.sub, cex.lab, cex.axis, cex.names, density are chosen by hit and trial method. (No need to remember such values)</p><p>8. The values for <strong>font</strong> and <strong>col</strong> depends on your choice. You may use <strong>Hex code of colous</strong> also to choose your favourite color. Example : <strong>Hex code for deep pink — #FF1493</strong></p><blockquote>To know Hex code of colors, click <a href="https://www.rapidtables.com/web/color/RGB_Color.html"><strong>here</strong></a></blockquote><h4>9. All Useful Codes -</h4><p>If you deal with frequency table and want to create a Vertical Simple Bar Chart, use following code -</p><pre><strong># Creating a Vertical Simple Bar Chart using frequency table </strong><br>a &lt;- table(data$Survived)<br><br>x &lt;- barplot(a , ylim = c(0,600) , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3 , font.axis = 2 , col.axis = &quot;dark blue&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; , cex.main = 1.8 , cex.lab = 1.5 , cex.sub = 1.15 , font.main = 2 , font.lab =2 , font.sub = 2 , col.main = &quot;red&quot; , col.lab = &quot;darkgreen&quot; , col.sub = &quot;magenta&quot;)    <br><br>text(x , a+16, labels = as.character(a) , cex = 1.4 , font = 2 , col = &quot;orange&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/825/1*bsW2Cy0641XVmYSrTdDowA.png" /><figcaption><strong>Output : 25</strong></figcaption></figure><p>If you deal with relative frequency table and want to create a Vertical Simple Bar Chart, use following code -</p><pre><strong># Creating a Vertical Simple Bar Chart using relative frequency table but adjust the limit of Y-axis , Otherwise it may create problem</strong><br>a &lt;- table(data$Survived)/length(data$Survived)<br><br>x &lt;- barplot(a , <strong>ylim = c(0,0.7)</strong> , names.arg = c(&quot;Not Survived&quot;,&quot;Survived&quot;) , las = 1 , col = &quot;spring green&quot; , border = &quot;spring green&quot; , density = 12 , cex.axis = 1.2 , cex.names = 1.3 , font.axis = 2 , col.axis = &quot;dark blue&quot;)<br><br>title(main = &quot;Distribution of Survived&quot; , sub = &quot;Fig : Bar Chart&quot; , xlab = &quot;Info about Survival of a Passenger -----&gt;&quot; , ylab = &quot;Frequency -----&gt;&quot; , cex.main = 1.8 , cex.lab = 1.5 , cex.sub = 1.15 , font.main = 2 , font.lab =2 , font.sub = 2 , col.main = &quot;red&quot; , col.lab = &quot;darkgreen&quot; , col.sub = &quot;magenta&quot;)    <br><br>text(x , a+16, labels = as.character(a) , cex = 1.4 , font = 2 , col = &quot;orange&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*hIIEDq9Au0RrY-yvz4Eeyw.png" /><figcaption><strong>Output : 26</strong></figcaption></figure><h4>10. References -</h4><p>I have read these things from some websites and Documentation, and I have compiled all gained knowledge in this single article. If you wish you may see the followings —</p><ul><li><a href="https://www.rdocumentation.org/packages/graphics/versions/3.6.2/topics/barplot">https://www.rdocumentation.org/packages/graphics/versions/3.6.2/topics/barplot</a></li><li><a href="http://howtoinr.weebly.com/customize-labels1.html">http://howtoinr.weebly.com/customize-labels1.html</a></li><li><a href="http://www.sthda.com/english/wiki/add-titles-to-a-plot-in-r-software">http://www.sthda.com/english/wiki/add-titles-to-a-plot-in-r-software</a></li><li><a href="https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781783988785/6/ch06lvl1sec69/displaying-values-on-top-of-or-next-to-the-bars">https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781783988785/6/ch06lvl1sec69/displaying-values-on-top-of-or-next-to-the-bars</a></li><li><a href="https://www.r-graph-gallery.com/210-custom-barplot-layout.html">https://www.r-graph-gallery.com/210-custom-barplot-layout.html</a></li></ul><p>Just do practice yourself !</p><p><a href="https://www.kaggle.com/pranjalpandey12/informative-and-decorative-simple-bar-chart-in-r"><strong>Link</strong></a> of my kaggle notebook.</p><p>If you find any mistake or you have any suggestions please let me know, you are always welcome.</p><h4>Thanks a lot for reading my article !</h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6cf45ef1fa75" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/creating-informative-and-decorative-simple-bar-chart-in-r-detailed-explanation-of-concepts-and-6cf45ef1fa75">Creating Informative and Decorative Simple Bar Chart in R (Detailed Explanation of Concepts and…</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Step by step procedure to perform Simple Linear Regression with Statistical Analysis in R]]></title>
            <link>https://medium.com/analytics-vidhya/step-by-step-procedure-to-perform-simple-linear-regression-with-statistical-analysis-in-r-340ef0a392d6?source=rss-4fa9dfe37370------2</link>
            <guid isPermaLink="false">https://medium.com/p/340ef0a392d6</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[r-programming]]></category>
            <dc:creator><![CDATA[Pranjal Pandey]]></dc:creator>
            <pubDate>Sat, 11 Jul 2020 15:40:17 GMT</pubDate>
            <atom:updated>2021-02-21T15:20:10.236Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KugvVm_M2RJ-AwZi.png" /><figcaption><strong>Image by </strong><a href="https://www.educba.com/"><strong>educba</strong></a></figcaption></figure><ol><li><strong>Introduction</strong></li></ol><p>Hello, This is my first article on Medium. In this article, I am going to show you how Simple Linear Regression, which is a very basic of Linear Regression algorithm frequently used in Machine Learning, is applied in solving real life problems. For this purpose, I’ll use a data set in which — “<strong>I have to predict Salary of an individual(say,a person) on the basis of one explanatory variable Years of Experience</strong>.” I am going to use <strong>Kaggle R-Notebook</strong> to do the above mentioned task. You may use R-studio in your PC to work offline.</p><p><strong>2. Loading Data set</strong></p><p>Link to download the data set for offline use — <a href="https://www.kaggle.com/rohankayan/years-of-experience-and-salary-dataset/download"><strong>https://www.kaggle.com/rohankayan/years-of-experience-and-salary-dataset/download</strong></a></p><p>Run the following command to create a R-object that will contain the whole data set -</p><pre><strong># Loading Data</strong><br>data = read.csv(&quot;../input/years-of-experience-and-salary-dataset/Salary_Data.csv&quot; , header = T)</pre><p>Now, Data will be successfully Loaded in the current Session and stored in object “data”.</p><p><strong>3. Loading Required Libraries</strong></p><pre>library(tidyverse)<br>library(caret)<br>library(MASS)<br>library(lmtest)<br>library(olsrr)<br>library(broom)</pre><p><strong>4. Exploring Data set</strong></p><pre><strong>#<em> </em>Random Inspection of Dataset<br></strong>sample_n(data,5)<strong># See the following output</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/839/0*sVgYNj6Nu3v2DXnD.png" /><figcaption><strong>Output-1</strong></figcaption></figure><p>In the given data set, The Target “Salary” and Predictor “YearsExperience” both are Quantitative Variable. Hence, we can apply Regression on the given data set. Now,</p><p><strong>Question is :</strong> Which type of Regression (Linear or Non-Linear) we must apply ?</p><p><strong>My suggestion :</strong> First of all try Linear Regression since it is simpler than Non-Linear. If it does not fit properly, then go for Non-Linear Regression.</p><p><strong>Next Question :</strong> There are many types of Linear Regression. Which will be better for this given data set ?</p><p><strong>Answer :</strong> Since I have only one Predictor so the possibility is either fit Simple Linear Regression or Polynomial (generally, Orthogonal Polynomial to avoid multi-collinearity) Regression in one variable.</p><p><strong>Question :</strong> Which one of the above two Linear Regression will perform better ?</p><p><strong>Answer :</strong> Plot Scatter Diagram between Predictor (on X-axis) and Target (on Y-axis). This will give a best idea.</p><p>Let’s do it as follows -</p><pre><strong># Scatter Plot<br></strong>plot(data$YearsExperience,data$Salary)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/0*9u0z7vrm-MLARavr.png" /><figcaption><strong>Output-2</strong></figcaption></figure><blockquote><em>The above plot shows Linear relationship between Predictor and Target. So Simple Linear Regression will perform better.</em></blockquote><p>Further, if you think that Polynomial model may perform better than Simple Linear Regression, Fit the Polynomial model, since if the polynomial model is not best for the given data set, the higher order terms will automatically become insignificant, which in turn shows that the best model for the given data set is Simple Linear Regression. (I will see you the same in later stage.)</p><p>SLR is based on certain assumptions which are as follows -</p><ol><li><strong>Linearity :</strong> Must be a linear relationship between Predictor and Target.</li><li><strong>Normality :</strong> Errors must be normally distributed.</li><li><strong>Homoscadasticity :</strong> Errors must have constant variance.</li><li><strong>Auto-correlation :</strong> Errors must be uncorrelated.(Errors must be independent)</li><li>Errors must have mean Zero.</li></ol><p>The above assumptions must be satisfied, because if any of the above assumptions is not satisfied, then we can not sure about the validity and reliability of obtained results. In such situation, either try to satisfy assumptions or adopt any other method of regression to get reliable result.</p><p>Some of these assumptions are either checked before fitting the model and some are after fitting as the case will be. This will be more clear in further steps.</p><p><strong>5. Checking Linearity assumptions by</strong></p><ol><li><strong>Visualization</strong> : Scatter plot between Predictor and Target helps us. (Already plotted above and found linearity.)</li><li><strong>Statistical Tests</strong> : Pearson — Correlation test helps. Use <strong>“cor.test()”</strong> function.</li></ol><p>Let’s try it.</p><pre><strong># Correlation Test</strong><br>cor.test(data$YearsExperience , data$Salary)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/0*GjnV6dD-l4XkeKNc.png" /><figcaption><strong>Output-3</strong></figcaption></figure><blockquote><em>Since, </em><strong><em>p-value is less than 0.05</em></strong><em> and sample correlation is </em><strong><em>0.9782416</em></strong><em>. Hence, High significant correlation (Highly Linearly Related) exists in the population.</em></blockquote><p>Rest assumptions will be checked after fitting the model.</p><p><strong>6. Fitting Model</strong></p><p>Before fitting, Split the given data set into <strong>train and test data</strong>, <strong>fit the model</strong> and <strong>obtain summary of model</strong> as follows -</p><pre><strong># Randomly Split the data into training and test set</strong></pre><pre>set.seed(123)<br>training.samples &lt;- data$Salary %&gt;%<br>  createDataPartition(p = 0.6, list = FALSE)<br>train.data  &lt;- data[training.samples, ]<br>test.data &lt;- data[-training.samples, ]<strong><br></strong></pre><pre><strong># Fit model</strong></pre><pre>model &lt;- lm(Salary ~ YearsExperience , data = train.data)</pre><pre>summary(model)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/0*S9WxK8YYr4kIKtpG.png" /><figcaption><strong>Output-4</strong></figcaption></figure><p>The above output shows that -</p><ol><li>Both the coefficients are statistically significant since p-value &lt; 0.05</li><li>The model is also statistically significant since p-value: 2.098e-13 &lt; 0.05</li><li>Adjusted R-squared: 0.9504 , This shows that 95.04% of the variation available in the given data set (in Salary) is explained by this Simple Linear Regression Model. <strong>Rest 5% variation in Salary is due to some other predictors (as post, age of the individuals) or due to random cause</strong>.</li></ol><p><strong>7. Checking Rest Assumptions</strong></p><p><strong>Normality assumption by -</strong></p><ol><li><strong>Visualization</strong> : Q-Q Plot of residuals helps us.</li><li><strong>Statistical Test</strong> : Shapiro-wilk-test on residuals helps us. Use <strong>“shapiro.test()”</strong> function.</li></ol><p>Let’s try it.</p><pre><strong># Visualization<br></strong>qqnorm(model$residuals)<br>qqline(model$residuals , col = &quot;steelblue&quot;, lwd = 2)<strong>  </strong></pre><pre><strong># Statistical Test<br></strong>shapiro.test(model$residuals)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/0*_2vnUE9Awjz7W5uQ.png" /><figcaption><strong>Output-5</strong></figcaption></figure><blockquote><em>Since p-value (0.1123) &gt; 0.05 , Hence, Accept the null hypothesis, i.e., Errors are normally distributed.</em></blockquote><p><strong>Homoscedasticity assumption by -</strong></p><ol><li><strong>Visualization</strong> : Residuals vs fitted values Plot helps us.</li><li><strong>Statistical Test</strong> : Breusch Pagan test helps us. Use <strong>bptest()</strong> function.</li></ol><p>Let’s do it.</p><pre><strong># Visualization<br></strong>plot(model$fitted.values , model$residuals)</pre><pre><strong># Statistical Test<br></strong>bptest(model)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/0*HnPqd5wE-7E5srQw.png" /><figcaption><strong>Output-6</strong></figcaption></figure><blockquote><em>Since p-value (0.0633) &gt; 0.05 , Hence, Accept null hypothesis, i.e., Errors have constant variance.</em></blockquote><p><strong>Auto-correlation assumption by -</strong></p><ol><li><strong>Visualization</strong> : Residual vs Order of taking observations.</li><li><strong>Statistical Test</strong> : Durbin-watson Test helps. Use <strong>durbinWatsonTest()</strong> function.</li></ol><p>Let’s try it.</p><pre><strong># Visualization<br></strong>plot(1:20 , model$residuals)</pre><pre><strong># Statistical Test<br></strong>dwtest(model,  alternative = c(&quot;two.sided&quot;))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/0*wlO9sMXAHgBLfjBK.png" /><figcaption><strong>Output-7</strong></figcaption></figure><blockquote><em>Since p-value (0.1349) &gt; 0.05 ; Hence, Accept null hypothesis, i.e., Errors are uncorrelated.</em></blockquote><p><strong>Assumption of Mean of errors -</strong></p><pre><strong># Calculating mean of errors<br></strong>mean(model$residuals)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/0*gw32QXrt7cb1ZBQA.png" /><figcaption><strong>Output-8</strong></figcaption></figure><blockquote><em>The above output shows that mean of errors is approximately zero.</em></blockquote><p>Now, all the assumptions of Simple Linear Regression has been satisfied.</p><p>One more task is to detect whether there is any <strong>influential point</strong> in the given data set. Because if influential point is present, then the result obtained above will become unreliable. Hence, there is a need for detection of influential point.</p><p><strong>Influential point detection by -</strong></p><ol><li><strong>Visualization</strong> : Cook’s distance plot helps us.</li><li><strong>Statistical Measure</strong> : Cook’s distance helps.</li></ol><p>Run the following command.</p><pre><strong># Visualization (Cook&#39;s distance plot)<br></strong>plot(model , 4)</pre><pre><strong># Statistical Measure<br></strong>model_dm &lt;- augment(model)</pre><pre><strong># Checking Highest Cook&#39;s distance<br></strong>max(model_dm$.cooksd)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/834/0*41-ChtdaZDf-oKJz.png" /><figcaption><strong>Output-9</strong></figcaption></figure><blockquote><strong><em>Maximum cook’s distance</em></strong><em> is </em><strong><em>0.186480962574431</em></strong></blockquote><blockquote><em>Since, for any observation in train data set, cook’s distance is not greater than 0.5, Hence, there is no influential point in the given data set.</em></blockquote><p><strong>8. Making Predictions</strong></p><p>Now, Simple Linear Regression model is ready. Apply it on test data set to check its performance on unseen data set, i.e., Determine how well this model will perform on unseen data.</p><p>This will again be done by two methods -</p><ol><li><strong>Visualization</strong> : predicted salary vs actual salary plot for test data set helps.</li><li><strong>Statistical Measure</strong> : R² , RMSE , MAE helps.</li></ol><pre><strong># Making prediction<br></strong>prediction &lt;- model %&gt;% predict(test.data)</pre><pre><strong># Visualization<br></strong>plot(test.data$Salary , prediction)<br>abline(lm(prediction ~ Salary, data = test.data), col = &quot;blue&quot;)</pre><pre><strong># Statistical Measure<br></strong>data.frame( R2 = R2(prediction, test.data$Salary),<br>            RMSE = RMSE(prediction, test.data$Salary),<br>            MAE = MAE(prediction, test.data$Salary))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/0*Z_8OVCtLwJVM0u-p.png" /><figcaption><strong>Output-10</strong></figcaption></figure><blockquote><em>We obtain : R² = 0.9701604 , which indicates a best fit.</em></blockquote><p>Since, this result is based on only one test data set. Hence, we can not sure that the model will perform better on all unseen data. To be more confident in this respect, we will use the method of K-fold cross validation to test the performance of model on different test data set.</p><p>This will be done as follows -</p><pre><strong># Define training control<br></strong>set.seed(123)<br>train.control &lt;- trainControl(method = &quot;repeatedcv&quot;, <br>                              number = 4, repeats = 3)<strong><br># Train the model<br></strong>model_cv &lt;- train(Salary ~ YearsExperience , data = data, method=&quot;lm&quot;,<br>                  trControl = train.control)</pre><pre><strong># Summarize the results<br></strong>print(model_cv)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/0*a_ba45GaXia08wFu.png" /><figcaption><strong>Output-11</strong></figcaption></figure><p><strong>Great Result !</strong></p><blockquote><strong><em>On an average, This Simple Linear Regression Model captures 97.29 % variability available in the target (Salary). That is, 97.29 % variability in Salary is due to the predictor “YearsExperience”. Rest variability is due to some other causes (like post/age of the individuals,etc.) or due to random causes.</em></strong></blockquote><p>Now, prepare a polynomial model as i have said earlier as follows-</p><pre><strong># Fitting Orthogonal polynomial model</strong></pre><pre>model_poly &lt;- lm(Salary ~ poly(YearsExperience,2) , data = train.data )</pre><pre><strong># Summarizing model<br></strong>summary(model_poly)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/811/0*iVLGNSyqV-H79Vb7.png" /><figcaption><strong>Output-12</strong></figcaption></figure><blockquote><em>The above output shows that </em><strong><em>the second order of predictor is not statistically significant</em></strong><em> since p-value (0.531) &gt; 0.05 ; Hence, Don’t fit polynomial model and go with Simple Linear Regression only.</em></blockquote><p><strong>Want to see my Kaggle R-Notebook : </strong>Click <a href="https://www.kaggle.com/pranjalpandey12/salary-vs-yearsexperience-slr-model"><strong>here</strong></a></p><h3>Thanks for reading my first article.</h3><p>If you find any mistake or you have any suggestions please let me know, you are always welcome.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=340ef0a392d6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/step-by-step-procedure-to-perform-simple-linear-regression-with-statistical-analysis-in-r-340ef0a392d6">Step by step procedure to perform Simple Linear Regression with Statistical Analysis in R</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>