<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aditya Jetely on Medium]]></title>
        <description><![CDATA[Stories by Aditya Jetely on Medium]]></description>
        <link>https://medium.com/@ajetely?source=rss-2d5085b9e12f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*QvUBG85IrF0aUOedV9UIIw.jpeg</url>
            <title>Stories by Aditya Jetely on Medium</title>
            <link>https://medium.com/@ajetely?source=rss-2d5085b9e12f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 19:13:45 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ajetely/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Digits_Recognition_Accuracy_Analysis]]></title>
            <link>https://medium.com/@ajetely/digits-recognition-accuracy-analysis-6efc3668da57?source=rss-2d5085b9e12f------2</link>
            <guid isPermaLink="false">https://medium.com/p/6efc3668da57</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[matplotlib]]></category>
            <category><![CDATA[numpy]]></category>
            <category><![CDATA[scikit-learn]]></category>
            <dc:creator><![CDATA[Aditya Jetely]]></dc:creator>
            <pubDate>Tue, 13 Oct 2020 09:21:43 GMT</pubDate>
            <atom:updated>2020-10-13T09:21:43.540Z</atom:updated>
            <content:encoded><![CDATA[<h3>Handwritten Digits Prediction Analysis</h3><p>A Brief Analysis of the Handwritten digits prediction results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GQLhc7ZB04HKB_PB" /><figcaption>Photo by <a href="https://unsplash.com/@antoine1003?utm_source=medium&amp;utm_medium=referral">Antoine Dautry</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Introduction</h4><p>In this notebook I have performed the analysis of Digits recognition accuracy, the dataset used in this notebook comes inbuilt with the scikit-learn library so it needs not to be downloaded.</p><h4>Problem Statement</h4><p>The Digits data set of the scikit-learn library provides numerous data-sets that are useful for testing many problems of data analysis and prediction of the results. Some Scientist claims that it predicts the digit accurately 95% of the times. Perform data Analysis to accept or reject this Hypothesis</p><h4>Importing Libraries</h4><pre>import numpy as np<br>import matplotlib<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>%matplotlib inline<br><br>sns.set_style(&#39;darkgrid&#39;)<br>matplotlib.rcParams[&#39;font.size&#39;] = 14<br>matplotlib.rcParams[&#39;figure.figsize&#39;] = (9, 5)<br>matplotlib.rcParams[&#39;figure.facecolor&#39;] = &#39;#00000000&#39;</pre><h4>Loading Data</h4><pre>from sklearn import datasets<br>digits = datasets.load_digits()<br>digits.data</pre><pre>array([[ 0.,  0.,  5., ...,  0.,  0.,  0.],<br>       [ 0.,  0.,  0., ..., 10.,  0.,  0.],<br>       [ 0.,  0.,  0., ..., 16.,  9.,  0.],<br>       ...,<br>       [ 0.,  0.,  1., ...,  6.,  0.,  0.],<br>       [ 0.,  0.,  2., ..., 12.,  0.,  0.],<br>       [ 0.,  0., 10., ..., 12.,  1.,  0.]])</pre><h4>Understanding the data</h4><pre>digits.data.shape</pre><pre>(1797, 64)</pre><p>We can see that our dataset has 1797 images and all the images are 8x8 in dimension.</p><pre>digits.target.shape</pre><pre>(1797,)</pre><p>We have 1797 labels for our target that contains values from 0 to 9.</p><h4>Let&#39;s see some Images and labels</h4><pre>plt.figure(figsize=(15,9))<br>for index, (image, label) in enumerate(zip(digits.data[0:6], digits.target[0:6])):<br>    plt.subplot(1, 6, index + 1)<br>    plt.imshow(np.reshape(image, (8,8)), cmap=&#39;gray&#39;)<br>    plt.title(f&#39;Training: {label}&#39;)</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*QCBNdkgn6vsL4zNMtfcGXQ.png" /></figure><h4>Splitting Data into Training and Test Sets</h4><p>Lets split our data so it can be used for training our model and later for testing purpose.</p><pre>from sklearn.model_selection import train_test_split<br>X_train, X_test, y_train, y_test = train_test_split(digits.data, digits.target, test_size=0.2, random_state=42)</pre><p>Let’s see our train data a bit.</p><pre>X_train</pre><pre>array([[ 0.,  0.,  3., ..., 13.,  4.,  0.],<br>       [ 0.,  0.,  9., ...,  3.,  0.,  0.],<br>       [ 0.,  0.,  0., ...,  6.,  0.,  0.],<br>       ...,<br>       [ 0.,  0.,  9., ..., 16.,  2.,  0.],<br>       [ 0.,  0.,  1., ...,  0.,  0.,  0.],<br>       [ 0.,  0.,  1., ...,  1.,  0.,  0.]])</pre><pre>y_train</pre><pre>array([6, 0, 0, ..., 2, 7, 1])</pre><h4>Modelling our dataset</h4><pre>from sklearn.linear_model import LogisticRegression # Loading the regressor</pre><pre>model = LogisticRegression() # creating instance of regression class</pre><pre>model.fit(X_train, y_train) # fitting the model</pre><pre>LogisticRegression()</pre><p>Making individual predictions.</p><pre>model.predict(X_test[0].reshape(1,-1))</pre><pre>array([6])</pre><p>Making random digits predictions.</p><pre>model.predict(X_test[0:6])</pre><pre>array([6, 9, 3, 7, 2, 1])</pre><pre>model.predict(X_test[15:21])</pre><pre>array([2, 3, 7, 8, 8, 4])</pre><pre>model.predict(X_test[45:51])</pre><pre>array([1, 3, 0, 6, 5, 5])</pre><p>Let’s visualize some of the results of the dataset.</p><pre>plt.figure(figsize=(15,9))<br>for index, (image, label) in enumerate(zip(X_test[0:6], y_test[0:6])):<br>    plt.subplot(1, 6, index + 1)<br>    plt.imshow(np.reshape(image, (8,8)), cmap=&#39;gray&#39;)<br>    plt.title(f&#39;Training: {label}&#39;)</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*hK8L1G3eqgRubXnji6tigw.png" /></figure><pre>plt.figure(figsize=(15,9))<br>for index, (image, label) in enumerate(zip(X_test[15:20], y_test[15:20])):<br>    plt.subplot(1, 5, index + 1)<br>    plt.imshow(np.reshape(image, (8,8)), cmap=&#39;gray&#39;)<br>    plt.title(f&#39;Training: {label}&#39;)</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*_nGJ128s1OcsQeIXLMLa-A.png" /></figure><p>Making predictions on the entire dataset</p><pre>predictions = model.predict(X_test)<br>predictions</pre><pre>array([6, 9, 3, 7, 2, 1, 5, 2, 5, 2, 1, 9, 4, 0, 4, 2, 3, 7, 8, 8, 4, 3,<br>       9, 7, 5, 6, 3, 5, 6, 3, 4, 9, 1, 4, 4, 6, 9, 4, 7, 6, 6, 9, 1, 3,<br>       6, 1, 3, 0, 6, 5, 5, 1, 3, 5, 6, 0, 9, 0, 0, 1, 0, 4, 5, 2, 4, 5,<br>       7, 0, 7, 5, 9, 5, 5, 4, 7, 0, 4, 5, 5, 9, 9, 0, 2, 3, 8, 0, 6, 4,<br>       4, 9, 1, 2, 8, 3, 5, 2, 9, 0, 4, 4, 4, 3, 5, 3, 1, 3, 5, 9, 4, 2,<br>       7, 7, 4, 4, 1, 9, 2, 7, 8, 7, 2, 6, 9, 4, 0, 7, 2, 7, 5, 8, 7, 5,<br>       7, 5, 0, 6, 6, 4, 2, 8, 0, 9, 4, 6, 9, 9, 6, 9, 0, 5, 5, 6, 6, 0,<br>       6, 4, 3, 9, 3, 8, 7, 2, 9, 0, 6, 5, 3, 6, 5, 9, 9, 8, 4, 2, 1, 3,<br>       7, 7, 2, 2, 3, 9, 8, 0, 3, 2, 2, 5, 6, 9, 9, 4, 1, 2, 4, 2, 3, 6,<br>       4, 8, 5, 9, 5, 7, 8, 9, 4, 8, 1, 5, 4, 4, 9, 6, 1, 8, 6, 0, 4, 5,<br>       2, 7, 1, 6, 4, 5, 6, 0, 3, 2, 3, 6, 7, 1, 9, 1, 4, 7, 6, 5, 8, 5,<br>       5, 1, 5, 2, 8, 8, 9, 9, 7, 6, 2, 2, 2, 3, 4, 8, 8, 3, 6, 0, 9, 7,<br>       7, 0, 1, 0, 4, 5, 1, 5, 3, 6, 0, 4, 1, 0, 0, 3, 6, 5, 9, 7, 3, 5,<br>       5, 9, 9, 8, 5, 3, 3, 2, 0, 5, 8, 3, 4, 0, 2, 4, 6, 4, 3, 4, 5, 0,<br>       5, 2, 1, 3, 1, 4, 1, 1, 7, 0, 1, 5, 2, 1, 2, 8, 7, 0, 6, 4, 8, 8,<br>       5, 1, 8, 4, 5, 8, 7, 9, 8, 6, 0, 6, 2, 0, 7, 9, 8, 9, 5, 2, 7, 7,<br>       1, 8, 7, 4, 3, 8, 3, 5])</pre><h4>Measuring Model Performance</h4><pre>score = model.score(X_test, y_test)<br>score</pre><pre>0.9694444444444444</pre><p>Our model has an accuracy of around 97% which is quite good.</p><h4>Conclusion</h4><ul><li>The accuracy obtained for the model is above 95% so the hypothesis provided in the problem statement holds True.</li><li>The accuracy is high for this dataset because it is relatively small and not a real-world dataset.</li><li>We can also use SVM for the same process and obtain results.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6efc3668da57" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[video-games-sales-analysis]]></title>
            <link>https://medium.com/@ajetely/video-games-sales-analysis-4b8b0b86eb3b?source=rss-2d5085b9e12f------2</link>
            <guid isPermaLink="false">https://medium.com/p/4b8b0b86eb3b</guid>
            <category><![CDATA[seaborn]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Aditya Jetely]]></dc:creator>
            <pubDate>Tue, 13 Oct 2020 04:01:13 GMT</pubDate>
            <atom:updated>2021-12-21T12:02:56.364Z</atom:updated>
            <content:encoded><![CDATA[<h3>Video Games Sales Analysis</h3><p>An attempt to extract useful information from the Video Games Sales dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*A4_vLxMHWrOiGQGi" /><figcaption>Photo by <a href="https://unsplash.com/@kellysikkema?utm_source=medium&amp;utm_medium=referral">Kelly Sikkema</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Introduction</h4><p>In this project, I will use the video games sales data for analysis. I will try to gain useful insights from this dataset and improve my skills while doing so. This analysis is part of the Zero to Pandas course offered by <a href="https://jovian.ml/?utm_source=medium">Jovian ML</a> and FreecodeCamp.</p><h4>Downloading the Dataset</h4><p>The dataset can be obtained from this Kaggle link: <a href="https://www.kaggle.com/gregorut/videogamesales/notebooks">https://www.kaggle.com/gregorut/videogamesales/notebooks</a> This dataset contains a list of video games with sales greater than 100,000 copies. It has sales records for North America, Europe, Japan, and the rest of the world in millions. Though we can conclude that some countries must have been excluded but it is not clear.</p><p>Let’s begin by downloading the data, and listing the files within the dataset.</p><pre>dataset_url = &#39;https://www.kaggle.com/gregorut/videogamesales/notebooks&#39;</pre><pre>import opendatasets as od<br>od.download(dataset_url)</pre><pre>Please provide your Kaggle credentials to download this dataset. Learn more: http://bit.ly/kaggle-creds<br>Your Kaggle username: aditya7jetely<br>Your Kaggle Key: ········<br><br><br>100%|██████████| 381k/381k [00:00&lt;00:00, 46.9MB/s]<br><br>Downloading videogamesales.zip to ./videogamesales</pre><p>The dataset has been downloaded and extracted.</p><pre>data_dir = &#39;./videogamesales&#39;</pre><pre>import os<br>os.listdir(data_dir)</pre><pre>[&#39;vgsales.csv&#39;]</pre><h4>Data Preparation and Cleaning</h4><p>This is the main step before the data analysis portion. In this step I will try to make the dataset ready for analysis, i.e. I would choose useful columns, handle missing or incorrect values and would perform parsing if necessary.</p><pre>import pandas as pd</pre><p>Loading the dataset into a pandas data frame.</p><pre>video_sales_df = pd.read_csv(&#39;./videogamesales/vgsales.csv&#39;)<br>video_sales_df.head() # Having a look at the first five rows of the dataframe</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*axCcKv0cnhVzeNnA_i2O4Q.png" /></figure><p>Evaluating the shape of our data frame.</p><pre>video_sales_df.shape</pre><pre>(16598, 11)</pre><p>We can see that our dataset has 16598 rows and 11 columns</p><p>List of columns in our data frame.</p><pre>video_sales_df.columns</pre><pre>Index([&#39;Rank&#39;, &#39;Name&#39;, &#39;Platform&#39;, &#39;Year&#39;, &#39;Genre&#39;, &#39;Publisher&#39;, &#39;NA_Sales&#39;,<br>       &#39;EU_Sales&#39;, &#39;JP_Sales&#39;, &#39;Other_Sales&#39;, &#39;Global_Sales&#39;],<br>      dtype=&#39;object&#39;)</pre><p>As the number of columns in our dataset is not very high we can use all the columns to some extend.</p><p>Now let&#39;s see some basic information about our dataset.</p><pre>video_sales_df.info()</pre><pre>&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;<br>RangeIndex: 16598 entries, 0 to 16597<br>Data columns (total 11 columns):<br> #   Column        Non-Null Count  Dtype  <br>---  ------        --------------  -----  <br> 0   Rank          16598 non-null  int64  <br> 1   Name          16598 non-null  object <br> 2   Platform      16598 non-null  object <br> 3   Year          16327 non-null  float64<br> 4   Genre         16598 non-null  object <br> 5   Publisher     16540 non-null  object <br> 6   NA_Sales      16598 non-null  float64<br> 7   EU_Sales      16598 non-null  float64<br> 8   JP_Sales      16598 non-null  float64<br> 9   Other_Sales   16598 non-null  float64<br> 10  Global_Sales  16598 non-null  float64<br>dtypes: float64(6), int64(1), object(4)<br>memory usage: 1.4+ MB</pre><p>As we can observe in the above result that the year column has the data type as a float which should have been datetime, so we must convert it into datetime by the help of pandas functions.</p><pre>video_sales_df[&#39;Year&#39;] =  pd.to_datetime(video_sales_df[&#39;Year&#39;], format=&#39;%Y&#39;, errors=&#39;coerce&#39;)</pre><p>Now we can confirm the data type of Year column.</p><pre>video_sales_df.dtypes</pre><pre>Rank                     int64<br>Name                    object<br>Platform                object<br>Year            datetime64[ns]<br>Genre                   object<br>Publisher               object<br>NA_Sales               float64<br>EU_Sales               float64<br>JP_Sales               float64<br>Other_Sales            float64<br>Global_Sales           float64<br>dtype: object</pre><p>We can see that the data type of our Year column is now datetime.</p><p>Now let us have some stats about the numeric columns of our data set.</p><pre>video_sales_df.describe()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*cNkpSpaV6nkfSdpEPJsUsw.png" /></figure><p>As we see that our Rank column is also numeric but it is not meant to perform statistics on it so we must remove it from the statistics info.</p><pre>video_sales_df_copy = video_sales_df.copy() # A copy dataframe to remove Rank column from statistical description</pre><pre>video_sales_df_copy.drop(columns = &#39;Rank&#39;, inplace = True) # droping the rank column</pre><pre>video_sales_df_copy.describe() # Now we dont have the Rank column in the statistical info</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*4SEab81leg31csp7-vB0Pw.png" /></figure><p>There doesn’t seem to be any problem with our numeric columns as of now. It is to be noted that all the sales are in millions.</p><h4>Exploratory Analysis and Visualization</h4><p>Before we can ask interesting questions about the dataset it would help to understand what the different columns of our dataset look like.</p><p>Let’s begin by importingmatplotlib.pyplot and seaborn.</p><pre>import seaborn as sns<br>import matplotlib<br>import matplotlib.pyplot as plt<br>%matplotlib inline<br><br>sns.set_style(&#39;darkgrid&#39;)<br>matplotlib.rcParams[&#39;font.size&#39;] = 14<br>matplotlib.rcParams[&#39;figure.figsize&#39;] = (9, 5)<br>matplotlib.rcParams[&#39;figure.facecolor&#39;] = &#39;#00000000&#39;</pre><h4>Genre</h4><p>Let&#39;s explore the genre column of our data frame to see what is the majority of games whose sales record are present.</p><pre>video_sales_df.Genre.value_counts()</pre><pre>Action          3316<br>Sports          2346<br>Misc            1739<br>Role-Playing    1488<br>Shooter         1310<br>Adventure       1286<br>Racing          1249<br>Platform         886<br>Simulation       867<br>Fighting         848<br>Strategy         681<br>Puzzle           582<br>Name: Genre, dtype: int64</pre><pre>plt.pie(video_sales_df.Genre.value_counts(), labels=video_sales_df.Genre.value_counts().index, autopct=&#39;%1.0f%%&#39;, pctdistance=1.1, labeldistance=1.2)<br>plt.title(&#39;Video Game Genre Share&#39;)<br>plt.ylabel(&#39; &#39;)<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*mdBtnZMVoyHumBwb1r4avA.png" /></figure><p>Action games have the highest count in the dataset will the Puzzle games have the least count. It could be due to the fact that many people are interested in video games that involve action in them as many consider puzzle-based games to be boring.</p><h4>Platform</h4><p>Let&#39;s analyse the number of games based on the platform.</p><pre>video_sales_df.Platform.value_counts()</pre><pre>DS      2163<br>PS2     2161<br>PS3     1329<br>Wii     1325<br>X360    1265<br>PSP     1213<br>PS      1196<br>PC       960<br>XB       824<br>GBA      822<br>GC       556<br>3DS      509<br>PSV      413<br>PS4      336<br>N64      319<br>SNES     239<br>XOne     213<br>SAT      173<br>WiiU     143<br>2600     133<br>NES       98<br>GB        98<br>DC        52<br>GEN       27<br>NG        12<br>WS         6<br>SCD        6<br>3DO        3<br>TG16       2<br>GG         1<br>PCFX       1<br>Name: Platform, dtype: int64</pre><pre>plt.figure(figsize = (15,9))<br>plt.title(&#39;Number of different games per game platforms&#39;)<br>sns.countplot(y=&#39;Platform&#39;, data=video_sales_df)<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*E6DSGaIH5i3QbIRQFEKKbA.png" /></figure><p>The most games are available for the DS and PS2 platforms which might mean that DS and PS2 are used by more customers But we can&#39;t be sure about that.</p><h4>Publisher</h4><p>Let&#39;s analyze the Publisher column to see which company publishes the what number of games. Let&#39;s consider publishers with game counts above 100 only.</p><pre>plt.figure(figsize=(15,12))<br>publishers = video_sales_df.Publisher.value_counts()<br>publishers = publishers[publishers.values &gt; 100]<br>publishers.plot(kind = &#39;barh&#39;)<br>plt.xlabel(&#39;Count&#39;)<br>plt.ylabel(&#39;Publishing Company&#39;)<br>plt.title(&#39;Publishing companies with more than 100 games&#39;)<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*foCcaJNYUS-v2OTqDZ5dag.png" /></figure><p>We see that Electronics Arts is the undisputed leader in terms of the video games sale count. One of the reason may be the popularity of its sports-based games like FIFA and Cricket</p><h4>Year</h4><pre>video_sales_df[&#39;Year&#39;].value_counts().sort_values(ascending=False)<br>video_sales_df[&#39;year_only&#39;] = video_sales_df[&#39;Year&#39;].dt.year<br>video_sales_df[&#39;year_only&#39;] = video_sales_df[&#39;year_only&#39;].astype(int, errors=&#39;ignore&#39;)</pre><pre>plt.figure(figsize = (15,9))<br>plt.title(&#39;Number of different games per game platforms&#39;)<br>sns.countplot(y=&#39;year_only&#39;, data=video_sales_df)<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*zIC8j5I3WEGZPf4vNke5yQ.png" /></figure><p>We can see that the sales were the highest in the years 2006 to 2011 and after that, we can see a downfall of the total sales.</p><h4>Sales</h4><p>Let&#39;s analyse the numeric sales column for North America, Europe, Japan, and the Rest of the World.</p><pre>fig, axes = plt.subplots(2,2, figsize= (18, 14))<br>axes[0, 0].scatter(video_sales_df.year_only, video_sales_df.NA_Sales, color = &#39;b&#39;)<br>axes[0, 0].set_title(&#39;North America Video Game Sales&#39;)<br>axes[0, 0].set_xlabel(&#39;Year&#39;)<br>axes[0, 0].set_ylabel(&#39;Sales in Million&#39;)<br>axes[0, 1].scatter(video_sales_df.year_only, video_sales_df.EU_Sales, color = &#39;r&#39;)<br>axes[0, 1].set_title(&#39;Europe Video Game Sales&#39;)<br>axes[0, 1].set_xlabel(&#39;Year&#39;)<br>axes[0, 1].set_ylabel(&#39;Sales in Million&#39;)<br>axes[1, 0].scatter(video_sales_df.year_only, video_sales_df.JP_Sales, color = &#39;g&#39;)<br>axes[1, 0].set_title(&#39;Japan Video Game Sales&#39;)<br>axes[1, 0].set_xlabel(&#39;Year&#39;)<br>axes[1, 0].set_ylabel(&#39;Sales in Million&#39;)<br>axes[1, 1].scatter(video_sales_df.year_only, video_sales_df.Other_Sales)<br>axes[1, 1].set_title(&#39;Other county Video Game Sales&#39;)<br>axes[1, 1].set_xlabel(&#39;Year&#39;)<br>axes[1, 1].set_ylabel(&#39;Sales in Million&#39;)<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*ebjLjw_Bv2Jj7mUNgkddWg.png" /></figure><p>In the above visualization, we saw the video game sales in the four different regions, the outliers in all the four regions are very few. We can see that sales have a higher number in the North American region.</p><h4>Asking and Answering Questions</h4><p>We’ve already gained several insights about the dataset and video game sales in different regions, let us further improve our insights by asking some interesting questions.</p><h4>Q1: What are the top five game genres?</h4><pre>video_sales_df.Genre.value_counts().head(5)</pre><pre>Action          3316<br>Sports          2346<br>Misc            1739<br>Role-Playing    1488<br>Shooter         1310<br>Name: Genre, dtype: int64</pre><pre>video_sales_df.Genre.value_counts().head(5).plot(kind = &#39;bar&#39;, color = &#39;g&#39;)<br>plt.xlabel(&#39;Genre&#39;)<br>plt.ylabel(&#39;Count&#39;)<br>plt.title(&#39;Top 5 game genre&#39;)<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*FRX18CYOzueCdZmG_9PuUw.png" /></figure><p><strong>Action</strong> and <strong>Sports</strong> are the top-grossing genres of video games followed by <strong>miscellaneous</strong>, <strong>role-playing</strong> and <strong>shooting</strong> games. This also shows why Electronics Art has a large share of sales because it mainly makes sports-based games.</p><h4>Q2: Which region has the largest share in the global video games sales?</h4><pre>Total_Video_Games_Sales = video_sales_df.Global_Sales.sum()<br>North_America_Sales_percent = (video_sales_df.NA_Sales.sum()*100)/Total_Video_Games_Sales<br>Europe_Sales_percent = (video_sales_df.EU_Sales.sum()*100)/Total_Video_Games_Sales<br>Japan_Sales_percent = (video_sales_df.JP_Sales.sum()*100)/Total_Video_Games_Sales<br>Other_Region_Sales_percent = (video_sales_df.Other_Sales.sum()*100)/Total_Video_Games_Sales</pre><pre>plt.figure(figsize=(16,9))<br>plt.pie([North_America_Sales_percent, Europe_Sales_percent, Japan_Sales_percent, Other_Region_Sales_percent], labels=[&#39;North America&#39;, &#39;Europe&#39;, &#39;Japan&#39;, &#39;Other Regions&#39;], autopct=&#39;%1.0f%%&#39;, pctdistance=1.1, labeldistance=1.2)<br>plt.title(&#39;Video Game Genre Share&#39;)<br>plt.ylabel(&#39; &#39;)<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*YvXAUwNkCuZwKT4IIVemsw.png" /></figure><p>Through this pie chart, we see that the major shareholder is <strong>North America</strong>. Though this result can be a bit biased because of the nature of the survey in which more importance might have been given to the North American area.</p><h4>Q3: What are the top 5 highest-grossing games in North America?</h4><pre>video_sales_NA_sorted_df = video_sales_df.sort_values(by = [&#39;NA_Sales&#39;], ascending = False)<br>video_sales_NA_sorted_df.head()[[&#39;Name&#39;, &#39;NA_Sales&#39;]]</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*9gE9CX7UyxROVfp55AAvMA.png" /></figure><p>The top-grossing game in North America is <strong>Wii Sports</strong> with a total sales of 41.49 million in North America followed by Super Mario Bros., Duck Hunt, Tetris and Mario Kart Wii.</p><h4>Q4: Which game has the highest sales in the other region(Excluding North America, Europe and Japan)?</h4><pre>video_sales_Other_sorted_df = video_sales_df.sort_values(by = [&#39;Other_Sales&#39;], ascending = False)<br>video_sales_NA_sorted_df.head(1)[[&#39;Name&#39;, &#39;Other_Sales&#39;]]</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*D2yvEUZKrOezh8tr7Bc3Bg.png" /></figure><p><strong>GTA: San Andreas</strong> has the highest sales in the other regions which are 10.57 million.</p><h4>Q5: Which game has the highest difference in their sales in North America and Europe?</h4><pre>video_sales_df[&#39;NA_EU_Sales_Diff&#39;] = abs(video_sales_df[&#39;NA_Sales&#39;] - video_sales_df[&#39;EU_Sales&#39;])<br>video_sales_df.head()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*ApC5zHBESkDGNwdpRyDcWg.png" /></figure><pre>video_sales_df.sort_values(by = [&#39;NA_EU_Sales_Diff&#39;], ascending = False).head(1)[[&#39;Name&#39;, &#39;NA_EU_Sales_Diff&#39;]]</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*Ifi8khbMu67SbNRPnXdqKg.png" /></figure><p><strong>Duck Hunt</strong> has the highest difference in their sales when the comparison is made between North America and Europe.</p><h4>Inferences and Conclusion</h4><ul><li>Based on this dataset we cant say that equal importance was given to all the regions of the world as it might be the case that the data was collected mostly from North American stores.</li><li>Games in the Action genre tend to fetch the most revenue followed by Sports based games.</li><li>Electronic Arts are the top-grossing publisher clearly because of their supper popular sports-based games.</li><li>GTA: San Andreas is the top-grossing game in the other Regions of the World.</li><li>Top-grossing game in North America is Wii Sports.</li><li>Duck Hunt has the highest difference in the sales made in Europe and North America which shows it is more popular in North America when compared with Europe.</li><li>These stats can be a bit biased based on the nature of data collection.</li></ul><h4>References and Future Work</h4><p>This dataset has a lot of information about the sales trend of video games which can be further exploited to gain useful insights.</p><ul><li>A region-based analysis can be performed in-depth to compare video games sales trend</li></ul><p>References:</p><ul><li>Stack Overflow Developer Survey: <a href="https://insights.stackoverflow.com/survey">https://insights.stackoverflow.com/survey</a></li><li>Pandas user guide: <a href="https://pandas.pydata.org/docs/user_guide/index.html">https://pandas.pydata.org/docs/user_guide/index.html</a></li><li>Matplotlib user guide: <a href="https://matplotlib.org/3.3.1/users/index.html">https://matplotlib.org/3.3.1/users/index.html</a></li><li>Seaborn user guide &amp; tutorial: <a href="https://seaborn.pydata.org/tutorial.html">https://seaborn.pydata.org/tutorial.html</a></li><li>open datasets Python library: <a href="https://github.com/JovianML/opendatasets">https://github.com/JovianML/opendatasets</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4b8b0b86eb3b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[EDA of the Iris Dataset]]></title>
            <link>https://medium.com/data-science/eda-of-the-iris-dataset-190f6dfd946d?source=rss-2d5085b9e12f------2</link>
            <guid isPermaLink="false">https://medium.com/p/190f6dfd946d</guid>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[seaborn]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-analysis]]></category>
            <dc:creator><![CDATA[Aditya Jetely]]></dc:creator>
            <pubDate>Mon, 12 Oct 2020 16:08:32 GMT</pubDate>
            <atom:updated>2020-10-12T18:33:35.865Z</atom:updated>
            <content:encoded><![CDATA[<h3>Yet Another Iris EDA</h3><h4>An attempt to extract useful insights from the super popular iris dataset</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HWeiO-tBxKeIvpVp" /><figcaption>Photo by <a href="https://unsplash.com/@zmachacek?utm_source=medium&amp;utm_medium=referral">Zdeněk Macháček</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>In this notebook, I have done the Exploratory Data Analysis of the famous <strong>Iris</strong> dataset and tried to gain useful insights from the data.</p><p>The features present in the dataset are:</p><ul><li>Sepal Width</li><li>Sepal Length</li><li>Petal Width</li><li>Petal Length</li></ul><h4>Importing Libraries</h4><pre>import pandas as pd<br>import numpy as np<br>import matplotlib<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>%matplotlib inline<br><br>sns.set_style(&#39;darkgrid&#39;)<br>matplotlib.rcParams[&#39;font.size&#39;] = 14<br>matplotlib.rcParams[&#39;figure.figsize&#39;] = (9, 5)<br>matplotlib.rcParams[&#39;figure.facecolor&#39;] = &#39;#00000000&#39;</pre><h4>Loading Data</h4><pre>iris_df = pd.read_csv(&quot;IRIS.csv&quot;)</pre><h4>Understanding the Data</h4><pre>iris_df.head()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*eDuHWp61UCA0sovwQxFTrA.png" /></figure><pre>iris_df.shape</pre><pre>(150, 5)</pre><p>We observe from this above line that our dataset has 150 rows and 5 columns</p><pre>iris_df.columns</pre><pre>Index([&#39;sepal_length&#39;, &#39;sepal_width&#39;, &#39;petal_length&#39;, &#39;petal_width&#39;,<br>       &#39;species&#39;],<br>      dtype=&#39;object&#39;)</pre><pre># Further inspecting the dataframe<br>def inspect_data(data):<br>    return pd.DataFrame({&quot;Data Type&quot;:data.dtypes,&quot;No of Levels&quot;:data.apply(lambda x: x.nunique(),axis=0), &quot;Levels&quot;:data.apply(lambda x: str(x.unique()),axis=0)})<br>inspect_data(iris_df)</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*njMAKDy1JvsCo_fj82Ib9Q.png" /></figure><p>Let&#39;s gain some more information about our dataset</p><pre>iris_df.info()</pre><pre>&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;<br>RangeIndex: 150 entries, 0 to 149<br>Data columns (total 5 columns):<br> #   Column        Non-Null Count  Dtype  <br>---  ------        --------------  -----  <br> 0   sepal_length  150 non-null    float64<br> 1   sepal_width   150 non-null    float64<br> 2   petal_length  150 non-null    float64<br> 3   petal_width   150 non-null    float64<br> 4   species       150 non-null    object <br>dtypes: float64(4), object(1)<br>memory usage: 6.0+ KB</pre><p>We see all the columns of our data set are free from null values and four of them are numeric while one is a categorical column which also is seen by the dtypes function.</p><pre>iris_df.dtypes</pre><pre>sepal_length    float64<br>sepal_width     float64<br>petal_length    float64<br>petal_width     float64<br>species          object<br>dtype: object</pre><h4>Statistical Insights</h4><pre>iris_df.describe(include=&#39;all&#39;)</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*sRkS-vQqDNNOgHfPvy3Fpg.png" /></figure><pre>iris_df.describe() #Performing just for numeric columns</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*kIWldzTpZKqqWCZYCm4Gdg.png" /></figure><h4>Checking for Duplicate values</h4><pre>iris_df.duplicated().sum() # Total no of duplicates in the dataset</pre><pre>3</pre><pre>iris_df[iris_df.duplicated()] #This shows the actual duplicate rows</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*NKWQWshtbzJy0iXxDfAIyg.png" /></figure><p>There are 3 duplicates in our dataset but removing them might break the balance of our dataset as we see that our dataset is balanced as in the describe function for the species column we saw the frequency of 50 which can be further proved by the following line of code.</p><pre>iris_df[&#39;species&#39;].value_counts()</pre><pre>Iris-setosa        50<br>Iris-versicolor    50<br>Iris-virginica     50<br>Name: species, dtype: int64</pre><p>So we can let the duplicate values as they are and proceed.</p><h4>Checking missing values</h4><p>We confirmed through the info function that our dataset is free from null values but let us further justify it.</p><pre>iris_df.isnull().sum(axis=0)</pre><pre>sepal_length    0<br>sepal_width     0<br>petal_length    0<br>petal_width     0<br>species         0<br>dtype: int64</pre><h4>Estimating Skewness and Kurtosis</h4><pre>iris_df.skew()</pre><pre>sepal_length    0.314911<br>sepal_width     0.334053<br>petal_length   -0.274464<br>petal_width    -0.104997<br>dtype: float64</pre><pre>iris_df.kurt()</pre><pre>sepal_length   -0.552064<br>sepal_width     0.290781<br>petal_length   -1.401921<br>petal_width    -1.339754<br>dtype: float64</pre><h4>Visualisations on target column</h4><p>Our target column is the species column because, in the end, we want to predict the correct species of flower.</p><pre>plt.title(&#39;Species Count&#39;)<br>sns.countplot(iris_df[&#39;species&#39;]);</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*ONiQPl_sSGoEF9H_1ZX3lg.png" /></figure><p>This further tells that our dataset is balanced with equal records for all three species.</p><h4>Visualising relations between variables</h4><p>Let&#39;s compare the sepal length and sepal width columns.</p><pre>plt.title(&#39;Comparison between sepal width and length&#39;)<br>sns.scatterplot(iris_df[&#39;sepal_length&#39;], iris_df[&#39;sepal_width&#39;]);</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*ui4QH7-jamJkDh6HH8TeDw.png" /></figure><p>Let&#39;s compare them again but this time lets compare on the basis on species as well.</p><pre>plt.figure(figsize=(16,9))<br>plt.title(&#39;Comparison between sepal width and length on the basis of species&#39;)<br>sns.scatterplot(iris_df[&#39;sepal_length&#39;], iris_df[&#39;sepal_width&#39;], hue = iris_df[&#39;species&#39;], s= 50);</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*PznqkcMDWJuEqsnT9ub5yQ.png" /></figure><p>From the above visualization, we can tell that the iris-setosa species has smaller sepal length but higher width. While we see Versicolor lies in almost middle for length as well as width. While Virginica has larger sepal lengths and smaller sepal widths.</p><p>Let&#39;s compare the petal length and petal width columns.</p><pre>plt.title(&#39;Comparison between petal width and length&#39;)<br>sns.scatterplot(iris_df[&#39;petal_length&#39;], iris_df[&#39;petal_width&#39;]);</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*2l-NF2qXOevoHMm6YMIDqA.png" /></figure><p>We can see two separate clusters but not sure about the species so let&#39;s bring the species into the equation as well.</p><pre>plt.figure(figsize=(16,9))<br>plt.title(&#39;Comparison between petal width and length on the basis of species&#39;)<br>sns.scatterplot(iris_df[&#39;petal_length&#39;], iris_df[&#39;petal_width&#39;], hue = iris_df[&#39;species&#39;], s= 50);</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*H14FetgHxIaAcRUF1eA3VA.png" /></figure><p>We see that setosa has the smallest petal length as well as petal widths, while Versicolor has average petal length and petal width while the virginica species has the highest petal length as well as petal width.</p><p>Now let&#39;s visualize all the columns relationship using pair plots.</p><pre>sns.pairplot(iris_df,hue=&quot;species&quot;,height=3);</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*BTApZqK7DmzU0-FhCwsEYQ.png" /></figure><p>Pair plot represents the relationship between our target and the variables. We can see that the setosa species has a large difference in its characteristics when compared to the other species, it has smaller petal width and length while its sepal width is high and its sepal length is low. Similar kind of conclusions can be drawn for the other species like the Versicolor species usually have average dimensions whether it is sepal or pedal. While virginica has high pedal width and length while it has small sepal width but large sepal length.</p><h4>Differentiating Numerical and Categorical Variables</h4><pre>categorical_columns = iris_df.dtypes[iris_df.dtypes == &#39;object&#39;].index<br>print(&#39;No of categorical fetures:&#39;,len(categorical_columns),&#39;\n&#39;)<br>print(categorical_columns)<br>print(&#39;*&#39;*100)<br><br>numerical_columns = iris_df.dtypes[iris_df.dtypes != &#39;object&#39;].index<br>print(&#39;No of numerical fetures:&#39;,len(numerical_columns),&#39;\n&#39;)<br>print(numerical_columns)</pre><pre>No of categorical fetures: 1 <br><br>Index([&#39;species&#39;], dtype=&#39;object&#39;)<br>****************************************************************************************************<br>No of numerical fetures: 4 <br><br>Index([&#39;sepal_length&#39;, &#39;sepal_width&#39;, &#39;petal_length&#39;, &#39;petal_width&#39;], dtype=&#39;object&#39;)</pre><p>This function just further supports our earlier observations.</p><h4>Summary of Numeric Features</h4><pre>iris_df[numerical_columns].describe()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*kIWldzTpZKqqWCZYCm4Gdg.png" /></figure><p>We can see that the mean and median values don&#39;t have a large difference among their values so no data transformation is required.</p><h4>Correlation Between Variables</h4><pre>iris_df.corr()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*EZfsR1Biyw0P4vLRCa16vg.png" /></figure><p>Further, we can visualise this correlation using a heatmap.</p><pre>fig = plt.figure(figsize = (15,9))<br>sns.heatmap(iris_df.corr(), cmap=&#39;Blues&#39;, annot = True);</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*ge2doHjqdMKi673xmqm1vg.png" /></figure><p>From the above heatmap, we see that petal_length and petal_width have a high correlation, petal_width and sepal_length have good correlation as well as petal_length and sepal_length have good correlations.</p><pre>cols = [&#39;sepal_width&#39;, &#39;sepal_length&#39;, &#39;petal_width&#39;, &#39;petal_length&#39;]<br>plt.figure(figsize=(20,4), dpi=100)<br>i = 1<br>for col in cols:<br>    plt.subplot(1,11,i)<br>    sns.distplot(iris_df[col])<br>    i = i+1<br>plt.tight_layout()<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*V33BMxTdoJ27eeGC2NbgLg.png" /></figure><p>We can observe the distribution of our columns through this distplot which shows that follow a normal distribution.</p><h4>Histograms</h4><p>Let&#39;s see the distribution of data for the various columns of our data set.</p><pre>fig, axes = plt.subplots(2, 2, figsize=(16,9))<br>axes[0,0].set_title(&quot;Distribution of Sepal Width&quot;)<br>axes[0,0].hist(iris_df[&#39;sepal_width&#39;], bins=5);<br>axes[0,1].set_title(&quot;Distribution of Sepal Length&quot;)<br>axes[0,1].hist(iris_df[&#39;sepal_length&#39;], bins=7);<br>axes[1,0].set_title(&quot;Distribution of Petal Width&quot;)<br>axes[1,0].hist(iris_df[&#39;petal_width&#39;], bins=5);<br>axes[1,1].set_title(&quot;Distribution of Petal Length&quot;)<br>axes[1,1].hist(iris_df[&#39;petal_length&#39;], bins=6);</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*JPZlMlwezj4aGEJY3OiXnw.png" /></figure><ul><li>The Highest frequency of sepal width is between 3.0 to 3.5 which is around 70.</li><li>The Highest frequency of sepal length is between 5.5 and 6.0 which is around 35.</li><li>The Highest frequency of petal width is between 0 to 0.5 which is around 50.</li><li>The Highest frequency of petal length is between 0 to 0.5 which is around 50.</li></ul><h4>Univariate Analysis of our columns</h4><pre>sns.FacetGrid(iris_df,hue=&quot;species&quot;,height=5).map(sns.distplot,&quot;petal_width&quot;).add_legend();</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*56I-zVgR5054I61NSWygpw.png" /></figure><p>We see the setosa is easily separable while some portions of Versicolor and virginica are mixed.</p><pre>sns.FacetGrid(iris_df,hue=&quot;species&quot;,height=5).map(sns.distplot,&quot;petal_length&quot;).add_legend();</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*4ZzqdJ0AAmpc0N1_HbxZ3Q.png" /></figure><p>Again we see that on the basis of petal length setosa is separable while the other two are still overlapping.</p><pre>sns.FacetGrid(iris_df,hue=&quot;species&quot;,height=5).map(sns.distplot,&quot;sepal_length&quot;).add_legend();</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*5l3mnBOVqgTv0KWvyGH7vg.png" /></figure><p>We see it is quite tough to separate the species on the basis of sepal_length alone.</p><pre>sns.FacetGrid(iris_df,hue=&quot;species&quot;,height=5).map(sns.distplot,&quot;sepal_width&quot;).add_legend();</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*gF_5UTWGu7JAHLTN6S7STQ.png" /></figure><p>While the overlapping of species is more intense in the case of sepal_width.</p><pre>fig, axes = plt.subplots(2, 2, figsize=(16,9))<br>sns.boxplot(  y=&quot;petal_width&quot;, x= &quot;species&quot;, data=iris_df,  orient=&#39;v&#39; , ax=axes[0, 0])<br>sns.boxplot(  y=&quot;petal_length&quot;, x= &quot;species&quot;, data=iris_df,  orient=&#39;v&#39; , ax=axes[0, 1])<br>sns.boxplot(  y=&quot;sepal_length&quot;, x= &quot;species&quot;, data=iris_df,  orient=&#39;v&#39; , ax=axes[1, 0])<br>sns.boxplot(  y=&quot;sepal_length&quot;, x= &quot;species&quot;, data=iris_df,  orient=&#39;v&#39; , ax=axes[1, 1])<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*6SxT3HaIrwcBBv6ChfxL_Q.png" /></figure><ul><li>Further, we see that the box plots describe that the setosa usually has smaller features with few outliers.</li><li>The Versicolor species has average features</li><li>The virginica species has the longest features widths and lengths as compared to others.</li></ul><p>We can further see the distributions using the violin plot on our dataset</p><pre>fig, axes = plt.subplots(2, 2, figsize=(16,9))<br>sns.violinplot(y=&quot;petal_width&quot;, x= &quot;species&quot;, data=iris_df,  orient=&#39;v&#39; , ax=axes[0, 0])<br>sns.violinplot(y=&quot;petal_length&quot;, x= &quot;species&quot;, data=iris_df,  orient=&#39;v&#39; , ax=axes[0, 1])<br>sns.violinplot(y=&quot;sepal_length&quot;, x= &quot;species&quot;, data=iris_df,  orient=&#39;v&#39; , ax=axes[1, 0])<br>sns.violinplot(y=&quot;sepal_length&quot;, x= &quot;species&quot;, data=iris_df,  orient=&#39;v&#39; , ax=axes[1, 1])<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*lOEU6C_GJAkkJS_vrOJtIA.png" /></figure><p>The kernel density of the data along with the full distribution of the data is shown through the violin plots. We can see the probability density of the various features.</p><h4>Conclusion</h4><ul><li>The dataset is balanced i.e. equal records are present for all three species.</li><li>We have four numerical columns while just one categorical column which in turn is our target column.</li><li>A strong correlation is present between petal width and petal length.</li><li>The setosa species is the most easily distinguishable because of its small feature size.</li><li>The Versicolor and Virginica species are usually mixed and are sometimes hard to separate, while usually Versicolor has average feature sizes and virginica has larger feature sizes.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=190f6dfd946d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/eda-of-the-iris-dataset-190f6dfd946d">EDA of the Iris Dataset</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Meteorological Data Analysis]]></title>
            <link>https://medium.com/@ajetely/meteorological-data-analysis-bbfb7de644b2?source=rss-2d5085b9e12f------2</link>
            <guid isPermaLink="false">https://medium.com/p/bbfb7de644b2</guid>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[numpy]]></category>
            <category><![CDATA[seaborn]]></category>
            <category><![CDATA[data-analysis]]></category>
            <dc:creator><![CDATA[Aditya Jetely]]></dc:creator>
            <pubDate>Mon, 12 Oct 2020 15:39:49 GMT</pubDate>
            <atom:updated>2020-10-12T15:39:49.980Z</atom:updated>
            <content:encoded><![CDATA[<h4>Analysis of the Weather data of Finland to check if the monthly average temperature and humidity have increased from the years 2006 to 2016 or not.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BGSHzNhrocLXsMVG" /><figcaption>Photo by <a href="https://unsplash.com/@nasa?utm_source=medium&amp;utm_medium=referral">NASA</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In this notebook I have performed the analysis of Meteorological Data, the dataset used in this notebook can be obtained from the following link: <a href="https://www.kaggle.com/muthuj7/weather-dataset">https://www.kaggle.com/muthuj7/weather-dataset</a></p><h4>Problem Statement</h4><p>Find whether the average Apparent temperature for a month starting from 2006 to 2016 and the average humidity for the same period have increased or not.</p><h4>Importing Libraries</h4><pre>import pandas as pd<br>import numpy as np<br>import matplotlib<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>%matplotlib inline<br><br>sns.set_style(&#39;darkgrid&#39;)<br>matplotlib.rcParams[&#39;font.size&#39;] = 14<br>matplotlib.rcParams[&#39;figure.figsize&#39;] = (9, 5)<br>matplotlib.rcParams[&#39;figure.facecolor&#39;] = &#39;#00000000&#39;</pre><h4>Loading Data</h4><pre>weather_df = pd.read_csv(&quot;weatherHistory.csv&quot;)<br>weather_df.head()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*LyQPo9SJpGi0LWIxq7FPiQ.png" /></figure><h4>Understanding the data</h4><pre>weather_df.shape</pre><pre>(96453, 11)</pre><p>We have 96453 rows and 11 columns but we are interested in only 3 columns which are Formatted Date, Apparent Temperature and Humidity. So lets just use these three columns from here on.</p><pre>weather_df.columns</pre><pre>Index([&#39;Formatted Date&#39;, &#39;Summary&#39;, &#39;Precip Type&#39;, &#39;Temperature (C)&#39;,<br>       &#39;Apparent Temperature (C)&#39;, &#39;Humidity&#39;, &#39;Wind Speed (km/h)&#39;,<br>       &#39;Wind Bearing (degrees)&#39;, &#39;Visibility (km)&#39;, &#39;Pressure (millibars)&#39;,<br>       &#39;Daily Summary&#39;],<br>      dtype=&#39;object&#39;)</pre><pre>weather_final_df = weather_df[[&#39;Formatted Date&#39;, &#39;Apparent Temperature (C)&#39;, &#39;Humidity&#39;]]<br>weather_final_df.head()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*H4baRFV-KLwvFXHVf-yOxQ.png" /></figure><p>Let&#39;s see some basic info about our dataset</p><pre>weather_final_df.info()</pre><pre>&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;<br>RangeIndex: 96453 entries, 0 to 96452<br>Data columns (total 3 columns):<br> #   Column                    Non-Null Count  Dtype  <br>---  ------                    --------------  -----  <br> 0   Formatted Date            96453 non-null  object <br> 1   Apparent Temperature (C)  96453 non-null  float64<br> 2   Humidity                  96453 non-null  float64<br>dtypes: float64(2), object(1)<br>memory usage: 2.2+ MB</pre><p>We see that we have 2 numeric and one categorical column and all of them are free from null values. Though the data type of the numeric columns is correct, there is a problem in the data type of Formatted date, which should have been DateTime, with which we will deal later.</p><p>The statistical description of our dataset</p><pre>weather_final_df.describe()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*Tv5l9jiboBcNgm4eXPKQ9w.png" /></figure><p>We don&#39;t see a large difference in the mean and median values for both of our numeric columns so they don&#39;t require any transformation.</p><h4>Estimating Skewness and Kurtosis</h4><pre>weather_final_df.skew()</pre><pre>Apparent Temperature (C)   -0.057302<br>Humidity                   -0.715880<br>dtype: float64</pre><pre>weather_final_df.kurt()</pre><pre>Apparent Temperature (C)   -0.706844<br>Humidity                   -0.462170<br>dtype: float64</pre><h4>Parsing the Formatting Date column</h4><pre>weather_final_df[&#39;Formatted Date&#39;] = pd.to_datetime(weather_final_df[&#39;Formatted Date&#39;],utc=True)<br>weather_final_df = weather_final_df.set_index(&#39;Formatted Date&#39;)<br>data = weather_final_df[[&#39;Apparent Temperature (C)&#39;,&#39;Humidity&#39;]].resample(&#39;MS&#39;).mean()<br>data</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*_OniNhTbb9V9XJpoVB0Wlg.png" /></figure><h4>Making new columns</h4><p>In this step, we will divide our Formatted time column into Year and Month columns because of the need of our problem statement</p><pre>weather_final_df = weather_final_df.reset_index()<br>weather_final_df[&#39;Formatted Date&#39;] = pd.to_datetime(weather_final_df[&#39;Formatted Date&#39;],utc = True)<br>weather_final_df[&#39;Month&#39;] = weather_final_df[&#39;Formatted Date&#39;].dt.month<br>weather_final_df[&#39;Year&#39;] = weather_final_df[&#39;Formatted Date&#39;].dt.year<br>weather_final_df.drop(columns= [&#39;index&#39;, &#39;level_0&#39;], inplace=True)<br>weather_final_df.head()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*wrY91Hl-Q71tDKSeOpU_4Q.png" /></figure><pre>weather_final_grouped_df = weather_final_df.groupby([&#39;Month&#39;,&#39;Year&#39;]).mean()<br>weather_final_grouped_df.head(11)</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*58D2X0P-SfXxnwNx6fOWIw.png" /></figure><h4>Variation in apparent temperature in different months</h4><pre>plt.figure(figsize=(21,12))<br>x_axis = np.arange(2006,2017)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Apparent Temperature (C)&#39;][:11].values, label = &#39;January&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Apparent Temperature (C)&#39;][11:22].values, label = &#39;February&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Apparent Temperature (C)&#39;][22:33].values, label = &#39;March&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Apparent Temperature (C)&#39;][33:44].values, label = &#39;April&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Apparent Temperature (C)&#39;][44:55].values, label = &#39;May&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Apparent Temperature (C)&#39;][55:66].values, label = &#39;June&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Apparent Temperature (C)&#39;][66:77].values, label = &#39;July&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Apparent Temperature (C)&#39;][77:88].values, label = &#39;August&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Apparent Temperature (C)&#39;][88:99].values, label = &#39;September&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Apparent Temperature (C)&#39;][99:110].values, label = &#39;October&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Apparent Temperature (C)&#39;][110:121].values, label = &#39;November&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Apparent Temperature (C)&#39;][121:132].values, label = &#39;December&#39;)<br>plt.legend(loc = &#39;lower right&#39;)<br>plt.title(&#39;Variation in apparent temperature in diffrent months&#39;)<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*TDtVnwqQdnsLTibVv3wUHw.png" /></figure><ul><li>Looking at the above visualization we can tell whether the apparent temperature for a given month from 2006 to 2016 increased or decreased.</li><li>For example, if we consider the month of December then we can tell that it has increased.</li><li>While if we see the month of October then it has actually decreased.</li></ul><h4>Variation in Humidity in different months</h4><pre>plt.figure(figsize=(21,12))<br>x_axis = np.arange(2006,2017)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Humidity&#39;][:11].values, label = &#39;January&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Humidity&#39;][11:22].values, label = &#39;February&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Humidity&#39;][22:33].values, label = &#39;March&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Humidity&#39;][33:44].values, label = &#39;April&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Humidity&#39;][44:55].values, label = &#39;May&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Humidity&#39;][55:66].values, label = &#39;June&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Humidity&#39;][66:77].values, label = &#39;July&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Humidity&#39;][77:88].values, label = &#39;August&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Humidity&#39;][88:99].values, label = &#39;September&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Humidity&#39;][99:110].values, label = &#39;October&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Humidity&#39;][110:121].values, label = &#39;November&#39;)<br>plt.plot(x_axis,weather_final_grouped_df[&#39;Humidity&#39;][121:132].values, label = &#39;December&#39;)<br>plt.legend(loc = &#39;lower right&#39;)<br>plt.title(&#39;Variation in apparent temperature in diffrent months&#39;)<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*Pwfqz_CAYRnOy0N2CgojyQ.png" /></figure><ul><li>Looking at the above visualization we can tell whether the average humidity for a given month from 2006 to 2016 increased or decreased.</li><li>For example, if we consider the month of February then we can tell that it has increased.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bbfb7de644b2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My First Open Source Experience]]></title>
            <link>https://medium.com/@ajetely/my-first-open-source-experience-8739388a4509?source=rss-2d5085b9e12f------2</link>
            <guid isPermaLink="false">https://medium.com/p/8739388a4509</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[ci]]></category>
            <dc:creator><![CDATA[Aditya Jetely]]></dc:creator>
            <pubDate>Tue, 01 Sep 2020 09:15:57 GMT</pubDate>
            <atom:updated>2020-09-01T09:15:57.271Z</atom:updated>
            <content:encoded><![CDATA[<p>An article about my open-source experience with StudentCodein(SCI).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j4mODWNxnUDNnpmI" /><figcaption>Photo by <a href="https://unsplash.com/@glenncarstenspeters?utm_source=medium&amp;utm_medium=referral">Glenn Carstens-Peters</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>What is Student Code-in(SCI)?</h3><p>Student Code-In is a global program that helps students grow with Open Source Contribution. It is a 2 months long Open-Source initiative which provides you with the best platform to improve your skills and abilities by contributing to a vast variety of Open Source Projects Projects. In this, all the registered participants would get an exquisite opportunity to interact with the mentors and the Organizing Team.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/1*pRe4bOBgcyCx6F5_VAL8aQ.jpeg" /><figcaption>Student Code-In Logo</figcaption></figure><h3>My Experience</h3><p>I joined SCI as a participant under the project called<strong> Survival Analysis with Python. </strong>The project admin was <a href="https://medium.com/u/f064ddbf5711">Akshat Anand</a> and I was mentored by <a href="https://medium.com/u/5163d3d15138">Rohan Mathur</a> and Ankur Chourasia. <a href="https://medium.com/u/e0b1917398a">Abhijit Tripathy</a> was a fellow participant along with me. Before starting the project I was a bit nervous as I had never contributed to open source but both the mentors and the Project Admin were so friendly and helpful that all my fear about open-source vanished as they were always ready to help us and they guided us throughout the project.</p><p>Through SCI I learnt a lot of things, not just about open source but also about project management, teamwork, time management and better code practices. I also acquired knowledge about various technologies as most of the elements used in our project were new to me.</p><p>I can for sure say that I had an awesome experience with SCI and I want to thank the organisers for creating this programme as well as for choosing me as a participant and I also want to thank all the members of my team without the support of whom this journey wouldn’t have been as awesome as it was.</p><p>To know about our project you can visit the following links:</p><ul><li><a href="https://github.com/cipheraxat/Survival-Analysis">cipheraxat/Survival-Analysis</a></li><li><a href="https://github.com/AdityaJ7/Dashboard_with_dash">AdityaJ7/Dashboard_with_dash</a></li></ul><p>PS: The results of SCI were declared today and I am one of the top 25 contributors.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8739388a4509" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Survival Functions 101]]></title>
            <link>https://medium.com/data-science/survival-functions-101-bd57fb8be888?source=rss-2d5085b9e12f------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd57fb8be888</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[survival-analysis]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Aditya Jetely]]></dc:creator>
            <pubDate>Sun, 21 Jun 2020 13:52:18 GMT</pubDate>
            <atom:updated>2020-06-22T14:58:12.147Z</atom:updated>
            <content:encoded><![CDATA[<h4>An introduction to one of the core parts of Survival Analysis.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5t3gwpjC4ETCjcendzS2SA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@srd844?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Stephen Dawson</a> on <a href="https://medium.com/s/photos/survival-analysis?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Before moving ahead with this post I would highly encourage you to be aware of Survival Analysis as the main focus of this post will be on Survival Functions though a brief intro to survival analysis has been provided.</p><h3><strong>What is Survival Analysis?</strong></h3><p>Survival Analysis is a modelling technique used to model time to an event ie. it is used to know how long will it take for an event to occur. The events could be time to death after a heart attack, time to get a job after graduation, the time till which a couple stays married or time till which a machine works properly, etc.</p><p>Survival Analysis consists of three parts:</p><ol><li>Collecting and processing survival data</li><li>Choosing a survival function</li><li>Performing analysis via different methods</li></ol><h3><strong>What is a Survival Function</strong></h3><p>It is a function which helps us to know that whether or not an object of interest is gonna survive beyond the specified time. This object of interest can be a patient, a machine or anything else.</p><p>Mathematically a survival function is denoted by S which is quite obviously a function of time. Survival function can be also interpreted as the probability that a certain object of interest will survive beyond a certain time t.</p><p>So it can be represented as:</p><p>S(<strong>t</strong>) = <strong>P</strong>(<strong>T</strong> &gt;<strong>t</strong>)</p><p>Here T is the random lifetime which is taken from the population on which the survival analysis has to be performed and hence it is non zero</p><p>The value of the function lies between 0 and 1(inclusive) and it is a non-increasing function.</p><p>The Survival Function is sometimes also referred to as <strong>survivor function </strong>or <strong>reliability function.</strong></p><p>The survival function S(t) can also be related to the cumulative distribution function F(t) in the following way:</p><p><strong>S(t) = 1 - F(t)</strong></p><p>Hence it is also called a <strong>complementary cumulative distribution function</strong>.</p><h3><strong>How to interpret a survival function?</strong></h3><p>Let&#39;s look at an arbitrary example to see how do we interpret a given survival function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/933/1*_k_lkhoAbt-OjDqnt8TWPA.jpeg" /><figcaption>Arbitrary Survival Function</figcaption></figure><p>After looking at the above survival function we can say that only 25% of the total population under observation survives longer than 4 months.</p><p>We can make more predictions like what percentage of the population survives more than 6 months which will be around 17%, etc.</p><p><strong>Note:</strong> This is not the only way of interpreting the survival function and other interpretations like median survival can also be made as per the need of the analyst.</p><h3>Types of survival functions:</h3><p>Survival functions are mainly divided into two categories: parametric survival functions and non-parametric survival functions. This differentiation is based on parameters which can be mean, standard deviation, median, etc.</p><p>Let&#39;s have a look at some of the parametric survival functions:</p><h4>1.Exponential survival function</h4><p>In this kind of function, the probability of failure is the same in every time interval, no matter what the age of the object of interest is, so the age of the subject does not affect the probability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/933/1*_k_lkhoAbt-OjDqnt8TWPA.jpeg" /><figcaption>Exponential survival function</figcaption></figure><h4>2.Weibull survival function:</h4><p>This function actually extends the exponential survival function to allow constant, increasing, or decreasing hazard rates where <strong>hazard rate </strong>is the measure of the propensity of an item to fail or die depending on the age it has reached.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*6MSbQggyhTh792OzBIcWfQ.jpeg" /><figcaption>Weibull survival function</figcaption></figure><h4>3.Other different survival functions</h4><p>We can choose other survival functions like normal, lognormal, log-logistic, and gamma depending on the data set which is under distribution. A log-logistic survival function is shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/731/1*GuGorc0_83D3JdeVKqWhIw.jpeg" /><figcaption>Log-logistic survival function</figcaption></figure><p><strong>Note: </strong>The second category of survival functions that is non-parametric survival functions has not been discussed here which is usually modelled using the famous Kaplan-Meier estimator which itself is a wide topic. To know about Kaplan-Meier estimator please visit <a href="https://towardsdatascience.com/understanding-kaplan-meier-estimator-68258e26a3e4"><strong>Understanding Kaplan-Meier Estimator</strong></a> by<a href="https://towardsdatascience.com/@pratikbaitha04"> Pratik Kumar</a>.</p><h4>Conclusion</h4><p>Survival Function forms the major part of the survival analysis and helps the analyst in gaining valuable knowledge related to the time-till events. A lot of important questions can be answered if the survival function is interpreted wisely.</p><p>This post is written as part of <a href="https://scodein.tech/">Student Code-In 2020</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/1*O5-ccKah-A15_SxSg7ljKw.jpeg" /></figure><p>Student Code-in is a global program to introduce university/pre-university students to open source software development.</p><h3>References</h3><ol><li><a href="https://lifelines.readthedocs.io/en/latest/Quickstart.html">https://lifelines.readthedocs.io/en/latest/Quickstart.html</a></li><li><a href="https://www.youtube.com/watch?v=MdmWdIV5k-I">https://www.youtube.com/watch?v=MdmWdIV5k-I</a></li><li><a href="https://en.wikipedia.org/wiki/Survival_function#:~:text=The%20survival%20function%20is%20a,survivor%20function%20or%20reliability%20function.">https://en.wikipedia.org/wiki/Survival_function#:~:text=The%20survival%20function%20is%20a,survivor%20function%20or%20reliability%20function.</a></li><li><a href="https://www.youtube.com/watch?v=XQfxndJH4UA">https://www.youtube.com/watch?v=XQfxndJH4UA</a></li><li>Survival Analysis: A Self-Learning Text by David Kleinbaum and Mitchel Klein.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd57fb8be888" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/survival-functions-101-bd57fb8be888">Survival Functions 101</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>