<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Arowolo Christopher Olalekan on Medium]]></title>
        <description><![CDATA[Stories by Arowolo Christopher Olalekan on Medium]]></description>
        <link>https://medium.com/@khrixres01?source=rss-48aaf22d12e6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*0UQ_70bFa7cyzLR3v2NxZA.jpeg</url>
            <title>Stories by Arowolo Christopher Olalekan on Medium</title>
            <link>https://medium.com/@khrixres01?source=rss-48aaf22d12e6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 09:02:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@khrixres01/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[UNDERSTANDING SQL JOIN]]></title>
            <link>https://medium.com/@khrixres01/understanding-sql-join-fa85b8f953fe?source=rss-48aaf22d12e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa85b8f953fe</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[data-analytics-tools]]></category>
            <category><![CDATA[sql-joins]]></category>
            <dc:creator><![CDATA[Arowolo Christopher Olalekan]]></dc:creator>
            <pubDate>Thu, 29 Feb 2024 13:06:03 GMT</pubDate>
            <atom:updated>2024-02-29T13:06:03.220Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This article is aimed at helping you understand SQL Joins and also providing instances of when to use it as a Data Analyst. Let’s get to it.</em></p><p><strong>SQL JOINS</strong> is one of the most important parts of Data Analysis using SQL, this is because most of the time, you would always have the need to pull information from two or more tables to get a complete picture of your data, and here is where SQL JOINS comes in.</p><p><strong>WHAT IS A JOIN</strong></p><p><strong>JOIN</strong> is a clause used to combine rows from two or more tables, although they can also be joined with themselves. They combine based on common columns in the tables. So what are the types of joins we have?</p><p><strong>Type of Joins</strong></p><p>Let’s look at some of the popular joins we have.</p><p>1. INNER JOIN</p><p>2. LEFT JOIN</p><p>3. RIGHT JOIN</p><p>4. FULL OUTER JOIN</p><p>5. SELF JOIN</p><p>6. CROSS JOIN</p><p>7. NATURAL JOIN</p><p>For the sake of this Article, we will be using the datasets below, you can reach me on X <a href="https://twitter.com/OgbeniHeroo">here</a> to get the dataset. This is the States and President Datasets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/376/1*PC5noDu4ek2-aVTVpM5ISg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/394/1*Evpcm_lXb31Bn0yXvxH8Xg.png" /><figcaption>LR- states data presidents data.</figcaption></figure><p><strong>INNER JOIN</strong></p><p>This type of join returns rows that have matching values between the columns in both tables. In other words, it returns a result that contains only the rows that have matching values in both tables. If there are no matching values, the rows will not be returned.</p><p>QUERY SYNTAX:</p><pre>SELECT t. column_A, t.column_B, t2.column_1, t2.column_2<br>FROM table1 AS t<br>INNER JOIN table2 as t2<br>ON t.column_name = t2.column_name;</pre><p>An example of this is in the image below.</p><pre>-- show the conntry, continents and presidents in the datasets.<br><br>SELECT s.country, s.continent,p.president<br>FROM states s <br>INNER JOIN presidents p<br>ON a.country = p.country;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*orZABFIY3IaKuHOqtSzowA.png" /></figure><p>As we can see, it returned just 7 rows but the states have 11 rows. This is because the inner join only returns matching columns and only 7 were matching. got it? right? next.</p><p><strong>LEFT JOIN</strong></p><p>This type of join returns all the rows in the left table and the matching rows in the right table. If there are rows in the right tables that do not hold a value, then Null is returned in the query response for such row. In other words, left join returns all the values in the left table and return matching values in the right table, should there be any missing value in the right table, then NULL is returned.</p><p>QUERY SYNTAX:</p><pre>SELECT t. column_A, t.column_B, t2.column_1, t2.column_2<br>FROM table1 AS t<br>LEFT JOIN table2 as t2<br>ON t.column_name = t2.column_name;</pre><p>In this instance, “table1” is the LEFT TABLE, so it returns all the values in the columns specified. If there are no matches in Table 2, then NULL values will be returned.</p><p>For example, see the image below.</p><pre>--show the country, continent and president columns using left join<br>SELECT s.country,s.contitnent, p.president<br>FROM states s<br>LEFT JOIN president p<br>ON a.country = p.country;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/915/1*bSQ9Tia7_Mv9tAB_zeLsbw.png" /></figure><p>In this example, we can see that it returned the 11 rows from the columns of the state, and also returned all the rows in the right table “president”, but non-matching columns were returned as NULL.</p><p><strong>RIGHT JOIN</strong></p><p>This type of join does the reverse of a LEFT JOIN. They return all the rows in the RIGHT table and return matching rows in the LEFT table, rows that are not a match, are returned as NULL values. In other words, right join returns all the values in the right table and return matching values on the left table, should there be no matching value in the left table, then NULL is returned.</p><p>QUERY SYNTAX:</p><pre>SELECT t. column_A, t.column_B, t2.column_1, t2.column_2<br>FROM table1 AS t<br>RIGHT JOIN table2 as t2<br>ON t.column_name = t2.column_name;</pre><p>In this instance, “table2” is the RIGHT table, and all its rows in the columns specified would be returned, then matching rows in the left table “table1” would also be returned, Null would be returned in the instance where there are no matching values, then NULL is returned in the output.</p><p>For example, see the image below.</p><pre>SELECT s.country, s.continent,p.president<br>FROM states s<br>RIGHT JOIN presidents p<br>ON s.country = p.country;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/779/1*XNObqutckmAattSvN6PjSw.png" /></figure><p>Here, the query returned all the records in the president table and returned all the matching rows in the states table also, however, the states table is longer than the president table, so rather than have NULL values on the 3 columns, it was truncated.</p><p><strong>FULL OUTER JOIN</strong></p><p>This type of join is used to return all rows in the columns specified in both tables combined. It is used when matching records are not in consideration, therefore NULL values are included.</p><p>QUERY SYNTAX:</p><pre>SELECT t.column_A, t.column_B, t2.column_1, t2.column_2<br>FROM table1 AS t<br>FULL OUTER JOIN table2 as t2<br>ON t.column_name = t2.column_name;</pre><p>In this instance, all the rows in the columns specified would be returned. For example, see the image below.</p><pre>SELECT s.country, s.continent, p.president<br>FROM states s<br>FULL OUTER JOIN presidents p<br>ON s.country = p.country;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/794/1*QP7zH8nB7cpwp6viuoqjOg.png" /></figure><p>The query returns the combined columns.</p><p><strong>SELF JOIN</strong></p><p>This is a type of join that is used when you join a table with itself. This is used in hierarchical relationships or to compare rows within the same table. For instance, in sales data, you want to compare the sales of electronics to the sales of other product categories.</p><p>QUERY SYNTAX:</p><pre>SELECT t.column_A, t2.column_A, t.column_B<br>FROM table1 AS t<br>JOIN table1 as t2<br>ON t.column_A = t2.column_A<br>AND t.column_A &lt;&gt; t2.column_A;</pre><p>In this instance, I am trying to compare values in column_B with each other assuming the column_A contains a metric.</p><p>For example, look at the image below.</p><p>Here we want to compare the independent year of all the countries in the states table that are in the same continent.</p><pre>SELECT DISTINCT s.country AS first_country, s2.country AS second_country, s.independent_year AS first_country_indp_year , s2.independent_year AS second_country_ind_year<br>FROM states s<br>JOIN states s2<br>ON s.continent = s2.continent<br>AND s.country &lt;&gt; s2.country;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I92vj8w2kS1WZltCYCGA8Q.png" /></figure><p>The query returns the comparison in their respective independent years, Egypt was not included as it does not have another African country to compare with.</p><p><strong>CROSS JOIN</strong></p><p>This type of join is used to return all possible combinations of two or more tables. This is rarely used as it leads to a large dataset as output.</p><p>QUERY SYNTAX:</p><pre>SELECT *<br>FROM table1 <br>CROSS JOIN table2;</pre><p>In this instance, the query would produce all possible combinations of the two tables used.</p><p>For example, look at the image below.</p><pre>SELECT *<br>FROM states<br>CROSS JOIN presidents;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*D-Bx4pP5zhJm7unP9ziZ2w.png" /></figure><p>This query returns all possible combinations of both rows and columns. This output often produces a large data.</p><p><strong>NATURAL JOIN</strong></p><p>This type of join is used to combine two tables based on the columns in the tables. It is important to know that the two tables must contain the same column name and datatype and lastly must not contain null values in the columns being joined.</p><p>QUERY SYNTAX:</p><pre>SELECT *<br>FROM table1<br>NATURAL JOIN table2;</pre><p>In this instance, the query produces rows where the values in the columns with the same name in both tables are equal. It functions the same way as the INNER JOIN, however, it does not use the “ON” clause and also does not include the same columns.</p><p>For example, check the image below.</p><pre>SELECT *<br>FROM states<br>NATURAL JOIN presidents;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/899/1*99Qyb_2Q5PWsI3EbuuBDVQ.png" /></figure><p>The query returns 7 rows, leaving out non-matching columns in the two tables, also redundant column in the president data is removed, leaving only the president column there.</p><p>yeah, looks like an inner join right, but the difference here is that inner join uses the “ON” clause but this doesn&#39;t, also, it returns distinct columns.</p><p>That&#39;s all for now.</p><p>Thank you for reading through.</p><p>follow me on x <a href="https://twitter.com/OgbeniHeroo">here</a> for more insightful articles like this.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa85b8f953fe" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Cleaning With Python]]></title>
            <link>https://medium.com/@khrixres01/data-cleaning-with-python-9128d50c632d?source=rss-48aaf22d12e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/9128d50c632d</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-cleaning-with-python]]></category>
            <category><![CDATA[data-wrangling]]></category>
            <category><![CDATA[data-analytics]]></category>
            <dc:creator><![CDATA[Arowolo Christopher Olalekan]]></dc:creator>
            <pubDate>Thu, 09 Nov 2023 15:03:56 GMT</pubDate>
            <atom:updated>2023-11-09T15:03:56.059Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How I cleaned one of the dirtiest data you can find on the internet using Pandas and Numpy</em></p><h3>Introduction</h3><p>What is Data Cleaning?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*wV4x2Ss_nNAYDhuQhYrriA.jpeg" /><figcaption>photo credit: google.com</figcaption></figure><p>Before we talk about what data cleaning is, one of the simple truths about data to every data professional is that data is rarely available in the form we all dream about. As a Data professional in Nigeria, data is never available, you don&#39;t get it on a platter, you either gather through surveys or scrap data off the internet. Neither of these methods would give you clean data a million times, this is why data cleaning is one of the most important processes to take after you have successfully gotten the data you need. Now, what really is data cleaning?</p><p>Data Cleaning is the process of identifying and correcting errors, inconsistencies, and inaccuracies in a dataset to improve its quality and reliability. It is a critical step in data preprocessing that helps ensure that data is accurate, complete, and consistent for effective analysis and decision-making.</p><h3><strong>Scope Of Data Cleaning</strong></h3><p>Data cleaning involves several processes, here are a few of them:</p><ol><li>Handling Missing Data: Addressing missing values by either imputing them using statistical techniques or removing the rows or columns containing missing data.</li><li>Removing Duplicates: Identifying and eliminating duplicate records to avoid redundancy and ensure data accuracy.</li><li>Standardizing Data: Converting data into a consistent format to facilitate comparisons and analysis.</li><li>Handling Outliers: Identifying and dealing with data points that deviate significantly from the rest of the dataset to prevent them from skewing analysis results.</li><li>Resolving Inconsistencies: Ensuring that the data type is uniform and conforms to the predefined type.</li></ol><p><strong><em>Cleaning The Movie Data.</em></strong></p><p>Data cleaning can be done with several tech stacks like SQL, Ms Excel, or Business Analysis tools like Power BI and Tableau. For this Project, I decided to use Python, because it possesses a variety of Libraries that can be deployed to fix a problem, some of the popular ones are Pandas and Numpy. The data was obtained from kaggle.com and was scraped off IMDb, Netflix, and Tvshows websites.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*heHatEdJQPoWWvecd5T69Q.jpeg" /><figcaption>movie data before cleaning</figcaption></figure><p>So it began,</p><ol><li>loaded the CSV file into my jupyter notebook</li></ol><pre>#first, i would import the data to my jupyter <br>import pandas as pd<br>import numpy as np<br><br>file_path = &quot;C:/Users/user/Desktop/archive122/movies.csv&quot; <br>#read the data into the notebook<br>movie_df = pd.read_csv(file_path)<br>#print the dataset<br>movie_df</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*0WAO-os3QpFg1n3W_Z-LXA.jpeg" /><figcaption>Data after Importing to my notebook.</figcaption></figure><p>2. I took a sneak peek at the dataset using the dotinfo() function</p><pre>#checking the dataset<br>movie_df.info()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/335/1*1M263IaOn1--YalmRN_IVQ.jpeg" /><figcaption>A sneak peek of the data.</figcaption></figure><p>3. Handled the Null Values by removing rows with null in the text data, I also replaced Null values with their mean value in the Numerical data columns, different strokes for different foes.. lol yeah? here is what it looked like..</p><pre>#first step in my cleaning process, is to get rid of null values. <br>#here we notice a drop in the numbers of non_null, meaning there are null values in there. lets take a look<br>movie_df.isna()<br><br>#this supports the claim, but lets look inwards to the count of these null cells<br>movie_df.isna().sum()<br>#There were enough Null values in the dataset.<br><br>#let us treat them one after the other. <br>#for the year column, we will drop the rows where year is null. <br><br>movie_df = movie_df.dropna(subset = [&#39;YEAR&#39;])<br>#Drop rows with Null values made more sense for an unbiase analysis. <br>#print the dataset<br>movie_df<br><br>#next is the genre column, imagine a movie without a genre classification, it should be dropped to avoid confusion <br>movie_df = movie_df.dropna(subset = [&#39;GENRE&#39;])<br>#print the dataset<br>movie_df<br><br>#Next is the rating column. <br>#Rating column can be replaced with the average rating in the column. <br>movie_df[&#39;RATING&#39;].fillna(movie_df[&#39;RATING&#39;].mean(), inplace = True)<br>#hit with a warning error.<br>#print dataset <br>movie_df<br><br>movie_df[&#39;RATING&#39;] = movie_df[&#39;RATING&#39;].round(1)<br>#print the dataset <br>movie_df<br><br>#up to the next column VOTES.i need to replace the null values with the mean value, however there is a problem, the column is a string,<br>#so i might need to convert first before finding the mean value. <br>try:<br>    #replace the &#39;,&#39; with &#39;&#39;.<br>    movie_df[&#39;VOTES&#39;] = movie_df[&#39;VOTES&#39;].str.replace(&#39;,&#39; , &#39;&#39;).astype(float)<br>except ValueError as e:<br>    #print error if any<br>    print(f&#39;Error: {e}&#39;)<br>#print the dataset<br>print(movie_df)<br><br>#next, i&#39;d get the mean value and replace the null values with it. <br>movie_df[&#39;VOTES&#39;].fillna(movie_df[&#39;VOTES&#39;].mean(), inplace = True)<br>#print to show changes<br>movie_df<br>#next, i want to round up to 1dp. <br>movie_df[&#39;VOTES&#39;] = movie_df[&#39;VOTES&#39;].round(1)<br>#print the dataset<br>movie_df<br><br>#next is the runtime column <br>movie_df[&#39;RunTime&#39;].fillna(movie_df[&#39;RunTime&#39;].mean(), inplace = True)<br>#print the dataset<br>movie_df<br>#next, i would round off to 1dp<br>movie_df[&#39;RunTime&#39;] = movie_df[&#39;RunTime&#39;].round(1)<br>#print the dataset<br>movie_df<br><br>#so Gross has so many null values in the column, i would replace the null with ZERO. <br>movie_df[&#39;Gross&#39;].fillna(0, inplace = True)<br>movie_df<br>#job is done on that. next i would be changing their data type to the appropriate one.</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/314/1*Hi5Nk-2t5x4IVPsjslNbpQ.jpeg" /><figcaption>After handling null values.</figcaption></figure><p>4. Handled Incorrect Data type. Some columns had incorrect data type, and some columns like year column, and gross column data type were not in the appropriate type and were converted.</p><p>Some columns needed to be cleaned before they could be converted, so I cleaned them before changing the data type.</p><pre>#the year is in string, it should be changed to the date data type.<br>movie_df[&#39;YEAR&#39;]<br>#to achieve that, first i must strip the column off any delimiter that is not a digit<br><br>#regex helps to filter the values in the columns and ensure they are trated as the condition set.<br>new_date = movie_df[&#39;YEAR&#39;].str.replace(&#39;[^0-9]&#39;, &#39;&#39;, regex = True)<br>#print the dataset<br>new_date<br># there are rows with extra values aside the dates, as shown below.<br><br># Trim the strings by removing extra digits behind the first 4.<br>trimmed_dates = [date[:4] for date in new_date]<br># Displaying the trimmed strings<br>for trimmed_date in trimmed_dates:<br>    print(trimmed_date)<br>#replace the values with the year column <br><br>#replace the column with the trimmed date values<br>movie_df.loc[:, &#39;YEAR&#39;] = trimmed_dates<br>#print the dataset<br>movie_df<br><br>#to double check the column<br>movie_df[&#39;YEAR&#39;].unique()<br><br>#next, i will change the datatype to datestamp <br>movie_df[&#39;YEAR&#39;] = movie_df[&#39;YEAR&#39;].astype(&#39;datetime64&#39;)<br>#print the dataset<br>movie_df<br><br>#next,the genre column. <br>#let me get the distinct values first.<br>movie_df[&#39;GENRE&#39;].unique()<br>#this is to show the extent of the dirty data.<br><br>#first i will get rid of the white spaces.<br>movie_df[&#39;GENRE&#39;] = movie_df[&#39;GENRE&#39;].str.strip()<br>#print the dataset<br>movie_df<br><br>#i want to spilt the key words into columns <br>#get the unique vales first<br>movie_df[&#39;GENRE&#39;].unique()<br><br>#i will split the keywords into columns <br># Split the keywords into separate columns using get_dummies<br>#get_dummies would split the keywords into columns and change it to a categorical data type, where 1 means present and 0 means otherwise<br>genre_df = movie_df[&#39;GENRE&#39;].str.get_dummies(&#39;, &#39;)<br>#print the new columns dataset<br>genre_df<br><br>#change the dataframe to a categorical data type.<br>genre_df = genre_df.astype(&#39;category&#39;)<br>#print the dataset<br>genre_df.info()<br><br>#next, i&#39;ll drop the old column and replace it with this new data frame<br>movie_df.drop(&#39;GENRE&#39;, axis = 1, inplace = True)<br>#print the dataset<br>movie_df<br><br>#then i would concat the new dataframe to the old dataframe<br>movie_df = pd.concat([movie_df, genre_df], axis=1)<br>#print the dataset after joining the new columns<br>movie_df<br><br>#next is the one-line column <br>#trim the whitespaces off <br>movie_df[&#39;ONE-LINE&#39;] = movie_df[&#39;ONE-LINE&#39;].str.strip()<br>#print the dataset<br>movie_df<br><br>#next,is the stars column<br>#first, get rid of the white spaces <br>movie_df[&#39;STARS&#39;] = movie_df[&#39;STARS&#39;].str.strip()<br>#print the dataset<br>movie_df<br><br>#lets take a closer look at the column again<br>movie_df[&#39;STARS&#39;]<br>#i need to replace the &#39;&#39;\n&#39;&#39; with blanks and also replace the &#39;|&#39; with blank<br>movie_df[&#39;STARS&#39;] = movie_df[&#39;STARS&#39;].str.replace(&#39;\n&#39;, &#39;&#39;).str.replace(&#39;|&#39;, &#39; &#39;)<br>movie_df[&#39;STARS&#39;]<br>#okay, this looks better. <br><br>#next, is the gross column, lets take a look<br>movie_df[&#39;Gross&#39;].unique()<br><br>#i will change the data type but first, have to strip  these string off.<br>movie_df[&#39;Gross&#39;] = movie_df[&#39;Gross&#39;].str.replace(&#39;$M&#39;, &#39;&#39;)<br><br>#check for the changes made<br>movie_df[&#39;Gross&#39;].unique()<br>#that did not go as planned, okay would try again<br><br>movie_df[&#39;Gross&#39;] = movie_df[&#39;Gross&#39;].str.replace(&#39;$&#39;,&#39;&#39;).str.replace(&#39;M&#39;,&#39;&#39;)<br>movie_df[&#39;Gross&#39;]<br>#okay, lets look at the unique values<br><br>movie_df[&#39;Gross&#39;].unique()<br>#next, i would change the data type to float<br><br>movie_df[&#39;Gross&#39;] = movie_df[&#39;Gross&#39;].astype(&#39;numeric&#39;)<br>#cannot convert cos of presence of some nan number, to pass that, we coerce the error<br><br>movie_df[&#39;Gross&#39;] = pd.to_numeric(movie_df[&#39;Gross&#39;], errors=&#39;coerce&#39;)<br># Round the &#39;float_column&#39; to 1 decimal place<br>movie_df[&#39;Gross&#39;] = movie_df[&#39;Gross&#39;].round(1)<br><br>#the values been replaced with NaN, next i would change the NaN to zero.<br>movie_df[&#39;Gross&#39;]<br>movie_df[&#39;Gross&#39;].fillna(0, inplace = True)<br>#print the dataset<br>movie_df</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/747/1*b3hwAqTjwhzbDg4FT6WDMg.jpeg" /><figcaption>After effecting the data type change and column cleaning.</figcaption></figure><p>5. Handled Duplicated Values. There were 3034 duplicated values in the dataset, I dropped all before proceeding to check by columns, this is because I noticed some of the values in the Movies column contain duplicated names, as the unique identifier in the table, duplicated names are not allowed. I dropped the rows that have duplicated names.</p><pre><br>#show the duplicated rows<br>movie_df[movie_df.duplicated()]<br>#drop duplicates <br>movie_df = movie_df.drop_duplicates(keep = &#39;first&#39;)<br>#view the changes made in the dataframe<br>movie_df<br><br>#several rows have been dropped, but to ensure there are no duplicates, we check by the columns<br>movie_df[&#39;MOVIES&#39;].duplicated()<br>#drop duplicates by columns<br>movie_df[&#39;MOVIES&#39;] = movie_df[&#39;MOVIES&#39;].drop_duplicates(keep = False)<br>#check for duplicate<br>movie_df[&#39;MOVIES&#39;].duplicated()<br><br>#i want to show the duplicated values in the column movies, for a closer look<br>duplicate = movie_df[movie_df.duplicated(subset = &#39;MOVIES&#39;, keep = False)]<br>duplicate<br>#the movie title columns contain NaN, this is unacceptable as the column is a unique identifier.<br><br>#i want to drop the NaN values from the dataset, but i would take a closer look before i do that<br>movie_df[&#39;MOVIES&#39;]<br><br>#there are definately NaN values there so i would drop them now. <br>movie_df = movie_df.drop_duplicates(subset = &#39;MOVIES&#39;, keep = False)<br>movie_df<br><br>#check to see if the changes were okay<br>movie_df[&#39;MOVIES&#39;].duplicated()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/1*T35rB871tnZpClVVuj5wyg.jpeg" /><figcaption>data after handling the duplicated rows.</figcaption></figure><p>After handling the duplicated data, I took a sneak peek at the dataset, before adding some quick data type corrections as shown below</p><pre>#next i want to round VOTES TO a whole munber<br>movie_df[&#39;VOTES&#39;] = movie_df[&#39;VOTES&#39;].round(0)<br><br>#then i want to round the runtime column to a whole number<br>movie_df[&#39;RunTime&#39;] = movie_df[&#39;RunTime&#39;].round(0)<br>movie_df<br><br>#Votes should be in int, not floats<br>movie_df[&#39;VOTES&#39;] = movie_df[&#39;VOTES&#39;].astype(&#39;int64&#39;)<br>movie_df<br><br>#Runtime is also supposed to be in minutes, the right data type for that is int.<br>movie_df[&#39;RunTime&#39;] = movie_df[&#39;RunTime&#39;].astype(&#39;int64&#39;)<br>movie_df</pre><p><strong>Final Look at the Data</strong></p><pre>movie_df.info()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/467/1*88f0qqx6vHqGfRRA9F_OHw.jpeg" /><figcaption>final peek at the data attribue.</figcaption></figure><pre>#final shape of the dataset<br>movie_df.shape</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/748/1*9O2Hnppe9rqW7davD6Pifg.jpeg" /><figcaption>final shape</figcaption></figure><pre>#final look at the dataset<br>movie_df</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*pvCNpnIqQFKZveQyBd7CNw.jpeg" /><figcaption>what the data looked like after cleaning.</figcaption></figure><p>I saved the cleaned CSV file to my local machine.</p><pre>#finally, i would save the dataset to my desktop<br>file_path = &#39;C:/Users/user/Desktop/cleaned_movie_data.csv&#39;<br>movie_df.to_csv(file_path, index = False)<br>#see you next time! caio!</pre><p>This is what I looks like on Ms Excel</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zvP4tBD-WxE54ahZbcv88g.jpeg" /><figcaption>After cleaning the data.</figcaption></figure><p>and it&#39;s a wrap!</p><p>To follow through with the process, my GitHub documentation contains a proper run-through with my jupyter notebook to help you relate better to the process.</p><p><a href="https://github.com/khrixres01/Data-Cleaning-Using-Python/tree/main">GitHub - khrixres01/Data-Cleaning-Using-Python: This is a the first important step in any data analysis process. Here, i used the most common python libraries in pandas and Numpy to Clean a very dirty date.</a></p><p>Thank you for reading through. Please let me know your thoughts and corrections.</p><p>kindly follow me on X</p><p>@ogbeniherooo</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9128d50c632d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>