<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ajibola Salami on Medium]]></title>
        <description><![CDATA[Stories by Ajibola Salami on Medium]]></description>
        <link>https://medium.com/@babagee500?source=rss-8ae1dd25a0df------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*cSOCNeKA39nBRK5S</url>
            <title>Stories by Ajibola Salami on Medium</title>
            <link>https://medium.com/@babagee500?source=rss-8ae1dd25a0df------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 19:30:44 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@babagee500/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Hypothesis Testing in an E-commerce Business]]></title>
            <link>https://medium.com/@babagee500/hypothesis-testing-in-an-e-commerce-business-1025bdab1f1d?source=rss-8ae1dd25a0df------2</link>
            <guid isPermaLink="false">https://medium.com/p/1025bdab1f1d</guid>
            <category><![CDATA[hypothesis-testing]]></category>
            <category><![CDATA[decision-making]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ecommerce]]></category>
            <dc:creator><![CDATA[Ajibola Salami]]></dc:creator>
            <pubDate>Sat, 08 Aug 2020 17:34:44 GMT</pubDate>
            <atom:updated>2020-08-10T08:49:20.365Z</atom:updated>
            <content:encoded><![CDATA[<p>Decision making is an indispensable part of every business, and every shrewd businessman knows that there are strategic processes to this — and that there are really no cut and dried rules. E-commerce businesses are a big partaker in the data economy, given their mode of operation, and online shops happen to be one of the most popular.</p><p>Hypothesis testing is basically checking how plausible an assumption is, and given the volume of data at the disposal of many of these businesses, it is likely that managers would need come up with good and timely decisions. This is where hypothesis testing becomes an invaluable tool.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*nkU_pcO5ol9XkfSnHEWiIA.png" /><figcaption>Image Source: 365 Data Science</figcaption></figure><p>This article is a simple example of testing assumptions in a an online retail shop. The data used is from online transactions between 01/12/2010 and 09/12/2011 from a UK-based store. More information on the data set can be obtained <a href="https://www.kaggle.com/carrie1/ecommerce-data">here</a></p><h3>Basic Structure of Data</h3><p>The data contains information like customer ID, item description, timestamp of transaction, quantity purchased, unit price, and customer’s country. Below is the first five rows of the data.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatapane.com%2FAjibola%2Freports%2Ffirstfiverows_e2afcb49%2Fembed%2F&amp;display_name=Datapane&amp;url=https%3A%2F%2Fdatapane.com%2FAjibola%2Freports%2Ffirstfiverows_e2afcb49%2F&amp;image=https%3A%2F%2Fdatapane.com%2Fstatic%2Fpython_report_illustration.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=datapane" width="800" height="625" frameborder="0" scrolling="no"><a href="https://medium.com/media/59cf1b10ab6c818711d811513f0a5296/href">https://medium.com/media/59cf1b10ab6c818711d811513f0a5296/href</a></iframe><p><strong><em>Key Features of the data:</em></strong></p><ol><li>The raw data contains 541909 rows</li><li>The <em>Quantity</em> and <em>UnitPrice </em>columns contain negative values, which ideally is not correct</li><li>There were missing values for <em>CustomerID</em> and <em>Description</em></li><li>Data contains very heavy outliers both on the left and right</li></ol><h3>Scenarios to Handle</h3><p>To make the article more precise, let’s imagine two simple scenarios:</p><ul><li>Let’s imagine the manager of the retail store decides to give some bonus to the highest paying customer in every country, and to do this he assumes that customer who has the highest number of order count per country are the highest paying customers. In other words, he believes that the average revenue generated from these customers is more than the revenue generated from the whole population of customers. Is there enough evidence to accept the manager’s hypothesis?</li><li>The assistant manager thinks the manager’s method is not comprehensive enough and opines that a customer may actually have high order count but not necessarily high number of units per order. So he recommends that customers with the highest average total units purchased should be given preference instead. The manager acknowledges his point but is not sure if there will be any major difference between them. So they decide to enlist the help of the data analyst for a quick hypothesis testing on their claims.</li></ul><h3>Data Cleaning</h3><p>The following commented code snippets are used to clean the data.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/de1af7e01fe91f8eea57b0bc81cdd4aa/href">https://medium.com/media/de1af7e01fe91f8eea57b0bc81cdd4aa/href</a></iframe><h3>Solution: Scenario One</h3><p>To reiterate, we want to check if the manager’s claim that customers with high count of orders per country generate higher revenue on average than all customers put together.</p><p>A quick view to the highest-ordering customers per country is generated thus:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f407c32a3f5ad287bd7fb16b76730e8d/href">https://medium.com/media/f407c32a3f5ad287bd7fb16b76730e8d/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatapane.com%2FAjibola%2Freports%2FHighestorderingcustomers%2Fembed%2F&amp;display_name=Datapane&amp;url=https%3A%2F%2Fdatapane.com%2FAjibola%2Freports%2FHighestorderingcustomers%2F&amp;image=https%3A%2F%2Fdatapane.com%2Fstatic%2Fpython_report_illustration.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=datapane" width="800" height="625" frameborder="0" scrolling="no"><a href="https://medium.com/media/58377e8eef18c423917cf75e66d3f7f9/href">https://medium.com/media/58377e8eef18c423917cf75e66d3f7f9/href</a></iframe><p>The hypothesized population mean revenue (assumed from the whole dataframe) is 12.77 while the mean revenue for highest-ordering customers is 13.15. The aim of the test is to see if this difference is just due to random variation or that it is actually inherent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*lquVNwjShSX3Qdz5qCNExA.png" /></figure><h4>Sample Distribution</h4><p>A sample distribution will offer a better view into the spread of the units</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*mn3I25g4fqygpFF9jY7wXw.png" /></figure><p>The plot shows the sample is somewhat bimodal and also skewed to the right. However to validate whether or not the sample is normally distributed, tests of normality will be carried out.</p><h4>Hypothesis Testing</h4><p>Before delving into the tests, let’s have a quick reminder on the steps involved:</p><ol><li>State hypothesis</li><li>Set decision criteria</li><li>Calculate Test-statistic and P-value</li><li>Make decision</li></ol><blockquote>Note that these four steps will be explicitly stated only in the main hypothesis tasks and not in the auxiliaries</blockquote><p>Since we do not have access to all the data and thus cannot obtain the population standard deviation, t-test is a good option to start off with, and the necessary assumptions needed to validate it goes thus:</p><ul><li>Each unit of the sample is an independent observation. (This assumption is simply implied)</li><li>The population from which the sample is drawn is normally distributed. (This is checked using both Shapiro Wilk’s test and the QQ-Plot)</li></ul><p><strong><em>Shapiro Wilk’s Test of Normality: </em></strong>The sample is tested for normality with the one-liner below. Note that sample size of 100 is used because the test is very sensitive to large sample size.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b2006b85649e72eef99704afa864bbb4/href">https://medium.com/media/b2006b85649e72eef99704afa864bbb4/href</a></iframe><p>And the result obtained is:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/83984b683b7d152978fcd1c1191c56c0/href">https://medium.com/media/83984b683b7d152978fcd1c1191c56c0/href</a></iframe><p>To further validate the above decision, a QQ-plot is displayed to see the sample distribution relative to the normal distribution</p><p><strong><em>QQ-plot:</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/634/1*0HZJV4CHu01NDxQlpMAFAA.png" /></figure><p>The sample clearly doesn’t follow the normal distribution (the red line), therefore a non-parametric test will be preferred for decision making</p><blockquote>Note that the Central Limit Theorem still encourages t-test if the sample size is large.</blockquote><p><strong>Wilcoxon Signed Rank Test</strong></p><p>This is a suitable non-parametric alternative to the one-sample t-test when the necessary assumptions are not met. Now the steps highlighted earlier will be followed.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/892a42cc567d09fa5d5df61ba4b2346a/href">https://medium.com/media/892a42cc567d09fa5d5df61ba4b2346a/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d48d0c2813ea3afd532e3f2373b6ba3b/href">https://medium.com/media/d48d0c2813ea3afd532e3f2373b6ba3b/href</a></iframe><blockquote>Wilcoxon Signed Rank test uses median instead of mean</blockquote><p><strong><em>Conclusion: </em></strong>With the result above, we can conclude that the managers assumption is not good enough, granted we took 5 percent risk of saying he was correct in the first place.</p><h3>Solution: Scenario Two</h3><p>As a reminder again, the assistant manager thinks the average quantity bought per order is much more granular and better for decision making. So to ascertain if the assistant manager’s idea is better, we will compare it to the manager’s.</p><p>The dataframe for customers with the highest average quantity purchased per country is generated thus:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1738fd4c0448f65e2508f1ecbf235d5c/href">https://medium.com/media/1738fd4c0448f65e2508f1ecbf235d5c/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatapane.com%2FAjibola%2Freports%2FHighestorderingcustomers_154ebd9a%2Fembed%2F%3Faccesstoken%3D5221c9c83d483055e4c50bb83e1fcfe74d18f9d3&amp;display_name=Datapane&amp;url=https%3A%2F%2Fdatapane.com%2FAjibola%2Freports%2FHighestorderingcustomers_154ebd9a%2F%3Faccesstoken%3D5221c9c83d483055e4c50bb83e1fcfe74d18f9d3&amp;image=https%3A%2F%2Fdatapane.com%2Fstatic%2Fpython_report_illustration.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=datapane" width="800" height="625" frameborder="0" scrolling="no"><a href="https://medium.com/media/527c2c341a4746fb62b906938639d04a/href">https://medium.com/media/527c2c341a4746fb62b906938639d04a/href</a></iframe><p>The mean revenue obtained for this customers is 26.67, which is quite greater than that for highest-ordering customers, as the bar chart below shows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*GDsQtkt_H1GC_86OtG9eaw.png" /></figure><blockquote>Despite this glaring difference between the two means, one cannot conclude that they are <strong>significantly</strong> different, until a test of hypothesis confirms it.</blockquote><h4>Sample Distribution</h4><p>As before, a sampling distribution can give us a better view into the data as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*demFhLFP1hdzPV3j71GLZg.png" /></figure><p>The graph shows that the revenue from highest average quantity customers is even more skewed than that of highest-ordering customers, leading to the mean being dragged higher.</p><h4>Hypothesis Testing</h4><p>Since the first distribution failed the normality test for t-test assumption, there is a chance that the residual of both distributions (which is a faster way of checking assumptions for two samples) will also fail the test. However, is advisable to still test.</p><blockquote>In addition to the earlier assumptions, we will also check if the variances of the two samples are equal, a common method being Levene’s test</blockquote><p><strong><em>Levene’s Test for Equal Variance: </em></strong>This test will be run first, because defaulting it does not totally discourage the use of a parametric test, as compared to normality test.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e2f94b7f1787ac563c4c4ba2c0bd87ba/href">https://medium.com/media/e2f94b7f1787ac563c4c4ba2c0bd87ba/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c23ba43ecb77710b2f1314dbc63073bc/href">https://medium.com/media/c23ba43ecb77710b2f1314dbc63073bc/href</a></iframe><p>Test shows non-equality of variance.</p><p><strong><em>Shapiro Wilk’s Test of Normality: </em></strong>Checking for normality for two samples can be done with the residual. Before going on with the test, let’s have a quick view of the histogram of the residuals:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/634/1*CjjpF4vaeEFC7k5mOaf-Lw.png" /></figure><p>The histogram looks somewhat bell-shaped, a subtle hint at normality. The code below generates the Shapiro Wilk’s test result:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/18b5ed16e07348f8ff1806e18f75e370/href">https://medium.com/media/18b5ed16e07348f8ff1806e18f75e370/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/da935e0994b9369acf93d55e9e5bc715/href">https://medium.com/media/da935e0994b9369acf93d55e9e5bc715/href</a></iframe><p>Result shows non-normality. It is always a good practice to combine point-estimate checks with visualization checks to increase certainty, hence, the need still for a QQ-plot still.</p><p><strong><em>QQ-plot:</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*CmccW1apyw2cSwMkbasEjA.png" /></figure><p>With this, we can confidently say the two samples do not follow the normal distribution. Hence we can opt for a non-parametric test.</p><p><strong>Wilcoxon MannWhitney Test</strong></p><p>This is an alternative to two-sample t-test. Like the Wilcoxon Signed Rank Test, it uses the median instead, because median is not affected by outliers. The result obtained goes thus:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/da47cf14ef821209bed4552f3e90d8b0/href">https://medium.com/media/da47cf14ef821209bed4552f3e90d8b0/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba09a447fef125ebb875c2475c6add26/href">https://medium.com/media/ba09a447fef125ebb875c2475c6add26/href</a></iframe><p><strong><em>Conclusion: </em></strong>With the test result confirming that the sample median values are not the same, we can conclude that the assistant managers assumption of rewarding customers who purchased the highest average quantity of goods is better.</p><p>Thank you for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1025bdab1f1d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Difference between EDA and Hypothesis Testing: a short conversation]]></title>
            <link>https://medium.com/@babagee500/the-difference-between-eda-and-hypothesis-testing-a-short-conversation-976a3738e046?source=rss-8ae1dd25a0df------2</link>
            <guid isPermaLink="false">https://medium.com/p/976a3738e046</guid>
            <dc:creator><![CDATA[Ajibola Salami]]></dc:creator>
            <pubDate>Mon, 27 Jul 2020 09:10:13 GMT</pubDate>
            <atom:updated>2020-07-27T09:10:13.035Z</atom:updated>
            <content:encoded><![CDATA[<p>In my conversation with a friend recently. we had this little argument about the difference between exploratory data analysis and hypothesis testing and I thought it is worth sharing. Though a lengthy conversation, I shortened it for the TLDRs. Plus technical terms are omitted so as to make it as simple as possible. Yea let’s dive in!</p><p><strong>Friend:</strong> What is really the difference between exploratory data analysis and hypothesis testing? Can you help clarify?</p><p><strong>Me:</strong> Yes, they are different. One is used to summarize the basic characteristics of your datasets while the other is used to verify those characteristics.</p><p><strong>Friend:</strong> No. I need a simple example. Can you paint a scenario?</p><p><strong>Me:</strong> Okay. Imagine we randomly select 1000 male and 1000 female LinkedIn users and calculate their respective average age, which average would be greater?</p><p><strong>Friend:</strong> And how does that answer my question?</p><p><strong>Me:</strong> It will.</p><p><strong>Friend:</strong> Ehn … one cannot be sure. Depends on the ages of those selected for each gender. One cannot be certain which will be greater.</p><p><strong>Me:</strong> Fantastic. You just answered your question.</p><p><strong>Friend:</strong> How? Be serious please.</p><p><strong>Me:</strong> Yes you have. We cannot be certain of the answer. That’s what exploratory analysis provides: summary and uncertainty.</p><p><strong>Friend:</strong> And are you saying hypothesis testing will tell which is greater?</p><p><strong>Me:</strong> Yes.</p><p><strong>Friend:</strong> Then hypothesis testing is some kind of magic.</p><p><strong>Me:</strong> (I laughed at this point) No. We will first calculate the averages of both. Then we test.</p><p><strong>Friend:</strong> Why test since you’ve already done the calculation and now know which is greater?</p><p><strong>Me:</strong> (Now I knew I had omitted some important details) Okay I get your point. But look at it from this angle. Lets’s say we did and calculated the average age of male users to be greater than that of female users, can we be so sure of the answer —</p><p><strong>Friend:</strong> — Why not?</p><p><strong>Me:</strong> — I mean to say if we select another random sample of 1000 each and find the averages, can we be certain that the male average age will still be greater than that of female?</p><p><strong>Friend:</strong> (He paused now, clearly in thought, and I thought my point was hitting him already. I smiled.) Wait. Wait. Why would we need to take another sample in the first place? We want to know the greater average age between male and female LinkedIn user and we already have that through the calculation. Why select another sample?</p><p><strong>Me:</strong> (Now I was brimming) You have a good point. But what’s the essence of data analytics and statistics if not for decision making? And in making decision we want to consider all possible units of our dataset. If what we are <strong>solely</strong> interested in is just those 1000 male and female LinkedIn users, then the calculated averages are enough to make our decisions. But if we want to make any decision based on the population of all LinkedIn users, we cannot use the calculated averages.</p><p><strong>Friend: </strong>Ehn … that’s fairly clear, but wait again. If we want to make a statement about all LinkedIn users, why not just use all LinkedIn users instead of samples? Isn’t that just intuitive enough.</p><p><strong>Me: </strong>(At this juncture I knew I was close to winning.) Well, in real life, you don’t always have access to the whole population. It’s either too dispersed, incessantly changing, or too expensive to cover. So to mitigate this difficulty we always use well collected samples.</p><p><strong>Friend:</strong> So what you trying to say is that, if after collecting those 1000 samples and the average age of male users is greater than that of female users, we cannot use it to generalize for the whole population of LinkedIn users?</p><p><strong>Me:</strong> Exactly! Now you get the point!</p><p><strong>Friend: </strong>So hypothesis testing is what you will use to be able to generalize?</p><p><strong>Me:</strong> You deserve a medal man!</p><p><strong>Friend:</strong> Wait. Don’t get excited yet. Now tell me how on earth you will use the result obtained from 1000 samples to generalize for millions of other people. How woefully correct can that be?</p><p><strong>Me:</strong> Well that’s Statistics. We do not promise you 100 percent accuracy. We only try as much as possible to minimize the errors.</p><p>At this point my friend did not say anything again. He just looked at me like:</p><p>LOL. So tell me, how would you have approached this conversation if you were in my shoes?</p><p><em>Originally published at </em><a href="https://www.linkedin.com/pulse/difference-between-eda-hypothesis-testing-short-salami-ajibola"><em>https://www.linkedin.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=976a3738e046" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploratory Data Analysis on Nigerian Graduates in the Labour Market]]></title>
            <link>https://medium.com/@babagee500/exploratory-data-analysis-on-nigerian-graduates-in-the-labour-market-90f4599dd594?source=rss-8ae1dd25a0df------2</link>
            <guid isPermaLink="false">https://medium.com/p/90f4599dd594</guid>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[nigeria]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[jobs]]></category>
            <dc:creator><![CDATA[Ajibola Salami]]></dc:creator>
            <pubDate>Fri, 19 Jun 2020 09:41:18 GMT</pubDate>
            <atom:updated>2020-06-19T09:41:18.267Z</atom:updated>
            <content:encoded><![CDATA[<p>Employment in Nigeria happens to be a big issue and most graduates, after fulfilling the basic requirements for an entry level job, still find it hard to secure one. And for those are able to, they either end up in an entirely new field (away from what they actually studied in school or desire) or in low-paying jobs. In reality, there is a high rate of unemployment in the country and some graduates tend to wander for a number of years before they are employed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/312/1*IJoqNcKAyaUeiFHVzXy79w.jpeg" /><figcaption>image source: currentschoolnews.com</figcaption></figure><p>While this project attempts to shed some light on the life of an average Nigerian graduate in the labour market, it is noteworthy to mention that it is relatively limited in scope due to the smallness of the data size compared to the population of graduates the country churns out every year . So the project will be more useful in the exploratory sense of data analysis than in gathering inference-worthy insights. However, the results obtained show considerable anecdotal evidence.</p><p><strong><em>Data Source</em></strong>: The data was obtained through a Google Form survey. The respondents are graduates who completed their degree within 2013–2017, recruited via email and social media, and a few offline. Find the full information on the data source <a href="https://www.kaggle.com/stutern/nigerian-gradaute-report-2018">here</a>.</p><h4>Questions To Answer</h4><p>The data contains a lot of categorical features, so deciding a handful but not exhaustive number of possible insights that could be drawn from the data will help make this project more direct and concise. So, apart from the necessary data preprocessing, the following questions will be answered through visualizations:</p><ol><li>Which job sectors offer most graduates entry-level job?</li><li>Which job sector do most graduates prefer?</li><li>At what rate do graduates switch from/to job sectors?</li><li>Is there any correlation between graduates’ first income and current income?</li><li>How is current income distributed by highest educational attainment?</li><li>Does working longer hour imply higher income?</li><li>Who earns more on average between employed and self-employed graduates?</li><li>Do graduates generally earn more with time and is there any variation by gender?</li><li>At what rate is qualification required to secure a job?</li><li>Through what medium do employed graduates mostly get their jobs?</li></ol><p>This is by no means an exhaustive list of the number of possible analysis that could be carried out from this data. The questions are only posed to help better define the path of the analysis .</p><h4><strong>Basic Structure of Data</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9zOs-u28ZLWdn5Y7PYtNJg.jpeg" /><figcaption>The first three rows of the data</figcaption></figure><p>The data contains 5219 rows and 36 columns, 32 of which are object data type, 2 floats and 2 integers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*UrG2V_830Cqlwqln2CPPjw.jpeg" /><figcaption>numerical variables</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*hWXhAEL0w4aldS6MmiDJ9w.jpeg" /><figcaption>categorical variables</figcaption></figure><p><strong>Key Observations:</strong></p><ol><li>Most of the variables have lengthy names.</li><li>There are couple of twins variables (that is columns with the same unique values) since the survey collected data on first employment and current employment. Example of such columns are <em>first employment monthly income level </em>and <em>current employment monthly income level.</em></li><li>Aside from Timestamp, some of the variables contain huge number of unique values.</li><li>Few variables have more than 50 percent of their data missing</li></ol><h4>Data Preprocessing</h4><p>As usual, data cleaning is an inevitable part of data analysis, barring a few instances when we have already clean data. In this case the cleaning ideally should start from the proposed questions to answer, meaning all data unnecessary to obtaining answers would be dropped. However, some of these data were useful in some earlier part of the cleaning, e.g when condensing the datasets. Plus renaming first would ease the burden of long typing. Thus the first step will be to rename the lengthy variables.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9458584082a54a56ca859f617c5bc50a/href">https://medium.com/media/9458584082a54a56ca859f617c5bc50a/href</a></iframe><p>Then the data was condensed to include only graduates who have completed their NYSC. The reasons for this are: a) Except those who are above thirty years old and others who may have some genuine excuse for exemption, almost all first degree graduates in Nigeria go for the programme, b) It is often a prerequisite in most local job openings, and c) It provides most graduates a platform to transition into the workforce.</p><p>Then the redundant variables were dropped, followed by replacing lengthy values with shorter ones and binning where necessary. Plus a couple other minor cleaning as shown below:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7a0b5565818134c3da12fce3eb7bf3af/href">https://medium.com/media/7a0b5565818134c3da12fce3eb7bf3af/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/992/1*Q1p-7XglaHEYjbO9Kmx7TQ.jpeg" /><figcaption>The data</figcaption></figure><p>The final part of the data cleaning was to fill the missing values. As stated earlier, some of the variables, the categorical variables precisely, have high proportion of missing values and a keen look at it shows that they are missing not at random. For instance, a person who has no current job would definitely leave all other columns depending on current job blank, such as current income, current job sector, etc. So these cells were filled directly and consistently across all the affected columns to show they are structural. On the other hand, the numerical missing data were filled via imputations.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d89cde41db31188769be5e1139f2596d/href">https://medium.com/media/d89cde41db31188769be5e1139f2596d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*6w7a-QTWvpmRuC1n63leMw.jpeg" /><figcaption>Cleaned Data</figcaption></figure><h4><strong>Feature Engineering</strong></h4><p>To further make sense out of the data, numerical features were engineered out of the income variables, which initially were objects of range of values. Mean income categories were created. However, that was not enough to portray the wider range of values that the data would show in an ideal situation, prompting the engineering also of columns of random values that fall within the original buckets. Plus the randomness will improve visualizations.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/79920fea7bdba02c17b790820ebbca84/href">https://medium.com/media/79920fea7bdba02c17b790820ebbca84/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/991/1*9VbD97pnRIzrt6GikNZ_HA.jpeg" /><figcaption>Ready Data</figcaption></figure><h4>Exploratory Analysis: Answers to Questions</h4><ol><li><strong>What are the top five job sectors that offer entry level jobs?</strong></li></ol><blockquote>Note that percentages were rounded to the nearest whole number and thus may show minor discrepancy in bar heights</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*F6To6Qa7QM5EZuaQ43P_4w.png" /><figcaption>Figure 1</figcaption></figure><p><strong><em>Key Observations: </em></strong>There are 21 job sectors altogether, and also common among them are <em>Telecommunications, Media/Creatives, Engineering, and Marketing/Communication.</em></p><p><strong>2. What are the job sectors graduates prefer most?</strong></p><p>Given the rate of unemployment in the country, many graduates tend to end up with the available and not the desirable. The chart below shows the five most preferred job sectors</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*X8pau8gARZktOKvsE1Jb_w.png" /><figcaption>Figure 2.1</figcaption></figure><p><strong><em>Key Observations: </em></strong>Noteworthy here are two things: the inclusion of <em>Oil&amp;Gas, </em>which was not among the top five in Figure 1 and only offered entry level jobs to 3 percent of the graduates, and the exclusion of <em>Education, </em>ranking low in the order of preference at 4 percent, as compared to what it offers.</p><p>To easily compare the difference between the two figures above, Figure 2.2 places each of the job sectors in both figures side by side. Plus in this case all job levels are considered as not all first jobs are of entry level</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/1*3nSPn1QEMZbptLlODfyXaQ.png" /><figcaption>Figure 2.2</figcaption></figure><p><strong><em>Key Observations: </em></strong>The bars for <em>Education</em> and <em>Oil&amp;Gas</em> show the stark difference between what graduates get and what they want in the job sector.</p><p>But one expects that as graduates move to new jobs they will be moving to their preferred job sectors, thus reducing the difference in the percentages on each pair of bars in Figure 2.2</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/786/1*bTs3PWRJQTSCYNHG7ykutA.png" /><figcaption>Figure 2.3</figcaption></figure><p><strong><em>Key Observations: </em></strong>While more percentage of graduates tend to be moving towards their preference as they switch job and thus closing the gap a little in job sectors like <em>Technology</em> and <em>Consulting</em>, same cannot be said for job sectors like <em>Oil&amp;Gas</em> and <em>Banking/Finance</em> where the preference percentage difference remains almost the same or bigger. That is to say these latter job sectors, so long the labour market absorbs new graduates, are likely to remain preferred.</p><p><strong>3. At what rate do graduates switch from/to job sectors?</strong></p><p>Figure 2.2 and 2.3 above only show the ratio between the job sectors graduates prefer and what they get, but does not show the rate at which people switch to and from different jobs. For instance, while we can from the above charts have an inkling and say more graduates are switching from <em>Education</em> to other job sectors given the reduction from 16 percent to 12 percent between first and current job, we can not say that for, say, <em>Technology, </em>which has the same percentage value. However given the increment in the total number of graduates having job as we move to the present, there is need to check the switch rate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*oK-XV4zvCvD8rxXSj4dmcA.png" /><figcaption>Figure 3.1</figcaption></figure><p><strong><em>Key Observations: </em></strong>The above shows that the supposed 4 percent difference in <em>Education</em> is a actually large in magnitude. Also interesting is the <em>Healthcare</em> sector, which has no switch percentage. This means two things: it’s either those whose first job is in healthcare remain in healthcare or the rate at which healthcare workers switch to other job sectors is the same rate at which it gains new entrants, which is quite more intuitive in the former case given the level of professionalism the job often requires.</p><p>Now switching is for those with more than one job so far. How about those who have only had one job? Comparing this first entrant rate to the switch rate above will give insight on the overall equilibrium of each of the job sectors:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/899/1*XwnEWOKWyswnF_lRzB0hnw.png" /><figcaption>Figure 3.2</figcaption></figure><p><strong><em>Key Observations: </em></strong>While most of the job sectors are tilted towards always gaining more graduates, <em>Education</em>, <em>Legal</em>, and <em>Manufacturing</em> are tilted towards losing more. Which in the case of <em>Education</em> is quite pathetic given that it offers most graduates their first job.</p><p><strong>4. Is there any correlation between graduates’ first income and current income?</strong></p><p>To answer this, let’s have a quick overview of the distribution of both incomes</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/909/1*qZKqX3aoZcYhSMX-CmQXUw.png" /><figcaption>Figure 4.1</figcaption></figure><p><strong><em>Key Observations: </em></strong>The original income data came in ranges, e.g, 20,000-49,000. To make it useful for numerical analysis, the mean was taken to represent the actual figure. However such figures make distributions quite static in display, hence the need for randomized figures (i.e random value that falls within that range).</p><blockquote>The essence of the randomized values is to improve visualization and were used only in cases where scatterplots were plotted. Elsewhere the mean values were used. Otherwise they are both clearly stated.</blockquote><p>Now to the relationship between incomes</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/845/1*0jksmFP1io3xo3wxCecfiA.png" /><figcaption>Figure 4.2</figcaption></figure><p><strong><em>Key Observations: </em></strong>The figure shows a positive correlation between <em>First Income </em>and <em>Current income</em>. That means graduates who earn higher in their first job are likely to earn higher in their current job and vice versa.</p><p><strong>5. How is current income distributed by highest educational attainment?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*tEV2FWtydWKAJocq6bipUw.png" /><figcaption>Figure 5</figcaption></figure><p><strong><em>Key Observations: </em></strong>BSc and HND have similar distribution, even though the average income for BSc holders is higher. Generally income increases as education increases, except for PhD, which has very few data points to back it up.</p><p><strong>6. Does working longer hour imply higher income?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*zY_VxrGxtee99FF-ujvhWA.png" /><figcaption>Figure 6</figcaption></figure><p><strong><em>Key Observations: </em></strong>The plot shows a weak relationship between work hours and income. Meaning graduates’ working duration does not really affect their income.</p><p><strong>7. Who earns more between employed and self-employed graduates?</strong></p><p>There tend to be some argument about who earns more between gainfully employed graduates and ones who venture into entrepreneurship.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*0fpN5URvD2VzVeIK2qOE6w.png" /><figcaption>Figure 7</figcaption></figure><p><strong><em>Key Observations: </em></strong>Apparently employed graduates earn more, and where both lines meet at the peak has no spread for self-employed graduates. But a closer look shows there is a larger variation in the earnings of the self-employed, which is likely due, for one, to individual’s successes and unstructured income plans as compared to corporate settings where income is structured.</p><p><strong>8. Do Graduates generally earn more with time and is there any variation by gender?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*y_jipIwvwJSLATip_YYyKA.png" /><figcaption>Figure 8</figcaption></figure><p><strong><em>Key Observations: </em></strong>The graph shows general increase in average income with time. However when separated by gender, male graduates earn more than the females.</p><p><strong>9. At what rate is qualification required to secure a job?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*s6eD4Ikq5tBr5EXmQc28Lg.png" /><figcaption>Figure 9</figcaption></figure><p><strong><em>Key Observations: </em></strong>Highly preferred job like <em>Oil&amp;Gas </em>requires 75 percent of the time that graduates have a qualification. Professional courses like <em>Healthcare, Legal</em> and <em>Banking/Finance</em> also have high rate of qualification requirement. <em>Technology, </em>with 54 percent, has a relatively low rate, underscoring the fact that technical abilities could be just as essential, likewise in the case of <em>Media/Creatives</em> and <em>Trade/Services</em> where creativity and business acumen could just be enough respectively.</p><p><strong>10. Through what channel do employed graduates get their jobs?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*XsHg1mTFbXnlYjEa3m8Kjw.png" /><figcaption>Figure 10</figcaption></figure><p><strong><em>Key Observations: </em></strong>Most graduates get their job through <em>Personal Contact</em>, which may further amplify the general notion that people get employed through ‘connection’. Little wonders that <em>Internship</em> is relatively low as getting one could be like getting a job itself.</p><h4>Conclusion:</h4><p>The analysis is basically exploratory and only gives a first overview of the data. To make decision-based analysis, a foray would be taken into inferential statistics and machine learning.</p><p>Thank you for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=90f4599dd594" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>