<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by ADELEKE ADEDEJI on Medium]]></title>
        <description><![CDATA[Stories by ADELEKE ADEDEJI on Medium]]></description>
        <link>https://medium.com/@adelekedejiimoleayo?source=rss-29e5af4a8360------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*pBNiWraosiwix4Pn</url>
            <title>Stories by ADELEKE ADEDEJI on Medium</title>
            <link>https://medium.com/@adelekedejiimoleayo?source=rss-29e5af4a8360------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 08:52:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@adelekedejiimoleayo/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Exploring the possibilities of utilizing ChatGPT in the field of data science]]></title>
            <link>https://medium.com/@adelekedejiimoleayo/exploring-the-possibilities-of-utilizing-chatgpt-in-the-field-of-data-science-3ec834dfd5ef?source=rss-29e5af4a8360------2</link>
            <guid isPermaLink="false">https://medium.com/p/3ec834dfd5ef</guid>
            <dc:creator><![CDATA[ADELEKE ADEDEJI]]></dc:creator>
            <pubDate>Wed, 24 Sep 2025 14:51:47 GMT</pubDate>
            <atom:updated>2025-09-24T14:51:47.359Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-jCKRU2yw8MzLHS0BLbAKA.jpeg" /></figure><p>What is CHATGPT?</p><p>The OpenAI-developed GPT-3 language model has a variation called ChatGPT. It is specially made for conversational applications like chatbots, virtual assistants, and question-and-answer programs. ChatGPT can be fine-tuned for certain tasks or conversations to increase its performance after being pre-trained on a sizable dataset of informal text. It can be applied to jobs like text production, data augmentation, language understanding and interpretation, and other tasks like creating synthetic data and enhancing the performance of current models. The production of chatbots, the creation of content, and language understanding are just a few of the many uses for ChatGPT.</p><p>WHO Creates CHATGPT?<br>OpenAI, a research organization dedicated to ensuring that artificial intelligence (AI) is developed in a manner that is secure and advantageous for all humanity, created ChatGPT. Elon Musk, Sam Altman, and Ilya Sutskever were among the technical pioneers who launched OpenAI, which is also backed by a number of illustrious investors and partners. A group of researchers and engineers at OpenAI was in charge of developing ChatGPT. They trained the model on a sizable corpus of text data using cutting-edge deep-learning methods. The concept was first introduced in a research article in 2019 and afterwards made available to the general public. The ChatGPT team has been working regularly to enhance the model by including fresh data and methodologies to increase its robustness and accuracy, as well as by tailoring it to various topics and languages. The most recent version of the model, ChatGPT-3, was released in June 2020 and has been extensively employed in numerous sectors and academic projects. It has attained cutting-edge performance on a variety of natural language processing benchmarks. To make it simple for developers to utilize and incorporate into their applications, OpenAI continuously releases updates and upgrades to the model in addition to an API and pre-trained models.</p><p>HOW WAS CHATGPT TRAINED?</p><p>Unsupervised learning, which entails training a model on a sizable dataset without explicit labels or annotations, was used to train ChatGPT. A dataset of more than 40 GB of text data, which includes a variety of materials like novels, papers, and websites, was used to train the model. The text data was tokenized, or divided into individual words or phrases, as part of the preprocessing stage before being used to train the model. During training, large amounts of text data are fed to the model, and its parameters are adjusted until the model can correctly predict the next word in a phrase given the preceding words. The model is adjusted when it is exposed to more and more data during the course of several iterations of this procedure. To enhance performance, the model’s architecture, including the number of layers and the size of the embeddings, is modified during the training phase.</p><p>The model may be fine-tuned for certain natural language processing tasks, including question answering, language translation, and text summarization. Once fully trained, the model was able to generate highly coherent and contextually appropriate text.</p><p>ChatGPT, like all language models, has some limitations. Some of the main limitations of the model include:</p><ol><li>Bias: ChatGPT, like all language models, has been trained on a dataset of text from the internet, which can contain biases and stereotypes. This can lead to the model generating text that is biased or insensitive.</li><li>Lack of understanding of certain topics: ChatGPT, like all language models, has been trained on a large dataset of text, but it may not have been exposed to all possible topics or information. This can lead to the model not having a complete understanding of certain topics or not being able to generate coherent text about them.</li><li>Limited interpretability: ChatGPT, like all deep learning models, can be difficult to interpret and understand how it arrived at its predictions. This can make it difficult to identify and fix errors or biases in the model.</li><li>Computational cost: ChatGPT is a large language model, which requires significant computational resources to run. This can make it difficult to use on low-powered devices or in resource-constrained environments.</li><li>Limited context: ChatGPT is trained to generate text based on the context of the input, but it may not always have enough context to generate accurate and coherent text.</li></ol><p>It’s important to keep in mind that these limitations are common to all language models, and the team behind ChatGPT continues to work on improving the model to overcome these limitations.</p><h3><strong>How Can ChatGPT Be Used as Data Scientist?</strong></h3><p>ChatGPT can be used as a data scientist in a number of ways:</p><ol><li>Text generation and data augmentation: ChatGPT can be used to generate large amounts of synthetic text data, which can be used to augment existing datasets for natural language processing tasks. This can be particularly useful for tasks such as text classification, language understanding and interpretation, and question answering.</li><li>Language Understanding: ChatGPT can be fine-tuned for specific natural languages processing tasks, such as question answering, language translation, and text summarization. This can be useful for tasks such as sentiment analysis, keyword extraction, and named entity recognition.</li><li>Text summarization: ChatGPT can be used to generate a summary of a given text, which can be useful for tasks such as summarizing long documents, articles or news.</li><li>Text completion: ChatGPT can be used to generate text based on a given prompt or context, which can be useful for tasks such as writing assistants, code completion, or even creating poetry or fiction.</li><li>Data Cleaning: ChatGPT can be used to automatically detect and correct errors in text data, such as typos, grammatical errors, or formatting inconsistencies. This can be useful for tasks such as data validation and cleaning, preparing data for analysis, or even training other models.</li><li>Synthetic data generation: ChatGPT can be used to generate synthetic data for tasks such as training models for natural language understanding and age, which is useful for tasks such as training chatbots and virtual assistants.</li><li>Language model fine-tuning: ChatGPT can be fine-tuned for specific use cases and languages, this can be useful for tasks such as language model fine-tuning, and language-specific use cases.</li></ol><p>It’s important to keep in mind that these are just some examples of how ChatGPT can be used as a data scientist, and the possibilities are almost endless, depending on the specific use case and the requirements of the task.</p><p>I will be using online competitions for Kaggle.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3ec834dfd5ef" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Data Science to Combat Customer Churn in the Utility Industry: A Case Study]]></title>
            <link>https://medium.com/@adelekedejiimoleayo/using-data-science-to-combat-customer-churn-in-the-utility-industry-a-case-study-69c4bc13cb75?source=rss-29e5af4a8360------2</link>
            <guid isPermaLink="false">https://medium.com/p/69c4bc13cb75</guid>
            <category><![CDATA[utility-industry]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[gas-and-electricity]]></category>
            <category><![CDATA[feature-engineering]]></category>
            <category><![CDATA[churn-prediction]]></category>
            <dc:creator><![CDATA[ADELEKE ADEDEJI]]></dc:creator>
            <pubDate>Sat, 06 Jan 2024 17:07:01 GMT</pubDate>
            <atom:updated>2024-01-06T17:07:01.746Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W00AfAJUKJwvh2lMNQipEg.jpeg" /><figcaption>Image From Adobe Stock</figcaption></figure><p>In the past year, the field of data science has played a pivotal role in driving business growth and aiding industries in customer retention by providing insights into areas where they should focus on generating new sales.</p><p><strong>Challenge:</strong> A gas and electricity utility company faces high customer churn, particularly among SMEs. Pricing changes are suspected as a key driver.</p><p><strong>Solution:</strong> Build a predictive model to identify churn-prone customers based on price sensitivity and accounting for a 20% discount retention strategy.</p><p>Goal: Actionable insights to diagnose churn and inform strategic decision-making, driving customer retention and business growth.</p><p>Let’s explore the exploratory data analysis (EDA) of the two datasets.</p><p>First, we’ll load the data from the client_data.csv and price_data.csv files into separate dataframes. Theclient_data dataframe contains a combination of numeric and categorical data, requiring preprocessing for future modeling. On the other hand, theprice_data dataframe consists of purely numeric data, with noticeable occurrences of zero values.</p><p>To facilitate further understanding, please refer to the dataset documentation available on my<a href="https://github.com/Gracebaytech/Customer-Churn-Prediction"> GitHub repository.</a> You can find both the dataset files and their corresponding documentation through the provided <a href="https://github.com/Gracebaytech/Customer-Churn-Prediction">GitHub link</a>.</p><p>Let’s proceed with loading and examining the datasets for a more detailed analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/429/1*pnOx2HwdYjCAeduej6cYbA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/362/1*jiD5TLKoQQ0t1d49svm5xg.png" /><figcaption><strong>Author by me</strong></figcaption></figure><p>We can see that all of thedatetime related columns are not currently in datetime format. We will need to convert these later.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*ob5nI61rUQ30KiJcI1SclQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/775/1*flPLxxGEoE8VdxtIA5sSkQ.png" /><figcaption>Author by me</figcaption></figure><p>The ‘describe’ method provides valuable insights into our client data. The main observation is that our data is heavily skewed, as indicated by the percentile values. However, the price data appears to be generally satisfactory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/466/1*eyxYd6sF5f1XaEyp3lHWfg.png" /><figcaption>Author by me</figcaption></figure><p>About 10% of the total customers have churned. (This sounds about right.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rBGodNpuVb7rMw6I0enmPg.png" /><figcaption>Author by me</figcaption></figure><p>Surprisingly, the customers who have churned are spread across five distinct values for channel_sales. Additionally, the value labeled MISSING, which signifies a missing value added by the team during dataset cleaning, has a churn rate of 7.6%. This feature could play a significant role in constructing our model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JxLZk-BAOG_QyXjyVUGTeg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GyeDOF5OOCtBDGTPkBFNHg.png" /><figcaption>Author by me</figcaption></figure><p>The consumption data exhibits a strong positive skewness, with a significantly elongated right tail extending towards higher values in the distribution. The values at both ends of the distribution are likely outliers. To gain a more detailed view of these outliers, we can use a standard plot. It is shown below.A boxplot, which is a standardized representation based on a five-number summary (minimum, first quartile (Q1), median, third quartile (Q3), and maximum), can effectively display the distribution. It not only identifies outliers and their values but also provides insights into the symmetry, tightness, and skewness of the data.</p><p>Similar to the consumption plots, forecast variables display high positive skewness, resulting in extended tails for higher values. In the upcoming exercise, we’ll implement transformations to address this skewness.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oXnPEK-v1ksAgYC4RWY9-A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bQJjYWYItMS9b-zygUZZgQ.png" /><figcaption>Author by me</figcaption></figure><p>Other univariate and bivariate analyses for the column can be found on my GitHub account.</p><p>Now, let’s proceed with our hypothesis investigation regarding the influence of price sensitivity on churn.</p><p>Firstly, we need to define what we mean by “price sensitivity.”. Given that we have consumption data for each company for the year 2015, we will construct new features to gauge “price sensitivity.”. These features will be based on the average consumption over different periods: the entire year, the last six months, and the last three months. This will provide us with a more nuanced understanding of each company’s sensitivity to price changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/890/1*_zbhcE44uu6IcCnq--AqQg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/721/1*5zoWed2jiFLGIS71J0abIQ.png" /><figcaption>Author by me</figcaption></figure><p>The code snippet aims to transform date columns in the client_df and price_df dataframes to the datetime type. For thisprice_df, it also involves creating mean average data for different periods (year, last 6 months, last 3 months) based on price variables. The results are then combined into a single dataframe named price_features containing various mean price features for further analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JEmo6z7b5poDly1WyrBzfQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qfTQ-xX7qm7La6VaYy5flw.png" /><figcaption>Author by me</figcaption></figure><p>From the correlation plot, shows a higher magnitude of correlation between other price sensitivity variables; however, overall, the correlation with churn is very low. This indicates that there is a weak linear relationship between price sensitivity and churn. This suggests that, for price sensitivity to be a major driver for predicting churn, we may need to engineer the feature differently.</p><h3>Feature engineering</h3><p>Off-peak prices represent the utility costs during periods of low demand. In the context of building features for analysis, one approach is to calculate the percentage change in off-peak prices from January to December. This calculation may reveal potential patterns or trends that could be indicative of future behaviours, such as customer churn.</p><p>Instead of examining differences across an entire year, we’ve now crafted features that assess mean average price disparities across distinct periods (off-peak, peak, and mid-peak). The December-January feature might unveil overarching patterns occurring throughout a whole year, while inter-time-period features may expose more nuanced patterns on a monthly scale.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*aiwRXWHDsSlDe410J_iT4A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/1*mmDEzO7dyppd1S9g34T6-Q.png" /></figure><p>I chose to calculate the maximum price change between months and time periods as a noteworthy feature. As a utility customer, abrupt price fluctuations between months can be inconvenient, and a substantial price increase within a short timeframe might influence customers to explore other utility providers for better deals. Given our objective of predicting churn in this scenario, I believe this feature could offer insightful information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/524/1*HTLWJNcZp9eNL8Ad1sMPUg.png" /><figcaption>Author by me</figcaption></figure><p>From the table below, it’s evident that companies with a tenure of 4 months or less are more prone to churn compared to those with a longer tenure. Notably, there’s approximately a 4% increase in the likelihood of churn when comparing the 4th and 5th months. This suggests that surpassing a 4-month tenure could be a significant milestone in retaining a customer for the long term.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/361/1*TantzsFLJwCxiGKShAORPQ.png" /></figure><p>The duration of a client’s tenure is a valuable feature for our model, as it impacts the probability of a client churning.</p><p>Dates as a datetime object are not useful for a predictive model, so we needed to use the datetimes to create some other features that may hold some predictive power.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/1*Tz4RT40rW02HK-PxG0Sf6g.png" /><figcaption>Author by me</figcaption></figure><p>Using intuition, you could assume that a client who has been an active client of PowerCo for a longer amount of time may have more loyalty to the brand and be more likely to stay. Whereas a newer client may be more volatile. Hence the addition of the months_activ feature.</p><p>As well as this, if we think from the perspective of a client with PowerCo, if you’re coming toward the end of your contract with PowerCo, your thoughts could go a few ways. You could be looking for better deals for when your contract ends, or you might want to see out your contract and sign another one. On the other hand, if you’ve only just joined, you may have a period where you’re allowed to leave if you’re not satisfied. Furthermore, if you’re in the middle of your contract, their may be charges if you want to leave, deterring clients from churning midway through their agreement. So, I think months_to_end will be an interesting feature because it may reveal patterns and behaviours about the timing of churn.</p><p>My belief is that if a client has made recent updates to their contract, they are more likely to be satisfied, or at least they have received a level of customer service to update or change their existing services. I believe this to be a positive sign; they are an engaged customer, and so I believe months_modif_prod will be an interesting feature to include because it shows the degree of how &#39;engaged&#39; a client is with PowerCo.</p><p>Finally, the number of months since a client last renewed a contract, I believe, will be an interesting feature because, once again, it shows the degree to which that client is engaged. It also goes a step further than just engagement; it shows a level of commitment if a client renews their contract. For this reason, I believe months_renewal will be a good feature to include.</p><p><strong>Handling Categorical Data Transformation</strong></p><p>In building a predictive model, categorical or string values must be converted into numerical representations for the model to process them effectively. As a data scientist, there are different methods for encoding categorical features, with the aim of maintaining compactness and discrimination.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/488/1*q6YVlxv7ybyGz7_c7m6wug.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/418/1*dKMymuAT1bo1Xpw7WHYFJw.png" /><figcaption>Author by me</figcaption></figure><p>One straightforward method is label encoding, where each category is mapped to an integer. However, caution is needed, as this introduces the concept of order into a feature, which may not be inherent (e.g., 0 &lt; 1 &lt; 2 &lt; 3…).</p><p>Alternatively, dummy variables, also known as one-hot encoding, can be employed to encode categorical features. This involves creating a new feature for each unique value in a categorical column and assigning either a 1 or 0 to indicate the company’s membership in that category.</p><p>In our case, with 8 categories, we generate 8 dummy variables from the column. However, examining the output reveals that the last 3 categories have low occurrences (11, 3, and 2, respectively). Given the dataset’s size (about 14,000 rows), these dummy variables would be predominantly 0, contributing little predictive power to the model.</p><p>Therefore, a decision is made to drop these three dummy variables. Similar consideration is given to the “Origin_up” feature, where the last 3 categories exhibit low frequency, leading to their removal after creating dummy variables.</p><p>We observed high skewness in some numerical variables. This needs to be addressed because certain predictive models, known as parametric models, assume that all input features are independent and normally distributed.</p><p>While skewness isn’t inherently problematic, it’s generally beneficial to treat highly skewed variables. This can enhance the efficiency of predictive models in converging to the optimal solution.</p><p>Several transformations can be applied to treat skewed variables, such as square root, cubic root, and logarithm transformations. For our purposes, we’ll apply a logarithmic transformation to the positively skewed features.</p><p>Please note that the logarithm of zero is undefined, so we’ll add a constant of 1 to all values to avoid this issue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/687/1*fqcUJfs5v8L2PCsUZ_tPXQ.png" /><figcaption>Author by me</figcaption></figure><p>We’ve noticed that the standard deviation of most of these features is quite large.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/1*28seSERv6QOq8uesrl0bGQ.png" /><figcaption>Author by me</figcaption></figure><p>Initially, we’ll examine the statistics of the skewed features to establish a basis for comparison before and after transformation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/696/1*UC8S6EDGAI49RIP-uTNUwQ.png" /></figure><p>Now we can see that for the majority of the features, their standard deviation is much lower after transformation. This is a good thing; it shows that these features are more stable and predictable now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*stLLlfHkCFa1bACWvyMNPw.png" /></figure><h3>MODEL DEVELOPMENT</h3><p>We have prepared a dataset with engineered features and are set to begin training predictive models. Our approach involves testing various models and evaluating their performance using accuracy metrics. The most promising models will undergo optimization in subsequent steps through the tuning of their hyperparameters. It’s essential to focus on training numerous models, including but not limited to Logistic Regression, Random Forest, and Naive Bayes GaussianNB, Naive Bayes_BernoulliNB, MLPClassifier, Nearest Neighbors, ExtraTreesClassifier, Linear SVC, rbf SVC, SGDClassifier, AdaBoost, Decision Tree, Decision Tree Gini, Decision Tree Entropy, Gradient Boosting, LightGBM, LinearDiscriminantAnalysis, QuadraticDiscriminantAnalysis, Perceptron, PassiveAggressiveClassifier, HistGradientBoosting, and Xgboost.</p><h3>Data sampling</h3><p>The first thing we want to do is split our dataset into training and test samples. The reason we do this is so that we can simulate a real-life situation by generating predictions for our test sample without showing the predictive model these data points. This gives us the ability to see how well our model can generalise to new data, which is critical.</p><p>A typical percentage to dedicate to testing is between 20 and 30; for this example, we will use a 75–25% split between train and test, respectively.</p><p><strong>Dealing with Imbalance:</strong></p><p>Imbalanced classes are a common problem in machine learning classification, where there is a disproportionate ratio of observations in each class. I will be using SMOTE (Synthetic Minority Over Sampling Technique).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/1*2EzXz0vinqB5PfyadKnZow.png" /></figure><h3>Model Training and Performance</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/805/1*NE5RRtgVVNE-9tr3kQtatA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ibBLtUJiHbazzY744oHmpg.png" /></figure><p>The chart shows that the XGB classifier has the highest accuracy score, with a score of 0.9085. This means that the XGB classifier is able to correctly classify 90.85% of the data points. The Random Forest model is the second most accurate, with a score of 0.9071. The LightGBM model is the third most accurate, with a score of 0.9058.</p><p>The other models on the chart have lower accuracy scores. The Passive Aggressive Classifier has the lowest accuracy score, with a score of 0.7031. This means that the passive-aggressive classifier is only able to correctly classify 70.31% of the data points.</p><p>The chart also shows a line that represents the average accuracy of all the models. The average accuracy score is 0.8746. This means that the average model is able to correctly classify 87.46% of the data points.</p><h3>Hyperparameter Tuning/Model Improvement</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1019/1*CsbMRWZSGgePK9IcXA8qfw.png" /></figure><p>The code snippet demonstrates the application of RandomizedSearchCV to an XGBoost classifier. It explores various hyperparameter combinations within the defined grid, performing a specified number of iterations (‘n_iter’). The search is conducted through cross-validation (‘cv=5’) to ensure robustness. The best parameters obtained from RandomizedSearchCV are then utilized to build the final XGBoost model. This tuned model is subsequently trained on the resampled training data (‘X_train_sm’, ‘y_train_sm’) and evaluated on the validation set (‘X_test’). The achieved accuracy is printed, reflecting the model’s enhanced predictive performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/718/1*SMCoFwACzymomBpkcq2gmw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*1K-PO3WRdCUA0IrMh6O3RQ.png" /></figure><p>A few key observations can be made from these results:</p><ul><li>Approximately 10% of the rows in the test set are churners (churn = 1).</li><li>Out of all the negative cases (churn = 0), we correctly predicted 3870 out of 4226 as negative, which is a positive outcome.</li><li>We have a high number of false negatives at 356, where we predicted a client would not churn (churn = 0), but they did (churn = 1). This is an area that needs improvement in the model.</li><li>In terms of false positives, where we predicted a client would churn but they didn’t, we only have 70 cases, which is excellent.</li><li>However, out of the 86 clients that churned in the test dataset, we were only able to correctly identify 70 out of those 426, indicating poor performance.</li><li>The accuracy score is high, but it can be misleading, hence the importance of precision and recall.</li><li>The precision score is 0.64, which is decent, but there’s room for improvement.</li><li>The recall and F1 score reveal that the classifier struggles to identify positive samples, which is a major concern for model improvement.</li><li>In summary, while we’re quite accurate in identifying clients who do not churn, we struggle to predict cases where clients do churn. A significant percentage of clients are incorrectly identified as not churning when they should be identified as churning. This suggests that the current set of features is not sufficiently discriminative to distinguish between churners and non-churners.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zt-zbQppcFVohH5Y4wlXzA.png" /></figure><p>From this chart, we can observe the following points:</p><ul><li>Net margin and consumption over 12 months are the top driver for churn in this model</li><li>Margin on power subscriptions is also an influential driver</li><li>Time seems to be an influential factor, especially the number of months they have been active, their tenure and the number of months since they updated their contract</li><li>The feature that our colleague recommended is in the top half in terms of how influential it is and some of the features built off of this actually outperform it</li><li>Our price sensitivity features are scattered around but are not the main driver of customer churning</li></ul><p>The last observation is important because it relates back to our original hypothesis:</p><pre>Is churn driven by the customers&#39; price sensitivity?</pre><p>Based on the output of the feature importance, it is not a main driver but a weak contributor. However, to arrive at a conclusive result, more experimentation is needed.</p><h3>Recommendations:</h3><p>1 Focus on Improving Churner Identification: Implement strategies like:</p><p><em>Cost-sensitive learning: assign higher weight to misclassifying churners.</em></p><p><em>Feature engineering: Enhance features that discriminate churners better (e.g., churn triggers, customer engagement metrics).</em></p><p>Investigate Price Sensitivity: Conduct further experiments with price-related features to understand their true influence on churn.</p><p>3 Model Optimization: Fine-tune XGBoost hyperparameters using techniques like Optuna or Hyperopt for further performance gains.</p><p>4 Explainable AI: Utilize tools like SHAP or LIME to understand individual feature contributions to churn predictions, aiding model interpretability and trust</p><p>All screenshots were taken by the author. The introductory image is on <a href="https://stock.adobe.com/search?k=regulations+health+and+safety&amp;search_type=autosuggest&amp;continue-checkout=1">Adobe Stock.</a></p><p>In case you missed my previous one on <a href="https://medium.com/@adelekedejiimoleayo/web-scraping-and-analysis-of-qatar-world-cup-2022-1f2c15f26025"><strong>World Cup Analysis</strong></a></p><p>I sincerely hope you’ve enjoyed it and that it has some value for you in some way. You can reach me on <a href="https://twitter.com/crown_deji?t=yyzLghHlVqixn6hSEI9AMw&amp;s=09"><strong>Twitter</strong></a>,<strong> </strong><a href="https://github.com/Gracebaytech"><strong>GitHub,</strong></a><strong> </strong>or <a href="http://www.linkedin.com/in/adelekeadedeji-09aba0195"><strong>Linkedin</strong></a> if you need anything, have a suggestion, or simply want to chat.</p><p>Thanks for reading.</p><p>Follow me to view more stories.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=69c4bc13cb75" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Uncovering Crime Patterns in Chicago: Insights to Improve Public Safety]]></title>
            <link>https://medium.com/@adelekedejiimoleayo/uncovering-crime-patterns-in-chicago-insights-to-improve-public-safety-3839ba2127b8?source=rss-29e5af4a8360------2</link>
            <guid isPermaLink="false">https://medium.com/p/3839ba2127b8</guid>
            <category><![CDATA[crime]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[chicago]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[ADELEKE ADEDEJI]]></dc:creator>
            <pubDate>Wed, 29 Mar 2023 10:17:39 GMT</pubDate>
            <atom:updated>2023-03-29T10:17:39.625Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/646/1*mtg86CtnjO0TNqV46RQPsA.jpeg" /></figure><h3><strong>Introduction:</strong></h3><p>Crime is a pervasive issue in Chicago, and it has been a source of concern for the Chicago Police Department (PD) for many years. The PD has been tracking reported crimes daily since 2001, and this project data file contains valuable insights that can be used to reduce the spate of violent crimes in the city. To be effective in combating crime, the PD needs to know crime patterns, where crimes are likely to occur, and equip their department appropriately. In this project, the PD has recruited me to conduct full data analytics to uncover insights that can be used to prepare for and respond to crimes.</p><p>Our main goal is to identify the types of crimes that the PD should prioritize, where these crimes are most likely to occur, and the days of the week and periods when they can expect them to occur. By gleaning valuable insights from the data, we hope to provide the Chicago PD with actionable information to help them improve public safety in the city.</p><h3>Introduction of data</h3><p>I will be using data from the Chicago crime dataset. This data holds 20 years of criminal history, including crime type and information about the crime&#39;s location. The file crime. csv contains 2278726 rows and 23 columns; each row corresponds to an individual crime that occurred in the city of Chicago.</p><pre># To load the dataet<br>df=pd.read_csv(&quot;C:/Users/Dell/Documents/Deji Folder/Workplace/crime_data_Proj1/crime_data_Proj1.csv&quot;)<br><br># To view the dataset<br>df.head()<br>print(df.info())<br><br>print(max(df[&quot;Year&quot;]),min(df[&quot;Year&quot;]))</pre><h3>Cleaning and Completing Data:</h3><p>At this point, we are well aware of our data. I know that it has some missing values and some features that are not important for this analysis.</p><p>I will perform different operations on it. E.g</p><p>● Exclude all features from the data where they are not needed for this analysis. E.g IUCR, FBI code etc</p><p>● I will form new Features like period, crime hour etc</p><p>● Out of the Population of the Crime Dataset, I will be using 100,000 as a sample of the dataset for analysis</p><p>● I will then drop a sample where there are missing values.</p><p><strong>Creating New Columns</strong></p><p>The code performs the following steps:</p><p>Import the necessary libraries: dateutil.parser.parse for parsing dates and datetime.datetime for working with dates and times. Extract the Date column from the dfsub DataFrame and store it in a variable tCol.</p><p>Create a list comprehension to iterate over the values in tCol, parse each date string using the parse function, convert it to a string representation using the ctime method of the datetime class, and extract the time period (AM or PM) from the original date string. Store the resulting list of tuples in a variable List.</p><p>Initialize empty lists for storing the day, month, and period values for each row in List.</p><p>Iterate over each row in List and extract the day and month values from the first element of the tuple (the string representation of the date). Use conditional statements to determine the period (Morning, Afternoon, Evening, Night, or Unknown) based on the time period and hour value extracted from the first element of the tuple. Append the extracted day, month, and period values to their respective lists.</p><p>Print the lengths of the dayList, monthList, and periodList lists to verify that they have the same length as the number of rows in dfsub.</p><p>Add new columns to dfsub for month, day, and period using the values in the monthList, dayList, and periodList lists.</p><p>Initialize an empty list for storing the day type (Weekend or Week-Day) for each row in dfsub. Iterate over each value in the day column of dfsub and use conditional statements to determine whether the day is a weekend (Saturday or Sunday) or a weekday.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/704/1*ARP0-tllEBzrBd2Ih_3M-Q.png" /></figure><p>Append the resulting day-type value to the weeklist list. Add a new column to dfsub for day type using the values in the weeklist list. Convert the Date and Updated On columns of dfsub to datetime objects using the pd.to_datetime function. Calculate the difference between the Updated On and Date columns of dfsub and store it in a new column called &quot;Day Difference&quot;.Extract the hour value from each datetime object in the &quot;Date&quot; column of dfsub using its .dt.hour attribute and store it in a new column called &quot;Crime_Hour&quot;.Extracts days of the month from each datetime object in &quot;Date&quot; column of dfsub using its .dt.day attribute and stores it in a new column called &quot;D_O_M&quot;.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*L8UjnOQ20xVv0HhTxxPyyw.png" /></figure><p>Our analysis of the data reveals that there are 35 different types of crimes reported to the Chicago PD in this dataset. We have also discovered that there are 146 different locations where these crimes occur, and 23 district circuits in the city where the crimes are reported.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*tKnLtHHaWqmyqZN1Gda_BQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/1*NxLV7dPXGs05trgsbCnmVg.png" /></figure><p>One of the most commonly reported crimes is theft, and we have found that thefts occur mostly on the streets, in residential areas such as apartments, and in stores. Most of these thefts involve monetary value, such as money, clocks, and other valuable items. Our analysis shows that theft is most common during mid-day and early afternoon (12 pm-3 pm), evening (5 pm-7 pm), and early night (8 pm-11 pm). Interestingly, the rate of theft increases towards the end of the month.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/915/1*8GMf-WbInF6HGela5aL9Bg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*QyiwHbMTQFqVMaQUNlro5w.png" /></figure><p>Battery is another prevalent crime in the city, and our analysis shows that it mostly occurs on the streets, in residential areas such as apartments, and in public places. Most of these batteries are simple assaults, such as beating up, and they occur mostly during the night and evening. Our analysis also shows that the South region of the city has more battery crimes than other regions.</p><h3><strong>INSIGHTS</strong></h3><p>Here are some insights on the crimes reported to the Chicago PD in this dataset:</p><ul><li>There are 35 different types of crimes reported.</li><li>There are 146 different locations where crimes have been reported.</li><li>There are 23 District circuits where crimes occur.</li><li>Theft is the most common type of crime reported, and it mostly occurs on the streets, and in residential areas like apartments, and stores. The stolen items are mostly of monetary value like money, clocks, etc. Theft occurs mostly during mid-day/early afternoon (12 pm-3 pm), evening (5 pm-7 pm), and early night (8 pm-11 pm). The rate of theft increases towards the end of the month.</li><li>Battery mostly happens on the streets, in residential areas like apartments, and in public places. Simple battery-like physical assault is the most reported type of battery, and it mostly happens during the night and evening. The southern region of the city has more battery crimes than others.</li><li>Narcotics mostly happen on the streets and sidewalks, especially early at night around 7 pm to 11 pm. Narcotics are most rampant in the west and north regions of the city, and they are mostly hard drugs like cannabis, crack, and heroin.</li><li>Robbery mostly happens on the streets, sidewalks, and public spaces like CTA trains and gas stations. Strong-armed robbery is the most reported type of robbery, and it occurs mostly during the early night and evening.</li><li>Criminal damage mostly happens on the streets, residential areas, and apartments. Vehicles and properties are the most common targets of criminal damage. It happens mostly at night and evening, and the south and west regions of the city have the highest occurrence of criminal damage crimes.</li></ul><h3><strong>RECOMMENDATIONS</strong></h3><p>Based on the insights gleaned from the data, the following recommendations are suggested to help reduce the spate of violent crimes reported in the city of Chicago:</p><ol><li>Increase Police Presence: Given the high incidence of crimes occurring in certain regions of the city, it is recommended that the Chicago PD increase their presence in those areas.</li><li>Enhance Security Measures: Theft and robbery are the most reported types of crime, and they mostly occur in public places. It is suggested that security measures be improved in those areas to deter criminal activities.</li><li>Focus on Preventive Measures: It is important to prevent crimes from occurring in the first place. The Chicago PD should focus on preventive measures such as community policing, public education, and outreach programs.</li><li>Collaborate with Other Agencies: The Chicago PD should collaborate with other agencies such as the DEA, FBI, and Homeland Security to tackle the issue of narcotics in the city.</li></ol><h3><strong>CONCLUSION</strong></h3><p>In conclusion, the spate of violent crimes reported in the city of Chicago is a major concern for the Chicago PD. The insights gleaned from the data suggest that theft, battery, narcotics, robbery, and criminal damage are the most reported types of crime in the city. The occurrence of these crimes mostly happens on the streets, in residential areas like apartments, and in public places. The south and west regions of the city have the highest incidence of crime. The suggested recommendations are aimed at reducing the spate of violent crimes reported in the city and making the city safer for its residents.</p><p>All the data and notebook are available on the GitHub account and the link to powerBi is <a href="https://app.powerbi.com/view?r=eyJrIjoiYThiYzY1YjgtODRhZC00N2M3LWI3MjktY2NmMzI5MzU0NjhjIiwidCI6Ijg3MTMxYjliLTg3NmUtNDdkZS1hMjlhLTExMGQ4ZDM5MWRiOCJ9">linked</a>.</p><p>All screenshots were taken by the author. The introductory image on <a href="https://stock.adobe.com/search?k=regulations+health+and+safety&amp;search_type=autosuggest&amp;continue-checkout=1">Adobe Stock.</a></p><p>In case you miss my previous one on <a href="https://medium.com/@adelekedejiimoleayo/web-scraping-and-analysis-of-qatar-world-cup-2022-1f2c15f26025"><strong>World Cup Analysis</strong></a></p><p>I sincerely hope you’ve enjoyed it and that it has some value for you in some way. You can reach me on <a href="https://twitter.com/crown_deji?t=yyzLghHlVqixn6hSEI9AMw&amp;s=09"><strong>Twitter</strong></a>,<strong> </strong><a href="https://github.com/Gracebaytech"><strong>GitHub,</strong></a><strong> </strong>or <a href="http://www.linkedin.com/in/adelekeadedeji-09aba0195"><strong>Linkedin</strong></a> if you need anything, have a suggestion, or simply want to chat.</p><p>Thanks for reading.</p><p>Follow me to view more stories….</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3839ba2127b8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Quality Assessment and Customer Segmentation for Sprocket Central]]></title>
            <link>https://medium.com/@adelekedejiimoleayo/data-quality-assessment-and-customer-segmentation-for-sprocket-central-674bd9b6616a?source=rss-29e5af4a8360------2</link>
            <guid isPermaLink="false">https://medium.com/p/674bd9b6616a</guid>
            <category><![CDATA[unsupervised-learning]]></category>
            <category><![CDATA[unsupervised-clustering]]></category>
            <category><![CDATA[marketing-strategies]]></category>
            <category><![CDATA[customer-segmentation]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[ADELEKE ADEDEJI]]></dc:creator>
            <pubDate>Sun, 12 Feb 2023 15:29:58 GMT</pubDate>
            <atom:updated>2023-02-12T15:29:58.479Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*9CnUrqim959mTkhKWFnlmg.jpeg" /><figcaption>Customer Segmentation — stock Adobe</figcaption></figure><p>Welcome to the world of data analysis and <strong>data</strong> visualization! In today’s digital era, data is considered to be the new oil. It is a valuable resource that provides insights into our business, and consumer behavior and helps us make better decisions. The process of collecting, analyzing, and visualizing data is known as data analysis. In order to perform data analysis, it is important to understand the variables and data types involved in the process. I’ll delve into identifying variables and data types in this article.</p><p><strong>Task 1: </strong>Data Quality Assessment (Assessment of Data Quality and Completeness in Preparation for Analysis)Check the data for accuracy and completeness so it can be used for analysis. Look for problems or missing information and list things to ask the client about. Offer suggestions to improve the data quality.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*29AlbJu_vVcaQ81d4HOK5Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/749/1*r-ZJ1ORVblzFyb3fJnpZTA.png" /></figure><p>The report is about reviewing data from Sprocket Central. Issues found include inconsistent customer IDs and inconsistent data types. Recommendations include using only customers listed in the “Customer Master” table, converting certain records to numeric values, and adding constraints to data types. The team will clean and transform the data, and it is suggested that you consult with Sprocket Central’s data expert to confirm assumptions.</p><p><strong>Task 2:</strong> Identify high-value customers using demographics and attributes.</p><p><strong>Key Points:</strong></p><p>Analyze the data distributions to understand the data.</p><p>Use feature engineering to improve the analysis.</p><p>Transform the data into a usable format.</p><p>Apply statistical or machine learning models.</p><p>Interpret the results and explain the findings.</p><p>Present results to the client.</p><p>Divide customers into segments.</p><p>Target high-value customers.</p><p>I am excited to take on this project and use my expertise in K-Means and PCA algorithms to identify high-value customers. I’ll examine the data’s patterns, create new variables, modify the data, and use statistical and machine learning techniques to uncover customer groupings. I am confident that I can help uncover valuable insights and define new customer segments for your organization;</p><h3>Steps in Data Exploration and Preprocessing</h3><ul><li>Identification of variables and data types</li><li>Analyzing the basic metrics</li><li>Missing value treatment</li><li>Non-Graphical Univariate Analysis</li><li>Graphical Univariate Analysis</li><li>Bivariate Analysis</li><li>Feature Engineering</li><li>Variable transformations</li><li>Outlier treatment</li></ul><p><strong>Identification of variables and data types</strong></p><p>Before I started, I was given a dataset in Excel with multiple sheets. To use it, I need to import some tools and load the dataset. Then, I will identify the variables and data types.</p><pre>data= pd.read_excel(&quot;KPMG_VI_New_raw_data_update_final.xlsx&quot;,sheet_name=None)<br><br># Prints all the sheets name in an ordered dictionary<br>print(data.keys())<br><br>Title_Sheet= pd.read_excel(&quot;KPMG_VI_New_raw_data_update_final.xlsx&quot;,sheet_name=&quot;Title Sheet&quot;)<br>Transactions=pd.read_excel(&quot;KPMG_VI_New_raw_data_update_final.xlsx&quot;,sheet_name=&quot;Transactions&quot;)<br>NewCustomerList=pd.read_excel(&quot;KPMG_VI_New_raw_data_update_final.xlsx&quot;,sheet_name=&quot;NewCustomerList&quot;)<br>CustomerDemographic=pd.read_excel(&quot;KPMG_VI_New_raw_data_update_final.xlsx&quot;,sheet_name=&quot;CustomerDemographic&quot;)<br>CustomerAddress=pd.read_excel(&quot;KPMG_VI_New_raw_data_update_final.xlsx&quot;,sheet_name=&quot;CustomerAddress&quot;)</pre><pre>Transactions.dtypes<br>CustomerDemographic.dtypes</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/339/1*NlvzNJzWi1I-1Eu9ZFVkYQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/430/1*TAVzhVNGmn-mW8Qdmy4Vvw.png" /></figure><p>changing the dtypes to the supposed types.</p><pre><br>def convert_object_to_int(df):<br>    for col in intVar:<br>        df[col] = pd.to_numeric(df[col], errors=&#39;coerce&#39;).fillna(0).astype(int)<br>    for col in floatVar:<br>        df[col] = pd.to_numeric(df[col], errors=&#39;coerce&#39;).astype(float)<br>    for col in Bolean:<br>        df[col] = df[col].astype(bool)<br>    for col in datetime:<br>        df[col] = pd.to_datetime(df[col], errors=&#39;coerce&#39;)<br>        <br>    return df<br>convert_object_to_int(Transactions)</pre><p>Analyzing key metrics helps me better understand the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A8LdQ59VBIN7t3QzELOguw.png" /></figure><p><strong>Missing value treatment</strong></p><p>Missing value treatment involves identifying and either replacing or removing the missing data points to ensure accurate analysis results.</p><pre>Transactions[&quot;standard_cost&quot;]=Transactions[&quot;standard_cost&quot;].fillna(Transactions[&quot;standard_cost&quot;].mean())<br> #fillling missing values with the mode<br>Transactions[&quot;product_class&quot;]=Transactions[&quot;product_class&quot;].fillna(Transactions[&quot;product_class&quot;].mode())<br><br># Filling the catogories varuable with the mode<br>CustomerDemographic[&quot;job_title&quot;]=CustomerDemographic[&quot;job_title&quot;].fillna(CustomerDemographic[&quot;job_title&quot;].mode())<br>CustomerDemographic[&quot;job_industry_category&quot;]=CustomerDemographic[&quot;job_industry_category&quot;].fillna(CustomerDemographic[&quot;job_title&quot;].mode())</pre><pre>CustomerDemographic[&quot;gender&quot;].replace(to_replace = [&quot;U&quot;,&quot;Femal&quot;,&quot;F&quot;], value = &quot;Female&quot;,inplace=True)<br>CustomerDemographic[&quot;gender&quot;].replace(to_replace = &quot;M&quot;, value = &quot;Male&quot;,inplace=True) </pre><p>Non-Graphical Univariate Analysis involves analyzing and summarizing the distribution of a single variable without using visual aids.</p><p><strong>Univariate Analysis</strong>Univariate Analysis is a statistical method used to analyze and understand individual variables in a dataset. It helps to determine the distribution, central tendency, and dispersion of the data, providing insights into the nature of the data. The aim of Univariate Analysis is to gain a deeper understanding of each variable and to identify potential trends and relationships between variables.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*Pp-c6r9_gm-WZ3xvhBECXA.png" /><figcaption>Online Order</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*3BSlkmqHnA9iB9mAfUM34g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*rfmqiUeNbvORkAI8kpfXMg.png" /></figure><p><strong>Bivariate Analysis</strong> is a statistical method used to examine the relationship between two variables. It helps in identifying if there is a connection between the two variables and what kind of relationship it is (positive, negative, or neutral). Bivariate Analysis is useful in understanding patterns and trends in data and making informed decisions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*JGeab9dC7q0MUDzT23w6kw.png" /></figure><pre><br>#merging the datasets to give  one<br>dateset=Transactions.merge(CustomerAddress,on=&quot;customer_id&quot;)<br>data=dateset.merge(CustomerDemographic,on=&quot;customer_id&quot;)</pre><p>Merging datasets means combining multiple datasets into a single, larger dataset. The process involves aligning and integrating the data from each source into a unified data set. This can be done through various methods such as joining, concatenating, or appending the datasets. The end result is a comprehensive dataset that provides a more complete picture of the information. I will merge the Transactions, Customer Addresses and Customer Demographics.</p><p><strong>Feature Engineering</strong></p><p>Feature engineering is the process of transforming raw data into meaningful and useful information. In order to extract useful information such as age, age group, frequency, and value, various techniques are applied to the dataset, such as data cleaning, data normalization, data transformation, and data aggregation. These techniques help to extract valuable insights and improve the accuracy of machine learning models.</p><pre>data[&quot;age&quot;] = data[&quot;transaction_date&quot;].dt.year - data[&quot;DOB&quot;].dt.year <br><br>def life_stage(age):<br>    if age &lt;= 24:<br>        return &#39;GenZ(0-24)&#39;<br>    elif age &lt;= 39:<br>        return &#39;Millennials(25-39)&#39;<br>    elif age &lt;=54:<br>        return &quot;Gen X(40-54)&quot;<br>    elif age &lt;= 74:<br>        return &#39;Baby Boomers(55-74)&#39;<br>    else:<br>        return &#39;Interwar(75-above)&#39;<br>data[&quot;Age_group&quot;]= data[&#39;age&#39;].apply(life_stage)<br><br>df= data.copy()<br><br><br><br><br># Calculate the margin for each product<br>df[&#39;Value&#39;] = df[&#39;list_price&#39;] - df[&#39;standard_cost&#39;]<br><br># Flag products with high margins<br>df[&#39;High_margin_product&#39;] = (df[&#39;Value&#39;] &gt;= df[&#39;Value&#39;].mean()).astype(int)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*Wk-1jSfyL0Olq8WO161XuA.png" /></figure><p><strong>Variable transformations and Data transformation ; </strong>Variable transformations refer to the process of changing the values or format of a variable in a dataset to make it suitable for analysis. This can involve scaling, normalization, encoding, or any other process that transforms the variable into a more usable form.</p><pre>df[&quot;online_order&quot;] = df[&quot;online_order&quot;].map({True: 1, False: 0})<br><br>df[&#39;order_status&#39;] = df[&#39;order_status&#39;].map({&#39;Approved&#39;: 1, &#39;Cancelled&#39;: 0})<br><br><br># check which columns are object type<br>obj_cols = df.columns[df.dtypes == &#39;object&#39;]<br><br># apply label encoding to object type columns<br>for col in obj_cols:<br>    df[col] = df[col].astype(&#39;category&#39;)<br>    df[col] = df[col].cat.codes<br><br># check the data after encoding<br>print(df.head())</pre><p>Before encoding, I will remove some features that I believe are not important.</p><p><strong>Outlier Treatment: </strong>The outline treatment using scipy in python involves importing the scipy library, defining the functions for data analysis, such as statistical tests, optimization techniques, and signal processing, and then applying these functions to the desired data.</p><pre>from scipy import stats<br>import numpy as np<br># remove the customer id column<br>new_df =df<br># remove outliers<br>z_scores = stats.zscore(new_df)<br>abs_z_scores = np.abs(z_scores)<br>filtered_entries = (abs_z_scores &lt; 3).all(axis=1)<br>new_df = new_df[filtered_entries]</pre><p>A good way to get an initial understanding of the relationship between the different variables is to explore how they correlate.</p><p>I calculate the correlation between our variables using corr method in the pandas library.</p><pre>plt.figure(figsize=(12,9))<br>sns.heatmap(finaldf.corr(),annot=True,cmap=&#39;RdBu&#39;)<br>plt.title(&#39;Correlation Heatmap&#39;,fontsize=14)<br>plt.yticks(rotation =0)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/838/1*QKAtD7trITyJBgoEnFoAVg.png" /></figure><p><em>let’s explore the correlation.</em></p><p>I see that there is a strong correlation between tenure and Age. In other words, older people tend to be more highly tenure.</p><p>How about Rank and Value?</p><p>Their correlation is 0.65. That means If you have a higher Value, you are more likely to have a higher level of Rank.</p><p>Correlation matrix is a very useful tool to analyze the relationship between features.</p><p>Now, I understand our dataset and have a general idea of it.</p><p>Standardization is a preprocessing step that involves transforming the data to have a mean of zero and a standard deviation of one. This helps to normalize the data and eliminate the effects of scale, making it easier to compare variables and interpret the results. Standardization can be performed using scikit-learn’s StandardScaler or by manually calculating the z-score for each data point.</p><pre>from sklearn.preprocessing import StandardScaler<br>new_df = new_df.drop_duplicates()<br>col_names = [&#39;tenure&#39;,&#39;property_valuation&#39;,&#39;age&#39;,&#39;Rank&#39;,&#39;gender&#39;,&#39;Value&#39;,&#39;state&#39;,&#39;past_3_years_bike_related_purchases&#39;]<br>features = finaldf [col_names]<br>scaler = StandardScaler().fit(features.values)<br>features = scaler.transform(features.values)<br>scaled_features = pd.DataFrame(features, columns = col_names)</pre><p>Now, I am all set to start building the segmentation model K-Means!</p><pre>import matplotlib.pyplot as plt<br>from sklearn.cluster import KMeans<br>from sklearn.metrics import silhouette_score<br>from sklearn.decomposition import PCA<br>from mpl_toolkits.mplot3d import Axes3D<br>SSE = []<br>for cluster in range(1,10):<br>    kmeans = KMeans(n_clusters = cluster, init=&#39;k-means++&#39;)<br>    kmeans.fit(scaled_features)<br>    SSE.append(kmeans.inertia_)<br># converting the results into a dataframe and plotting them<br>frame = pd.DataFrame({&#39;Cluster&#39;:range(1,10), &#39;SSE&#39;:SSE})<br>plt.figure(figsize=(12,6))<br>plt.plot(frame[&#39;Cluster&#39;], frame[&#39;SSE&#39;], marker=&#39;o&#39;)<br>plt.xlabel(&#39;Number of clusters&#39;)<br>plt.ylabel(&#39;Inertia&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*ySSe0-rPezwH82Ur6SGx-g.png" /></figure><p>The elbow in the graph is the three -cluster mark. This is the only place until which the graph is steeply declining while smoothing out afterward.</p><p>Let’s perform K-Means clustering with 3 clusters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/703/1*a3p3-Gk-8vwOyIVXlFONaA.png" /></figure><p>The silhouette coefficient of this model is <strong>0.13, </strong>indicating reasonable cluster separation.</p><p>I will create a new data frame with the original features and add a new column with the assigned clusters for each point.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/943/1*_dc0xJ87-4WpJmJI8TJLRw.png" /></figure><p>Then I must visualize the data to identify the distinct traits of customers in each segment:</p><pre>x_axis = scaled_features[&#39;age&#39;]<br>y_axis = scaled_features[&#39;Value&#39;]<br>plt.figure(figsize = (10, 8))<br>sns.scatterplot(x_axis, y_axis, hue = scaled_features[&#39;Segment K-means&#39;], palette = [&#39;g&#39;, &#39;r&#39;,&#39;c&#39;])<br>plt.title(&#39;Segmentation K-means&#39;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/617/1*R_s9zxin1cFtMTAZ931TgA.png" /></figure><p>In the next section, I will combine PCA and K-Means to try to get a better result.</p><h3>PCA with K-Means for Better Visualization</h3><pre>df_std= scaled_features[[&#39;tenure&#39;,&#39;property_valuation&#39;,&#39;age&#39;,&#39;Rank&#39;,&#39;gender&#39;,&#39;Value&#39;,&#39;state&#39;,&#39;past_3_years_bike_related_purchases&#39;]]<br><br>pca = PCA()<br>pca.fit(df_std) </pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/697/1*yY5J5aOy05YwkrRJmF-MkA.png" /></figure><p>I observe that the first component explains around 20% of the variability of the data. The second one is 18% and so on.</p><p>I now can plot the cumulative sum of explained variance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*eguewtVwv1XjoLOc0yjgWA.png" /></figure><p>80 % of the variance of the data is explained by the first 3 components. Let’s keep the first 3 components for our further analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*4-CiRo0_ETkuBUyO6Kv7AA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*vXhNVgqyH5jLFTJsD3SeRg.png" /></figure><p>In our analysis, we found that there is a strong positive correlation between Component1, Rank, and Value. This suggests that individuals who purchase high-end products tend to have a career-focused mindset. The second component is mainly determined by the variables of Tenure and Age. Finally, for the third component, Property Valuation and State proved to be the most significant features. Although Gender and Past_3_Years Bike Load had a negative impact, they still played an important role in the analysis. With this information, we now have a clear understanding of the relationships between the components and variables in the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/664/1*zv9YeLq5IdofAuSQtZ2QvA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*bR3CFjEUd0T5T4ryrw5OTw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/979/1*aHXuaA3mCCjK0RzCxJ3Nwg.png" /></figure><p>The silhouette coefficient of this model is <strong>0.28, </strong>indicating reasonable cluster separation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/608/1*ppbg60Zyl5FRqNg1mvMfwg.png" /></figure><p>When I used K-means clustering without applying PCA, we were only able to differentiate the green segment. However, after using PCA, the division between the different segments became much clearer. This highlights the key goal of PCA, which is to simplify the data by combining multiple variables into fewer components. As the famous quote by Seth Godin states, “Don’t find customers for your products, find products for your customers.” The same concept applies to data analysis, where it is crucial to understand the needs and preferences of your data to make meaningful insights.</p><p>As a data scientist, based on the previous customer segmentation analysis, I will now extend the model to a new customer set. Before proceeding, I will conduct an exploratory data analysis on the new customer data and identify the important features for the analysis. This will ensure that the model is accurate and relevant for the new customer data set.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/645/1*p1LrrqiKMJKmwoDhhYx9kw.png" /></figure><p>The next step is standardization and feature selection using the previous PCA model</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/1*5YNPDYk2NqnJ9L2EkJlQEQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/608/1*To3tb5XurcVOrG7p4PGWvA.png" /></figure><p><strong>Task 3:</strong> Data Insights and Presentation: Using visualisations to present insights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/904/1*Zxq7OcJ9cxldANOu1o1uOw.png" /></figure><h3>Trends in the underlying data</h3><p>Based on the data, we can see that the total number of customers and total Spending has been steadily increasing over time. Additionally, the top brands product size and product class have remained consistent, with some brands outperforming others. The target customer with the highest customer value is likely to be a valuable target for marketing and growth strategies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/969/1*QzeNmieK21AUS9B-A5tpdQ.png" /></figure><h3>Customer segment with the highest customer value</h3><p>Based on the data, we can see that the customer segment with the highest customer value is likely to be older, male customers with a high level of spending.</p><h3>Marketing and growth strategy</h3><p>Given these insights, I recommend that Sprocket Central Pty Ltd focus its marketing and growth efforts on older, male customers with a high level of spending. This could include targeted advertising and promotions, as well as expanding the product offerings in this segment. Additionally, focusing on expanding the product size and class offerings for the top-performing brands may also be a worthwhile strategy to drive growth.</p><p>It would also be beneficial for the company to continue monitoring and analyzing customer demographics and spending trends to ensure that its marketing and growth efforts are aligned with the evolving needs and preferences of its customer base.</p><p>Overall, by focusing on the customer segment with the highest customer value and regularly monitoring and analyzing customer data, Sprocket Central Pty Ltd can effectively drive its marketing and growth strategies to achieve long-term success.</p><p>All screenshots were taken by the author. The introductory image is made On Stock Adobe by Author</p><p>All the data used is available on the GitHub account</p><p>In case you miss my previous story on <a href="https://medium.com/@adelekedejiimoleayo/uncovering-public-sentiment-through-sentiment-analysis-in-nigerian-elections-3697dca8fd7d"><strong>Uncovering Public Sentiment through Sentiment Analysis in Nigerian Elections</strong></a></p><p>I sincerely hope you’ve enjoyed it and that it has some value for you in some way. You can reach me on <a href="https://twitter.com/crown_deji?t=yyzLghHlVqixn6hSEI9AMw&amp;s=09"><strong>Twitter</strong></a>,<strong> </strong><a href="https://github.com/Gracebaytech"><strong>GitHub,</strong></a><strong> </strong>or <a href="http://www.linkedin.com/in/adelekeadedeji-09aba0195"><strong>Linkedin</strong></a> if you need anything, have a suggestion, or simply want to chat.</p><p>Thanks for reading.</p><p>Follow me to view more stories….</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=674bd9b6616a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Uncovering Public Sentiment through Sentiment Analysis in Nigerian Elections]]></title>
            <link>https://medium.com/@adelekedejiimoleayo/uncovering-public-sentiment-through-sentiment-analysis-in-nigerian-elections-3697dca8fd7d?source=rss-29e5af4a8360------2</link>
            <guid isPermaLink="false">https://medium.com/p/3697dca8fd7d</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[sentiment-analysis]]></category>
            <category><![CDATA[politics]]></category>
            <category><![CDATA[nigeria-election]]></category>
            <dc:creator><![CDATA[ADELEKE ADEDEJI]]></dc:creator>
            <pubDate>Thu, 02 Feb 2023 13:50:39 GMT</pubDate>
            <atom:updated>2023-02-02T14:12:17.497Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2l3IgCg3k6X4SM4KXqOPzA.png" /><figcaption>Author By Me Through Canva</figcaption></figure><p>Nigeria&#39;s elections are a significant event in the country’s political landscape. They determine the leadership of the country and shape its future trajectory. Technology has become more important in elections lately, especially for sharing and getting information to the public. Sentiment analysis is a technology that helps us understand how people feel about something, like an election. This article will show how we can use it to learn what people in Nigeria think about their elections and how it can give us important information. We will delve into how sentiment analysis works, its benefits and limitations, and its potential impact on the election process. This article will explain sentiment analysis and its role in Nigerian elections. It will also show why public opinion is important for the country’s future.</p><p>Sentiment analysis is a way to figure out how people feel about something by looking at their words. For example, if someone writes a tweet about a candidate running for president, sentiment analysis can tell us if the person writing the tweet feels good or bad about that candidate. By looking at lots of tweets and other information from the internet, we can get a better idea of what lots of people think about different candidates and the things they care about the most.</p><h3>What is sentiment analysis?</h3><p>Sentiment analysis is a tool that helps us understand people’s opinions, feelings, and emotions about something. It does this by looking at what people write or say and figuring out what they’re feeling. It uses special computer tools to do this, like language processing and text analysis.</p><p>This tool helps us find out what people are saying about a topic, like the Nigerian elections, and how they feel about it. Are they happy? Are they sad? Are they mad? Sentiment analysis helps us answer these questions by giving us a better understanding of what people are saying.</p><p>Sentiment analysis is used in a variety of concepts and industries. Some examples include:</p><ul><li>Marketing and customer service: Companies use sentiment analysis to understand how customers feel about their products and services, and to respond accordingly.</li><li>Social media monitoring: Sentiment analysis can be used to track brand mentions and gauge public opinion on social media platforms.</li><li>Politics: Sentiment analysis can be used to track public opinion on political issues and candidates.</li><li>Finance: Sentiment analysis can be used to track public opinion on companies, sectors and markets.</li><li>Healthcare: Sentiment analysis can be used to track public opinion on healthcare issues and pharmaceutical products.</li><li>Entertainment: Sentiment analysis can be used to track public opinion on movies, TV shows, and music.</li><li>Human Resources: Sentiment analysis can be used to track employee engagement and satisfaction with the company, its culture, and its management.</li></ul><p>During an election campaign, sentiment analysis can help track how people feel about what the candidates are saying. For example, it can analyze speeches, debates, and interviews to see which topics are important to people and how they feel about the candidates. This can give us a better understanding of what people like or don’t like about each candidate.</p><p>Sentiment analysis can also be used to see what people are saying about the candidates on social media. By looking at tweets, posts, and comments on platforms like Twitter and Facebook, we can see how people feel about each candidate and what they are saying. This can help us understand if people are generally happy or unhappy with each candidate and where they might be getting support or opposition.</p><p>Additionally, when people talk about something, they often have strong feelings about it. These feelings can be either positive or negative. The strength of these feelings is called <strong>“polarity.”</strong></p><p>Polarity can range from -1 to +1. A negative feeling would be a -1, a neutral feeling would be a 0, and a positive feeling would be a +1.</p><p>Another important aspect of people’s feelings is how personal they are. This is called <strong>“subjectivity.”</strong> If someone has a very personal connection to something, their feelings about it will be more subjective. This means that their experience with that thing is unique to them and may be different from someone else’s experience.</p><p>I will be using Twitter’s API (application programming interface) to extract tweets for sentiment analysis. analysis.</p><p>To use the Twitter API, you must register for a developer account and create a new project. This will give you access to the API keys and tokens you need to access the API.</p><h3>First, let’s import all the python libraries that we will use throughout the program.</h3><ol><li><strong>Pandas</strong> — For data analysis and data manipulation.</li><li><strong>Matplotlib </strong>— For data visualization.</li><li><strong>Seaborn</strong> — It’s based on matplotlib and provides a high-level interface for data visualization.</li><li><strong>WordCloud</strong> — For visualizing text data in the form of clouds.</li></ol><h3>Natural Language Processing</h3><ol><li><strong>A group of libraries for natural language processing is called the &quot;Natural Language Toolkit&quot; (nltk).</strong></li><li><strong>stopwords</strong> — A group of words that don’t give a sentence meaning.</li></ol><p>Once I have my API keys and tokens, I’ll need to authenticate the API using the appropriate libraries for my programming language. This typically involves passing your API keys and tokens as arguments to a function or method.</p><pre># Import tweepy to work with the twitter API<br>import tweepy as tw<br># Import numpy and pandas to work with dataframes<br>import numpy as np<br>import pandas as pd<br># Import seaborn and matplotlib for viz<br>from matplotlib import pyplot as plt<br><br><br># Authenticate<br>auth = tw.OAuthHandler(consumer_key, consumer_secret)<br># Set Tokens<br>auth.set_access_token(access_token, access_token_secret)<br># Instantiate API<br>api = tw.API(auth, wait_on_rate_limit=True)</pre><p><strong>Search for tweets:</strong> Once authenticated, I can use the API to search for tweets using specific keywords, hashtags, or other criteria. This can be done using the API’s search endpoint, which allows me to specify the search query and the number of tweets I want to retrieve. In this case, I will be using #NigeriaElection,#NigeriaDecides etc</p><pre>keywords=[&quot;#NigeriaElection&quot;,&quot;#NigeriaDecides&quot;,&quot;Nigeria politics&quot;,&quot;#tinubu&quot;,&quot;#atiku&quot;,&quot;#peterobi&quot;]<br>import time<br><br>Tweet=[]<br>for keyword in keywords:<br>    query = tw.Cursor(api.search_tweets, q=keyword). items(2000)<br>    for tweet in query:<br>        tweets = {&#39;Tweet&#39;:tweet.text, &#39;Timestamp&#39;:tweet.created_at} <br>        #print(tweets)<br>        Tweet.append(tweets)</pre><p><strong>Retrieve the tweets:</strong> Once I’ve searched for tweets, you can retrieve the tweets using the API’s GET method. The API will return the tweets in JSON format, which can be parsed and analyzed using appropriate libraries and tools.</p><pre>nltk.download(&#39;stopwords&#39;)<br>nltk.download(&#39;wordnet&#39;)<br>stop_words = stopwords.words(&#39;english&#39;)<br>custom_stopwords = [&#39;RT&#39;,&quot;#NigeriaElection&quot;,&quot;#NigeriaDecides&quot;,&quot;Nigeria politics&quot;,&quot;#tinubu&quot;,&quot;#atiku&quot;,&quot;#peterobi&quot;]</pre><p><strong>Pre-processing: </strong>Before analyzing the tweets, the tweets must be pre-processed. This can include cleaning the text, removing stopwords, stemming or lemmatizing the words, and handling emojis or emoticons.</p><pre>def preprocess_tweets(tweet, custom_stopwords):<br>    processed_tweet = tweet<br>    processed_tweet.replace(&#39;[^\w\s]&#39;, &#39;&#39;)<br>    processed_tweet = &quot; &quot;. join(word for word in processed_tweet.split() if word not in stop_words)<br><br>    processed_tweet = &quot; &quot;. join(word for word in processed_tweet.split() if word not in custom_stopwords)<br>    processed_tweet = &quot; &quot;. join(Word(word).lemmatize() for word in processed_tweet.split())<br>    return(processed_tweet)<br>df[&#39;Processed Tweet&#39;] = df[&#39;Tweet&#39;].apply(lambda x: preprocess_tweets(x, custom_stopwords))<br>df.head()</pre><p>See the results: Once you’ve looked at all the tweets, you can show what you found using pictures, graphs, and lists. This way, you can understand what people are saying, see if there are any patterns, and figure out what it all means</p><pre># Calculate polarity and subjectivity<br>df[&#39;polarity&#39;] = df[&#39;Processed Tweet&#39;].apply(lambda x: TextBlob(x).sentiment[0])<br>df[&#39;subjectivity&#39;] = df[&#39;Processed Tweet&#39;].apply(lambda x: TextBlob(x).sentiment[1])<br>df[[&#39;Processed Tweet&#39;, &#39;Tinubu&#39;, &#39;Atiku&#39;,&quot;Obi&quot;, &#39;polarity&#39;, &#39;subjectivity&#39;]].head()<br><br>display(df[df[&#39;Tinubu&#39;]==1][[&#39;Tinubu&#39;,&#39;polarity&#39;,&#39;subjectivity&#39;]].groupby(&#39;Tinubu&#39;).agg([np.mean, np.max, np.min, np.median]),<br>df[df[&#39;Atiku&#39;]==1][[&#39;Atiku&#39;,&#39;polarity&#39;,&#39;subjectivity&#39;]].groupby(&#39;Atiku&#39;).agg([np.mean, np.max, np.min, np.median]),<br>       df[df[&#39;Obi&#39;]==1][[&#39;Obi&#39;,&#39;polarity&#39;,&#39;subjectivity&#39;]].groupby(&#39;Obi&#39;).agg([np.mean, np.max, np.min, np.median]))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/553/1*qlab2n4NYthg2GQUZUkYhw.png" /></figure><p>These values refer to the sentiment analysis of tweets related to three Nigerian political candidates, Tinubu, Atiku, and Obi, using a sentiment analysis tool. The polarity score measures the sentiment expressed in the tweet on a scale of -1 (strongly negative) to 1 (strongly positive). The subjectivity score measures the subjectivity of the tweet on a scale of 0 (objective) to 1 (subjective).</p><p>In the case of Tinubu, the mean polarity is 0.10, indicating that the sentiment expressed in the tweets is slightly positive. The maximum polarity score is 0.85 and the minimum polarity score is -1, indicating that there is a wide range of sentiments expressed in the tweets about Tinubu. The mean subjectivity score is 0.29, indicating that the tweets are moderately subjective.</p><p>For Atiku, the mean polarity score is 0.05, indicating a neutral sentiment. The maximum polarity score is 0.8 and the minimum polarity score is -0.5, indicating that there is a moderate range of sentiment expressed in the tweets about Atiku. The mean subjectivity score is 0.37, indicating that the tweets are subjective.</p><p>For Obi, the mean polarity score is -0.03, indicating slightly negative sentiment. The maximum polarity score is 1 and the minimum polarity score is -1, indicating that there is a wide range of sentiments expressed in the tweets about Obi. The mean subjectivity score is 0.27, indicating that the tweets are moderately subjective.</p><p><strong>Note: I extracted 12998 tweets; this analysis is based on the tweets and it is two-week tweets</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*X1BH5xRvVGQ7tWADg2v7AQ.png" /></figure><p>Let&#39;s check the <strong>polarity trends</strong> of the candidates over two weeks of tweets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uPICQk6APa0kIA7e6UqqZQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eq9bEM76l4gG2memjvYPvQ.png" /></figure><p>Around Jan 31, 2023, at 04: 50 Tinubu has the highest positive mean polarity score is 0.63 and on Jan 26, 2023, at 10:21 has the lowest mean polarity of -0.36.</p><p>Atiku</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vtHHFgTLaO03oA3o42e0vw.png" /></figure><p>Around Jan 31, 2023, at 04: 50 Akitu has the highest positive mean polarity score is 0.31 and on Jan 26, 2023, at 10:21 has the lowest mean polarity of -0.45.</p><p>Obi</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3lvnknlXs8pyd9sfhvD_wg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZV5V85cGOHtaFjnOlxbOoQ.png" /></figure><p>Around Jan 31, 2023, at 04: 50 Obi has the highest positive mean polarity score is 0.40 and on Jan 26, 2023, at 10:21 has the lowest mean polarity of -0.40.</p><p>In conclusion, the sentiment analysis of these tweets suggests that Tinubu has a slightly positive sentiment, Atiku has a neutral sentiment, and Obi has a slightly negative sentiment. The subjectivity of the tweets about all three candidates is moderate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*UfoqsQaxHRpFhVIHc6FZGA.png" /><figcaption>Tinubu Word Clouds</figcaption></figure><p>there have 95950 words in all text</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*Lywed8d_y5dp4WEi-J4ZPg.png" /><figcaption>Atiku Word Clouds</figcaption></figure><p>there have 13116 words in all text</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*K8v8pf1M4yMncUSBD3lWOw.png" /><figcaption>Obi Word Clouds</figcaption></figure><p>there have 375645 words in all text</p><p>The words that are used the most in positive tweets will be shown in a larger font size, while the words used in negative tweets will be shown in a smaller font size.</p><p>By analyzing the Word Clouds for each candidate, one can get a good sense of the overall sentiment towards that candidate as well as what issues or topics are most associated with them. This information can be useful for political campaigns, media organizations, and other entities that are interested in understanding public sentiment towards political candidates.</p><p><strong>Conclusion</strong></p><p>In conclusion, sentiment analysis is a valuable tool that helps us understand public opinion and feelings towards a specific topic, such as the Nigerian elections. Analyzing social media posts, comments, and tweets can give us a better understanding of what people are saying about each candidate, what topics are important to them, and how they feel about those topics. The process involves authenticating the Twitter API, importing necessary libraries, and utilizing natural language processing techniques to extract the sentiment of the tweets. Sentiment analysis can be used in a variety of industries, including politics, marketing, finance, and entertainment. However, it is important to note that sentiment analysis has its limitations and can sometimes produce incorrect results, which is why it is important to consider other factors when making decisions based on the results.</p><p><strong>Key Takeaways:</strong></p><ol><li>Sentiment analysis is a tool that helps us understand public opinion and feelings towards a specific topic.</li><li>It involves analyzing social media posts, comments, and tweets to extract the sentiment of the text.</li><li>Sentiment analysis has its benefits and limitations and should be used in conjunction with other information to make informed decisions.</li><li>It is important to authenticate the Twitter API and utilize natural language processing techniques when conducting sentiment analysis.</li><li>Sentiment analysis can be used in various industries, including politics, marketing, finance, and entertainment.</li></ol><p>All screenshots were taken by the author. The introductory image is made On Canva by Author</p><p>All the data used is available on the GitHub account</p><p>In case you miss my previous one on <a href="https://medium.com/@adelekedejiimoleayo/web-scraping-and-analysis-of-qatar-world-cup-2022-1f2c15f26025"><strong>World Cup Analysis</strong></a></p><p>I sincerely hope you’ve enjoyed it and that it has some value for you in some way. You can reach me on <a href="https://twitter.com/crown_deji?t=yyzLghHlVqixn6hSEI9AMw&amp;s=09"><strong>Twitter</strong></a>,<strong> </strong><a href="https://github.com/Gracebaytech"><strong>GitHub,</strong></a><strong> </strong>or <a href="http://www.linkedin.com/in/adelekeadedeji-09aba0195"><strong>Linkedin</strong></a> if you need anything, have a suggestion, or simply want to chat.</p><p>Thanks for reading.</p><p>Follow me to view more stories….</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3697dca8fd7d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Safe and Sound Data Showdown]]></title>
            <link>https://medium.com/@adelekedejiimoleayo/safe-and-sound-data-showdown-68afe132ed4?source=rss-29e5af4a8360------2</link>
            <guid isPermaLink="false">https://medium.com/p/68afe132ed4</guid>
            <category><![CDATA[mining-industry]]></category>
            <category><![CDATA[organization]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[health-and-safety]]></category>
            <dc:creator><![CDATA[ADELEKE ADEDEJI]]></dc:creator>
            <pubDate>Tue, 17 Jan 2023 10:26:03 GMT</pubDate>
            <atom:updated>2023-01-17T10:26:03.096Z</atom:updated>
            <content:encoded><![CDATA[<p>The goal of this article is to improve the safety of employees working within a large mining organization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*SLDoN_L1U7ofcEnuoEMu1Q.jpeg" /><figcaption>AUTHOR FROM ADOBE STOCK</figcaption></figure><h3><strong>EXECUTIVE SUMMARY</strong></h3><p>The mining industry is an important contributor to the economy, but it also poses inherent risks to health and safety. By using data analytics, the mining organization can identify patterns and trends in safety incidents and take appropriate interventionary actions to mitigate these risks. Three crucial areas were found in this review: Factors that contribute to reported safety accidents, how the general business can be improved based on safety performance, and how machine learning can be used to avoid accidents in the future.</p><p><strong>Key findings</strong> :Incidents are more likely to occur during certain seasons, particularly in regions with extreme weather; accidents tend to increase at the end of the month and in the morning; the most dangerous jobs in this mining organization are those involving fixed plants and underground mining; and worker fatigue is a top concern in the mining industry. To improve business, the mining organization should focus on reducing worker fatigue and increasing communication and collaboration between workers and staff.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*degQPwBC6vuKOZ3XsBOM0g.jpeg" /></figure><h3>INTRODUCTION</h3><p>The mining industry is a vital sector that provides employment for millions of people worldwide and supports the economies of several nations. However, it also faces significant challenges, such as mistrust from the public and high rates of injury among workers. Occupational health and safety is a major concern in the mining industry due to the hazardous nature of the work and the potential for accidents to occur. Injuries in the mining industry can have a major impact on the productivity, health, and financial well-being of workers and their families. Many industry stakeholders have been working to improve the occupational health and safety of mining employees to reduce the number of accidents and injuries. However, statistics from developed economies such as China and the United States suggest that there is still much work to be done to improve the safety of miners and reduce the number of mining-related deaths.</p><h3><strong>PROBLEM STATEMENT:</strong></h3><p><strong>T</strong>he mining industry has been worried about accidents happening at various site locations. Reported accidents have been tracked daily since the year 2000 and have been provided in the project data file. The mining industry would like to drastically reduce the spate of violent accidents reported on the mining site. Being effective involves knowing accident patterns and where they are likely to occur. They have recruited me to conduct full data analytics and uncover insights from the data that can be used to effectively prepare for and respond to an accident. My task is to use the provided datasets to uncover insights that reveal useful information about the safety of mine-site employees/contractors. This mining organization is looking to answer a broad range of questions relating to the analysis of these safety-related datasets, from the highest level to the lowest.</p><p><strong>A</strong>. analyzes the provided datasets to uncover insights into what features contribute to the observed safety incidents.</p><p><strong>B.</strong> Develop approaches that allow the safety team to provide insights/trends/actions back to the wider business on the safety performance across time.</p><p><strong>Data Provided (Introduction to the dataset)</strong></p><p>There are six datasets given for this project work. There is the dataset and brief information about it.</p><p><strong>safety_events.csv</strong></p><p>From 2000 to 2022, each significant incident and reportable injury safety event is included in this dataset.</p><p>The columns of this dataset contain data related to the context surrounding each row, which represents a specific safety event that has occurred. This context includes the event’s date and time, the location where it happened, the body part(s) involved, the type of injury, the person or workgroup involved, the agency/mechanism/nature of the injury, as well as a few other details.</p><p>The majority of the columns should be fairly self-explanatory. The “LTIDays” column, on the other hand, denotes the lost-time-injury (LTI) days, which are essentially the number of days of work the employee missed because of the injury they sustained in connection with this safety event.</p><p>The <em>‘TimeBand’</em> column represents a rough period that the employee involved with this safety event has been employed within the business. This range of time should be able to be calculated much more accurately using</p><p>the <em>‘emp_start_end_dates.csv’</em>, but some employees (specific contractors) might not be represented in that dataset, so that’s where this <em>‘TimeBand’</em> column could potentially act as a less accurate proxy.</p><p><strong>production_data.csv</strong></p><p>The daily production information (daily tonnes moved) tallied at each of the various mine sites is included in this dataset. The closest measurement to what actually occurred on-site is the recorded and reconciled tonnes moved, which are the actual tonnes moved. The budgeted tonnes basically represent the tonnes that should be moved, as determined, set, and planned at the beginning of the financial year. The 3-month forward-looking forecast for the tonnes being moved is represented by the short-term forecasted tonnes moved. The “half 2 forecast tonnes moved” refers to the forecast for tonnes moved for the second half. This forecast was made in January using the actuals, budget, and 6-month forecast from the first half (June — Jan).</p><p>Typically the actual and planned tonnes would be the most utilised measures, but the other actions should be utilised within your analysis if you deem it appropriate.</p><p>If any sites are missing entire years worth of this production data, this was purposely done due to the determined unreliability of the data during these years, hence their removal.</p><p><strong>emp_start_end_dates.csv</strong></p><p>Although this dataset is very straightforward, it can be a valuable source of data for your analysis. The dates that each employee started and ended working for this mining organization are listed in each row, along with the name of the employee. You can assume that this employee is still employed right now by this mining organization if the end date is NA.</p><p>These dates can be used to accurately calculate each employee’s total length of employment. When combined with the ‘person workgroup.csv’, these dates can also be used to find out when specific employees left the company, as well as how many times they left at various points in time.</p><p><strong>person_workgroup.csv</strong></p><p>This dataset represents the workgroup that a specific employee belongs to. Each row represents a specific employee, the workgroup they were working within, and the start/end dates that they were working within this workgroup. However, this dataset can be messy in some instances.</p><p>An employee can also be employed within multiple workgroups during their employment within this mining organisation. This is why you will see the same employee being recorded against different workgroups within this dataset. An employee being assigned to a different workgroup likely indicates that the employee is switching roles, switching departments, or perhaps even switching to a different mine site entirely.</p><p>This is important information to keep in mind, because switching roles/departments / mine-sites could potentially put the individual, or the workgroups involved, under increased stress, potentially leading to increased safety events. Similarly, such a switch could be a “breath of fresh air” for the individual/workgroup, or help provide the workgroup with some extra resourcing, perhaps leading to a decrease in safety events.</p><p><strong>employee_roster.csv</strong></p><p>This dataset represents the roster for a particular individual, and it also contains the record of leave taken by this particular employee. Each row in this dataset relates to a specific employee, and a specific day of their employment within this mining organisation. The scheduled work hours represent the hours that this employee contractually should be working during these days of employment.</p><p>The value <em>‘RDO’</em> that’s seen within this scheduled work hours column indicates the employee’s rostered day off (RDO). In the resources sector, it’s very common that individuals will fly to the mine site, work for some amount of time, and then fly back home to enjoy their time off. This work is commonly referred to as FIFO work (fly-in-fly-out work). The days that an individual is working on-site will be indicated by a number within the scheduled work hours column (typically being equal to 12, indicating 12 hours worked). Then, the values within the scheduled work hours column that are equal to the string “RDO” indicate a normal day off for this employee. An employee who works in an official capacity would generally have a number recorded in this column from Mon-Fri, but then an “RDO” value on the Sat / Sun.</p><p><strong>labour_hours_worked.csv</strong></p><p>This dataset represents the monthly labour hours worked across different sites, and these labour hours worked have been split by contractors/staff. This dataset, being a <em>monthly</em> total of hours worked, is a lot higher-level relative to the other datasets that have been provided. However, this is still an important source of information because, generally speaking, not a lot of information is always recorded surrounding contractors working within an organisation. This means that these monthly hours worked by contractors might be the most accurate piece of information that the organisation can use to gauge the working patterns of contracting employees.</p><p><strong>site_location.csv</strong></p><p>This dataset contains the <em>approximate</em> location of each mine site/operation. The given locations are <em>approximate</em>, meaning that the actual location of the safety events that have occurred at these specific sites, could have occurred up to ~50km away from these provided latitude/longitude locations.</p><p><strong>1.2 Approach</strong></p><p>The Approach I will be using is to answer the question to consider. These are the questions. This is by no means an exhaustive list — this list is intended to give you an idea of what sort of questions the safety team are interested in understanding from this data. I was encouraged to get creative with the direction of my analysis.</p><h3><strong>Analyse the provided datasets to uncover insights into what features are contributing to the observed safety incidents.</strong></h3><p>Does the time of year/month/week/day, public holidays, daylight savings, etc. correlate with increased incident likelihood?</p><p>Can we better understand where / when incidents are occurring? With which teams? And, related to what types of activities?</p><p>Is there any pattern within the prevalence of certain types of injuries? Are any specific body parts always involved? How is injury severity seen to differ?</p><p>Does the weather (strong winds, high temperature, lots of rain) impact the frequency, or severity, of safety incidents?</p><p>Do production rates influence the safety outcomes on-site?</p><p>Do new employees suffer more safety incidents compared to more experienced employees? How do their safety incidents differ?</p><p>When employees move in/out of a workgroup, is there any change in the observed safety incidents?</p><p>When individuals come off leave or have a significant change in their roster, does this affect the nature/prevalence of their safety incidents?</p><p>Does the amount of leave taken within a team affect the observed safety incidents within that team?</p><p>Is there any correlation between the number of hours worked (by contractors/staff) and the amount/severity of safety incidents?</p><p>Do incidents tend to reduce after a large number of incidents have been recorded, or after a significantly severe incident was recorded?</p><h3><strong>Develop approaches that allow the safety team to provide insights/trends/actions back to the wider business on the safety performance across time.</strong></h3><p>Can we conclude that safety performance has been improving across the business over time? How do we know this? Are there any points in time where we observed large improvements in safety performance?</p><p>Can we flag when / where certain types of incidents are increasing so we can trigger some sort of interventionary action (extra education, preventative measures, risk controls, etc.)?</p><p>Are there any types of data that you believe we should be recording to help understand/reduce safety incidents, that we’re not?</p><h3>2.0 ANALYSIS</h3><p><strong>2.1 Analyses the provided datasets to uncover insights into what features are contributing to the observed safety incidents:</strong></p><p><strong>1. Does the time of year/month/week/day, public holidays, daylight savings, etc. correlate with increased incident likelihood?</strong></p><p>I will be using the safety dataset safety.csv and the worker roaster data. To explore this question, first I started by converting all the date/ time features in the safety dataset and roaster to date_time data_types in pandas. After which I will create a new data frame. This new data frame is will be formed from date-time features, I will extract the month of the incidents, week of the incidents, time of the year, daylight etc. for both datasets. since RDO mean the off-day of the workers</p><p>The time of the year will be</p><p><strong>Winter</strong> — December, January and February.</p><p><strong>Spring</strong> — March, April and May.</p><p><strong>Summer</strong> — June, July and August.</p><p><strong>Autumn </strong>— September, October and November.</p><p><strong>HYPOTHESIS: </strong>To compare and check whether year/month and day increased the likelihood. This will be checked by comparing the day of work with the day of the accident.</p><p>I will find the probability of an accident occurring in that period.</p><p>- P(A) = Probability of an event (event A) occurring.</p><p>- f = Number of ways an event can occur (frequency)</p><p>- N = the Total number of outcomes possible.</p><p>For the Roaster dataset, This means FIFO workers work for like 4 hours to 11 hours. Even if workers work for only 1 min, accidents still can occur. I will add FIFO to the working day. I will only subtract RDO from it.</p><p>The value <em>‘RDO’</em> that’s seen within this scheduled work hours column indicates the employee’s rostered day off (RDO). In the resources sector, commonly, individuals will fly to the mine site, work for some amount of time, and then fly back home to enjoy their time off. This work is commonly referred to as FIFO work (fly-in-fly-out work). The days that an individual is working on-site will be indicated by a number within the scheduled work hours column (typically being equal to 12, implying 12 hours worked).</p><p><strong>Does the time of the year affect the likelihood?</strong></p><p>I will plot the count plot of the total days that work happen on the site and also the days that accident happens to do the probability of occurrence for each time of the year.</p><p>These are the count plot for both.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/627/1*uDXSevcm6GzmIMi6GP6v8w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*cK3lYnbcrBfXuhsTTNcUIA.png" /></figure><p>This graph shows that there is more work during the summer of the year and it records more accidents in the winter. Let&#39;s check for likelihood using the probability</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/585/1*aPtCuOBNlSPiGyxGdJWEdQ.png" /></figure><p>Does Month affect the likelihood:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*rgp_4CKbvuqjLblGnhzziQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/407/1*jaYieyvq61jKlsqoPssVrQ.png" /></figure><p>c) Does Daylight saving affect the likelihood: There are only 10 Locations out of 15 site records of accidents. That sites that are not recorded as groups as null Locations. Countplot of accidents in the daylight saving area.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*UvQ0DtBmf_a0wdN4yjEvqA.png" /></figure><p>Does Holiday affect the likelihood:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*TqZgNXnsyWw9Wb2R1_R5gA.png" /></figure><p><strong>2. Can we better understand where / when incidents are occurring?</strong></p><p>With which teams? And, related to what types of activities? I will first plot the area of the accident. It is only 10 locations were given from the data out of 15 reported sites affected</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QesLqCeTsp9H2dv3WifdSQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TMNp-sFjvz7MXC6IB63ZRg.png" /></figure><p>The Blue indicator shows approx. 50 km to the site. I will concentrate on each site depending on the number of occurrences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*NNthjQkgvGd5hGJ3FJN4Xg.png" /></figure><p>A site with hashed key (bf4065199d7003b) has the highest accident with 2843, it is located.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*59uOT8AMyRn52hJJO0kPzQ.png" /></figure><p><strong>With which teams? And, related to what types of activities</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*_XGcKlDtf-N-SRYdEAg3NA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*6POsRXJ24wzA8NUiSAqYoQ.png" /></figure><p>The graphs show at this named site, Team 764 has more accidents and there are activities involved at the site.</p><p><strong>Second Hihgest Site(49d11ae6ef35a6f5) : Its Location</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/963/1*RMiLZdRdwlUfinECYNFzkA.png" /></figure><p><strong>With which teams? And, related to what types of activities?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*kjjYYPCxe4ZSQwzaOYVcmg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*8aQ3bsZXvWJuQH0nYJeKUw.png" /></figure><p>Team 633 has the highest accident while Underground activities have the highest accident.</p><p>Site 3(6d4b7ee99aebdab7) with highest incidents: its Location</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1012/1*8kWRdbYPSHTocERe4IrA_A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*8aQ3bsZXvWJuQH0nYJeKUw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*8aQ3bsZXvWJuQH0nYJeKUw.png" /></figure><p>Team 325 has the highest accident while Unspecified agencies’ activities caused more accident</p><h4><strong>Is there any pattern within the prevalence of certain types of injuries? Are any specific body parts always involved? How is injury severity seen to differ?</strong></h4><p>For this question, let me first define the Definition of prevalence Prevalence, sometimes referred to as prevalence rate, is the proportion of persons in a population who have a particular disease or attribute at a specified point in time or over a specified period. Prevalence differs from incidence in that prevalence includes all cases, both new and preexisting, in the population at the specified time, whereas incidence is limited to new cases only. Lost Time Injury refers to incidents that result in a disability or an employee missing work due to an injury. Employees who return to work after sustaining an injury may also be counted as a Restricted Work Injury if they are unable to completely perform their job duties as outlined in their job description. Also, only injuries deemed to be work-related will count as a Lost Time Injury. Accidents that happen off the clock that affect an employee’s ability to work will not be counted in this metric.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/990/1*ZoTBbgSWPoXEBfFLL5HnfA.png" /></figure><p><strong>First_aid Injury:</strong></p><p>Are any specific body parts always involved</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*C0sPRYVd8SCvokOYMBclZQ.png" /></figure><p><strong>No_ Treatment</strong></p><p>Are any specific body parts always involved?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*WBe2exgmVVmstaFFJRrtTw.png" /></figure><p><strong>Medical Treatment</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*UUh2wpSv3fTPpmD1qJT25Q.png" /></figure><p><strong>Restricted Work injury</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*MlOi4wTP7IBWeoDqn11JGg.png" /></figure><p><strong>Non-work</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*UkxRaMZAQa1KU5yhLGW8yQ.png" /></figure><p><strong>Occupational Injury/illness</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/903/1*rvUfUCbFTmhjuqdttVGGqg.png" /></figure><p><strong>Severity Rate of the injury:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/1*zB1ayiRmc1x7sVK4wUSTKA.png" /></figure><p><strong>Does the weather (strong winds, high temperature, lots of rain) impact the frequency, or severity, of safety incidents?</strong></p><p>In Brisbane, the summer started in December and ends in Feb, there is more rain. Perth has a hot-summer Mediterranean climate. It is the windiest city in Australia. It is also considered to be the third windiest capital city in the world. Perth is also the sunniest city in Australia. Winnipeg region is the coldest city in the world. With an average annual temperature of 38 degrees, Winnipeg is far from being the coldest city on Earth. (Chicago’s average annual temperature, by comparison, is about 50 degrees, depending on the location.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*ZUfAIplGvBwyQ-B0eoS68Q.png" /></figure><h4><strong>Do production rates influence the safety outcomes on-site?</strong></h4><p><strong>I will be using production data and safety data. First of all, I will define a threshold for production. To define a threshold I find the max. actual prod and min actual prod. I will be the median production as the yard. Using histogram</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZFkNj2zWFvoX0NH6ANwgbg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tUU1aP3_GV7MJPR3NbOmAA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gd_-bto614r5kfNyuogjiA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s7IqNo5IIVliW6xjLGwZxg.png" /></figure><p><strong>Do new employees suffer more safety incidents compared to more experienced employees? How do their safety incidents differ?</strong></p><p>I will the Employee start and end date for this question for the end date that is null, I assumed that the person is still working in the mining organization. First I get the number of days for the person using a threshold of 90 days i.e month to check whether. After which based on the name of the worker I merged.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*sPamo4KpMjUQEokgQhesIA.png" /></figure><p><strong>Is there any correlation between the number of hours worked (by contractors/staff) and the amount/severity of safety incidents?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/1*2fnLvEpfJKMowaiqH4-dPw.png" /></figure><p>There is a correlation</p><p><strong>Do incidents tend to reduce after a large number of incidents have been recorded, or after a significantly severe incident was recorded</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*90DZBWb2hwE2v_wLxbBsiQ.png" /></figure><p>No, the accident increase with time.</p><h3><strong>B. Develop approaches that allow the safety team to provide insights/trends/actions back to the wider business on the safety performance across time.</strong></h3><p>Can we conclude that safety performance has been improving across the business overtime? How do we know this? Are there any points in time where we observed large improvements in safety performance</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/871/1*Xge5IL523vVIUcKE6875yA.png" /></figure><p>Can we flag when / where certain types of incidents are increasing so we can trigger some sort of interventionary action (extra education, preventative measures, risk controls, etc.)?</p><p><strong>For First Aid</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/819/1*EQloKoG7O78e_pK0iGSdSw.png" /></figure><p><strong>No Treatment</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/807/1*nvLGu6V-Z6mNHuykqdA1sQ.png" /></figure><p><strong>Medical Treatment</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*WDbyLATuprNsSK7RJtZeVQ.png" /></figure><p><strong>Lost Time Injury</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/810/1*YGTG6l_sb93z3UcYXbKWGA.png" /></figure><h3><strong>CONCLUSION</strong></h3><p>The weather presents some challenging hazards to safe mining operations: lightning poses a risk to personnel involved in heavy equipment operation, explosives handling and construction activities; heavy rain can close mines and bog down access roads; winds disrupt blasting; high temperatures can affect staff and machinery. However, with effective planning, the disruption caused by adverse weather can be minimized and risks mitigated. Mine operators can now proactively and effectively manage pending storm risks with technology which tracks observed lightning, delivers proximity alerts, and forecasts activity over the coming hour. Enhance ground control protection by identifying key elements in the control of these hazards and requiring employers to maintain a record of significant seismic events in addition to incidents of ground instability Require contractor to prepare a formal plan to manage hazards that cause occupational illness, including requirements for worker and supervisor training and communication To reduce those risks, the mining Organization should use a consistent change management process when introducing new technology and/or a change in the process. Based on the information provided by safety incident records, which will include training for all workers (including change contractors and staff) periodically and conducting regular risk assessments, the mining organization can take proactive measures to address and mitigate hazards. It is also important to continuously monitor and analyze safety incident data, in order to identify patterns and trends and take appropriate interventionary actions such as providing extra education, implementing preventative measures and implementing risk controls. The mining organization should also ensure that contractors and staff are properly trained and aware of the hazards and risks present in the mining operations. Additionally, the mining organization should have a consistent change management process in place to ensure that any changes made to the technology, equipment or processes are carefully evaluated and implemented in a way that minimizes the risk of safety incidents. By implementing these measures, the mining organization can proactively manage the risks posed by adverse weather and other hazards and create a safer working environment for its employees and contractors.</p><p>All screenshots were taken by the author. The introductory image on <a href="https://stock.adobe.com/search?k=regulations+health+and+safety&amp;search_type=autosuggest&amp;continue-checkout=1">Adobe Stock.</a></p><p>All the data used is available on the GitHub account</p><p>In case you miss my previous one on <a href="https://medium.com/@adelekedejiimoleayo/web-scraping-and-analysis-of-qatar-world-cup-2022-1f2c15f26025"><strong>World Cup Analysis</strong></a></p><p>I sincerely hope you’ve enjoyed it and that it has some value for you in some way. You can reach me on <a href="https://twitter.com/crown_deji?t=yyzLghHlVqixn6hSEI9AMw&amp;s=09"><strong>Twitter</strong></a>,<strong> </strong><a href="https://github.com/Gracebaytech"><strong>GitHub,</strong></a><strong> </strong>or <a href="http://www.linkedin.com/in/adelekeadedeji-09aba0195"><strong>Linkedin</strong></a> if you need anything, have a suggestion, or simply want to chat.</p><p>Thanks for reading.</p><p>Follow me to view more stories….</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=68afe132ed4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[WEB SCRAPING AND ANALYSIS OF QATAR WORLD CUP 2022]]></title>
            <link>https://medium.com/@adelekedejiimoleayo/web-scraping-and-analysis-of-qatar-world-cup-2022-1f2c15f26025?source=rss-29e5af4a8360------2</link>
            <guid isPermaLink="false">https://medium.com/p/1f2c15f26025</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[web-scraping]]></category>
            <category><![CDATA[power-bi]]></category>
            <category><![CDATA[mongodb]]></category>
            <dc:creator><![CDATA[ADELEKE ADEDEJI]]></dc:creator>
            <pubDate>Fri, 23 Dec 2022 22:49:28 GMT</pubDate>
            <atom:updated>2022-12-24T18:42:45.297Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oINHs75Ju4oL6W9HsnxdIQ.jpeg" /><figcaption>Images on UNSPLASH</figcaption></figure><p>Every four years, the men’s national teams of FIFA’s member associations compete in the FIFA World Cup, an international football competition. The FIFA World Cup, which is in its 22nd year. The event was held in Qatar from November 20 through December 18, 2022. 64 matches are played by 32 teams.</p><p>For my data, I shall scrape the internet.</p><p>How does web scraping work? Web scraping is a computerized technique for gathering copious volumes of data from websites. Most of this data is in an unstructured format, which is HTML, and is transformed into structured data in a spreadsheet or database so that it may be used in different applications. To collect data from websites, web scraping can be done in a variety of ways.</p><p>I will be using three Python libraries.</p><p>1. Requests</p><p>2. BeautifulSoup</p><p>3. Pandas_read Html</p><p>Requests is a straightforward yet sophisticated HTTP library. Queries make it incredibly simple to send HTTP/1.1 requests. Both form-encoding your POST data and manually adding query strings to your URLs are not required. Both HTTP connection pooling and keep-alive are fully automated.</p><p>The Python module BeautifulSoup extracts data from HTML and XML files. It provides natural means of traversing, searching, and altering the parse tree in conjunction with your preferred parser. It frequently spares programmers several hours or even days of labour.</p><pre>pip install beautifulsoup4<br></pre><p>Pandas read Html: Converting an HTML table into a pandas DataFrame is simple and rapid with the pandas read html() function. Without knowing how to scrape a website’s HTML, this tool can be useful for swiftly combining tables from numerous websites. However, structuring and cleaning the data before analysis can present some difficulties. I’ll go over how to use pandas read html() to read and clean a few HTML tables from Wikipedia so you can use them for additional numerical analysis in this article.</p><p>I will be scraping this football <a href="https://fbref.com/en/comps/1/World-Cup-Stats">website.</a></p><p>I want to scrape data for each player that plays in the World Cup game.</p><p>Importing of python Libraries</p><pre>import requests<br>from bs4 import BeautifulSoup<br>import pandas as pd</pre><p>Now, the first thing you’ll want to do is import some necessary packages — BeautifulSoup and requests.</p><pre>standing_url=&quot;https://fbref.com/en/comps/1/World-Cup-Stats&quot;<br>data=requests.get(standing_url)<br>data.status_code</pre><p>Next, you’ll want to make a <em>get </em>request to retrieve your webpage and then pass the contents of the page through BeautifulSoup so that it can be parsed.</p><p>Before that, I will read_html to read the League table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/1*_cDplyjbCnejnyRKjvUumw.png" /></figure><pre># To scrabe the league Table for the World Cup<br>WC_league=pd.read_html(data.text, match=&quot;League Table&quot;)[0]<br></pre><p>Looking through the website, this is the last table on the site. I need to get the URL for each team that plays in the World Cup. This table contains the national team. In HTML, there is what we called “href”. The href attribute link (short for “Hypertext Reference”) <strong>indicates the relationship between pages to search engines</strong>. href is an attribute of the anchor tag and contains two components: the URL (the actual link). To get the href for the team. I will select the table, then I will get the a tag attributes.</p><pre>soup=BeautifulSoup(data.text)<br># selecting the legue table<br>standing_table=soup.select(&quot;table.stats_table&quot;)[-1]<br># finding all the a tag in the table<br>links=standing_table.find_all(&quot;a&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*VQCXZMEaWNQEQN5AjCx3MA.png" /></figure><p>we’ll use the find_all method. This method retrieves all the elements on the page that match our specifications. Here we use the tag name and the class to retrieve all the quotes. Using this syntax, we can even specify more href to filter:</p><pre># filter the the href to get the important url needed<br>links=[l for l in links if &quot;/squads/&quot; in l]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/1*x5n-ayret5G8luhayceihA.png" /></figure><p>From the list of the team URL. I will first be scraping through the first team to know what I need.</p><p>The team’s name is France.</p><p>Next, you’ll want to make a <em>get </em>request to retrieve your webpage and then pass the contents of the page to read_html in pandas.</p><p>I want to create the table for “Shooting”, “Goalkeeping”, “Passing”, “Pass Types”, “Goal and Shot Creation”, “Defensive Actions”,</p><p>“Possession” and “Miscellaneous Stats”.</p><p>I will extract the information I need from that table and then merge the table to get the information for team France.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/915/1*8Up5lE783DHRjIz7ZlvbDA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/936/1*ejpCsm2HfkshpatSFAJeiA.png" /></figure><h3>Grabbing more Data</h3><p>Despite the fact that each URL page only contains team-related information, I’m thinking you’ll want to scrape more than that. We must go to the following URL page in order to receive more quotes. Again, depending on the website you’re scraping, there are many ways to go about doing this.</p><p>Instead of using “https://fbref.com/en/squads/b1b36dcd/2022/Argentina-Men-Stats&quot;, we can use “https://fbref.com/en/squads/b1b36dcd/2022/France-Men-Stats&quot; and alter the page number in another form.</p><p>We are scraping all of the team’s URL sites for this example. This is the complete task’s code:</p><pre>import time<br><br>all_players=[]<br>all_keeper=[]<br><br><br>standing_url=&quot;https://fbref.com/en/comps/1/World-Cup-Stats&quot;<br>data=requests.get(standing_url)<br>League_Table=pd.read_html(data.text, match=&quot;League Table&quot;)[0]<br>soup=BeautifulSoup(data.text)<br>standing_table=soup.select(&quot;table.stats_table&quot;)[-1]<br>links=standing_table.find_all(&quot;a&quot;)<br>links=[l.get(&quot;href&quot;) for l in links]<br>links=[l for l in links if &quot;/squads/&quot; in l]<br>team_urls=[f&quot;https://fbref.com{l}&quot; for l in links]<br><br>for team in team_urls:<br>    team_name=team.split(&quot;/&quot;)[-1].replace(&quot;-Men-Stats&quot;,&quot;&quot;).replace(&quot;-&quot;,&quot; &quot;)<br>    data=requests.get(team)<br>    shoot=pd.read_html(data.text, match=&quot;Shooting&quot;)[0]<br>    shoot.columns=shoot.columns.droplevel()<br>    keep=pd.read_html(data.text, match=&quot;Goalkeeping&quot;)[0]<br>    keep.columns=keep.columns.droplevel()<br>    Pass=pd.read_html(data.text, match=&quot;Passing&quot;)[0]<br>    Pass.columns=Pass.columns.droplevel()<br>    PaT=pd.read_html(data.text, match=&quot;Pass Types&quot;)[0]<br>    PaT.columns=PaT.columns.droplevel()<br>    goal=pd.read_html(data.text, match=&quot;Goal and Shot Creation&quot;)[0]<br>    goal.columns=goal.columns.droplevel()<br>    defen=pd.read_html(data.text, match=&quot;Defensive Actions&quot;)[0]<br>    defen.columns=defen.columns.droplevel()<br>    pos=pd.read_html(data.text, match=&quot;Possession&quot;)[0]<br>    pos.columns=pos.columns.droplevel()<br>    stats=pd.read_html(data.text, match=&quot;Miscellaneous Stats&quot;)[0]<br>    stats.columns=stats.columns.droplevel()<br>    play=pd.read_html(data.text, match=&quot;Playing Time&quot;)[0]<br>    play.columns=play.columns.droplevel()<br>    stand=pd.read_html(data.text,match=&quot;Standard Stats&quot;)[0]<br>    stand.columns=stand.columns.droplevel()<br><br><br>    n_stand=[&#39;Player&#39;, &#39;Pos&#39;, &#39;Age&#39;, &#39;Club&#39;, &#39;MP&#39;, &#39;Starts&#39;, &#39;Min&#39;, &#39;90s&#39;, &#39;Gls&#39;,<br>        &#39;Ast&#39;, &#39;G-PK&#39;, &#39;PK&#39;,&#39;CrdY&#39;, &#39;CrdR&#39;]<br>    _stand=stand[n_stand]<br><br>    n_shoot=[&#39;Player&#39;,&#39;Sh&#39;, &#39;SoT&#39;, &#39;G/Sh&#39;, &#39;G/SoT&#39;]<br>    _shoot=shoot[n_shoot]<br><br>    n_keep=[&#39;Player&#39;, &#39;Pos&#39;, &#39;Age&#39;, &#39;MP&#39;, &#39;Starts&#39;, &#39;Min&#39;, &#39;90s&#39;,&#39;SoTA&#39;, <br>    &#39;Saves&#39;, &#39;Save%&#39;, &#39;W&#39;, &#39;D&#39;, &#39;L&#39;, &#39;CS&#39;, &#39;CS%&#39;]<br>    _keep=keep[n_keep]<br><br>    n_ass=Pass.iloc[:,4:7]<br>    n_ass[&quot;Player&quot;]=Pass[&#39;Player&#39;]<br><br>    n_defen=[&#39;Player&#39;,&#39;Tkl&#39;, &#39;TklW&#39;,&#39;Blocks&#39;, &#39;Int&#39;,<br>        &#39;Tkl+Int&#39;, &#39;Clr&#39;, &#39;Err&#39;]<br><br>    _defen=defen[n_defen]<br><br>    n_pos=[&quot;Player&quot;,&#39;Touches&#39;,&#39;Succ&#39;, &#39;Att&#39;, &#39;Succ%&#39;]<br>    _pos=pos[n_pos]<br><br>    n_stats=[&#39;Player&#39;,&#39;Fls&#39;, &#39;Fld&#39;,<br>        &#39;Off&#39;, &#39;Crs&#39;, &#39;Int&#39;, &#39;TklW&#39;, &#39;PKwon&#39;, &#39;PKcon&#39;, &#39;OG&#39;,&#39;Won&#39;,<br>        &#39;Lost&#39;,]<br>    _stats=stats[n_stats]<br><br><br>    match=[_shoot,n_ass,_defen,_pos,_stats,_stand]<br>    team_match=merge_mult_dfs(match)<br><br><br><br>    team_match[&quot;Team&quot;]=team_name<br>    new=team_match.query(&quot;Player != &#39;Squad Total&#39; and Player !=&#39;Opponent Total&#39;&quot;)<br><br>    Keeping=_keep.query(&quot;Player != &#39;Squad Total&#39; and Player !=&#39;Opponent Total&#39;&quot;)<br><br>    all_players.append(new)<br>    all_keeper.append(Keeping)<br><br>    time.sleep(3)<br><br><br><br><br><br><br><br></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*iP9Pw-eNqUzo_cndzyuBIg.png" /></figure><h3>Storing the Data</h3><p>The scraping is finished, but I assume you want to save the data in a file so you can utilize it later rather than just printing it out in the console.</p><p>There are various methods for doing this. The simplest method is to compile the information for each quote into a list and then append that list to another list. The list of lists can then be quickly converted into a DataFrame and exported as a.csv file.</p><p>The dataset will be kept by me in a database named MongoDB.</p><p>MongoDB is a source-available cross-platform document-oriented database program. Classified as a NoSQL database program, MongoDB uses JSON-like documents with optional schemas.</p><p><strong>Install PyMongo Module</strong></p><p>Firstly, we need to run connect with the MongoDB client, so we need to install the pymongo module using this command.</p><pre>import pymongo<br>from pymongo import MongoClient<br></pre><p><strong>Step 2 — Connect to MongoDB</strong></p><p>We will import the module pymongo to connect with MongoDB to insert records in the database.</p><pre>cluster=MongoClient(&#39;mongodb+srv://Gracebay04:15996@gracebay.zyepmqz.mongodb.net/?retryWrites=true&amp;w=majority&#39;)<br>db=cluster[&quot;World_Cup&quot;]<br>collections=db[&quot;GoalKeeper_Record&quot;]</pre><p>We will pass MongoDB connection details and connection to the database and collection to insert records.</p><p><strong>Step 3 — Reading CSV File</strong></p><p>We will insert CSV file data to MongoDB, so first, we will read CSV files and convert data into JSON. Import the CSV module at the top of the file to read the CSV file.</p><p>We will CSV file name user using DictReader() method. The DictReader() function returns a CSV reader object.</p><pre>import csv<br><br>with open(&#39;all_keeper_WC.csv&#39;, &#39;r&#39;) as f:<br>    reader = csv.DictReader(f)<br><br>    #get fieldnames from DictReader object and store in list<br>    headers = reader.fieldnames<br><br>    for each in reader:<br>     rows={}<br>     for field in headers:<br>         rows[field]=each[field]</pre><p>We will iterate(loop) over the CSV reader object and create JSON data to insert multiple records into the MongoDB database.</p><p><strong>Step 4 — Insert Data into MongoDB</strong></p><p>Finally, we will insert JSON rows data into MongoDB using the insert_many() method.</p><p><strong>Step 5 — Insert CSV data into MongoDB</strong></p><p>Final code to import CSV data into MongoDB.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QHf79t9XV3xv6JLz5jgfHA.png" /></figure><p>I will be using Power Bi for my Dashboard</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CU8WtX5DP9NXebPioI0cOQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1IrthXqtsREESx1lkQvaWg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pcc3jUoF6Nr34hYOHptk2A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wGgTLD2JQKqZgDkfrhGlvg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9xs31I2lEMYltFEpvF_3lA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4wWV4_ajPMplqeK97JxwZw.png" /></figure><p>All the data scraped is available on the GitHub account and the link to powerBi is <a href="https://app.powerbi.com/view?r=eyJrIjoiYWM1ZjMwMTYtNzkwNy00NmEyLWI3M2ItY2JlNGY5Y2IwMTIxIiwidCI6Ijg3MTMxYjliLTg3NmUtNDdkZS1hMjlhLTExMGQ4ZDM5MWRiOCJ9"><strong>here</strong></a></p><p>In case you miss my previous on <a href="https://medium.com/@adelekedejiimoleayo/a-comprehensive-project-for-data-analysis-tracks-70b035d3232c">SQL</a> and marketing Campaign</p><p>I sincerely hope you’ve enjoyed it and that it has some value for you in some way. You can reach me on <a href="https://twitter.com/crown_deji?t=yyzLghHlVqixn6hSEI9AMw&amp;s=09"><strong>Twitter</strong></a>,<strong> </strong><a href="https://github.com/Gracebaytech"><strong>GitHub,</strong></a><strong> </strong>or <a href="http://www.linkedin.com/in/adelekeadedeji-09aba0195"><strong>Linkedin</strong></a> if you need anything, have a suggestion, or simply want to chat.</p><p>Thanks for reading.</p><p>Follow me to view more stories….</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1f2c15f26025" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A comprehensive project for data analysis Tracks]]></title>
            <link>https://medium.com/@adelekedejiimoleayo/a-comprehensive-project-for-data-analysis-tracks-70b035d3232c?source=rss-29e5af4a8360------2</link>
            <guid isPermaLink="false">https://medium.com/p/70b035d3232c</guid>
            <dc:creator><![CDATA[ADELEKE ADEDEJI]]></dc:creator>
            <pubDate>Tue, 06 Dec 2022 11:17:05 GMT</pubDate>
            <atom:updated>2022-12-06T11:17:05.669Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kBjl3WUOHtV87sLVj3kavA.jpeg" /><figcaption>image on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>I was advised to study SQL when I was first introduced to the field of data science because it is a vital skill. With personal research and feedback from my professional network in the area, I’ve found that most data scientists learn SQL from those who study database management or data engineering.</p><p>Before beginning to learn SQL as a data scientist, I often had no idea where to begin when I wanted to build a SQL project, like many high school projects.</p><p>In this narrative about the SQL portfolio project, I want to show how to construct a new SQL database from a CSV or Excel file and then query it on a data scientist’s desk. Most data scientists use PyCharm or Jupyter as their notebooks for their work and also build a dashboard with a Visualization Tool (Power Bi)</p><p><strong>What is SQL?</strong></p><p>Structured Query Language, abbreviated as SQL, is a domain-specific language used in programming and designed for managing data held in a relational database management system or for stream processing in a relational data stream management system.</p><p><strong>What is a database</strong>: An orderly collection of data is what makes up a database. A database does not have to be digital to be referred to as such. An excellent example is a phone book, which contains information about people and businesses along with a phone number. A database in its most basic form is a to-do list. Digital databases have grown in significance since their introduction in the 1960s, as ever-increasing volumes of data regarding even the most routine of procedures are recorded.</p><p>A database management system (DBMS) is software used to manage digital databases (DBMS). A DBMS is what someone thinks about PostgreSQL or MySQL. When you use DBMS software to store data on subjects that make sense to you or your organization, you create a database. For instance, your organization may use PostgreSQL to keep track of the inventory of the cell phones you sell. In this instance, PostgreSQL was used as your DBMS to create an inventory database.</p><p>In this study, I’ll be utilizing SQLite, a database engine created in the C programming language. Generally speaking, the basic engine of SQLite is less than one megabyte in size.</p><p><strong>Problem Statement:</strong> The International Olympic Committee (IOC) is worried about having several CSV and is apprehensive about extracting information from various CSV. A database is essential for them. I’ve been hired to handle data storage and database queries.</p><h3>Introduction.</h3><p>A zip file containing six files with various datasets that include records and data for every Olympic event during the course of the Games’ 120-year history. You may download the zip file from <a href="https://www.kaggle.com/datasets/heesoo37/120-years-of-olympic-history-athletes-and-results">Kaggle</a>.</p><p>A list of the CSV files is provided.</p><p><strong>Athletics Biography:</strong> Each Olympic athlete’s name, sex, birth date, height, weight, nationality, description, and any relevant special remarks are included in this dataset.</p><p><strong>Athlete Events Results: </strong>This dataset includes data based on each event’s outcome, including the edition year and season, edition ID, the kind of sport, the type of event, and other information. Athlete name, results ID, athlete id, medal, and whether the sport is a team sport or not.</p><p><strong>Olympic Games Medal Tally</strong>: The medal each individual or team has earned are listed in this dataset. information such as the edition’s year and season, edition ID, country, country noc, the total number of medals earned, and gold, silver, and bronze awards.</p><p><strong>Results</strong>: This dataset includes data on each event’s outcomes. comparable to result_id, event title, edition year, edition id, sports, sports URL, result location, result participants, result details, result description, start date, end date, and time.</p><p><strong>Country:</strong> All nation that has ever participated in the tournament</p><p><strong>Games:</strong> Each game in each edition year is described in this dataset. Details such as edition name, edition id, year, city, nation flag URL, country noc, start date, finish date, is held, competition end date, and competition start date.</p><h3>Importing Python Libariers Package:</h3><p>I’ll be using a Jupyter notebook and importing several libraries for this project. Let’s learn more about these libraries:</p><p>Sqlite3: A lightweight disk-based database named SQLite is a C library that enables access to the database using an alternate form of the SQL query language without the requirement for a separate server process. Some programs allow internal data storage using SQLite. Additionally, I will be using SQLite to create an application prototype.</p><p>Pandas: view my previous post on <a href="https://medium.com/@adelekedejiimoleayo/marketing-campaigns-of-a-portuguese-banking-institution-dataset-analysis-report-using-python-f2ac996c574b">EDA linkages</a></p><pre># Importing of libraries<br>import pandas as pd<br>import sqlite3<br>import os</pre><p>First, I need to create a pathway to work that is more reason why I import OS</p><p>First, I need to create a new database and open a database connection to allow sqlite3 to work with it. <strong>Call sqlite3.connect()</strong> to create a connection to the database <strong>Olympic_data.db</strong> in the current working directory, implicitly creating it if it does not exist.</p><p>The returned<strong> Connection</strong> object connector represents the connection to the on-disk database. To execute SQL statements and fetch results from SQL queries, we will need to use a database cursor. <strong>Call connector.cursor()</strong> to create the<strong> Cursor:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/918/1*u_bsiDYHEvETlh7W2LLkDQ.png" /><figcaption>SOURCE BY AUTHOR\</figcaption></figure><p>I pass this flag value to the <em>detect_ types</em> (PARSE_COLNAMES) parameter of connect() to look up a converter function by using the type name, parsed from the query column name, as the converter dictionary key. The type name must be wrapped in square brackets ([]) and also pass this flag value to the <em>detect_types (</em>PARSE_DECLTYPES<em>)</em> parameter of connect() to look up a converter function using the declared types for each column. The types are declared when the database table is created. sqlite3 will look up a converter function using the first word of the declared type as the converter dictionary key.</p><p>Importing Data into Data Frames:</p><p>To start working on data first we need to import data from CSV files to pandas DataFrame. This will be done using pandas’ read_csv method</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*livynTexqo2bLBsW0CkMVw.png" /><figcaption>Image by Author</figcaption></figure><p>Now that we’ve got a database connection and a cursor and also import the dataset, we can create a database table Althete_Bio with columns for s/n, athlete_id, name, sex, born, height, weight, country, country_noc, description, and special_notes. For simplicity, we can just use column names in the table declaration — thanks to the <strong>flexible typing</strong> feature of SQLite, specifying the data types is optional. Execute the CREATE TABLE statement by calling cur. execute(...):</p><pre>**Table Creation** <br>SYNTAX:<br>      CREATE TABLE Name_of_table(column names and properties )<br></pre><p>I can verify that the new table has been created by querying the sqlite_master table built-in to SQLite, which should now contain an entry for the Athlete_Bio table definition). Execute that query by calling <strong>curr. execute(…)</strong>, assign the result to<strong> check</strong> and call <strong>check.fetchone() </strong>to fetch the resulting row:</p><pre><br>check = curr.execute(&quot;SELECT name FROM sqlite_master&quot;)<br>check.fetchone()</pre><p>After this, I will insert my values in the table, the traditional way is to insert them row by row or sample by sample. SQL accept data in tuples then I need to convert each row to a tuple then.</p><pre># Inserting many row at onces<br>curr.executemany(insertQuery,df_Bio_Tuple)</pre><p>Then creating the imitative values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/763/1*PxEbcxAWqukhsbERhylYBQ.png" /></figure><p>Notice that? placeholders are used to bind data to the query. Always use placeholders instead of <a href="https://docs.python.org/3/tutorial/inputoutput.html#tut-formatting">string formatting</a> to bind Python values to SQL statements, to avoid <a href="https://en.wikipedia.org/wiki/SQL_injection">SQL injection attacks</a></p><p>After inserting all the rows/samples into the table created, then be commit</p><p>The COMMIT command is used <strong>to close out the current transaction and commit the changes to the database</strong></p><pre># Remember to commit the transaction after executing INSERT.<br>connector.commit()</pre><p>We can verify that the new rows were inserted by executing a<strong> SELECT</strong> query, this time iterating over the results of the query and converting it to a pandas data frame.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/963/1*VzBB_KarXWoa1p_r9kUN4A.png" /></figure><p>For the other five data sets, the exact same steps will be taken again.</p><p>Let me first explain some fundamental concepts, including the PRIMARY KEY. To uniquely identify each record in a table, a key is chosen. Primary key columns are not permitted to contain null values. It maintains distinct values over the whole column.</p><p>Although there are circumstances when a table might determine its main key using two columns, a table is typically assumed to have just one primary key.</p><p>Foreign Key</p><p>A foreign key is a column that is utilized as the primary key in one database but not another. Both duplicate values and numerous nulls are acceptable.</p><p>After inserting all the datasets then I will close the connection.</p><pre># Closing the connector<br>connector.commit()<br>connector.close()</pre><h3>Exploring the data</h3><p>When I explore the data, I’ll do so in a different way. A magic function is what Jupyter Notebook refers to this as.</p><p>A magic function is what?</p><p>In the Jupyter kernel, magic functions are pre-defined functions that carry out specified instructions. There are two different types of magic: line-oriented and cell-oriented, both preceded with % and %.</p><p>I need to first install the Ipython-SQL and SQLAlchemy modules. If you’re using a Jupyter notebook, Anaconda. Installed on the anaconda are the two libraries</p><pre>!pip install ipython-sql<br>!pip install SQLAlchemy</pre><p>This is how would we do it in case we have enabled SQL magic functions.</p><p>Ordinary IPython assignment works for single-line <strong>%sql</strong> queries:</p><pre>%sql select  name from sqlite_master where type=&#39;table&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*TfsG4yheeQ5fZHkBEPWoFA.png" /></figure><p>For multi-line queries, I need to use %%SQL and &lt;&lt; syntax</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*GdfPidmbrx3dk7mx8A9hjQ.png" /></figure><p>Pandas SQL magic has a nice integration with the pandas’ library. The result from the SQL query can be converted to a regular pandas data frame via DataFrame call.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*8buiVVY-cU1SdUl1y7ioKg.png" /></figure><p><strong>I will be moving to explore the tables in the database.</strong></p><h3>SELECT BASICS</h3><p>We use SELECT to query the database. Here is a basic SELECT statement to get data from the database. We say which columns we want to select and from which table. The * means we select all columns.</p><p>Assuming I want data for the result data</p><pre># selecting the table for Results<br>score=%sql select *from RESULTS ;<br>score</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*gRemjMLfaq9k3dbG8kXNXA.png" /></figure><h3>LIMIT</h3><p>This returns all our rows and all columns. Not very useful as there are 7329 rows. If we wanted to take a quick look at the kind of data we have, we could run the above with LIMIT to only return a specified number of rows. Here we return 2 rows:</p><pre>score=%sql select *from RESULTS limit 2;<br>score</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*cxuRkyGf_zRp9NZSlVt9fQ.png" /></figure><h3>SELECT SPECIFIC COLUMNS</h3><p>So we have fewer rows, but by using * we still see all the columns, which makes it hard to see the most interesting columns. We can replace * with the specific columns we want to view.</p><pre>#SELECT SPECIFIC COLUMNS FOR MEDAL<br>querry=%sql select Edition,country ,year,gold,silver,bronze from Games_Medal_Rally;<br><br>querry.DataFrame()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*g9kN5Idmx2J8vxeBxf9QNw.png" /></figure><h3>ORDER BY</h3><p>So far we have a list of records in alphabetical order by name (because that’s how the data in the CSV was and how it was saved to the bale). It would be more interesting to see the top 5 rows based on some metric. We can add the total column to the query to sort by it. This needs to go before LIMIT.</p><pre>#SELECT SPECIFIC COLUMNS FOR MEDAL OREDER BY total medal<br>querry=%sql select Edition,country ,year,gold,silver,bronze,total from Games_Medal_Rally ORDER BY total;<br><br>querry.DataFrame()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*iqgEz5fwAlW-3eKzkCLIHg.png" /></figure><h3>WHERE</h3><p>If we wanted to see this for a particular country, can filter on the team using WHERE. Let’s see for. So it’s WHERE [the column we want to match]=[value we want to match]:</p><pre>#SELECT SPECIFIC COLUMNS FOR MEDAL OREDER BY total medal and where clause<br>querry=%sql select country ,Edition,year,gold,silver,bronze,total from Games_Medal_Rally Where country=&#39;Nigeria&#39;  ORDER BY total;<br><br>querry.DataFrame()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*WPYP-c-HRSOht8OqHRzNbA.png" /></figure><h3>LIKE</h3><p>In case I am selecting a name WHERE the name is equal to a particular person, We can also use LIKE to match a pattern. For example, say we wanted to get a name Mark, we can replace = with LIKE and put ‘Mark%’ as what we want to match. This will look for all names that start with ‘Mark’. The % is a wildcard that matches any number of characters after that.</p><pre># LIKE<br>mark= %sql select athlete_id,country,name,born from Athlete_Bio Where name LIKE &#39;Mark%&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*_PiHNMk9BBZaN4pp9dxiAg.png" /></figure><p><strong>GROUP BY</strong></p><p>Up until now, we have been returning single rows (each representing a table). If we want to see the total of some metric for a particular country or person across all events we can use GROUP BY with the SUM function performed.</p><pre>querry=%sql select Edition,country ,year,SUM(gold)from Games_Medal_Rally GROUP BY country ORDER BY SUM(gold) DESC;<br><br>querry.DataFrame()</pre><p>JOIN</p><p>INNER JOIN: This is a type of join that returns the data that is common between two tables based on the relationship of the Primary Key and the Foreign Key.</p><p>LEFT JOIN: a join that favours the first table mentioned (Which is the Left Table). It returns all the IDs in the Left table and only what’s common in the Right Table.</p><p>RIGHT JOIN type of join that favours the last table mentioned (Which is the Right Table). It returns all the IDs in the Right table and only what’s common in the Left Table.</p><p>FULL JOIN: this does not favour any of the tables. It returns data for all the IDs in both the Right table &amp; left the table while filling whatever isn’t available with “null”.</p><h3>How to Use the Dataset for a complete athlete dataset</h3><ul><li><strong>Olympic_Athlete_Event_Results.csv</strong> — Contains basic athlete-to-result data (including athlete_id and result_id)</li><li>LEFT JOIN <strong>Olympic_Athlete_Event_Results.csv</strong> file with <strong>Olympic_Athlete_Bio.csv</strong> (on athlete_id) for complete information of the athlete (i.e. height, weight, date of birth when participating in the event)</li><li>LEFT JOIN <strong>Olympic_Athlete_Event_Results.csv</strong> with <strong>Olympic_Results.csv</strong> (on result_id) to obtain detailed information about the result (i.e. start date, sport, result_participants, format, time</li><li>Aggregate information to obtain features such as:</li><li><strong>BMI</strong> — from height and weight</li><li><strong>Age of participation</strong> — start date of the result subtract the date of birth of the athlete</li></ul><p><strong>Data cleaning and completion:</strong></p><p>Data preparation and completion: At this stage, we are fully aware of our data. We are aware that it has some missing values.</p><p>Na represents missing values.</p><p>On it, I’ll make various corrections. For instance, remove any entries from the data for which we have no start date and end date information.</p><p>Exclude any records for which we do not have a birth date. To calculate each Athlete’s age, subtract the start date from birthdate.</p><p>Exclude all records from the data when we don’t have any medal information.</p><p>Fill in the blank age values with the average age of other athletes.</p><p>Fill in the empty height values for women and men with the average height of women and men athletes.</p><p>Fill in the missing weight numbers for women and men with the average weight of female and male athletes competing in the same sport.</p><p>Brief Analysis Using Microsoft Power Bi (DashBoard)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*jG_CwKs8IaJlSiXS-tVgAQ.png" /></figure><p>You can view my project along with the jupyter notebook on my <a href="https://github.com/Gracebaytech"><strong>GitHub</strong></a></p><p>All screenshots were taken by the author. The introductory image on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70b035d3232c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Marketing campaigns of a Portuguese banking institution Dataset Analysis Report Using Python]]></title>
            <link>https://medium.com/@adelekedejiimoleayo/marketing-campaigns-of-a-portuguese-banking-institution-dataset-analysis-report-using-python-f2ac996c574b?source=rss-29e5af4a8360------2</link>
            <guid isPermaLink="false">https://medium.com/p/f2ac996c574b</guid>
            <category><![CDATA[data-storytelling]]></category>
            <category><![CDATA[insight-data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <dc:creator><![CDATA[ADELEKE ADEDEJI]]></dc:creator>
            <pubDate>Tue, 08 Nov 2022 14:01:35 GMT</pubDate>
            <atom:updated>2022-11-08T14:01:35.174Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is EDA?</h3><p>Exploratory data analysis (EDA) is about getting an overall understanding of data. EDA includes exploring data to find its main characteristics and identifying patterns and visualizations. EDA provides meaningful insights into data that can be used in a variety of applications, e.g., machine learning. Python can be effectively used to do EDA as it has a rich set of easy-to-use libraries like Pandas, Seaborn, Numpy, and Matplotlib. In this workshop, we will cover the basics of EDA using a real-world data set, including, but not limited to, correlating, converting, completing, correcting, creating, and charting the data. In addition, we will be using Jupyter Notebooks (an open-source web application that allows you to create and share documents that contain live code, equations, visualizations, and narrative text).</p><h3>Problem statement:</h3><p>A bank wants to promote its deposit-term rate with the client. I will be analyzing this data as a data analyst to find insights that can be used to promote term deposits effectively. The company is interested in the insights that would help them in their marketing campaigns.</p><h3>Introduction of data:</h3><p>Data is taken from Kaggle (<a href="https://www.kaggle.com/datasets/yufengsui/portuguese-bank-marketing-data-set">https://www.kaggle.com/datasets/yufengsui/portuguese-bank-marketing-data-set</a>). This is related to direct marketing campaigns for a Portuguese banking institution. The marketing campaigns were based on phone calls. Often, more than one contact with the same client was required to determine if the product (a bank term deposit) would be (or not be) subscribed.</p><p><strong>Bank client data:</strong></p><p>1. Age: client&#39;s age, and it is a numerical value</p><p>2. Job: The kind of Job the Clients do; it is a categorical variable, like Admin, Management, Housemaid, Entrepreneur, Students, Blue-collar, Self-employed, Retired, Technician, Services and Unknown</p><p>3. Marital: The marital status of the clients is a categorical variable. i.e. Married, Divorced, Single; Note Divorced means divorced or widowed)</p><p>4. Education: it is categorical i.e. Unknown, Secondary, Primary, Tertiary)</p><p>5. Default: The client has credit in default? (Binary: Yes, No)</p><p>6. Balance: average yearly balance, in euros (numeric)</p><p>7. Housing: has a housing loan? (Binary: Yes, No)</p><p>8. Loan: Has a personal loan? (Binary: Yes, No )</p><p><strong>Related to the last contact of the current campaign:</strong></p><p>9. Contact: contact communication type (categorical: unknown, telephone cellular)</p><p>10. Day: last contact day of the month (numeric)</p><p>11. Month: last contact month of the year (categorical: Jan, Feb, Mar, …, Nov, Dec)</p><p>12. Duration: last contact duration, in seconds (numeric)</p><p><strong>Other attributes of the current campaign:</strong></p><p>13. Campaign: number of contacts performed during this campaign and for this client (numeric, includes the last contact)</p><p>14. Pdays: number of days that passed by after the client was last contacted from a previous campaign (numeric, -1 means the client was not previously contacted)</p><p>15. Previous: number of contacts performed before this campaign and for this client (numeric)</p><p>16. Poutcome: outcome of the previous marketing campaign (categorical: unknown, other, failure, success)</p><p><strong>Output variable (desired target):</strong></p><p>17. y: has the client subscribed to a term deposit? (binary: yes, no)</p><h3>Importing Python Libariers Package:</h3><p>I installed libraries to perform different operations on data. Let me explain these libraries a bit:</p><p>Pandas is a Python package that provides fast, flexible, and expressive data structures designed to make working with “relational” or “labelled” data both easy and intuitive.</p><p>Numpy: NumPy is a library for the Python programming language, adding support for large, multi-dimensional arrays and matrices, along with a large collection of high-level mathematical functions to operate on these arrays.</p><p>Seaborn: Seaborn is a Python data visualization library based on matplotlib. It provides a high-level interface for drawing attractive and informative statistical graphics.</p><p>Matplotlib: Matplotlib is a Python 2D plotting library that produces publication-quality figures in a variety of hardcopy formats and interactive environments across platforms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3otM4hZ4Xg0rEDheMKDMgA.png" /><figcaption>Image 1. Screenshot of the python libraries imported. Image by Author</figcaption></figure><h3>Importing Data into Data Frames:</h3><p>To start working with data, first we need to import data from CSV files into a Pandas DataFrame. This will be done using Pandas’ read_csv method. We will further learn how different delimiters are used by this function.</p><p>We need to make sense of our data in what it looks like. We will explore some more pandas’ functions here like</p><p>● See data in tabular form by using. head() to view the top five samples and. tail() to view the last 5 samples.</p><figure><img alt="To view the top dataset and the last dataset sample" src="https://cdn-images-1.medium.com/max/1024/1*nb7hSzQS2LguIMXdUtFMcg.png" /><figcaption>Image 2. Screenshot of the top 5 samples and last 5 samples. Image by author</figcaption></figure><p>● Overall summary of DataFrame using .info( )</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qIcFXOWASdzAh1aXl92pXA.png" /><figcaption>Image 3. Screenshot of statistical information of the data. source by me</figcaption></figure><p>The information helps us know the type of dataset and general information. In this bank data, there are 7 numerical variables and 9 categorical variables.</p><p><strong>Data Cleaning</strong></p><p>I want to find out if there are any missing (null) values in columns. Check using Pandas’ isnull. and also<strong> </strong>duplicate values.</p><p><strong>ANALYSIS</strong></p><p>Our Analysis will be in three stages;</p><p>1. Univariate</p><p>2. Bi-variate</p><p>3. Multivariate</p><h3>Univariate Analysis:</h3><p>The univariate analysis explores each variable in a data set, separately. It looks at the range of values, as well as the central tendency of the values. It describes the pattern of response to the variable. It describes each variable on its own. Examples of our Univariate Analysis are count plots, boxplots, histograms, etc.</p><p>From this dataset</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*0EC4mM7V3CMARe5BeFqx7A.png" /><figcaption>Image by Author: Visualize the histogram of the client’s Age</figcaption></figure><p>In this graph, the dashed circle shows the range of most customers’ ages, which explains that most people in the bank are in the age bracket of 35–50.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*Sw0x3TEPCLOSxZcGciIwPw.png" /><figcaption>Image by author</figcaption></figure><p>This graph shows that the most common job categories are blue-collar and management jobs. A blue-collar worker is a person from the working class who performs manual labour. Examples are farmers, mechanics, power plant operators, etc. They are based on hourly wages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/713/1*SS39NmpCy1xg8fpCOJBBuw.png" /><figcaption>Image by author</figcaption></figure><p>It explains that most customers in the marketing campaign are secondary holders.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*ohLdVN4K0iXaQqktFmcMbA.png" /><figcaption>Image by author</figcaption></figure><p>Married customers are involved in the marketing campaign.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/1*U9uCvShURM3VVzKRbwxEdQ.png" /><figcaption>Image by author</figcaption></figure><p>This graph illustrates that only 11.70 per cent of the customers subscribed to the deposit term offered by the marketing campaign.</p><p>In bivariate analysis, we will know the reason why and what the marketing campaign can do to promote the rate of subscription.</p><p><strong>Bi-variate Analysis:</strong></p><p>According to the definition, the bivariate analysis examines any concurrent relationship between two variables or qualities. This study explores the relationship between two variables as well as the depth of this relationship to figure out if there are any discrepancies between the two variables and any causes of these differences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*F8Z5_0OVvVbl4Ue7cxslmQ.png" /><figcaption>Image by author</figcaption></figure><p>This graph shows the relationship between deposit (the target variable) and the job category.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*LdLCWCOgGpSXumouVD3P5Q.png" /><figcaption>Image by author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*CJueVyANVa4GXBTHmwm4ig.png" /><figcaption>Image by author</figcaption></figure><p>This graph shows that clients with loans, either personal or housing, tend not to show interest in deposit subscriptions. Both charts show that customers require a housing loan mostly, and their loan is small.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*1XgXzs8KEBj3UWRjUD0JFw.png" /><figcaption>Image by author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*vtPflDjTXaTmRCelGX3AhA.png" /><figcaption>Image by author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*w5P_UnDN_macbcbLrtV6BA.png" /><figcaption>Image by author</figcaption></figure><p>It shows that no matter the age or balance in euros of the client, it has the same patterns concerning the deposit subscription.</p><h3>Multivariate Analysis</h3><p>Multivariate analysis is based on the observation and analysis of more than one statistical outcome variable at a time. In design and analysis, the technique is used to perform trade studies across multiple dimensions while taking into account the effects of all variables on the responses of interest.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*3FmjBRnDJMleIeyk54LyCg.png" /><figcaption>Image by author</figcaption></figure><h3>Correlation</h3><p>Correlation is a statistical term describing the degree to which two variables move in coordination with one another.</p><p>There are two types of correlation</p><p>1. Spearman correlation</p><p>2. Pearson correlation</p><p>Spearman’s rank Correlation: Spearman’s rank correlation measures the strength and direction of association between two ranked variables. It gives the measure of the monotonicity of the relation between two variables.</p><p>Pearson Correlation: Pearson Correlation measures the strength of the linear relationship between two variables.</p><p>In this case, I will be using spearman’s rank correlation</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*_1K7wwvU4IVN-dujcnGY-g.png" /><figcaption>Image by author</figcaption></figure><p><strong>Duration, previous, balance</strong>, and <strong>pdays </strong>are positively correlated to deposit, while <strong>credit default, personal loan</strong>, and <strong>housing loan</strong> are negatively correlated to deposit.</p><h3>DASHBOARD</h3><p>The main use of a dashboard is <strong>to show a comprehensive overview of data from different sources</strong>. Dashboards are useful for monitoring, measuring, and analyzing relevant data in key areas. I will be using two Python libraries. Dash and plotly</p><p><strong>DASH</strong></p><p>Dash is <strong>an open-source Python framework used for building analytical web applications</strong>. It is a powerful library that simplifies the development of data-driven applications. It’s especially useful for Python data scientists who aren’t very familiar with web development.</p><p><strong>PLOTLY</strong></p><p>The Plotly Python library is an interactive, open-source plotting library that supports over 40 unique chart types covering a wide range of statistical, financial, geographic, scientific, and 3-dimensional use cases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*_5tCrpANYxRpTFsnCSJIPw.png" /><figcaption>Image by author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*mM1sPTvc5sItbIu9v5AuPg.png" /><figcaption>Image by author</figcaption></figure><figure><img alt="Dashboard for the bank" src="https://cdn-images-1.medium.com/max/975/1*x93prby49cig-bIY50f2rQ.png" /><figcaption>Image by author</figcaption></figure><h3>INSIGHT and CONCLUSION.</h3><p>From the various chart above, the following can be derived;</p><p>1. Most of the clients that are involved in this marketing campaign are in the age range of 35–50.</p><p>2. Most of the client&#39;s job categories are blue-collar, managerial, and technician.</p><p>3. Since it is mostly blue-collar and technician job categories, this is the reason why their educational level is mostly secondary. A blue-collar worker is a manual worker, like a farmer. Plant operator etc</p><p>4. Clients or customers with loans, either housing or personal, tend not to subscribe to this deposit due to paying for their existing loan.</p><p>5. Both married and single people subscribe to the deposit terms.</p><p>6. Only 11.70 % of the bank customers subscribed to the deposit terms.</p><p>7. With or without a deposit term, the age follows the same pattern.</p><p>8. The client who subscribes to the deposit term is more likely to be married. This can be a result of the responsibility they have at hand.</p><p>9. Clients with low balances tend to subscribe more than those with high –balances.</p><p>The rate of deposit-term subscriptions decreases as people get older.</p><h3>RECOMMENDATION</h3><p>The company should focus more on youths aged 35–50.</p><p>The company should focus more on clients with high-income balances.</p><p>The company should shift its focus to the client without any form of a loan.</p><p>You can view my project along with the jupyter notebook on my <a href="https://github.com/Gracebaytech"><strong>GitHub</strong></a></p><p>You can also connect with me on <a href="https://www.linkedin.com/in/adeleke-adedeji-09aba0195"><strong>LinkedIn</strong></a><strong> </strong>and Twitter</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f2ac996c574b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>