<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Auwal Auwal Sulaiman on Medium]]></title>
        <description><![CDATA[Stories by Auwal Auwal Sulaiman on Medium]]></description>
        <link>https://medium.com/@Khalboy10?source=rss-53882316f7f2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Mq449uYEwlJ4ea55OXOaDQ.jpeg</url>
            <title>Stories by Auwal Auwal Sulaiman on Medium</title>
            <link>https://medium.com/@Khalboy10?source=rss-53882316f7f2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 12:51:46 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@Khalboy10/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Basic Introduction to Reinforcement Learning]]></title>
            <link>https://medium.com/@Khalboy10/basic-introduction-to-reinforcement-learning-c37c12924dd7?source=rss-53882316f7f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/c37c12924dd7</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <dc:creator><![CDATA[Auwal Auwal Sulaiman]]></dc:creator>
            <pubDate>Fri, 21 Jul 2023 17:46:12 GMT</pubDate>
            <atom:updated>2023-07-21T17:46:12.962Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/468/1*5rKQLlJ0ruK78PNryXh_ZA.png" /></figure><p>Reinforcement learning is a type of machine learning where an algorithm learns to make decisions by interacting with an environment. The goal is to maximize a reward signal that the algorithm receives for its actions.</p><h4>Fundamental Concept In Reinforcement Learning:</h4><ol><li>Agent: Think of the agent as a learner or decision-maker. It’s like a little creature or robot trying to figure out how to perform well in a specific environment.</li><li>Environment: The environment is like a playground or a simulation where the agent operates. It provides the rules, challenges, and feedback to the agent.</li><li>Actions: The agent can take different actions in the environment. These actions might include moving in a certain direction, making a choice, or performing specific tasks.</li><li>Rewards: Whenever the agent takes an action, the environment gives it feedback in the form of a reward or a score. The reward can be positive, negative, or zero, depending on how well the agent’s action aligns with the desired goal.</li><li>Learning Process: The agent’s objective is to learn which actions lead to the highest rewards over time. It does this through trial and error. By exploring different actions and observing the rewards, the agent gradually learns to make better decisions.</li><li>Policy: The agent develops a strategy called a “policy” that guides its actions. The policy is like a set of rules or a plan that the agent follows to maximize its rewards.</li><li>Optimization: The agent continuously improves its policy based on the feedback received from the environment. It adjusts its decision-making process to achieve better outcomes.</li><li>Goal: The ultimate goal of reinforcement learning is for the agent to find the best policy that maximizes its rewards, allowing it to perform well and accomplish the desired tasks in the given environment.</li></ol><p>9. State: In the interaction between the agent and the environment, a “state” represents the current situation or condition of the environment. It contains all the relevant information that the agent needs to make decisions.</p><p>10. Reward Function: The reward function is a crucial component of reinforcement learning. It defines the immediate feedback the agent receives after taking an action in a specific state. The reward signal is what the agent uses to learn and improve its policy.</p><h4>Q Learning</h4><p>Q-learning is a popular and fundamental algorithm in the field of reinforcement learning. It is a model-free method used to learn an optimal action-selection strategy (policy) for an agent interacting with an environment to maximize cumulative rewards. The algorithm learns to estimate the value of taking a specific action in a particular state, known as the action-value function (Q-function).</p><p><strong>Steps Of Q-learning:</strong></p><ol><li>Setup: Q-learning involves an agent and an environment. The environment provides states (s) and possible actions (a) the agent can take. The agent explores the environment and interacts with it, receiving immediate rewards (r) for each action it takes.</li><li>Q-Table: The agent maintains a data structure called the Q-table. It is a table that stores the estimated Q-values for all state-action pairs. Initially, these values are usually set to arbitrary values or initialized randomly.</li><li>Action Selection: At each time step, the agent observes the current state of the environment and selects an action based on an exploration-exploitation trade-off. It can either choose an action randomly (exploration) to explore new possibilities or pick the action with the highest Q-value for the current state (exploitation) to follow its current knowledge.</li></ol><h4>Conclusion</h4><p>Q-learning is a powerful and straightforward algorithm for reinforcement learning, especially in discrete state and action spaces. However, it has limitations when applied to environments with continuous state and action spaces, as the Q-table becomes infeasible to store and update. To handle more complex scenarios, deep reinforcement learning algorithms, such as Deep Q Networks (DQNs), have been developed, combining Q-learning with deep neural networks to approximate the action-value function.</p><p>Visit <a href="https://github.com/arewadataScience/ML-4-Beginners-Arewa-Datascience/tree/main/8-Reinforcement/1-QLearning">https://github.com/arewadataScience/ML-4-Beginners-Arewa-Datascience/tree/main/8-Reinforcement/1-QLearning</a> to see a hands-on implementation of Q Learning algorithm.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c37c12924dd7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Time Series Forecasting : Introduction To ARIMA model]]></title>
            <link>https://medium.com/@Khalboy10/time-series-forecasting-introduction-to-arima-model-a01b840023b9?source=rss-53882316f7f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/a01b840023b9</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[time-series-forecasting]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Auwal Auwal Sulaiman]]></dc:creator>
            <pubDate>Thu, 20 Jul 2023 20:45:59 GMT</pubDate>
            <atom:updated>2023-07-20T20:45:59.628Z</atom:updated>
            <content:encoded><![CDATA[<h3>Time Series Forecasting : Introduction To ARIMA model</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/468/1*MIBRErTDqtW3DShMnTneNg.png" /></figure><p>Time series forecasting is a method of predicting future values based on historical data that is recorded over time at regular intervals (such as daily, monthly, or yearly). The goal is to uncover patterns and trends in the data, so we can make predictions about what might happen in the future.</p><blockquote>Note: The key idea is that events in the past can give us valuable insights into what might happen in the future.</blockquote><h4>Steps Of Time Series Forecasting:</h4><ol><li>Collecting Data: The first step is to collect data over a period of time. This could be anything that changes over time, like temperature, sales numbers, or website traffic.</li><li>Visualizing the Data: Once we have the data, we can plot it on a graph. By looking at the graph, we might see patterns like seasonality (repeating patterns) or trends (upward or downward movements).</li><li>Data Preprocessing: Time series data may have missing values or irregularities. We need to clean the data and make it consistent for accurate forecasting.</li><li>Choosing a Model: There are various mathematical models to analyze and forecast time series data. One popular model is the ARIMA (AutoRegressive Integrated Moving Average) model, which takes into account past values, differences between observations, and moving averages.</li><li>Training the Model: We split our data into a training set (past data) and a testing set (future data). The model is then trained on the training set to learn from past patterns and relationships.</li><li>Making Predictions: After training, the model is ready to make predictions. It uses the patterns and relationships it learned during training to forecast future values.</li><li>Evaluating the Model: To know how well the model is performing, we compare its predictions to the actual values in the testing set. Metrics like MAPE (Mean Absolute Percentage Error) or RMSE (Root Mean Squared Error) help us measure the accuracy of the forecast.</li><li>Adjusting the Model: If the model’s accuracy is not satisfactory, we can adjust the model parameters or try different models until we achieve a more accurate forecast.</li></ol><p><strong>Some key concepts and techniques in time series forecasting:</strong></p><p>1. Trend: The trend represents the long-term movement or direction of the data over time. It shows whether the values are generally increasing, decreasing, or staying relatively constant.</p><p>2. Seasonality: Seasonality refers to repeating patterns in the data that occur at regular intervals. For example, sales of air conditioners might show seasonality with higher values during summer and lower values during winter.</p><p>3. Stationarity: A time series is considered stationary if its statistical properties, such as mean, variance, and autocorrelation, remain constant over time. Stationarity is essential for many time series models to work effectively.</p><p>4. Autocorrelation: Autocorrelation measures the relationship between an observation and its lagged (previous) values. A high autocorrelation indicates that the current value is influenced by past values, which is a common characteristic in time series data.</p><p>5. Moving Average (MA): The moving average is a technique used to smooth out fluctuations in the data and identify trends. It involves calculating the average of a fixed number of consecutive data points.</p><p>6. Autoregressive (AR): The autoregressive model uses past values of the time series to predict future values. It assumes that the current value is a linear combination of its past values and a random error term.</p><p>7. Model Evaluation: To evaluate the accuracy of a time series forecasting model, we use metrics like Mean Absolute Percentage Error (MAPE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and others. These metrics help us understand how well the model is performing and compare different models.</p><h3>ARIMA Model</h3><p>ARIMA stands for AutoRegressive Integrated Moving Average. It is a popular and powerful time series forecasting model that can help us predict future values based on past data.</p><h4>What ARIMA Stands For:</h4><p><strong>AutoRegressive (AR):</strong></p><ul><li>“Auto” means “self” or “own.” In this context, it refers to the relationship between a data point and its previous data points.</li><li>The AR part of ARIMA uses past values of the time series to predict future values.</li><li>For example, if we have a daily temperature dataset, the AR part looks at how yesterday’s temperature influences today’s temperature.</li></ul><p><strong>Integrated (I):</strong></p><ul><li>The “Integrated” part is about making the data stationary, which means making the statistical properties constant over time.</li><li>Non-stationary data can have trends or seasonality, which can make forecasting difficult. So, the I part transforms the data to remove these patterns.</li><li>This transformation is often done by taking the difference between consecutive data points.</li></ul><p><strong>Moving Average (MA):</strong></p><ul><li>The MA part focuses on the relationship between the data points and the past forecast errors (the difference between the predicted value and the actual value).</li><li>It helps to capture any short-term changes or fluctuations in the data.</li><li>For example, if we have a daily stock price dataset, the MA part looks at how past forecast errors affect today’s stock price.</li></ul><p>The ARIMA model combines these three components to create a forecasting model that considers the data’s past values, differences between observations, and past forecast errors to make predictions about future values.</p><pre># importing our libraries<br><br>import pandas as pd<br>import numpy as np<br>import matplotlib.pyplot as plt<br>from statsmodels.tsa.arima.model import ARIMA<br>from sklearn.metrics import mean_absolute_percentage_error<br><br># loading our dataset<br><br>currency = pd.read_csv(&#39;data/currency.csv&#39;)<br>currency.head()<br><br># changing our time column to datetime format<br><br>currency[&#39;Time&#39;] = pd.to_datetime(currency[&#39;Time&#39;])<br>currency.set_index(&#39;Time&#39;, inplace=True)<br><br># ploting our data<br><br>plt.figure(figsize=(12, 6))<br>plt.plot(currency[&#39;GEMS_GEMS_SPENT&#39;])<br>plt.xlabel(&#39;Date&#39;)<br>plt.ylabel(&#39;GEMS_GEMS_SPENT&#39;)<br><br>plt.show()<br><br># Split the data into train and test sets<br><br>train_data = currency[:&#39;2017-12-24&#39;]<br>test_data = currency[&#39;2018-02-24&#39;:]<br><br># Fit the ARIMA model<br><br>model = ARIMA(train_data, order=(5, 1, 0))<br>model_fit = model.fit()<br><br># Make predictions<br><br>predictions = model_fit.predict(start=&#39;2018-02-24&#39;, end=&#39;2020-12-01&#39;, dynamic=False)<br><br># Plot the predicted values along with the original data<br><br>plt.figure(figsize=(10, 6))<br>plt.plot(train_data, label=&#39;Training Data&#39;)<br>plt.plot(test_data, label=&#39;Test Data&#39;)<br>plt.plot(predictions, label=&#39;Predictions&#39;)<br>plt.xlabel(&#39;Time&#39;)<br>plt.ylabel(&#39;GEMS_GEMS_SPENT&#39;)<br>plt.legend()<br><br>plt.show()<br><br># calculate the MAPE<br><br>actual_values = test_data.to_numpy().flatten()<br>predicted_values = predictions.to_numpy()<br>mape = np.mean(np.abs((predicted_values - actual_values) / actual_values)) * 100<br><br>print(&#39;Mean Absolute Percentage Error (MAPE):&#39;, mape)</pre><h4>Conclusion</h4><p>Time series forecasting is used in various fields, such as finance, economics, meteorology, and sales forecasting. It helps organizations and individuals make better decisions by providing insights into future trends and patterns.</p><p>ARIMA is a flexible and widely used model for time series forecasting, and it can handle various types of time series data, including those with trends, seasonality, and irregular patterns. By understanding the patterns and relationships in the data, ARIMA helps us make more informed decisions and predictions about the future.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a01b840023b9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction To Sentiment Analysis]]></title>
            <link>https://medium.com/@Khalboy10/introduction-to-sentiment-analysis-1bf4df00a33c?source=rss-53882316f7f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/1bf4df00a33c</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[sentiment-analysis]]></category>
            <dc:creator><![CDATA[Auwal Auwal Sulaiman]]></dc:creator>
            <pubDate>Mon, 03 Jul 2023 04:56:57 GMT</pubDate>
            <atom:updated>2023-07-03T04:56:57.265Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/468/1*AE-JSrYXau0SyMUsvvUBGA.png" /></figure><h4>What is Sentiment Analysis?</h4><p>Sentiment analysis, also known as opinion mining, is a technique used to determine the overall sentiment or emotional tone expressed in a piece of text. It involves analyzing words, phrases, and context to understand whether the sentiment of the text is positive, negative, or neutral.</p><blockquote>sentiment analysis helps computers understand whether a piece of text, such as a sentence, a review, or a social media post, conveys a positive, negative, or neutral feeling.</blockquote><h4>How Sentiment Analysis Works:</h4><ol><li>Text Input: The sentiment analysis process begins with a piece of text as input. This could be a single sentence, a paragraph, or even an entire document.</li><li>Text Processing: The text is then processed to prepare it for sentiment analysis. This may involve removing irrelevant words (like common articles or prepositions) and punctuation, as well as splitting the text into individual words or tokens.</li><li>Sentiment Classification: The processed text is analyzed to determine the sentiment of each word or phrase. This is typically done by comparing the words in the text with a pre-defined sentiment lexicon or a set of pre-labeled training data.</li><li>Aggregation: Once the sentiment of individual words or phrases is determined, the overall sentiment of the text is calculated. This can be done by taking an average of the individual sentiment scores or applying more sophisticated algorithms that consider the context and relationships between words.</li><li>Sentiment Labeling: Finally, the sentiment analysis system assigns a sentiment label to the text based on the calculated sentiment score. This label could be positive, negative, or neutral, indicating the overall emotional tone of the text.</li></ol><h4>Applications Of Sentiment Analysis:</h4><ul><li><strong>Customer feedback analysis:</strong> Analyzing customer reviews or feedback to understand the sentiment towards a product, service, or brand.</li><li><strong>Social media monitoring:</strong> Tracking sentiment on social media platforms to gauge public opinion about a topic or event.</li><li><strong>Market research:</strong> Assessing sentiment in online discussions or surveys to understand consumer preferences and trends.</li><li><strong>Reputation management:</strong> Monitoring online sentiment to manage the reputation of individuals, organizations, or brands.</li><li><strong>Voice of the customer analysis:</strong> Analyzing sentiment in customer support interactions to identify customer satisfaction or dissatisfaction.</li></ul><h4>Conclusion</h4><p>Sentiment analysis helps automate the process of understanding and categorizing emotions in text, providing valuable insights for businesses and organizations to make informed decisions, improve customer experiences, and gain a better understanding of public sentiment.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1bf4df00a33c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Intoduction To NLP]]></title>
            <link>https://medium.com/@Khalboy10/intoduction-to-nlp-98a6cc7507b3?source=rss-53882316f7f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/98a6cc7507b3</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[textblob]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Auwal Auwal Sulaiman]]></dc:creator>
            <pubDate>Mon, 03 Jul 2023 04:41:46 GMT</pubDate>
            <atom:updated>2023-07-03T04:41:46.284Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/468/1*BhPoUWKSckUpJ1UFSq-JRA.png" /></figure><h4>What is NLP?</h4><p>Natural Language Processing (NLP) is a field of study that focuses on how computers can understand and work with human language. It involves teaching computers to process, interpret, and generate human language in a way that is similar to how people do.</p><p>NLP helps computers understand and communicate with humans using words and sentences. It enables machines to read, listen, and comprehend text or speech, allowing them to perform tasks like understanding the meaning of a sentence, answering questions, translating languages, summarizing text, or even generating human-like responses.</p><p>NLP involves various techniques and algorithms that enable computers to break down language into smaller parts, such as words or phrases, and analyze their meanings and relationships. It uses pattern recognition, machine learning, and linguistic rules to understand the structure, context, and sentiment of language.</p><p>For example, when you use voice assistants like Siri or Alexa, they utilize NLP to understand your spoken commands or questions and respond accordingly. NLP algorithms process the audio input, convert it into text, analyze the meaning of the words, and generate appropriate responses.</p><blockquote>NLP makes it possible for computers to understand, interpret, and generate human language, making interactions with technology more natural and enabling a wide range of applications that involve language processing and communication.</blockquote><h4>Applications Of NLP</h4><ol><li>Text Understanding: NLP helps computers understand and interpret written text. This includes tasks such as part-of-speech tagging (identifying the role of each word in a sentence), named entity recognition (identifying names of people, places, organizations, etc.), and syntactic parsing (analyzing the grammatical structure of sentences).</li><li>Sentiment Analysis: NLP allows computers to analyze the sentiment or emotion expressed in a piece of text. Sentiment analysis can determine whether a statement is positive, negative, or neutral, enabling applications like social media sentiment analysis, customer feedback analysis, and brand monitoring.</li><li>Machine Translation: NLP facilitates the automatic translation of text from one language to another. Machine translation systems employ NLP techniques to understand the meaning of sentences in the source language and generate equivalent sentences in the target language. Popular translation services like Google Translate rely on NLP algorithms.</li><li>Question Answering: NLP enables computers to understand questions asked in natural language and provide relevant answers. Question answering systems analyze the question’s structure, identify keywords, and retrieve or generate appropriate responses. This application is widely used in virtual assistants, search engines, and customer support chatbots.</li><li>Text Summarization: NLP algorithms can condense lengthy documents or articles into shorter summaries that capture the main points. Text summarization can be extractive (selecting important sentences) or abstractive (generating new sentences). It helps users quickly grasp the essence of large volumes of information.</li><li>Chatbots and Virtual Assistants: NLP plays a vital role in developing conversational agents, such as chatbots and virtual assistants. These systems utilize NLP to understand user queries or commands, generate appropriate responses, and engage in human-like conversations. NLP helps make interactions more intuitive and natural.</li><li>Information Extraction: NLP techniques can extract structured information from unstructured text. For example, named entity recognition and relation extraction algorithms can identify specific entities (such as people, organizations, or dates) and their relationships from a document, enabling tasks like data mining, knowledge graph construction, and information retrieval.</li><li>Speech Recognition: NLP is involved in converting spoken language into written text. Speech recognition systems use techniques like acoustic modeling and language modeling to transcribe spoken words into text, enabling applications like voice assistants, voice commands, and transcription services.</li></ol><h4>What is Textblob?</h4><p>TextBlob is a popular Python library that provides a simple and intuitive API for common natural language processing (NLP) tasks. It is built on top of other NLP libraries, such as NLTK (Natural Language Toolkit), and offers a range of functionalities for tasks like sentiment analysis, part-of-speech tagging, noun phrase extraction, and language translation.</p><p>TextBlob offers a user-friendly and high-level API, making it accessible for beginners in NLP. It simplifies many common NLP tasks, allowing developers to quickly prototype and implement language processing functionality in their applications.</p><h4>How to Create simple Chatbot Using Textblob</h4><pre>from textblob import TextBlob<br>print(&quot;Hello, I am Khalboy, the simple robot.&quot;)<br>print(&quot;You can end this conversation at any time by typing &#39;quit&#39;.&quot;)<br><br>while True:<br>    user_input = input(&quot;&gt; &quot;)<br>    user_input_blob = TextBlob(user_input)<br>    if user_input.lower() == &#39;quit&#39;:<br>        print(&quot;Goodbye! Have a great day!&quot;)<br>        break<br>    elif user_input_blob.polarity &lt;= -0.5:<br>        response = &quot;Oh dear, that sounds bad. &quot;<br>        print(response)<br><br>    elif user_input_blob.polarity &lt;= 0:<br>        response = &quot;Hmm, that&#39;s not great. Can you tell more&quot;<br>        print(response)<br><br>    elif user_input_blob.polarity &lt;= 0.5:<br>        response = &quot;Well, that sounds positive. &quot;<br>        print(response)<br><br>    elif user_input_blob.polarity &lt;= 1:<br>        response = &quot;Wow, that sounds great. &quot;<br>        print(response)</pre><h4>Conclusion</h4><p>These are just a few examples of the wide-ranging applications of NLP. It empowers computers to process, understand, and generate human language, enabling applications that involve text understanding, sentiment analysis, translation, question answering, summarization, chatbots, and much more.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=98a6cc7507b3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Basic Introduction To Clustering]]></title>
            <link>https://medium.com/@Khalboy10/basic-introduction-to-clustering-f787a0068658?source=rss-53882316f7f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/f787a0068658</guid>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Auwal Auwal Sulaiman]]></dc:creator>
            <pubDate>Sun, 11 Jun 2023 17:22:27 GMT</pubDate>
            <atom:updated>2023-06-11T17:22:27.719Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*f4tWXIjdQcK0PHUG" /><figcaption>Photo by <a href="https://unsplash.com/@deepmind?utm_source=medium&amp;utm_medium=referral">Google DeepMind</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Introduction</h4><p>Clustering is an unsupervised machine learning technique used in data analysis to group similar objects or data points together. It’s like organizing items based on their similarities and putting them into separate groups or categories.</p><p>The goal of clustering is to find natural groupings or patterns in the data without having predefined categories or labels. By identifying these groups, we can gain insights into the structure and relationships within the data. Clustering can help us discover hidden patterns, detect anomalies or outliers, segment customer groups, or even analyze genetic data to identify subgroups of patients.</p><p>Imagine you have a collection of different fruits, such as apples, oranges, and bananas. If you wanted to group these fruits based on their similarities, you might consider factors like color, size, and shape. Clustering algorithms work in a similar way, but they use mathematical calculations to determine similarities between data points based on their features or attributes.</p><blockquote>Note: Clustering is an exploratory technique, meaning it helps us understand and uncover patterns in the data but does not provide explicit answers or predictions. The quality of the clustering results depends on the algorithm used, the choice of features, and the interpretation of the clusters by domain experts.</blockquote><h4>Clustering Algorithms</h4><p>Clustering algorithms use various mathematical techniques and distance measures to calculate similarities between data points. These algorithms aim to maximize the similarity within each group (intra-cluster similarity) while minimizing the similarity between different groups (inter-cluster dissimilarity). Below are explanations of some clustering algorithms:</p><p><strong>K-Means Clustering:</strong></p><p>K-Means is one of the most widely used clustering algorithms. It aims to partition the data into K clusters, where K is a pre-defined number chosen by the user. The algorithm starts by randomly selecting K centroids as the initial cluster centers. Then, it iteratively assigns each data point to the nearest centroid and updates the centroid based on the mean of the points assigned to it. This process continues until the centroids no longer change significantly or a maximum number of iterations is reached.</p><p><strong>Agglomerative Clustering:</strong></p><p>Agglomerative Clustering is a hierarchical clustering algorithm that starts with each data point as a separate cluster and merges them iteratively based on their similarity. At each step, the two closest clusters are combined until all data points belong to a single cluster. The similarity between clusters is typically measured using distance-based metrics such as Euclidean distance or linkage criteria like Ward’s linkage or average linkage.</p><p><strong>DBSCAN (Density-Based Spatial Clustering of Applications with Noise):</strong></p><p>DBSCAN is a density-based clustering algorithm that groups together data points that are close to each other in densely populated regions and identifies outliers as noise points. It works by defining a neighborhood around each data point based on a distance threshold and a minimum number of points required to form a dense region. It then expands clusters by connecting neighboring points based on the defined criteria.</p><p><strong>Gaussian Mixture Models (GMM):</strong></p><p>GMM is a probabilistic clustering algorithm that assumes the data points are generated from a mixture of Gaussian distributions. It models each cluster as a Gaussian distribution and estimates the parameters (mean and covariance) using an expectation-maximization algorithm. GMM assigns data points to the clusters based on the probability of belonging to each distribution.</p><p><strong>Spectral Clustering:</strong></p><p>Spectral Clustering combines elements of graph theory and matrix algebra to perform clustering. It views the data points as nodes in a graph and measures the similarity between points using the graph Laplacian matrix. Spectral Clustering then uses eigenvalue decomposition or k-means clustering on the eigenvectors to assign points to clusters. It can handle complex geometric structures and is particularly effective in clustering data with nonlinear boundaries.</p><blockquote>Note: These are just a few examples of clustering algorithms, and there are many more available depending on the specific requirements and characteristics of the data. Each algorithm has its own strengths, weaknesses, and assumptions, and the choice of algorithm should be based on the nature of the data and the goals of the analysis.</blockquote><h4>Example On How To Do Clustering:</h4><p>In this our example we are using the Agglomerative algorithm to cluster our dataset.</p><pre># we import pandas library, and we load and explore our dataset<br><br>import pandas as pd<br><br>df = pd.read_csv(&#39;Nigerian_songs.csv&#39;)<br>df.head()<br></pre><pre># we preprocess/clean the data<br><br>df = df[df[&#39;artist_top_genre&#39;] != &#39;Missing&#39;]<br>df = df[(df[&#39;artist_top_genre&#39;] == &#39;afro dancehall&#39;) | (df[&#39;artist_top_genre&#39;] == &#39;afropop&#39;) | (df[&#39;artist_top_genre&#39;] == &#39;nigerian pop&#39;)]<br>df = df[(df[&#39;popularity&#39;] &gt; 0)]<br><br>from sklearn.preprocessing import LabelEncoder<br>le = LabelEncoder()<br><br>X = df.loc[:, (&#39;artist_top_genre&#39;,&#39;popularity&#39;,&#39;danceability&#39;,&#39;acousticness&#39;,&#39;loudness&#39;,&#39;energy&#39;)]<br>X[&#39;artist_top_genre&#39;] = le.fit_transform(X[&#39;artist_top_genre&#39;])</pre><pre># We choose and apply our clustering method(Agglomerative Clustering)<br><br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>from sklearn.cluster import AgglomerativeClustering<br><br>clustering = AgglomerativeClustering(n_clusters=3)<br>cluster_labels = clustering.fit_predict(X)<br>cluster_labels</pre><pre># we visualize the clusters<br><br>plt.figure(figsize=(10, 6))<br>sns.scatterplot(data=df, x=&#39;popularity&#39;, y=&#39;danceability&#39;, hue=cluster_labels, palette=&#39;Set1&#39;)<br>plt.xlabel(&#39;popularity&#39;)<br>plt.ylabel(&#39;danceability&#39;)<br>plt.title(&#39;Agglomerative Clustering - popularity vs danceability&#39;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*_kBSVXO1fjKWCg9wr9pV5w.png" /></figure><pre># we calculate the Silhouette Score<br><br>from sklearn.metrics import silhouette_score<br><br>score = silhouette_score(X, cluster_labels)<br>print(&quot;Silhouette Score:&quot;, score)<br><br>Output:<br>Silhouette Score: 0.5435156033436574</pre><h4>Conclusion</h4><p>Clustering is an unsupervised machine learning technique used in data analysis to group similar objects or data points together. The goal of clustering is to find natural groupings or patterns in the data without having predefined categories or labels</p><p>In customer segmentation, a company might have a large customer dataset and wants to identify distinct groups of customers based on their purchasing behavior. By clustering the data, the company can discover different customer segments, such as frequent buyers, occasional buyers, or high-value customers. This information can then be used to tailor marketing strategies or personalize services for each customer segment.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f787a0068658" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing Classification in ML]]></title>
            <link>https://medium.com/@Khalboy10/introducing-classification-in-ml-78baab6e4c8a?source=rss-53882316f7f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/78baab6e4c8a</guid>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[scikit-learn]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[classification]]></category>
            <dc:creator><![CDATA[Auwal Auwal Sulaiman]]></dc:creator>
            <pubDate>Sun, 28 May 2023 15:06:38 GMT</pubDate>
            <atom:updated>2023-05-28T15:06:38.955Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sigE7rC3IjgqOkrd" /><figcaption>Photo by <a href="https://unsplash.com/@deepmind?utm_source=medium&amp;utm_medium=referral">Google DeepMind</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Intoduction</h4><p>Classification is a fundamental task in machine learning that involves categorizing or labeling data into predefined classes or categories based on certain features or characteristics. The goal of classification is to develop a model that can learn patterns and relationships within the data and then use that knowledge to accurately predict the class or category of future instances.</p><p>In classification, the data is typically represented by a set of features or attributes, which serve as the basis for making predictions. These features can be numerical or categorical, and they provide information about the data points or instances. For example, in a classification problem of predicting whether an email is spam or not, the features could include the presence of certain keywords, the length of the email, or the sender’s address.</p><p>To create a classification model, a machine learning algorithm is trained on a labeled dataset, where each data point is associated with its correct class or category. During the training phase, the algorithm learns patterns and relationships by analyzing the feature values and their corresponding classes. The trained model captures this knowledge in the form of decision boundaries or rules that separate different classes in the feature space.</p><p>Classification algorithms can vary in complexity and approach. Some popular algorithms include logistic regression, decision trees, random forests, support vector machines (SVM), k-nearest neighbors (KNN), and neural networks. Each algorithm has its own strengths, weaknesses, and assumptions, making them suitable for different types of data and problem domains.</p><h4>Example of some classification algorithms and a match dataset:</h4><blockquote>Note:We will explain how the dataset would work with the chosen classification algorithm.</blockquote><pre>import pandas as pd<br><br>table = {<br>    &#39;Dataset&#39; : [&#39;Iris&#39;, &#39;Breast Cancer&#39;, &#39;Titanic Survival&#39;],<br>    &#39;Question&#39; : [&#39;Can we classify iris flowers by species?&#39;, &#39;Can we diagnose breast cancer?&#39;, &#39;Can we predict passenger survival?&#39;],<br>    &#39;Classification Technique&#39; : [&#39;K-Nearest Neighbors&#39;, &#39;Support Vector Machines&#39;, &#39;Logistic Regression&#39;]<br>}<br><br>df = pd.DataFrame(table)<br>df<br><br><br>Output:<br><br>            Dataset                                  Question  Classification Technique<br>0              Iris  Can we classify iris flowers by species?   K-Nearest Neighbors<br>1     Breast Cancer            Can we diagnose breast cancer?   Support Vector Machines<br>2  Titanic Survival        Can we predict passenger survival?   Logistic Regression</pre><h4>Iris</h4><p>The Iris dataset is a well-known dataset in machine learning and consists of measurements of four features (sepal length, sepal width, petal length, and petal width) of three different species of iris flowers (setosa, versicolor, and virginica). The goal is to classify iris flowers into their respective species based on these measurements.</p><p>To apply the K-Nearest Neighbors (KNN) classification algorithm to the Iris dataset, we would first split the dataset into training and testing sets. The training set would be used to train the KNN model on the known measurements and corresponding species labels. The testing set would be used to evaluate the performance of the trained model on unseen data.</p><p>By applying the KNN algorithm to the Iris dataset, we can effectively classify iris flowers based on their measurements and achieve a classification model that can identify the species of an iris flower with a certain level of accuracy.</p><h4>Breast Cancer</h4><p>The Breast Cancer dataset contains measurements of various features extracted from breast mass images, along with corresponding labels indicating whether the mass is benign or malignant. The goal is to develop a classification model that can accurately diagnose breast cancer based on these features.</p><p>To apply the Support Vector Machines (SVM) classification algorithm to the Breast Cancer dataset, we would first split the dataset into training and testing sets. The training set would be used to train the SVM model on the known measurements and corresponding labels. The testing set would be used to evaluate the performance of the trained model on unseen data.</p><p>The performance of the SVM model can be evaluated using various metrics such as accuracy, precision, recall, and F1 score. These metrics provide insights into how well the model diagnoses breast cancer based on the given dataset.</p><p>By applying the SVM algorithm to the Breast Cancer dataset, we can develop a classification model that can assist in diagnosing breast cancer based on the extracted features from breast mass images. The trained SVM model would be able to predict whether a given breast mass is benign or malignant, potentially aiding in early detection and treatment decisions.</p><h4>Titanic Survival</h4><p>The Titanic Survival dataset contains information about passengers aboard the Titanic, including features such as age, sex, passenger class, and whether they survived or not. The goal is to develop a classification model that can predict whether a passenger would have survived the Titanic disaster based on these features.</p><p>To apply the Logistic Regression classification algorithm to the Titanic Survival dataset, we would first split the dataset into training and testing sets. The training set would be used to train the logistic regression model on the known features and corresponding survival labels. The testing set would be used to evaluate the performance of the trained model on unseen data.</p><p>Logistic Regression is a widely used classification algorithm that is suitable for binary classification problems. It models the probability of an instance belonging to a certain class using the logistic function. In the case of the Titanic dataset, logistic regression would estimate the probability of a passenger surviving the disaster based on the given features.</p><p>Once the logistic regression model is trained, we can use it to predict the survival outcome for new passengers by providing their corresponding features as input to the model. The model will calculate the probability of survival and assign a class label (survived or not survived) based on a threshold probability.</p><p>By applying the logistic regression algorithm to the Titanic Survival dataset, we can develop a classification model that can predict passenger survival based on features such as age, sex, and passenger class. The trained logistic regression model would be able to estimate the likelihood of survival for individual passengers, providing valuable insights into the factors that influenced survival during the Titanic disaster.</p><h4>Solvers In Logistic Regression</h4><p>solvers refer to the optimization algorithms used to find the optimal coefficients or parameters of the logistic regression model. The objective of logistic regression is to find the best-fitting parameters that minimize the error between the predicted probabilities and the actual class labels.</p><p>The choice of solver depends on factors such as the size of the dataset, the number of features, the regularization technique used (L1 or L2), and the computational resources available. It is recommended to experiment with different solvers and compare their performance in terms of convergence speed and accuracy on your specific dataset.</p><h4>List of some commonly used solvers:</h4><ol><li>Liblinear</li><li>LBFGS (Limited-memory Broyden–Fletcher–Goldfarb–Shanno)</li><li>Newton-CG</li><li>Saga</li><li>sag</li></ol><blockquote>Liblinear and Lbfgs: we will compare and contrast these two solvers</blockquote><p>The liblinear solver is designed for solving linear classification and regression problems. It supports both L1 and L2 regularization and is suitable for binary classification tasks. It solves the optimization problem by updating one coordinate (feature) at a time while holding others fixed. It performs well for small to medium-sized datasets and can handle high-dimensional feature spaces. liblinear is suitable for working on a linear classification or regression problem and want to utilize L1 or L2 regularization. It performs well for small to medium-sized datasets.</p><p>The lbfgs solver is suitable for solving optimization problems with smooth and convex objective functions. It is commonly used for logistic regression and other machine learning tasks. lbfgs works with numeric matrices or tensors. It is typically used with dense input data structures. lbfgs is a good option for working on a problem that requires optimization with a smooth and convex objective function, such as logistic regression. It provides efficient optimization for moderate-sized datasets and is well-suited for dense data structures.</p><h4>Example of classification algorithm: Code</h4><pre>import pandas as pd<br>from sklearn.linear_model import LogisticRegression<br>from sklearn.model_selection import train_test_split, cross_val_score<br>from sklearn.metrics import accuracy_score, precision_score, confusion_matrix, classification_report, precision_recall_curve<br>from sklearn.svm import SVC<br>import numpy as np<br><br>cuisines_df = pd.read_csv(&#39;cleaned_cuisines.csv&#39;)<br>cuisines_df.head()<br><br>cuisines_label_df = cuisines_df[&#39;cuisine&#39;]<br>cuisines_label_df.head()<br><br>cuisines_feature_df = cuisines_df.drop([&#39;Unnamed: 0&#39;, &#39;cuisine&#39;], axis=1)<br>cuisines_feature_df.head()<br><br>X_train, X_test, y_train, y_test = train_test_split(cuisines_feature_df, cuisines_label_df, test_size=.3)<br><br>model = LogisticRegression(multi_class=&#39;ovr&#39;, solver=&#39;lbfgs&#39;)<br>model.fit(X_train, np.ravel(y_train))<br>y_pred = model.predict(X_test)</pre><pre># checking the accuracy of the test data<br>accuracy = model.score(X_test, y_test)<br>print(f&#39;Accuracy is {accuracy}&#39;)<br><br>Output:<br>Accuracy is 0.804837364470392<br><br># printing the classification report<br>print(classification_report(y_test,y_pred))<br><br>Output:<br>            precision    recall  f1-score   support<br><br>     chinese       0.74      0.74      0.74       242<br>      indian       0.92      0.89      0.90       249<br>    japanese       0.74      0.73      0.73       219<br>      korean       0.84      0.82      0.83       248<br>        thai       0.79      0.85      0.82       241<br><br>    accuracy                           0.81      1199<br>   macro avg       0.80      0.80      0.80      1199<br>weighted avg       0.81      0.81      0.81      1199</pre><h4>Conclusion</h4><p>Classification has a wide range of applications across various fields. It is used for spam detection, sentiment analysis, credit scoring, medical diagnosis, image recognition, fraud detection, and many other tasks where identifying and categorizing data into different classes is important.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=78baab6e4c8a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Linear Regression in Machine Learning]]></title>
            <link>https://medium.com/@Khalboy10/linear-regression-in-machine-learning-61255ed130f7?source=rss-53882316f7f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/61255ed130f7</guid>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Auwal Auwal Sulaiman]]></dc:creator>
            <pubDate>Wed, 24 May 2023 12:31:23 GMT</pubDate>
            <atom:updated>2023-05-24T12:31:23.665Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/464/1*7pln464ZzJ5FHG-eWBcE5A.png" /></figure><h4>Inroduction</h4><p>Regression is a statistical analysis technique used to model the relationship between a dependent variable and one or more independent variables. It predict continuous numerical values based on input variables. The dependent variable, also known as the response variable or outcome variable, is the variable that you want to predict or explain. The independent variables, also called predictor variables or features, are the variables used to predict or explain the dependent variable.</p><p>Regression is like trying to figure out how different things are related to each other using numbers. Regression is useful because it helps us understand how different things can be connected and predict what might happen based on what we already know.</p><blockquote>The goal of regression analysis is to find the best-fit line or curve that represents the relationship between the independent variables and the dependent variable.</blockquote><p>There are various types of regression techniques, each suited to different scenarios and data types. Here are a few common regression models:</p><ol><li>Simple Linear Regression: This type of regression involves a single independent variable and a linear relationship with the dependent variable. It assumes that the relationship between the variables can be approximated by a straight line.</li><li>Multiple Linear Regression: It extends simple linear regression by incorporating multiple independent variables. It assumes a linear relationship between the dependent variable and the independent variables, but with more than one predictor.</li><li>Polynomial Regression: In polynomial regression, the relationship between the independent and dependent variables is modeled as an nth-degree polynomial. It allows for curved relationships between variables.</li><li>Logistic Regression: Unlike the previous types, logistic regression is used for predicting categorical outcomes or probabilities. It is employed when the dependent variable is binary (e.g., yes/no) or falls into a limited number of categories.</li></ol><blockquote>Note: In this blog we will focus on simple linear regression.</blockquote><h4>Installations and configurations</h4><p>The following software and libraries should be available on your computer:</p><ol><li><strong>Install Python</strong>. Ensure that <a href="https://www.python.org/downloads/">Python</a> is installed on your computer, at least python 3=.</li><li><strong>Install Visual Studio Code</strong>. Make sure you have Visual Studio Code installed on your computer.</li><li><strong>Install Scikit-learn</strong>, by following <a href="https://scikit-learn.org/stable/install.html">these instructions</a>.</li><li><strong>Other library to install are: </strong><a href="https://www.youtube.com/watch?v=Nui_gII-bxs&amp;pp=ygUkaG93IHRvIGluc3RhbGwgcGFuZGFzIGFuZCBtYXRwbG90bGli"><strong>Matplotlib, Numpy, Pandas,</strong></a><strong> and</strong></li><li><strong>Install Jupyter Notebook</strong>. You will need to <a href="https://pypi.org/project/jupyter/">install the Jupyter package</a>.</li></ol><h4>Working with a Library: Scikit-learn</h4><p>Scikit-learn is a library that provides an <a href="https://scikit-learn.org/stable/modules/classes.html#api-ref">extensive API</a> to help data scientists perform ML tasks. From their <a href="https://scikit-learn.org/stable/getting_started.html">website</a>, “Scikit-learn is an open source machine learning library that supports supervised and unsupervised learning. It also provides various tools for model fitting, data preprocessing, model selection and evaluation, and many other utilities”.</p><p>There are many types of regression methods, and which one to use depends on the question you want to answer. If you want to predict the probable weight for a person of a given age, use linear regression. If you want to determine whether a particular disease is communicable, use logistic regression. More on these variations in a bit.</p><h3>How to create simple linear regression model</h3><p>To create a regression model that plots the relationship between the waistline and the number of situps accomplished using the Linnerud dataset in Scikit-learn, you should follow these steps:</p><blockquote>Note: we are using Scikit-learn linnerud dataset in this tutorial, but you can use any dataset of your choice. our goal is to demonstrate how to create simple linear regression.</blockquote><h4>Import the necessary libraries:</h4><p>from sklearn.datasets import load_linnerud</p><p>import matplotlib.pyplot as plt</p><p>from sklearn.linear_model import LinearRegression</p><p>from sklearn.model_selection import train_test_split</p><h4>Load the Linnerud dataset:</h4><p>data = load_linnerud()</p><h4>Extract the relevant data and target variables:</h4><p>waistline = data.data[:, 0]</p><p>situps = data.target[:, 0]</p><h4>Create a scatter plot to visualize the relationship:</h4><p>plt.scatter(waistline, situps)</p><p>plt.xlabel(‘Waistline’)</p><p>plt.ylabel(‘Situps’)</p><p>plt.title(‘Relationship between Waistline and Situps’)</p><p>plt.show()</p><h4>Split the data into train and test:</h4><p>X_train, X_test, y_train, y_test = model_selection.train_test_split(waistline, situps, test_size=.33)</p><h4>Create a regression model and fit the data:</h4><p>model = LinearRegression()</p><p>model.fit(waistline, situps)</p><p>Predict the situps based on the waistline values:</p><p>situps_predicted = model.predict(waistline)</p><h4>Plot the regression line on the scatter plot:</h4><p>plt.scatter(waistline, situps)</p><p>plt.plot(waistline, situps_predicted, color=’red’, linewidth=2)</p><p>plt.xlabel(‘Waistline’)</p><p>plt.ylabel(‘Situps’)</p><p>plt.title(‘Relationship between Waistline and Situps’)</p><p>plt.show()</p><h4>Example of simple linear regression</h4><pre># importing the required libraries<br>import numpy as np<br>import matplotlib.pyplot as plt<br>from sklearn import datasets, model_selection, linear_model<br><br># loading the dataset and assigning features to X and target to y<br>X, y = datasets.load_linnerud(return_X_y=True)<br>X, y<br><br><br><br>Output:<br>Output exceeds the size limit. Open the full output data in a text editor<br>(array([[  5., 162.,  60.],<br>        [  2., 110.,  60.],<br>        [ 12., 101., 101.],<br>        [ 12., 105.,  37.],<br>        [ 13., 155.,  58.],<br>        [  4., 101.,  42.],<br>        [  8., 101.,  38.],<br>        [  6., 125.,  40.],<br>        [ 15., 200.,  40.],<br>        [ 17., 251., 250.],<br>        [ 17., 120.,  38.],<br>        [ 13., 210., 115.],<br>        [ 14., 215., 105.],<br>        [  1.,  50.,  50.],<br>        [  6.,  70.,  31.],<br>        [ 12., 210., 120.],<br>        [  4.,  60.,  25.],<br>        [ 11., 230.,  80.],<br>        [ 15., 225.,  73.],<br>        [  2., 110.,  43.]]),<br>array([[191.,  36.,  50.],<br>        [189.,  37.,  52.],<br>        [193.,  38.,  58.],<br>        [162.,  35.,  62.],<br>        [189.,  35.,  46.],<br>...<br>        [202.,  37.,  62.],<br>        [176.,  37.,  54.],<br>        [157.,  32.,  52.],<br>        [156.,  33.,  54.],<br>        [138.,  33.,  68.]]))</pre><pre># selecting the first columns of X and y<br>X, y = X[:, np.newaxis, 0], y[:, np.newaxis, 0]<br>X, y<br><br><br>Output:<br>Output exceeds the size limit. Open the full output data in a text editor<br>(array([[162.],<br>        [110.],<br>        [101.],<br>        [105.],<br>        [155.],<br>        [101.],<br>        [101.],<br>        [125.],<br>        [200.],<br>        [251.],<br>        [120.],<br>        [210.],<br>        [215.],<br>        [ 50.],<br>        [ 70.],<br>        [210.],<br>        [ 60.],<br>        [230.],<br>        [225.],<br>        [110.]]),<br>array([[36.],<br>        [37.],<br>        [38.],<br>        [35.],<br>        [35.],<br>...<br>        [37.],<br>        [37.],<br>        [32.],<br>        [33.],<br>        [33.]]))</pre><pre># splitting, training and predicting the data<br>X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, test_size=.37) <br>model = linear_model.LinearRegression()<br>model.fit(X_train, y_train)<br>prediction = model.predict(X_test)<br><br>print(prediction[ :5])<br>print(y_test[ : 5])<br><br><br>Output:<br>[[37.39169076]<br> [35.04957008]<br> [37.04613197]<br> [33.01461277]<br> [37.04613197]]<br>[[36.]<br> [36.]<br> [33.]<br> [34.]<br> [37.]]</pre><pre># checking error percentage<br>from sklearn.metrics import mean_squared_error, r2_score<br><br>mse = mean_squared_error(y_test, prediction)<br>print(f&#39;Mse: {mse}&#39;)<br><br>rmse = np.sqrt(mse)<br>print(f&#39;Rmse: {rmse}&#39;)<br><br>r2 = r2_score(y_test, prediction)<br>print(f&#39;R2: {r2}&#39;)<br><br><br>Output:<br>Mse: 3.1715806176012187<br>Rmse: 1.7808932078036623<br>R2: -0.4194486680173286</pre><pre>plt.scatter(X_test, y_test, color=&#39;red&#39;)<br>plt.plot(X_test, prediction, color=&#39;black&#39;, linewidth=3)<br><br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/543/1*AtkShV5YnHirtXow8T4CVA.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=61255ed130f7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction To Visualization]]></title>
            <link>https://medium.com/@Khalboy10/introduction-to-visualization-be75c23db3eb?source=rss-53882316f7f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/be75c23db3eb</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[python-programming]]></category>
            <dc:creator><![CDATA[Auwal Auwal Sulaiman]]></dc:creator>
            <pubDate>Sun, 14 May 2023 19:41:17 GMT</pubDate>
            <atom:updated>2023-05-14T19:41:17.417Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jkV3-0tVxk36IhfL" /><figcaption>Photo by <a href="https://unsplash.com/@kmuza?utm_source=medium&amp;utm_medium=referral">Carlos Muza</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>visualization is an important aspect of data analysis. In data analysis, plotting and visualization are used to explore and understand patterns and relationships in the data, identify outliers and trends, and communicate findings to others in a clear and concise way.</p><p>There are various tools and libraries available for visualization, such as Matplotlib, Seaborn, Plotly, and ggplot2. These libraries provide a wide range of options for creating different types of visualizations, including scatter plots, line charts, bar charts,histograms, heatmaps, and more.</p><p>Use the best chart to suit your data’s structure and the story you want to tell.</p><ul><li>To analyze trends over time: <strong>line</strong></li><li>To compare values: <strong>bar, column, pie, scatterplot</strong></li><li>To show how parts relate to a whole:<strong> pie</strong></li><li>To show distribution of data: <strong>scatterplot, bar</strong></li><li>To show trends: <strong>line, column</strong></li><li>To show relationships between values:<strong> line, scatterplot, bubble</strong></li></ul><blockquote>Before plotting charts you need to import some modules and libraries</blockquote><pre>import pandas as pd<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br><br># These are what we are going to be using throughout this blog</pre><h4>Visualizing Quantities</h4><p>Visualizing quantities is the use of visual representations, such as charts, graphs, and diagrams, to display numerical data in a way that is easily understandable and interpretable. It involves converting numerical values into visual elements to help viewers to understand patterns, relationships, and trends within the data.</p><p>There are various types of charts that can be used to represent quantities, including: Bar charts, line charts, scatter plots, and more. Now lets look at some examples:</p><pre>df = pd.read_csv(&#39;birds.csv&#39;)<br><br>plt.tick_params(labelbottom=False,bottom=False)<br>plt.xlabel(&#39;birds&#39;)<br>plt.ylabel(&#39;maxwingspan&#39;)<br>plt.title(&#39;Max Wingspan in centimeter&#39;)<br>for i in range(len(df)):<br>    x = df[&#39;Name&#39;][i]<br>    y = df[&#39;MaxWingspan&#39;][i]<br><br>    if df[&#39;MaxWingspan&#39;][i] &lt; 500:<br>        plt.scatter(x, y)<br><br>    if df[&#39;Name&#39;][i] == &#39;Snow goose&#39;:<br>        plt.text(x, y, df[&#39;Name&#39;][i])<br>            <br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*qPYvYfbySINt33vpUfz2bQ.png" /><figcaption>Scatter plot</figcaption></figure><pre>snowGeese = df[df[&#39;Name&#39;] == &#39;Snow goose&#39;]<br>snowGeese.plot(kind=&#39;bar&#39;, title=&#39;Snow Geese&#39;, stacked=True)<br><br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/530/1*oyVhjmEais_jvCPSlD2xTQ.png" /><figcaption>Bar chart</figcaption></figure><pre>maxwingspan = birds[&#39;MaxWingspan&#39;]<br>maxwingspan.plot(color=&#39;green&#39;)<br><br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/561/1*I2Kzq3hDSemIe56A8Q1iaQ.png" /><figcaption>Line chart</figcaption></figure><h4>Visualizing Distributions</h4><p>Visualizing distributions is another way to dig into data, how the data is organized along an axis. It represent the spread, shape, and characteristics of a dataset. It focuses on understanding the underlying distribution of values within the data and provides insights into its central tendency, variability, and any patterns. Let use histogram to see some examples.</p><pre>df = pd.read_csv(&#39;Popular_Baby_Names.csv&#39;)<br><br>df[&#39;Year of Birth&#39;].plot(kind=&#39;hist&#39;, bins=20, figsize=(10,8))<br><br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*O8ZiiFrt4r-1QLiYMl7PFw.png" /><figcaption>Histogram</figcaption></figure><pre>filtered = df[(df[&#39;Year of Birth&#39;] == 2015) &amp; (df[&#39;Rank&#39;] &lt; 40)]<br>filtered[&#39;Count&#39;].plot(kind=&#39;hist&#39;, bins=40, figsize=(10,8))<br><br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*AlfnH0BKBAjG3fRzoqvv5g.png" /><figcaption>Histogram</figcaption></figure><pre>filteredBirds = birds[(birds[&#39;MaxBodyMass&#39;] &gt; 1) &amp; (birds[&#39;MaxBodyMass&#39;] &lt; 60)]<br>filteredBirds.plot(kind=&#39;hist&#39;, bins=40, figsize=(10,8))<br><br># i used Birds.csv in this example</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*jI_O2_eKGdAA1pitPsFFzQ.png" /><figcaption>Histogram</figcaption></figure><h4>Visualizing Proportions</h4><p>Visualizing proportions is the representation of the relative sizes or proportions of different categories or groups within a dataset. It aims to provide a visual understanding of how the parts contribute to the whole or how different groups compare in terms of their composition. We will use pie chart and donut chart in this example.</p><pre>gender = df[&#39;Gender&#39;].value_counts()<br>plt.pie(gender, labels=gender.index, autopct=&#39;%.1f%%&#39;)<br>plt.title(&#39;Gender&#39;)<br><br>plt.show()<br><br># we use Popular_Baby_Names.csv as our dataset</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*1Gxhso9GJNEpeAtNXg_e_Q.png" /><figcaption>Pie chart</figcaption></figure><pre>plt.pie(year_of_birth, labels=year_of_birth.index, autopct=&#39;%.1f%%&#39;, wedgeprops=dict(width=0.3))<br>plt.title(&#39;Year of Birth&#39;)<br><br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/415/1*BKRrFzvoZJGcpCvCG2hxVA.png" /><figcaption>Donut chart</figcaption></figure><h4>Visualizing Relationships</h4><p>Visualizing relationships involves representing the connections, associations, or dependencies between variables or entities in a dataset. It aims to visually depict how two or more variables interact or relate to each other, providing insights into patterns, trends, correlations, or causal relationships.</p><pre>honey = pd.read_csv(&#39;Honey.csv&#39;)<br><br>sns.relplot(data=honey, x=&#39;year&#39;, y=&#39;totalprod&#39;, hue=&#39;state&#39;)<br>plt.title(&#39;Honey Production Over the Years&#39;)<br>plt.xlabel(&#39;Year&#39;)<br>plt.ylabel(&#39;Total Production&#39;)<br><br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/563/1*gDOyBROU--CIUGiz9PO34w.png" /><figcaption>Scatter plot</figcaption></figure><pre>sns.relplot(x=&#39;state&#39;, y=&#39;totalprod&#39;, data=honey, kind=&#39;line&#39;, height=10, aspect=2)<br>plt.title(&#39;Total Production of Each state&#39;)<br><br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_tjdJka2OZWz_rG7XEEecw.png" /><figcaption>Line chart</figcaption></figure><blockquote>Note: Prior to visualizing your data, you need to ensure that it has been cleaned and prepared.</blockquote><p>To make a meaningful visualization, you need to do the following:</p><ol><li>Choose the right chart type</li><li>Avoid deceptive charting</li><li>Work with color</li><li>Style your charts for readability</li><li>Build animated or 3D charting solutions</li><li>Build a creative visualization</li></ol><h4>Conclusion</h4><p>visualizing data improve our understanding by transforming numerical information into visual representations. It enables us to explore quantities, distributions, proportions, and relationships within a dataset. By choosing appropriate visualization techniques, we can uncover insights, discover patterns, and effectively communicate complex information to others. Visualization helps us make data-driven decisions, identify outliers or anomalies, and gain a deeper understanding of the underlying patterns and structures in our data.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be75c23db3eb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Basic Introduction On Data Preparation]]></title>
            <link>https://medium.com/@Khalboy10/basic-introduction-on-data-preparation-b8c04aa08bcf?source=rss-53882316f7f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8c04aa08bcf</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-preparation]]></category>
            <dc:creator><![CDATA[Auwal Auwal Sulaiman]]></dc:creator>
            <pubDate>Thu, 04 May 2023 17:19:53 GMT</pubDate>
            <atom:updated>2023-05-04T17:19:53.617Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*a0apa4tHTl4JmCoR" /><figcaption>Photo by <a href="https://unsplash.com/@lukechesser?utm_source=medium&amp;utm_medium=referral">Luke Chesser</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Raw data may contain some fault that will cause challenges in analysis and modeling, this data can be categorized as <strong><em>dirty;</em></strong> and will need to be cleaned up.</p><h4>Importance Of Cleaning Data</h4><p><strong>Ease of use and reuse: </strong>When data is well arrange<strong> </strong>it’s easier to search, use, and share with others.</p><p><strong>Consistency:</strong> Data science often requires working with more than one dataset; datasets from different sources need to be joined together. Consistency ensure that the Data are still useful when merged into one dataset.</p><p><strong>Model accuracy:</strong> Data that are well cleaned and arranged improves the accuracy of models that rely on it.</p><h4>Common Cleaning Goals and Strategies</h4><p><strong>Exploring a dataset: </strong>Data exploration helps in discovering Data that needs to be cleaned up. Exploring a dataset helps in gaining an understanding of the data, including the range of values, the distribution of variables, the presence of missing values or outliers, and other important characteristics.</p><blockquote>In data analysis process, it helps in understanding the data and gaining and insights into the underlying patterns and relationships.</blockquote><p><strong>Formatting: </strong>It helps in creating clear, concise, and visually appealing content that effectively communicates information and clear and easy-to-understand charts, graphs, and tables.</p><p><strong>Duplications:</strong> This occure due to: errors in data entry, copying and pasting information, or merging multiple datasets; which result in inaccurate statistics and metrics, leading to incorrect conclusions and decisions, and also increase the risk of errors and inconsistencies in data analysis and reporting.</p><blockquote>It is important to identify and remove duplicate from the dataset, This can be done using various techniques such as sorting and filtering.</blockquote><p><strong>Missing Data:</strong> This can occur due to various reasons such as data entry errors, data loss, non-response in surveys, or incomplete data collection. missing data can lead to biased or inaccurate results. There are numerous reasons for why data may be missing and the actions that are taken to resolve these missing values can be dependent on how and why they went missing in the first place.</p><h4>Conclusion</h4><p>Data preparation is the process of transforming raw data into a format that is suitable for analysis. It is a crucial step in data analysis, as the quality of the data preparation can significantly impact the accuracy and reliability of any subsequent analysis.</p><blockquote>Data preparation is an iterative process that often requires significant time and effort. However, investing time in data preparation can greatly improve the quality and accuracy of subsequent analysis.</blockquote><blockquote>Exloring a dataset is a crucial step in the data analysis process, and can help in gaining a better understanding of the data and identifying important patterns and relationships that can be used to make informed decisions.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8c04aa08bcf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction Of Machine Learning: A Beginners Guide]]></title>
            <link>https://medium.com/@Khalboy10/introduction-of-machine-learning-a-beginners-guide-b13392b31149?source=rss-53882316f7f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/b13392b31149</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Auwal Auwal Sulaiman]]></dc:creator>
            <pubDate>Sat, 01 Apr 2023 20:23:14 GMT</pubDate>
            <atom:updated>2023-04-01T20:23:14.540Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/993/1*X1eIoGnpn8a5w7fSE7QSHQ.png" /></figure><p>Machine Learning is a subfield of Artificial Intelligence devoted on building algorithms that learn; that is algorithm that leverage data to improve performance on a set of tasks.</p><p>Computers are dump machines, because we have to provide them with detailed, step by step instructions on exactly what to do. Machine Learning is a process that overcomes this conventional way and make computers smart and give them the ability to think.</p><blockquote>Tom Mitchell, an american computer scientist defines ML as- “A Computer program is said to learn from experiance E with respect to some task T and some performance measure P if its performance on T as measured by P improves with experiance E”.</blockquote><p>The main objective of ML algorithms is to teach the computers on how to learn by themselves. These algorithms can be categorized into the following:</p><p>* Supervised Learning</p><p>* Unsupervised Learning</p><p>* Reinforcement Learning</p><p>In supervised learning, we use the labeled datasets to train algorithms to classify data or predict outcomes accurately. in simple words, it is a process that helps the machine to learn to map an input to a perticular output.</p><p>In unsupervised learning, no supervision is provided in this type of machine learning. This means no training is given to the machine. Therefore machines are restricted to find the hidden structure in unlabeled data by themselves.</p><p>Reinforcement Learning is about taking suitable action to maximize reward in a particular situation. it is called reward base learning. in simple words, reinforcement learning is like teaching a child to do something, you provide reward to him if he does that thing the way you desire, else you punish him by not providing any rewards.</p><h4>History of Artificial Intelligence and Machine Learning</h4><p>1950s: Early development of artificial intelligence (AI) and machine learning (ML) concepts and algorithms.</p><p>1956: The term “artificial intelligence” is coined by John McCarthy at the Dartmouth Conference.</p><p>1958: John McCarthy developed the AI programming language (Lisp) and published the paper “programs with common sense”.</p><p>1960s: The concept of artificial neural networks is introduced, laying the groundwork for deep learning.</p><p>1963: John McCarthy started the AI Lab at Stanford.</p><p>1966: Joseph Weizenbaum created the first chatbot “ELIZA”.</p><p>1970s-1980s: Machine learning algorithms, such as decision trees and logistic regression, are developed and used in practical applications such as image recognition and speech recognition.</p><p>1980: the first national conference of the American Association of AI was held at Stanford University.</p><p>1987–1993: The second AI winter.</p><p>1990s: The concept of support vector machines (SVMs) is introduced, and machine learning techniques begin to be applied in areas such as credit scoring and fraud detection.</p><p>1997: IBM Deep Blue beat world chess champion, Gary Kasparov, and became the first computer to beat a world chess champion.</p><p>2000s: The rise of big data leads to the development of new techniques such as ensemble methods and deep learning, which enable the processing of large amounts of data.</p><p>2010s: The use of ML becomes widespread in areas such as natural language processing, computer vision, and recommendation systems. Advances in hardware, particularly the development of graphical processing units (GPUs), make it possible to train deep neural networks much faster than before.</p><p>2020s: Machine learning continues to advance, with new breakthroughs in areas such as generative models, reinforcement learning, and explainable AI.</p><h4>Fairness and Unfairness in Machine Learning</h4><p>Fairness in machine learning refers to the idea that machine learning algorithms should not systematically discriminate against certain groups of people based on their gender, race, ethnicity, age, religion, or other characteristics. it is the prevention of manipulating data to support bias conclusion. Fairness is the goal to ensure that machine learning models are unbiased.</p><blockquote>Achieving fairness in machine learning can be challenging because machine learning algorithms are often trained on historical data that reflects biases. To address this issue, researchers and practitioners have developed a range of techniques for measuring and mitigating bias in machine learning.</blockquote><p>Unfairness in machine learning refers to situations where machine learning models produce biased outcomes, often against certain groups of people based on their gender, race, ethnicity, age, religion, or other characteristics. This can occur for several reasons, such as:</p><ul><li>Biased training data</li><li>Incorrect assumptions</li><li>Limited transparency</li><li>Lack of divercity</li></ul><h4>How Machine Learning Works</h4><p>Machine learning systems are made up of three major parts: Model, Parameters and Learner</p><p>Model: it is the system that makes predictions or identifications.</p><p>Parameters: They are the signals used by model to form its decisions.</p><p>Learner: it is the system that adjusts the parameters, and in turn the model by looking at differences in predictions versus actual outcome.</p><h4>Machine Learning workflow:</h4><ul><li>Problem Defination: The firststep in the ML workflow is to define and understand the problem that we are going to solve. You will have to start by analyzing the goals and the why factor behind a particular problem statement.</li><li>Data Collection: Once the goal is clearly defined, one has to start getting the data that is needed from various available data sources.</li><li>Data Preparation: The type and quality of data that is used in a ML model affect the output considerably. in this step, one explores, pre-processes, conditions, and transforms data prior to modelling and analysis. this step is the most time consuming.</li><li>Data Visualization: This is where graphs are build in other to identify patterns and trends in data, which leads to clearer understanding and reveals important insights.</li><li>ML Modelling: In this step, ML algorithm is train to make predictions based on the provided data. it is a crucial step that will determine the quality and accuracy of future predictions in new situations.</li><li>Feature Engineering: It is a process to achieve a set of features by performing mathematical, statistical, and heuristic procedures.</li><li>Model Deployment: This is the process of putting a Machine Learning model into a production environment to make data-driven decisions in a more automated way.</li></ul><h4>Conclusion</h4><p>Machine Learning is simply how computers think and execute a task without being programmed to. it is a subset of artificial intelligence that involves algorithms and models that can automatically analyze and learn data to make decisions without human intervention.</p><p>some implementations of machine learning uses data and neural networks in a way that mimics the working of a biological brain.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b13392b31149" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>