<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Maryam Abubakar on Medium]]></title>
        <description><![CDATA[Stories by Maryam Abubakar on Medium]]></description>
        <link>https://medium.com/@maryamsabubakar53?source=rss-1ad073d95cce------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*PMfktTz1bWh2mwYr.jpg</url>
            <title>Stories by Maryam Abubakar on Medium</title>
            <link>https://medium.com/@maryamsabubakar53?source=rss-1ad073d95cce------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 11:18:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@maryamsabubakar53/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Reinforcement learning]]></title>
            <link>https://medium.com/@maryamsabubakar53/reinforcement-learning-b102275ad2c2?source=rss-1ad073d95cce------2</link>
            <guid isPermaLink="false">https://medium.com/p/b102275ad2c2</guid>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ａｉ]]></category>
            <dc:creator><![CDATA[Maryam Abubakar]]></dc:creator>
            <pubDate>Sat, 22 Jul 2023 07:32:57 GMT</pubDate>
            <atom:updated>2023-07-22T07:32:57.086Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*M1niW41vMAigOsyUCSpHlw.jpeg" /><figcaption><a href="https://www.google.com/url?sa=i&amp;url=https%3A%2F%2Fwww.indianai.in%2Fwhat-is-reinforcement-learning%2F&amp;psig=AOvVaw2_136S6LlHFvRX0PpQWqoG&amp;ust=1690094578146000&amp;source=images&amp;cd=vfe&amp;opi=89978449&amp;ved=0CBEQjRxqFwoTCODc0ofboYADFQAAAAAdAAAAABAE">Source</a></figcaption></figure><p>Reinforcement learning is an area of Machine Learning. It is about taking appropriate behavior to maximize benefit in a certain situation. It is used by various applications and computers to determine the best possible behavior or course to take in a given situation. Reinforcement learning differs from supervised learning in that the training data contains the answer key, allowing the model to be trained with the correct answer, whereas in reinforcement learning, there is no answer and the reinforcement agent decides what to do to complete the given task. It is obligated to learn from its experience in the absence of a training dataset.</p><p>Reinforcement Learning (RL) is an examination of decision making. It involves understanding how to act in a situation to reap the most benefits. Data for RL is gathered from machine learning systems that employ a trial-and-error process. Input for either supervised or unsupervised machine learning does not include data.</p><p>Algorithms used in reinforcement learning determine the next course of action based on results. The algorithm receives feedback after each step that aids in determining whether the decision it made was good, bad, or indifferent. It is a useful method for automated systems that must make numerous tiny judgments without human supervision.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/508/1*4bQvYwKXwXqGxopB1RoO-w.png" /><figcaption><a href="https://www.javatpoint.com/reinforcement-learning">source</a></figcaption></figure><h3>Applications and examples of reinforcement learning</h3><p>Although reinforcement learning has attracted a lot of attention in the field of artificial intelligence, there are still many limitations to its adoption and use in the actual world. Despite this, there are several research articles on theoretical applications, and some effective use cases exist.</p><p>Current use cases include, but are not limited to, the following:</p><ul><li>gaming</li><li>resource management</li><li>personalized recommendations</li><li>robotics</li></ul><h3>Common reinforcement learning algorithms</h3><p>Instead of referring to a single algorithm, the discipline of reinforcement learning comprises a number of algorithms that employ somewhat diverse methodologies. Their approaches to navigating their environments account for the majority of the disparities.</p><p><strong>State-action-reward-state-action (SARSA). </strong>Giving the agent a policy is the first step in this reinforcement learning technique. The policy is essentially a probability that informs it of the likelihood that specific behaviors will provide rewards or advantageous situations.</p><p><strong>Q-learning.</strong>This method of reinforcement learning adopts a different strategy. Since the agent doesn’t receive a policy, its investigation of its surroundings is more independent.</p><p><strong>Deep Q-Networks.</strong>These algorithms combine approaches from reinforcement learning with neural networks. They make use of reinforcement learning’s self-directed environment exploration. Future actions are based on a random sampling of advantageous prior behaviors that the neural network has learned.</p><h3>How is reinforcement learning different from supervised and unsupervised learning?</h3><p>Although reinforcement learning is regarded as a distinct branch of machine learning, it shares several characteristics with the other three, which can be divided into the following four domains:</p><ol><li><strong>Supervised learning. </strong>Algorithms are trained on a body of labeled data in supervised learning. The only properties that supervised learning algorithms can learn are those that are included in the data set. Image recognition models are typical supervised learning applications. These models learn to detect common characteristics of specified forms after receiving a batch of tagged photos.</li><li><strong>Unsupervised learning. </strong>Developers use unsupervised learning to let algorithms loose on completely unlabeled data. Without being instructed what to search for, the algorithm learns by documenting its own observations regarding data properties.</li><li><strong>Semisupervised learning. </strong>This method uses a pragmatic approach. Developers enter a small corpus of labeled training data as well as a larger corpus of unlabeled data. The algorithm is then told to apply what it has learned from the labeled data to the unlabeled data and make conclusions from the entire set.</li><li><strong>Reinforcement learning.</strong>This takes a very different approach. It places an agent in an environment with explicit parameters defining useful and nonbeneficial behaviour, as well as an overarching goal to achieve. In some aspects, it is comparable to supervised learning in that developers must offer algorithms well defined goals as well as establish rewards and punishments. This means that more explicit programming is necessary than in unsupervised learning. However, once these parameters are chosen, the algorithm acts autonomously, making it far more self-directed than supervised learning algorithms. As a result, while reinforcement learning is occasionally referred to as a subset of semisupervised learning, it is more commonly recognized as its own sort of machine learning.</li></ol><h3>Conclusion</h3><p>Reinforcement Learning is one of the most fascinating and valuable aspects of machine learning. In RL, the agent explores the environment without the need for human intervention. It is the primary learning algorithm in Artificial Intelligence. However, there are specific situations in which it should not be use, such as when there is enough data to answer the problem and other ML methods may be employed more efficiently. The fundamental issue with the RL algorithm is that certain of the settings, such as delayed feedback, may alter the speed of learning.</p><p>Thank you!!!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b102275ad2c2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to Time Series Analysis in Machine learning]]></title>
            <link>https://medium.com/@maryamsabubakar53/introduction-to-time-series-analysis-in-machine-learning-5242ef31918b?source=rss-1ad073d95cce------2</link>
            <guid isPermaLink="false">https://medium.com/p/5242ef31918b</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[time-series-analysis]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Maryam Abubakar]]></dc:creator>
            <pubDate>Fri, 14 Jul 2023 15:07:46 GMT</pubDate>
            <atom:updated>2023-07-14T15:07:46.774Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*Ub9IDgh3IjgBdbG4VRbIkQ.png" /><figcaption><a href="https://www.business-science.io/business/2018/12/04/time-series-forecasting.html">Source</a></figcaption></figure><h3>Introduction</h3><p>While there are many sophisticated tools and techniques used for data analysis, including machine learning (ML), statistical time series analysis is one of the methods that is usually chosen. We’ve all probably heard someone claim that the cost of certain goods has gone down or up over time. These goods might be anything from gasoline, diesel, to gold, silver, edibles, etc. Time-series data can also be interpreted in real-time or in the actual world because time is a physical number while the components, coefficients, parameters, and features of time-series data are mathematical values.</p><p>Time Series Data Analysis is a way of studying the characteristics of the response variable with respect to time as the independent variable. Use the time variable as a point of reference for estimating the target variable for the purposes of predicting or forecasting. A time-based set of commands is represented by a time-series. It would be Horus, Minutes, Seconds, Years, Months, Weeks, and Weeks. It is a finding based on the discrete time pattern of subsequent intervals.</p><blockquote><strong><em>A </em></strong><a href="https://www.analyticsvidhya.com/blog/2021/10/a-comprehensive-guide-to-time-series-analysis/"><strong><em>time series</em></strong></a><strong><em> is nothing but a sequence of various data points that occurred in a successive order for a given period of time …….</em></strong></blockquote><h4><strong>Time-Series Analysis Examples</strong></h4><p>Consider the following example: The major goal in the financial domain is to recognize trends, seasonal behavior, and correlation using time series analysis techniques and developing filters based on forecasts, which includes;</p><ol><li><strong>To predict predicted utilities</strong>- It is required to have accurate and trustworthy future projections such as asset prices, variation in usage, products in demand in statistical form through market research, and time-series dataset for perfect and successive trading.</li><li><strong>Simulate series</strong>- After obtaining statistical output data from financial time series, simulations of future events can be created. It assists us in determining the number of deals, estimated trading costs and profits, required financial and technical investment, various trading dangers, and so on.</li><li><strong>Assume relationship</strong>- Recognizing the link between the time series and other quantities provides us with trading signals to improve the current trading style. Estimated trades can be inferred for a particular period for projecting a widespread to reduce transaction costs, for example, to know the spread of a foreign currency pair and its fluctuation with a proposal.</li></ol><p><strong>How Should Time Series Be Analyzed?</strong><br>To conduct a time series analysis, we must first undertake the following steps:</p><ol><li>Gathering and cleansing the data</li><li>Creating a visual representation of time vs key feature</li><li>Observing the series’ stationarity</li><li>Creating charts to better comprehend its nature.</li><li>Model construction — AR, MA, ARMA, and ARIMA</li><li>Prediction insight extraction</li></ol><h4><strong>Time Series Analysis Components</strong></h4><p>Let’s have a look at the different components of Time Series Analysis-</p><p><strong>Trend:</strong> A continuous timeline in which there is no defined interval and any divergence within the presented dataset. The general trend would be Positive, Negative, or Null Trend<br><strong>Seasonality:</strong> When a dataset’s regular or fixed interval shifts within a continuous timeframe. Is it a bell curve or a saw tooth?<br><strong>Cyclical:</strong> There is no defined interval, there is variability in movement and its pattern.<br><strong>Irregularity:</strong> Unexpected situations/events/scenarios, as well as surges in a short period of time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*iAPZNz3O1LacPBbSKkxfew.png" /><figcaption><a href="https://www.analyticsvidhya.com/blog/2021/10/a-comprehensive-guide-to-time-series-analysis/">Source</a></figcaption></figure><h3>Data Types of Time Series</h3><p>There are two major types — stationary and non-stationary.</p><p><strong>Stationary:</strong> A dataset should adhere to the following guidelines if it lacks Trend, Seasonality, Cyclical, and Irregularity components of the time series.</p><p>1. During the analysis, <strong>the mean</strong> value of them should be entirely constant in the data.<br>2. <strong>The variance</strong> should be constant in relation to time.<br>3. <strong>Covariance</strong> is a measurement of the connection between two variables.</p><p><strong>Non-stationary</strong>: A dataset is non-stationary if either the mean-variance or the covariance changes with time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*vbFJd4D8fjQ0ttpg83ggpQ.png" /><figcaption><a href="https://www.analyticsvidhya.com/blog/2021/10/a-comprehensive-guide-to-time-series-analysis/">source</a></figcaption></figure><p>Thank You!!!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5242ef31918b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sentiment Analysis using NLTK]]></title>
            <link>https://medium.com/@maryamsabubakar53/sentiment-analysis-using-nltk-2a41801388c8?source=rss-1ad073d95cce------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a41801388c8</guid>
            <category><![CDATA[nltk]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[sentiment-analysis]]></category>
            <dc:creator><![CDATA[Maryam Abubakar]]></dc:creator>
            <pubDate>Mon, 26 Jun 2023 18:10:03 GMT</pubDate>
            <atom:updated>2023-06-26T18:10:03.024Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*r_rqGVUqDeyN3cf_lOSNXQ.jpeg" /><figcaption><a href="https://www.google.com/url?sa=i&amp;url=https%3A%2F%2Fwww.kdnuggets.com%2F2021%2F06%2Fcreate-deploy-sentiment-analysis-app-api.html&amp;psig=AOvVaw1ivEfwnlkuGEHq51f9rudA&amp;ust=1687801929318000&amp;source=images&amp;cd=vfe&amp;ved=0CBMQjhxqFwoTCKjQpsr-3v8CFQAAAAAdAAAAABAJ">source</a></figcaption></figure><h4>What Is Sentiment Analysis?</h4><p>Sentiment analysis (also known as opinion mining) is a type of natural language processing (NLP) approach that determines whether data is positive, negative, or neutral. Sentiment analysis on textual data is frequently used to assist organizations in monitoring brand and product sentiment in consumer feedback and understanding customer demands.</p><p><strong>NLTK</strong> means Natural Language Processing Toolkit is a package in python made especially for text-based analysis. These are namely the steps needed:</p><p>1.Importing Necessary Modules<br>2. Importing Dataset<br>3. Data Preprocessing and Visualization<br>4. Model Building<br>5. Prediction</p><h3>1. Importing Necessary Modules:</h3><p>Here we are importing all the basic import modules required depending on your data.</p><pre>import nltk<br>import random<br>import pandas as pd<br>import time<br>from bs4 import BeautifulSoup<br>import re<br><br>from nltk.corpus import stopwords<br>from nltk.sentiment import SentimentIntensityAnalyzer<br><br>from nltk.tokenize import word_tokenize<br>from sklearn.feature_extraction.text import TfidfVectorizer<br>from sklearn.model_selection import train_test_split<br>from sklearn.naive_bayes import MultinomialNB<br>from sklearn.metrics import classification_report<br>from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score</pre><h3>2. Importing Dataset:</h3><p>Now you import the dataset using Panda</p><pre>df = pd.read_csv(&#39;your_data_here.csv&#39;)<br>or<br>dataset = pd.read_csv(&#39;your_data_here.csv&#39;)</pre><h3>3. Data Preprocessing and Visualization:</h3><p>Data preprocessing changes the data into a format that can be processed in data mining, machine learning, and other data science tasks more easily and efficiently.</p><p>After having access to the data, we have clean it. We could quickly determine the total number of missing values in the dataset by using the ‘isnull().sum()’ technique.</p><pre>dataset.isnull().sum()</pre><p>then we can either replace the null values or remove the respective rows which contain the null values, to drop we use</p><pre>dataset = data.dropna()</pre><p>more on data Visualization in <a href="https://medium.com/@maryamsabubakar53/many-analytical-tasks-such-as-data-summarization-exploratory-data-analysis-and-model-output-af7288ad10db?source=your_stories_page">here</a></p><h3>4. Model Building:</h3><p>Before starting to develop the model we need let’s do a quick task first. We are aware that in order for us to properly understand and then classify the review, humans need articles, determinants, conjunctions, punctuation marks, etc. But with machines, this is not the case. Therefore, they don’t truly require these to categorize the sentiment; instead, if they are present, they literally become confused. Thus, the ‘nltk’ library is required in order to do this task, just like any other sentiment analysis.</p><p>‘Natural Language Processing Toolkit’ is what the acronym NLTK stands for. This is one of the essential libraries for performing Sentiment Analysis or any text-based ML project. So, with the aid of this library, you can start by removing the punctuation and then the words that don’t add anything meaningful to the text. you start by using a function called “punc_clean” to remove all punctuation from each review. The following code will implement the same:</p><pre>import nltk<br>def punc_clean(text):<br>    import string as st<br>    a=[w for w in text if w not in st.punctuation]<br>    return &#39;&#39;.join(a)<br>dataset[&#39;review&#39;] = dataset[&#39;review&#39;].apply(punc_clean)<br>dataset.head(2)</pre><p>Therefore, the punctuation marks are removed by the code above. Next, we must eliminate any terms that don’t add meaning to the statement. These terms are referred to as “stopwords.” but we want to keep the word “not” so we use this code.</p><pre>def remove_stopword(text):<br>    stopword=nltk.corpus.stopwords.words(&#39;english&#39;)<br>    stopword.remove(&#39;not&#39;)<br>    a=[w for w in nltk.word_tokenize(text) if w not in stopword]<br>    return &#39; &#39;.join(a)<br>data[&#39;review&#39;] = data[&#39;review&#39;].apply(remove_stopword)</pre><p>As a result, we are currently only one step behind the model building. The next goal is to assign a sentiment score to each word in every review. In order to implement it, we must use the ‘TfidVectorizer’ library from the’sklearn’ module, which is contained in the ‘feature_extraction.text’. The ‘TfidVectorizer’ documentation should be thoroughly read to gain a thorough grasp of the library.</p><pre>from sklearn.feature_extraction.text import TfidfVectorizer<br>vectr = TfidfVectorizer(ngram_range=(1,2),min_df=1)<br>vectr.fit(data[&#39;review&#39;])<br>vect_X = vectr.transform(data[&#39;review&#39;])</pre><p>It is now time to construct the model. Since the sentiment analysis is a binary classification, a “1” indicates a positive review and a “0” indicates a negative review.</p><pre>from sklearn.linear_model import LogisticRegression<br>model = LogisticRegression()<br>clf=model.fit(vect_X,data[&#39;sentiment&#39;])<br>clf.score(vect_X,data[&#39;sentiment&#39;])*100</pre><h3>5. Prediction:</h3><p>So to clarify the performance of the model I have used two simple sentences “I love bread” and “I hate bread” which clearly refer to positive and negative sentiment.</p><p>More here: <a href="https://www.analyticsvidhya.com/blog/2021/06/sentiment-analysis-using-nltk-a-practical-approach/">https://www.analyticsvidhya.com/blog/2021/06/sentiment-analysis-using-nltk-a-practical-approach/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a41801388c8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Natural Language Processing]]></title>
            <link>https://medium.com/@maryamsabubakar53/natural-language-processing-83d4a1a1ae0e?source=rss-1ad073d95cce------2</link>
            <guid isPermaLink="false">https://medium.com/p/83d4a1a1ae0e</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Maryam Abubakar]]></dc:creator>
            <pubDate>Fri, 23 Jun 2023 17:23:04 GMT</pubDate>
            <atom:updated>2023-06-23T17:23:04.417Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/796/1*esjTSVXg__JNbX_eZDm6Dw.png" /><figcaption><a href="https://www.google.com/url?sa=i&amp;url=https%3A%2F%2Fwww.freepik.com%2Ffree-photos-vectors%2Fnatural-language-processing&amp;psig=AOvVaw3uZ2Q8x9dXMXYiVB5dJ4kz&amp;ust=1687547748927000&amp;source=images&amp;cd=vfe&amp;ved=0CBMQjhxqFwoTCJCfp67L1_8CFQAAAAAdAAAAABAr">Source</a></figcaption></figure><p>Natural Language Processing (NLP) is in every part of our life from our home to our work. We send voice commands to control our smartphones, virtual assistants in our homes, and even our cars. NLP and Machine Learning (ML) are used by voice-activated applications like Alexa, Siri, and Google Assistant to respond to our questions, add events to our calendars, and call the contacts that we state in our voice commands.</p><h3>Natural language processing definition</h3><p><strong>Natural language processing</strong> (<strong>NLP</strong>) is an interdisciplinary subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to process and analyze large amounts of natural language data. The goal is a computer capable of “understanding” the contents of documents, including the contextual nuances of the language within them. The technology can then accurately extract information and insights contained in the documents as well as categorize and organize the documents themselves. <a href="https://en.wikipedia.org/wiki/Natural_language_processing"><em>wikipedia</em></a><em>.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*8A_L947ndRIoXiM4.png" /><figcaption><a href="https://www.encora.com/hs-fs/hubfs/Blog-EncoraML.png?width=800&amp;name=Blog-EncoraML.png">source</a></figcaption></figure><h3>The evolution of natural language processing</h3><p>NLP dated back to the mid-20th century, that’s from</p><p><strong>1950s.</strong> In this decade, Alan Turing created the Turing Test to determine whether or not a machine might be truly intelligent. This is when natural language processing first emerged. The test uses computerized interpretation and natural language generation as IQ criteria.</p><p><strong>1950s </strong>to <strong>1990s </strong>majority of NLP relied on rules, which were built by linguists and used to guide how computers processed language.</p><p><strong>1990s.</strong>Natural language processing (NLP) technology was developed using a more statistical method rather than a top-down, language-first approach as a result of advances in computing. The ability of computers to build rules based on linguistic statistics became faster. During this decade, data-driven natural language processing gained popularity. Instead of focusing on linguistics, natural language processing now takes a more engineering-based approach and draws from a wider range of scientific fields.</p><p><strong>2000–2020s</strong>. The term “natural language processing” has become significantly more widespread. Natural language processing has many practical uses thanks to advancements in computing power. Modern approaches to NLP combine statistical techniques with traditional linguistics.</p><h3>Natural language techniques</h3><p>NLP includes a wide range of techniques used for analyzing human language. You’ll probably come across the following methods most frequently in the field:</p><ul><li><strong>Sentiment analysis</strong>: This is a NLP technique that analyzes text to identify its sentiments, which can be either “positive,” “negative,” or “neutral.” it mostly used in businesses to better understanding of customer feedback.</li><li><strong>Summarization</strong>: This NLP technique summarize a longer text like reports and articles, to make it manageable for time-sensitive readers.</li><li><strong>Keyword extraction</strong>: A natural language processing (NLP) technique that analyze a text to determine the most important keywords or phrases. Search engine optimization (SEO), social media monitoring, and business intelligence all frequently make use of keyword extraction.</li><li><strong>Tokenization</strong>: The process of breaking characters, words, or subwords to “tokens” that a computer software can understand. Common NLP activities like word modeling, vocabulary development, and frequent word occurrence are made possible by tokenization.</li></ul><h3>NLP benefits</h3><p>NLP offers both organizations and customers a number of advantages, whether it’s used to quickly translate a text between languages or to provide business insights by performing a sentiment analysis on hundreds of reviews.</p><p>Other benefits include:</p><ul><li>improved accuracy and efficiency of documentation;</li><li>ability to automatically make a readable summary of a larger, more complex original text;</li><li>useful for personal assistants such as Alexa, by enabling it to understand spoken word;</li><li>enables an organization to use chatbots for customer support;</li><li>easier to perform sentiment analysis; and</li><li>provides advanced insights from analytics that were previously unreachable due to data volume.</li></ul><h4>NLP limitations</h4><p>NLP has a wide range of applications, although it is far from perfect. Many NLP technologies, in fact, struggle to interpret sarcasm, emotion, slang, context, errors, and other confusing expressions. This means that NLP is mostly limited to unambiguous situations that don’t require a significant amount of interpretation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=83d4a1a1ae0e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Clustering in Machine Learning]]></title>
            <link>https://medium.com/@maryamsabubakar53/clustering-in-machine-learning-2ec8dc3897ce?source=rss-1ad073d95cce------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ec8dc3897ce</guid>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[clustering-algorithm]]></category>
            <dc:creator><![CDATA[Maryam Abubakar]]></dc:creator>
            <pubDate>Sun, 11 Jun 2023 12:39:44 GMT</pubDate>
            <atom:updated>2023-06-11T12:39:44.242Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*cRbLt-FeMsOIGneTjNl0aA.png" /><figcaption><a href="https://www.google.com/url?sa=i&amp;url=https%3A%2F%2Fwww.javatpoint.com%2Fclustering-in-machine-learning&amp;psig=AOvVaw3gWUE_-bh8aJUWeedCk5eg&amp;ust=1686504890407000&amp;source=images&amp;cd=vfe&amp;ved=0CBEQjRxqFwoTCNiz_rieuf8CFQAAAAAdAAAAABAJ">Source</a></figcaption></figure><p>Clustering is a process of arranging a group of objects in such that an objects in the same group are more similar to each other than to the objects in any other group. We refers to a group as cluster and each cluster has a unique ID called the cluster ID.</p><p>Clustering is a type of unsupervised learning method where the algorithms deals with the unlabeled dataset.</p><blockquote>Clustering algorithm has some similar to the classification algorithm, but the difference is the type of dataset that we are using. In classification algorithms, we work with the labeled data set, whereas in clustering, we work with the unlabeled dataset.</blockquote><p>To understand clustering more, less look at a shopping mall, we can see how different items/product with the same similarity in usage are put together such as the diary products section, vegetable section, the toiletries etc. These helps to find out things easily. The clustering technique also works in the same way.</p><p>The clustering technique can be widely used in various tasks. Some most common uses of this technique are:</p><ul><li>Market Segmentation</li><li>Statistical data analysis</li><li>Social network analysis</li><li>Image segmentation</li><li>Anomaly detection, etc.</li></ul><h3>Types of Clustering Methods</h3><ol><li><strong>Partitioning Clustering</strong></li><li><strong>Density-Based Clustering</strong></li><li><strong>Distribution Model-Based Clustering</strong></li><li><strong>Hierarchical Clustering</strong></li><li><strong>Fuzzy Clustering</strong></li></ol><h4>Partitioning Clustering</h4><p>It is a type of clustering that divides the data into non-hierarchical groups. The most common example of partitioning clustering is the <a href="https://github.com/arewadataScience/ML-4-Beginners-Arewa-Datascience/blob/main/5-Clustering/2-K-Means/README.md"><strong>K-Means Clustering algorithm</strong>.</a> By using a sequence of observations, K-Means is used to separate and partition groupings of data into ‘k’ clusters. Each observation attempts to arrange a particular data point into a cluster that is closest to its nearest “mean,” or the cluster’s focal point.</p><h4>Density-Based Clustering</h4><p>As long as the dense region can be connected, the density-based clustering method joins the extremely dense areas into clusters, resulting in distributions of any shape. This algorithm accomplishes that by finding several clusters in the dataset and joining the regions with high densities into clusters. The dense areas in data space are divided from each other by sparser areas.</p><h4>Distribution Model-Based Clustering</h4><p>In the distribution model-based clustering method, the data is divided based on the probability of how a dataset belongs to a particular distribution. The grouping is done by assuming some distributions commonly <strong>Gaussian Distribution</strong>.</p><h4>Hierarchical Clustering</h4><p>Because there is no need to specify the number of clusters to be produced, hierarchical clustering can be utilized as an alternative to partitioned clustering. The dataset is divided into clusters in this technique to form a tree-like structure known as a dendrogram. The observations or any number of clusters can be selected by cutting the tree at the correct level. The Agglomerative Hierarchical algorithm is the most common example of Hierarchical clustering</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*-1y9QuZ206ysgzhB178aVQ.png" /><figcaption><a href="https://www.javatpoint.com/clustering-in-machine-learning">source</a></figcaption></figure><h4>Fuzzy Clustering</h4><p>A data object may be a member of more than one group or cluster when using fuzzy clustering. Every dataset has a set of membership coefficients that are based on how much of a member a dataset is in a cluster. The Fuzzy C-means algorithm, also referred to as the Fuzzy k-means algorithm, is an example of this kind of clustering.</p><h3>Clustering Algorithms</h3><ol><li><strong>K-Means algorithm:</strong> The k-means algorithm is one of the most popular clustering algorithms.</li><li><strong>Mean-shift algorithm:</strong> Mean-shift algorithm tries to find the dense areas in the smooth density of data points.</li><li><strong>DBSCAN Algorithm:</strong> It stands <strong>for Density-Based Spatial Clustering of Applications with Noise</strong>.</li><li><strong>Expectation-Maximization Clustering using GMM:</strong> This algorithm can be used as an alternative for the k-means algorithm.</li><li><strong>Agglomerative Hierarchical algorithm:</strong> The Agglomerative hierarchical algorithm performs the bottom-up hierarchical clustering.</li><li><strong>Affinity Propagation:</strong> It is different from other clustering algorithms as it does not require to specify the number of clusters.</li></ol><p><strong>Applications of Clustering in different fields:</strong></p><ol><li><strong>Marketing:</strong> It can be used to define and discover consumer segments..</li><li><strong>Biology:</strong> It can be used for classification among different species of plants and animals.</li><li><strong>Libraries:</strong> It is used in grouping different books base on different on different topics.</li><li><strong>City Planning: </strong>Using their geographic locations and other criteria, it is used to categorize houses and analyze their values.</li><li><strong>Earthquake studies: </strong>By learning the earthquake-affected areas we can determine the dangerous zones.</li><li><strong>Image Processing</strong>: Clustering is a technique for grouping similar photos together, classifying images based on content, and identifying patterns in image data.</li><li><strong>Genetics:</strong> Clustering is a technique for grouping genes with similar expression patterns and identifying gene networks that work together in biological processes.</li><li><strong>Finance:</strong> With clustering we can group market section based on customer behavior, then identify different patterns in stock market data and analyze risk in investment portfolios.</li><li><strong>Customer Service:</strong> Inquiries and complaints from clients are categorized using clustering, which also helps identify common problems and create solutions.</li><li><strong>Manufacturing</strong>: Clustering is used to group similar products together, which helps identify common problems and target market.</li></ol><p>And many more</p><h3><strong>Conclusion</strong></h3><p>Clustering is an unsupervised learning approach where grouping is done on similarities basis. Here the machine learns from the existing data and does not need any training while classification is a supervised learning approach where a specific label is provided to the machine to classify new observations. Here the machine needs proper testing and training for the label verification</p><p>Thank you.</p><blockquote>Read more: <a href="https://www.javatpoint.com/clustering-in-machine-learning">https://www.javatpoint.com/clustering-in-machine-learning</a></blockquote><blockquote><a href="https://www.geeksforgeeks.org/clustering-in-machine-learning/">https://www.geeksforgeeks.org/clustering-in-machine-learning</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ec8dc3897ce" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Classification in Machine Learning: An Introduction]]></title>
            <link>https://medium.com/@maryamsabubakar53/classification-in-machine-learning-an-introduction-d4a9f56cae8?source=rss-1ad073d95cce------2</link>
            <guid isPermaLink="false">https://medium.com/p/d4a9f56cae8</guid>
            <category><![CDATA[logistic-regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[classification-algorithms]]></category>
            <category><![CDATA[machine-learning-ai]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Maryam Abubakar]]></dc:creator>
            <pubDate>Sat, 03 Jun 2023 10:15:37 GMT</pubDate>
            <atom:updated>2023-06-03T10:15:37.854Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/593/1*vKLMyX17iNx9O4T0DSzrvA.jpeg" /><figcaption><a href="https://res.cloudinary.com/dyd911kmh/image/upload/v1663850409/Difference_between_Classification_and_Regression_98d8677bfd.png">Source</a></figcaption></figure><p>Artificial intelligence and machine learning are using data to understand society as well as predict business outcomes and drive decision-making and growth in different aspect of life. Classification in machine learning is one of the commonly used supervised machine learning processes. which helps in categorizing data into different classes. Applications using this are email spam detection, medical diagnostic test, fraud detection, image classification, and speech recognition etc.</p><h3>Supervised Learning</h3><p>In Supervised machine learning, the machine is trained with the help of labeled input and output data to analyze the training data and predict accurate outcomes for new and unseen data. In this approach past data is used to train the algorithm to categorize file types, such as images, words, and documents, and then predict outcomes by learning the new data via learning patterns. Also, these supervised approaches are valuable in developing business applications. However, training the model can take time, and the process can become even more challenging while classifying big data.</p><p>The supervised machine learning algorithms can be broadly classified into two categories: Regression and classification. <a href="https://emeritus.org/blog/artificial-intelligence-and-machine-learning-classification-in-machine-learning/#:~:text=In%20machine%20learning%2C%20classification%20is,datasets%20of%20input%20and%20output.">Here</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/296/1*ghK9uVFfXneVpQk-eplNYQ.jpeg" /><figcaption><a href="https://www.simplilearn.com/tutorials/machine-learning-tutorial/classification-in-machine-learning">source</a></figcaption></figure><blockquote>Regression</blockquote><blockquote>A regression problem is used to predict the output for real or continuous values. Moreover, this technique aims to map a predictive relationship between dependent and independent variables. <a href="https://emeritus.org/blog/artificial-intelligence-and-machine-learning-classification-in-machine-learning/#:~:text=In%20machine%20learning%2C%20classification%20is,datasets%20of%20input%20and%20output.">Here</a></blockquote><ul><li>The prediction task is a <strong><em>regression</em></strong> when the target variable is continuous. An example can be the prediction of the salary of a person given their education degree, previous work experience, geographical location, and level of seniority.</li></ul><h3>Classification</h3><p>Based on training data given, the Classification algorithm is a Supervised Learning technique used to categorize new observations. In classification, a program uses the dataset to categorize new observations into various classes or groups. This require two or more categories of a given data set. Therefore, it generates a probability score to assign the data into a specific category, such as spam or not spam, yes or no, disease or no disease, red or green, male or female, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*7DxHilvYNk5jFk3f1wq4Pg.png" /><figcaption><a href="https://res.cloudinary.com/dyd911kmh/image/upload/v1663850409/Difference_between_Classification_and_Regression_98d8677bfd.png">Source</a></figcaption></figure><h3>Some Applications of Machine Learning Classification Problems</h3><ul><li>Image classification</li><li>Fraud detection</li><li>Document classification</li><li>Spam filtering</li><li>Facial recognition</li><li>Voice recognition</li><li>Medical diagnostic test</li><li>Customer behavior prediction</li><li>Product categorization</li><li>Malware classification</li></ul><h3>Types of Classification Tasks in Machine Learning</h3><ul><li>Binary Classification</li><li>Multi-Class Classification</li><li>Multi-Label Classification</li><li>Imbalanced Classification</li></ul><h3>Binary Classification</h3><p>Binary classification in machine learning has only two possible outcomes. For example, yes or no, true or false, spam or not spam.</p><p>Examples include:</p><ul><li>Email spam detection (spam or not).</li><li>Churn prediction (churn or not).</li><li>Conversion prediction (buy or not).</li></ul><p>Popular algorithms that can be used for binary classification include:</p><ul><li>Logistic Regression</li><li>k-Nearest Neighbors</li><li>Decision Trees</li><li>Support Vector Machine</li><li>Naive Bayes</li></ul><p>We will use <a href="https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_blobs.html">make_blobs() function</a> in scikit-learn to generate a synthetic binary classification dataset. <a href="https://machinelearningmastery.com/types-of-classification-in-machine-learning/">Here</a></p><p>The example below generates a dataset with 1,000 examples that belong to one of two classes, each with two input features.</p><pre>in<br><br># example of binary classification task<br>from numpy import where<br>from collections import Counter<br>from sklearn.datasets import make_blobs<br>from matplotlib import pyplot<br># define dataset<br>X, y = make_blobs(n_samples=1000, centers=2, random_state=3)<br># summarize dataset shape<br>print(X.shape, y.shape)<br># summarize observations by class label<br>counter = Counter(y)<br># plot the dataset and color the by class label<br>for label, _ in counter.items():<br> row_ix = where(y == label)[0]<br> pyplot.scatter(X[row_ix, 0], X[row_ix, 1], label=str(label))<br>pyplot.legend()<br>pyplot.show()</pre><pre>out<br><br>(1000, 2) (1000,)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*ToZOagWaq46NGNpgJl2RdQ.png" /></figure><h3>Multi-Class Classification</h3><p>Multi-class is a type of classification problem with more than two outcomes, here each outcome is assigned to only one label.</p><p>Examples include:</p><ul><li>Face classification.</li><li>Plant species classification.</li><li>Optical character recognition.</li></ul><p>Popular algorithms that can be used for multi-class classification include:</p><ul><li>k-Nearest Neighbors.</li><li>Decision Trees.</li><li>Naive Bayes.</li><li>Random Forest.</li><li>Gradient Boosting.</li></ul><p>We can use the same code and change the counter to 3 instead of 2</p><pre>in<br><br>X, y = make_blobs(n_samples=1000, centers=3, random_state=3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*IdyOJuc2ffbGwic5lb3MhQ.png" /></figure><h3>Multi-Label Classification</h3><p>Multi-label is a type of classification problem that may have more than one class label assigned to the data. Here the model will have multiple outcomes.</p><p>For example</p><ul><li>A book or a movie can be categorized into multiple genres.</li><li>An image can have multiple objects.</li></ul><p>Popular algorithms that can be used for multi-Label classification include</p><ul><li>Multi-label Decision Trees</li><li>Multi-label Random Forests</li><li>Multi-label Gradient Boosting</li></ul><pre># example of a multi-label classification task<br>from sklearn.datasets import make_multilabel_classification<br># define dataset<br>X, y = make_multilabel_classification(n_samples=1000, n_features=2, n_classes=3, n_labels=2, random_state=1)<br># summarize dataset shape<br>print(X.shape, y.shape)<br># summarize first few examples<br>for i in range(10):<br> print(X[i], y[i])<br># plot the dataset and color the by class label<br>for label, _ in counter.items():<br> row_ix = where(y == label)[0]<br> pyplot.scatter(X[row_ix, 0], X[row_ix, 1], label=str(label))<br>pyplot.legend()<br>pyplot.show()</pre><pre>out<br><br>(1000, 2) (1000, 3)<br>[18. 35.] [1 1 1]<br>[22. 33.] [1 1 1]<br>[26. 36.] [1 1 1]<br>[24. 28.] [1 1 0]<br>[23. 27.] [1 1 0]<br>[15. 31.] [0 1 0]<br>[20. 37.] [0 1 0]<br>[18. 31.] [1 1 1]<br>[29. 27.] [1 0 0]<br>[29. 28.] [1 1 0]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/543/1*lzRFB-KlOZ1O6DQV-w8YOw.png" /></figure><h3>Imbalanced Classification</h3><p>Most machine learning algorithms assume equal data distribution. When the data distribution is not equal, it leads to imbalance. An imbalanced classification problem is a classification problem where the distribution of the dataset is skewed or biased. This method employs specialized techniques to change the composition of data samples.</p><p>Examples include:</p><ul><li>Fraud detection.</li><li>Outlier detection.</li><li>Medical diagnostic tests.</li></ul><pre># example of an imbalanced binary classification task<br>from numpy import where<br>from collections import Counter<br>from sklearn.datasets import make_classification<br>from matplotlib import pyplot<br># define dataset<br>X, y = make_classification(n_samples=1000, n_features=2, n_informative=2, n_redundant=0, n_classes=2, n_clusters_per_class=1, weights=[0.99,0.01], random_state=1)<br># summarize dataset shape<br># print(X.shape, y.shape)<br># summarize observations by class label<br>counter = Counter(y)<br># print(counter)<br># summarize first few examples<br>for i in range(10):<br> print(X[i], y[i])<br># plot the dataset and color the by class label<br>for label, _ in counter.items():<br> row_ix = where(y == label)[0]<br> pyplot.scatter(X[row_ix, 0], X[row_ix, 1], label=str(label))<br>pyplot.legend()<br>pyplot.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*6zD8BsikCtlSF0kiS2jzbA.png" /></figure><h3>Learners in Classification Problems</h3><h3>Lazy Learners</h3><p>Lazy learners store the training data and wait until a testing dataset appears. The primary aim of lazy learning is to continuously update the dataset with new entries. However, as the data is continually updated, it becomes outdated frequently. Thus, these algorithms take comparatively less time to train and more time to predict. Lazy learning algorithms are beneficial when working with large, changing datasets with a smaller set of queried attributes. Lazy learning is easy to maintain and can be applied to multiple problems.</p><h3>Eager Learners</h3><p>Eager learners construct a classification layer before receiving the training and testing the dataset. Before it observes the input queries, eager learning builds an explicit description of the training function based on the training data. Because it is building a classification model, eager learning takes more time to train the dataset and less time to predict as compared to the lazy learning system. Eager learning is required to commit to a single hypothesis that covers the entire instance space.</p><h3>Conclusion</h3><p>Writing this blog is one of the requirement on Arewa Data Science Academy, as a fellow, writing what we have learn during the week is a most, but putting together is blog, researching different site for info, really help my understanding on classification in machine learning. So here are some interesting site to check out.</p><p>Here: <a href="https://emeritus.org/blog/artificial-intelligence-and-machine-learning-classification-in-machine-learning">https://emeritus.org/blog/artificial-intelligence-and-machine-learning-classification-in-machine-learning</a></p><p>Also: <a href="https://machinelearningmastery.com/types-of-classification-in-machine-learning">https://machinelearningmastery.com/types-of-classification-in-machine-learning</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d4a9f56cae8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Cleaning: Python]]></title>
            <link>https://medium.com/@maryamsabubakar53/data-cleaning-python-593bc47a7b71?source=rss-1ad073d95cce------2</link>
            <guid isPermaLink="false">https://medium.com/p/593bc47a7b71</guid>
            <category><![CDATA[data-cleansing]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python-programming]]></category>
            <dc:creator><![CDATA[Maryam Abubakar]]></dc:creator>
            <pubDate>Sun, 21 May 2023 12:56:32 GMT</pubDate>
            <atom:updated>2023-05-21T12:56:32.756Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*-9pT8jLany3uKm-iRoVezg.jpeg" /><figcaption><a href="https://www.google.com/url?sa=i&amp;url=https%3A%2F%2Fwww.linkedin.com%2Flearning%2Fpython-vs-r-for-data-science%2Fdata-cleaning-and-manipulation-in-python-vs-r&amp;psig=AOvVaw36ucDrt-dYITIlEuAZE9b_&amp;ust=1684751476358000&amp;source=images&amp;cd=vfe&amp;ved=2ahUKEwiIusS7mob_AhWipicCHXCkBv8Qr4kDegUIARDCAQ">Source</a></figcaption></figure><p>Data cleaning means fixing or dropping bad data or unimportant data in your dataset. Data scientists spend a significant amount of effort cleansing datasets and transforming them into usable formats. Indeed, many data scientists claim that the initial steps of getting and cleansing data account for 80% of the work.</p><p>Bad data could be:</p><ul><li>Empty cells</li><li>Data in wrong format</li><li>Wrong data</li><li>Duplicates</li></ul><h3>First, check for missing dates</h3><p>After importing the necessary libraries.</p><p>You will first need to take steps to check for missing dates, we will be using the <a href="https://storage.googleapis.com/kaggle-data-sets/3259501/5670167/bundle/archive.zip?X-Goog-Algorithm=GOOG4-RSA-SHA256&amp;X-Goog-Credential=gcp-kaggle-com%40kaggle-161607.iam.gserviceaccount.com%2F20230521%2Fauto%2Fstorage%2Fgoog4_request&amp;X-Goog-Date=20230521T090727Z&amp;X-Goog-Expires=259200&amp;X-Goog-SignedHeaders=host&amp;X-Goog-Signature=6d9405e1866ec55143ee75e26f6b53f3467eeb55fa9c6ba511991c695f4a9c152f0555d2cfc07e94e6750136d0b85a9e9a1da43c4b83d97455ee619e70f90146ca3286f6fe5e4b62e60cd6d6fb52f065db475257fc0b53f97881a992e5cd4d578d1d39d1a78667cb387210f928851c86749dfc4f612e42e70ecc24801a520e945cc28d68856beb59472ec72cfa71445f9e7af403d4cbffffac6ef046b1cf602d652c8ceaeb17f9aa0ed7ec77d381fb720f60f3cd7e5702d9b5f6c733aa897db44471216310b0c02ce8a19630f3534932e30f96dadc3f80700e9cacd9df2e071d4395a69014578217d54617ed1f4ea0652cd02cd0fb0d38dacf8cbda9645daccc">df_arabica_clean</a> dataset. Which is rename as coffee.csv.</p><pre>import numpy as np<br>import pandas as pd<br>import matplotlib.pyplot as plt<br><br>coffee = pd.read_csv(&#39;coffee.csv&#39;)<br>coffee.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j09tlmXiRJA2j3QaL0XDQQ.png" /></figure><p>Now we can see the columns of this dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1Yv-LrL8liW8s_2K4YRSYQ.png" /></figure><p>The last part shows that we have 5 rows and 41 coloums, this is from the head() function. When we look at the tail() function we can see the last 5 rows of the dataset.</p><pre>coffee.tail()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AMZixnxVdG7oOUMXRGgZpQ.png" /></figure><p>To get all the information on the data, we use info() function. it will get you the general information you need.</p><pre>coffee.info()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t1olQ4Dh06Q4Pr4geWIG5g.png" /></figure><p>You can see our dataset has an object value and some integers.</p><p>We can also use the shape() function to simply knows the number of rows and columns in the dataset.</p><pre>coffee.shape</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/254/1*H0fVQ2H8AjP3Llociysr3A.png" /></figure><p>To check an empty cell or missing data.</p><pre>coffee.isnull().sum()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FLQwEW1GUHfIN19efmjaOw.png" /></figure><p>Now we can choose to drop some unnecessary data, that we don&#39;t need from our dataset with.</p><pre>to_drop = [&#39;Unnamed: 0&#39;,&#39;Harvest Year&#39;,<br>           &#39;Grading Date&#39;,&#39;Owner&#39;,<br>           &#39;Processing Method&#39;,&#39;Status&#39;,<br>           &#39;Aftertaste&#39;,&#39;Uniformity&#39;,<br>           &#39;Clean Cup&#39;,&#39;Sweetness&#39;,&#39;Overall&#39;,<br>           &#39;Acidity&#39;]<br><br>coffee.drop(to_drop, inplace=True, axis=1)</pre><p>This will remove the columns specified in bracket, so now you can check in info() function to see if the columns are dropped.</p><p>We can also make a new DataFrame from the dataset, those that consist of only the data we will be working with.</p><pre>new_columns = [&#39;Country of Origin&#39;, &#39;Company&#39;,&#39;Number of Bags&#39;, &#39;Moisture Percentage&#39;, &#39;Flavor&#39;, &#39;Color&#39;,&#39;Total Cup Points&#39;,&#39;Expiration&#39;]<br>coffee = coffee.drop([c for c in coffee.columns if c not in new_columns],axis=1)<br>coffee.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e1zCwce2O1NNu0Qn5PdBXA.png" /></figure><p>You can see now your dataset is more readable and neat.</p><p>Depending on what your goal is, a lot for things can be achieve with a clean and organize dataset. we can visualize this data with different charts or think of the story you want your data to tell.</p><p>Example.</p><pre>x = coffee[&#39;Color&#39;]<br>y = coffee[&#39;Flavor&#39;]<br><br>plt.barh(x,y)<br><br><br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/1*8536AG98f8luWtCAIlg1yw.png" /></figure><p>Thank you,</p><p>Happy practise.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=593bc47a7b71" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Choose the Right Data Visualization]]></title>
            <link>https://medium.com/@maryamsabubakar53/how-to-choose-the-right-data-visualization-ab17212459f1?source=rss-1ad073d95cce------2</link>
            <guid isPermaLink="false">https://medium.com/p/ab17212459f1</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[matplotlib]]></category>
            <dc:creator><![CDATA[Maryam Abubakar]]></dc:creator>
            <pubDate>Sun, 14 May 2023 16:52:50 GMT</pubDate>
            <atom:updated>2023-05-14T16:52:50.217Z</atom:updated>
            <content:encoded><![CDATA[<p>Data visualizations is the process of creating graphical representations of information and data. Data visualizations is an important component of a data analysis because it helps to communicate data in a way that’s easier for a viewer to interpret and draw conclusions.</p><blockquote>There are many chart types available, each with its own strengths and use cases. One of the trickiest parts of the analysis process is choosing the right way to represent your data using one of these visualizations. <a href="https://chartio.com/learn/charts/how-to-choose-data-visualization/">source</a></blockquote><p>The type of data visualization technique you are to use depends on the the type of data you have and the story you are want to tell or the information you want pass. There are numerous visual elements like charts, graphs, and maps to use.</p><p>Use the best chart to suit your data’s structure and the story you want to tell.</p><ul><li>To analyze trends over time: You can use line chart.</li><li>To compare values: You can use bar, column, pie and scatterplot.</li><li>To show how parts relate to a whole: You can use pie chart.</li><li>To show distribution of data: You can use scatterplot and bar chart.</li><li>To show trends: You can use line and column chart.</li><li>To show relationships between values: You can use line, scatterplot and bubble.</li></ul><h4>Visualization with Python</h4><p>The most common libraries use in python are Matplotlib and Seaborn. Each of these libraries needs to be install and import.</p><p>To install</p><pre>pip install Matplotlib <br>pip install Seaborn</pre><p>To import we use</p><pre>import pandas as pd<br>import matplotlib.pyplot as plt<br>import Seaborn as sns</pre><p>For these next examples we will be using tips database from <a href="http://www.geeksforgeeks.org/data-visualization-with-python/">here.</a></p><p><strong>Bar Chart:</strong> Bar charts represent numerical values that are compared to each other. The length of the bar represents the value of each variable.</p><pre>import pandas as pd<br>import matplotlib.pyplot as plt<br><br><br># reading the database<br>data = pd.read_csv(&quot;tips.csv&quot;)<br><br># Bar chart with day against tip<br>plt.bar(data[&#39;day&#39;], data[&#39;tip&#39;])<br><br>plt.title(&quot;Bar Chart&quot;)<br><br># Setting the X and Y labels<br>plt.xlabel(&#39;day&#39;)<br>plt.ylabel(&#39;tips&#39;)<br><br># Adding the legends<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/562/1*qbmr-JN4cyuuHCI4vPmDKw.png" /></figure><p><strong>Pie charts: </strong>Pie chart<strong> </strong>are ideal for illustrating proportions, or part-to-whole comparisons. For viewers who require a more thorough explanation of the data, pie charts fall short in their ability to display complex</p><p>Follow for more</p><p>Thank you</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ab17212459f1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting started with machine learning in python]]></title>
            <link>https://medium.com/@maryamsabubakar53/getting-started-with-machine-learning-in-python-93034401cb44?source=rss-1ad073d95cce------2</link>
            <guid isPermaLink="false">https://medium.com/p/93034401cb44</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-preparation]]></category>
            <category><![CDATA[pandas]]></category>
            <dc:creator><![CDATA[Maryam Abubakar]]></dc:creator>
            <pubDate>Sat, 29 Apr 2023 12:30:57 GMT</pubDate>
            <atom:updated>2023-04-29T12:30:57.749Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XLrAmGwW3m5DR-2Oaub5ww.png" /><figcaption>Working With Python — <em>Sketch note by </em><a href="https://twitter.com/nitya"><em>@nitya</em></a></figcaption></figure><p>Machine learning deals with a lot of Data and this data can come in many forms, which can either be tabular data, Plain Text or an image. All these kinds of data can be processed with python to give out meaningful information.</p><p>Machine learning is a branch of artificial intelligence that focuses on how a computer system can learn based on the amount of data it receives. Machine learning deals with a lot of data However, It could be impossible for raw data from its source to be free from inconsistency and hence complete. The task of cleaning up the data such that it is simple to use, consistent, and accurate is left to the data scientists. This will lead to data exploration, formatting, replacement and removing all duplicates, that’s where python and its libraries come in.</p><p>Python is a general-purpose programming language, its simplicity make it one of the best for beginners. Python has many additional libraries that can assist you in solving a variety of practical problems, such as extracting data from a ZIP archive or converting a picture to grayscale. In addition to data science, Python is also often used for web development.</p><p>Python has a lot of libraries that are useful in machine learning but the most useful of all are <strong>Panda </strong>and <strong>Numpy </strong>especially when working with Array, DataFrames or Tabular data. We also have <strong>Matplotlib </strong>which is used for visualization and plotting graphs and <strong>SciPy </strong>library<strong> </strong>for a lot of scientific functions.</p><p>All packages above need to be installed before importing them.</p><p>To Import use the following set of code:</p><pre>import numpy as np<br>import pandas as pd<br>import matplotlib.pyplot as plt<br>from scipy import ... # you need to specify exact sub-packages that you need</pre><p>Importing them as np, pd and plt are totally optional but is the standard that most programmers use.</p><blockquote>Pandas is a well-known open-source Python library that provides a wide range of capabilities to make data analysis more effective. The Pandas package is mostly utilized for pre-processing data activities like cleaning, transforming, and manipulating data. As a result, it is a very useful tool for analysts and data scientists. The two most popular data structures in Pandas — Series and DataFrame</blockquote><h4><strong>Pandas Series</strong></h4><p>Pandas Series are one-dimensional, Homogenous (Series elements must be of the same data type) and Size-immutable because once created, the size of the Series cannot be changed just like a List or a Numpy Array, the main difference is that series also has an index, and when we operate on series (eg., add them), the index is taken into account. Index can be as simple as integer row number or it can have a complex structure, such as date interval.</p><pre>import pandas<br><br>fruits = pandas.Series([&quot;mango&quot;,&quot;apples&quot;, &quot;oranges&quot;, &quot;bananas&quot;, &quot;watermelon&quot;])<br><br>print(&quot;Fruit series:&quot;)<br>print(fruits)<br><br>#FLOAT SERIES<br>temperature = pandas.Series([92.5, 22.11, 28.101, 5.965])<br><br>print(&quot;\nTemperature series:&quot;)<br>print(temperature)<br><br>#INTEGER SERIES<br>factors_of_24 = pandas.Series([1,2,3,4,6,8,12,24])<br><br>print(&quot;\nFactors of 24 series:&quot;)<br>print(factors_of_24)<br><br>print(&quot;Type of this data structure is:&quot;, type(factors_of_24))</pre><pre>Fruit series:<br>0         mango<br>1        apples<br>2       oranges<br>3       bananas<br>4    watermelon<br>dtype: object<br><br>Temperature series:<br>0    92.500<br>1    22.110<br>2    28.101<br>3     5.965<br>dtype: float64<br><br>Factors of 24 series:<br>0     1<br>1     2<br>2     3<br>3     4<br>4     6<br>5     8<br>6    12<br>7    24<br>dtype: int64<br>Type of this data structure is: &lt;class &#39;pandas.core.series.Series&#39;&gt;</pre><h4>Pandas DataFrame</h4><p>DataFrame is two-dimensional, Heterogeneous (DataFrame elements can have different data types) and size-mutable because elements can be added or dropped. We can say that DataFrame is a collection Series with the same index, we can also join several Series together to form DataFrame. There are several operations to be performed on a DataFrame.</p><pre>import pandas as pd<br><br>fruits_maryam = [&quot;mango&quot;, &quot;oranges&quot;, &quot;guavas&quot;]<br>fruits_khadija = [&quot;orange&quot;, &quot;pineapple&quot;, &quot;strawberries&quot;]<br>index = [&quot;A&quot;, &quot;B&quot;, &quot;C&quot;]<br>all_fruits = {&quot;Maryam&#39;s&quot;: fruits_maryam, &quot;Khadija&#39;s&quot;: fruits_khadija}<br><br>fruits_default_index = pd.DataFrame(all_fruits)<br>print(&quot;Dataframe with default indices:\n&quot;, fruits_default_index, &quot;\n&quot;)<br><br>new_fruits = pd.DataFrame(all_fruits, index = index)<br>print(&quot;Dataframe with given indices:\n&quot;, new_fruits, &quot;\n&quot;)</pre><pre>Dataframe with default indices:<br>       Khadija&#39;s Maryam&#39;s<br>0        orange    mango<br>1     pineapple  oranges<br>2  strawberries   guavas <br><br>Dataframe with given indices:<br>       Khadija&#39;s Maryam&#39;s<br>A        orange    mango<br>B     pineapple  oranges<br>C  strawberries   guavas </pre><h4>GETTING DATA</h4><p>Python has a great way of load data from the disk or from a website , for example reading data from the disk is as simple as this.</p><pre>df = pd.read_csv(&#39;file.csv&#39;)</pre><p>CSV stands for Comma-Separated Values and is a common way of storing formatted data. Using Jupiter notebook we can visualize the data in the file.csv but panda has additional functionality to view the the head in tabular form using df.head() .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AgF2e_rIayCMTJ3g_OlRQQ.png" /><figcaption>Data Preparation — <em>Sketch note by </em><a href="https://twitter.com/nitya"><em>@nitya</em></a></figcaption></figure><h3>Data Preparation</h3><p>After loading data the next thing is preparing the data. it’s necessary to convert <strong>raw data</strong> into a <strong>clean</strong> <strong>data</strong> set, that&#39;s to take care of missing data, duplicate data and inaccurate data, these enable data scientists to share data and reuse the data.</p><h4>Common cleaning goals and strategies</h4><ol><li>Exploring a dataset: this give you a general idea of the data by querying, visualization and sampling.</li><li>Formatting: data can have inconsistencies depending on where it comes from, so formatting problems can involve resolving whitespace, dates, and data types. These solved problem for searching and representing the values.</li><li>Duplications: duplicate data usually occurs when two or more dataset are joined together, these duplicate should be removed so as to produced accurate results.</li><li>Missing Data: Data isn’t always complete; that is, certain values may be missing or null. There are set of techniques for dealing with the missing data and completing the data.</li></ol><p>There are numerous ways of handling data inconsistences with python.</p><h4>Thank you.</h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=93034401cb44" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to Machine Learning]]></title>
            <link>https://medium.com/@maryamsabubakar53/introduction-to-machine-learning-43f2f2ca5a55?source=rss-1ad073d95cce------2</link>
            <guid isPermaLink="false">https://medium.com/p/43f2f2ca5a55</guid>
            <category><![CDATA[history-of-technology]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Maryam Abubakar]]></dc:creator>
            <pubDate>Sun, 02 Apr 2023 09:29:59 GMT</pubDate>
            <atom:updated>2023-04-02T09:29:59.609Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/826/1*66xwQPBd_elRigZi-Uk5-w.jpeg" /><figcaption><a href="https://www.google.com/url?sa=i&amp;url=https%3A%2F%2Fwww.freecodecamp.org%2Fnews%2Fwhat-is-machine-learning-for-beginners%2F&amp;psig=AOvVaw2M0kxj5wUDpdpBgRT9Wt5n&amp;ust=1680512744363000&amp;source=images&amp;cd=vfe&amp;ved=0CBEQjhxqFwoTCPC89vvriv4CFQAAAAAdAAAAABBQ">Machine learning</a></figcaption></figure><p>Machine learning (ML) is the subset of artificial intelligence (AI) that focuses on building systems with learning abilities so as to improve their performance based on historical data given to them. Artificial intelligence refers to systems or machines with human intelligence or show a level of human intelligence. Artificial intelligence (AI) and machine learning are often used interchangeably, but machine learning is a subset of Artificial intelligence. Machine learning focus on teaching a machine how to perform a specific task by providing data which gives accurate results by identifying patterns while Artificial intelligence is a field of computer science that makes a computer system that can mimic human intelligence.</p><p>The Artificial intelligence system or computer sometimes called Robots are not required to be pre-programmed, but they use machine learning algorithms and deep learning to make decisions.</p><p>Today, <a href="https://www.analyticsinsight.net/the-difference-between-artificial-intelligence-and-machine-learning/">machine learning</a> is at work all around us. We interact with banks, shop online, or use social media. Machine learning algorithms come into play to make our experience efficient, smooth, and secure. Machine learning and the technology around it are developing rapidly, and we’re just beginning to scratch the surface of its capabilities.</p><p><a href="https://www.salesforce.com/eu/blog/2020/06/real-world-examples-of-machine-learning.html"><strong>Machine Learning: Six Real-World Examples</strong></a></p><h4><strong>1. Image recognition</strong></h4><p>Image recognition is a well-known and widespread example of machine learning in the real world.</p><p>Real-world examples</p><ul><li>Label an x-ray as cancerous or not</li><li>Assign a name to a photographed face (aka “tagging” on social media)</li><li>Recognize handwriting by segmenting a single letter into smaller images</li></ul><h4>2. Speech recognition</h4><p>Machine learning can translate speech into text. Certain software applications can convert live voice and recorded speech into a text file.</p><p>Real-world examples</p><ul><li>Voice search</li><li>Voice dialing</li><li>Appliance control</li></ul><p>Some of the most common uses of speech recognition software are devices like Google Home or Amazon Alexa.</p><h3>3. Medical diagnosis</h3><p>Machine learning can help with the diagnosis of diseases. Many physicians use chatbots with speech recognition capabilities to discern patterns in symptoms.</p><p>Real-world examples</p><ul><li>Assisting in formulating a diagnosis or recommends a treatment option</li><li>Oncology and pathology use machine learning to recognize cancerous tissue</li><li>Analyze bodily fluids</li></ul><h3>4. Statistical arbitrage</h3><p>Arbitrage is an automated trading strategy that’s used in finance to manage a large volume of securities.</p><p>Real-world example:</p><ul><li>Algorithmic trading which analyzes a market microstructure</li><li>Analyze large data sets</li><li>Identify real-time arbitrage</li></ul><h3>5. Predictive analytics</h3><p>Machine learning can classify available data into groups, which are then defined by rules set by analysts. When the classification is complete, the analysts can calculate the probability of a fault.</p><p>Real-world examples:</p><ul><li>Predicting whether a transaction is fraudulent or legitimate</li><li>Improve prediction systems to calculate the possibility of fault</li></ul><p>Predictive analytics is one of the most promising examples of machine learning. It’s applicable for everything; from product development to real estate pricing.</p><h3>6. Extraction</h3><p>Machine learning can extract structured information from unstructured data. Organizations amass huge volumes of data from customers. A machine learning algorithm automates the process of annotating datasets for predictive analytics tools.</p><p>Real-world examples of extraction:</p><ul><li>Generate a model to predict vocal cord disorders</li><li>Develop methods to prevent, diagnose, and treat the disorders</li><li>Help physicians diagnose and treat problems quickly</li></ul><p>Typically, these processes are tedious. But machine learning can track and extract information to obtain billions of data samples.</p><h4><strong>History of Machine learning</strong></h4><p>The history of machine learning begins in 1943 with the publication of the first mathematical model of neural network by Welter Pitts and Warren McCulloch in their academic article, “A logical calculus of the ideas immanent with nervous activity”.</p><p>The Turing Test, developed by Alan Turing in 1950, is used to assess a computer’s level of intelligence. A computer needs to be able to trick a person into thinking it is also a person in order to pass the test.</p><p>The belief that AI could fix a wide range of issues peaked in the 1950s and lasted until the middle of the 1970s. Marvin Minsky asserted in 1967 that “the problem of creating ‘artificial intelligence’ will substantially be solved within a generation.” Marvin Minsky’s Computation: Finite and Infinite Machines was published in 1967 by Prentice-Hall in Englewood Cliffs, New Jersey.</p><p>By the 1970s activity on Artificial intelligence was low because of reasons including limited computing power, some existing programs were challenged and limited amounts of data.</p><p>Work on machine learning evolved from a knowledge-driven to a data-driven approach in the 1990s. Scientists began developing computer systems to evaluate massive volumes of data to make conclusions as well as learn from the results.</p><p>In 1997, IBM’s Deep Blue stunned the world by defeating the world chess champion.</p><p>Geoffrey Hinton first used the phrase “deep learning” in 2006 to describe new methods that enable computers to “see” and recognize objects and text in photos and videos. With the introduction of smartphones in about 2007, the volume of data started to grow quickly and became more accessible. Algorithms changed quickly as compute power increased. The once-wild days of the past started to become a true discipline as the profession started to mature.</p><p>Microsoft claimed in 2010 that their Kinect technology could track 20 human features at a rate of 30 times per second, enabling users to connect with the computer through gestures and movements. The next year, Watson from IBM defeated Jeopardy’s human opponents.</p><p>Facebook created DeepFace in 2014, a computer system that can detect or authenticate people in photographs to the same degree as humans.</p><p>Almost every aspect of our lives is now impacted by machine learning and AI. On this day, it is crucial to fully comprehend the dangers and potential repercussions that these algorithms may have on people’s lives. According to Brad Smith, a Microsoft employee, “Information technology creates concerns that go to the core of fundamental human-rights protections like privacy and freedom of expression. For tech businesses that produce these products, these problems increase responsibility. They also, in our opinion, demand the creation of standards on their usage and careful government regulation</p><blockquote><a href="https://www.lightsondata.com/the-history-of-machine-learning/#:~:text=Machine%20learning%20history%20starts%20in,by%20Donald%20Hebb%20is%20published.">2015</a> — Present day</blockquote><blockquote>Amazon launched its own machine learning platform in 2015. Microsoft also created the Distributed Machine Learning Toolkit, which enabled the efficient distribution of machine learning problems across multiple computers.</blockquote><blockquote>Then more 3,000 AI and Robotics researchers, endorsed by Stephen Hawking, Elon Musk and Steve Wozniak (among many others), signed an open letter warning of the danger of autonomous weapons which select and engage targets without human intervention.</blockquote><blockquote>In 2016 Google’s artificial intelligence algorithm beat a professional player at the Chinese board game Go, which is considered the world’s most complex board game and is many times harder than chess. The AlphaGo algorithm developed by Google DeepMind managed to win five games out of five in the Go competition.</blockquote><blockquote>Waymo started testing autonomous cars in the US in 2017 with backup drivers only at the back of the car. Later the same year they introduced completely autonomous taxis in the city of Phoenix.</blockquote><blockquote>In 2020, while the rest of the world was in the grips of the pandemic, open AI announced a ground-breaking natural language processing algorithm GPT-3 with a remarkable ability to generate human-like text when given a prompt. Today, GPT-3 is considered the largest and most advanced language model in the world, using 175 billion parameters and Microsoft Azure’s AI supercomputer for training</blockquote><h4><strong>Fairness in Machine Learning</strong></h4><p>Fairness in machine learning refers to numerous approaches to correct algorithmic bias in machine learning-based automated decision systems. Decisions made by computers during a machine-learning process may be considered unjust if they are based on sensitive variables. Definitions of fairness and bias, like many other ethical ideas, are always contentious. When a choice affects people’s lives, fairness and bias are deemed important. The topic of algorithmic bias in machine learning is well known and well studied. Outcomes may be skewed by a variety of circumstances and thus be regarded unfair in relation to specific groups or individuals. One example is how social media sites give customized news to users.</p><p>Thank you for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=43f2f2ca5a55" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>