<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Bhanumathi Ramesh on Medium]]></title>
        <description><![CDATA[Stories by Bhanumathi Ramesh on Medium]]></description>
        <link>https://medium.com/@bhanu0925?source=rss-4dfd629c989b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*22oMLDKZnlb1F-01d_-vcw.jpeg</url>
            <title>Stories by Bhanumathi Ramesh on Medium</title>
            <link>https://medium.com/@bhanu0925?source=rss-4dfd629c989b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 04:54:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bhanu0925/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Evolution of Natural Language Processing]]></title>
            <link>https://medium.com/@bhanu0925/evaluation-of-natural-language-processing-845660fbc26?source=rss-4dfd629c989b------2</link>
            <guid isPermaLink="false">https://medium.com/p/845660fbc26</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Bhanumathi Ramesh]]></dc:creator>
            <pubDate>Sun, 27 Mar 2022 18:53:54 GMT</pubDate>
            <atom:updated>2022-03-27T18:59:46.241Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1vQDUPSssosjR8bCMu8WYg.jpeg" /></figure><p><strong>1950</strong>: Alan Turing wrote a paper describing a test for a “thinking” machine. He stated that if a machine could be part of a conversation using a teleprinter, and it imitated a human so completely that there are no noticeable differences, then the machine could be considered capable of thinking.</p><p><strong>1952</strong>: T<a href="https://www.swarthmore.edu/NatSci/echeeve1/Ref/HH/HHmain.htm">he Hodgkin-Huxley model</a> showed how the brain uses neurons in forming an electrical network. These events helped inspire the idea of Artificial Intelligence (AI), <a href="http://www.dataversity.net/fundamentals-natural-language-processing-natural-language-generation/">Natural Language Processing</a> (NLP), and the evolution of computers.</p><p>NLP was not born overnight, it has evolved as time and technology advanced and continues to evolve today. The below diagram highlights some of the significant algorithms created along the NLP journey.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yFZBXOZqxjpsxsq3.png" /></figure><p>NLP started with very simple models like the <strong>Bag-of-Words</strong> model which simply counts the occurrence of each word within a document. However, real-world applications have the corpora of millions of documents that may have millions of unique words within them. It became difficult to represent the count of each word in such a large matrix, and if you haven’t tried it, the results are quickly dominated by common words such as “is”, “a”, “the”, etc. This led to the introduction of another technique called <strong>TF-IDF </strong>which eliminated the common words or “stop words”. As these techniques did not include semantic relationships between words, another advanced technique, <strong>Co-occurrence Matrix, </strong>was created. This technique generated word embeddings to track the sentiment or the context of the text. Co-occurrence matrices required a large amount of memory to store the resulting matrix, which eventually led to other more advanced techniques involving neural networks.</p><p><strong>Word2Vec </strong>was among the first prediction-based modeling techniques in NLP. It included techniques that are frequently used today such as <strong>Skip Gram </strong>and<strong> Continuous Bag of Words. </strong>These techniques leverage neural networks as their foundation and consider the semantics of the text. They also use a <strong>FastText </strong>algorithm (developed by Facebook), that uses character-level information to generate the text representation. The word is considered as a bag of character n-grams in addition to the word itself.</p><p>ELMo came into existence with a key idea of solving the problem of homonyms in word representation. Take these sentences, for example, #1, “I like to play cricket” and #2, “I am watching the Julius Cesar play”. The word ‘play’ has different meanings.</p><p>Along with these developments, also came the <strong>Transformer Models </strong>in the form of encoders and decoders. It is a model that uses attention to boost the speed of training and outperformed the Google Neural Machine Translation model in specific tasks. Similarly, BERT (Bidirectional Encoder Representations from Transformers) uses encoder representations of the transformer network and has marked a new era in NLP by breaking several records in handling language-based tasks.</p><p>Recently, researchers from Carnegie Mellon &amp; Google developed an Attention Network-based architecture called the <strong>XLNet</strong>. Published in 2019, it claims to outperform BERT in 20 different tasks. At the same time, a Chinese company published another Attention Network-based model called ERNIE 2.0, they claim to outperform both BERT and XLNet in 16 different tasks.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=845660fbc26" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Natural Language Processing(NLP) — An Introduction]]></title>
            <link>https://medium.com/@bhanu0925/the-natural-language-processing-nlp-an-introduction-cd562e725873?source=rss-4dfd629c989b------2</link>
            <guid isPermaLink="false">https://medium.com/p/cd562e725873</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[arificialinterlligence]]></category>
            <dc:creator><![CDATA[Bhanumathi Ramesh]]></dc:creator>
            <pubDate>Thu, 25 Nov 2021 20:05:30 GMT</pubDate>
            <atom:updated>2021-11-25T20:05:30.672Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Natural Language Processing(NLP) — An Introduction</h3><p><em>A different language is a different vision of life…..</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a6t2Ty0LyZHf9VGGATBGkg.png" /></figure><p>Before we dive deep into Natural Language Processing and its techniques and implementation, it is important to know the overview of the topic.</p><p><strong><em>What is NLP?</em></strong></p><p>According to Wikipedia,</p><blockquote><strong>Natural language processing</strong> (<strong>NLP</strong>) is a subfield of <a href="https://en.wikipedia.org/wiki/Linguistics">linguistics</a>, <a href="https://en.wikipedia.org/wiki/Computer_science">computer science</a>, and <a href="https://en.wikipedia.org/wiki/Artificial_intelligence">artificial intelligence</a> concerned with the interactions between computers and human language, in particular how to program computers to process and analyze large amounts of <a href="https://en.wikipedia.org/wiki/Natural_language">natural language</a> data.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/239/1*N0MWVgpG45K1RljCR0Tabw.png" /><figcaption>Natural Language Processing</figcaption></figure><p><strong><em>Need for NLP</em></strong></p><p>Natural Language Processing (NLP) is a branch of Artificial Intelligence (AI) that enables machines to understand human language. Its goal is to build systems that can make sense of the text and automatically perform tasks like translation, spell check, or topic classification. NLP has changed the way we interact with computers and it’ll continue doing so in the future. These AI technologies will be the underlying force for transformation from data-driven to intelligence-driven endeavors, as they shape and improve communication technology in the years to come.</p><p><strong><em>Real-World Applications of NLP</em></strong></p><p>Below are some of the real-world examples of NLP but not limited</p><ul><li>Contextual Advertisements</li><li>Email Clients — spam filtering, smart reply</li><li>Social Media — removing adult content, hate speech, opinion mining</li><li>Search Engines</li><li>Chatbots</li></ul><p><strong><em>Common NLP tasks</em></strong></p><ul><li><strong>Text/Document Classification</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/313/1*15hoLqc30OkHpo23URyv4w.png" /><figcaption>Text/Document Classification</figcaption></figure><p>Document classification is an example of Machine Learning (ML) in the form of Natural Language Processing (NLP). By classifying text, we are aiming to assign one or more classes or categories to a document, making it easier to manage and sort. This is especially useful for publishers, news sites, blogs, or anyone who deals with a lot of content.</p><ul><li><strong>Sentiment Analysis</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*AarBoewnhYz0Qxjo1x52NA.png" /><figcaption>Sentiment Analysis</figcaption></figure><p>Sentiment analysis also referred to as opinion mining, is an approach to natural language processing (<a href="https://searchbusinessanalytics.techtarget.com/definition/natural-language-processing-NLP">NLP</a>) that identifies the emotional tone behind a body of text. This is a popular way for organizations to determine and categorize opinions about a product, service, or idea.</p><ul><li><strong>Information Retrieval</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*SprYF3uhNAWdGUVPMeizJQ.jpeg" /><figcaption>Information Retrieval</figcaption></figure><p>Information retrieval (IR) may be defined as a software program that deals with the organization, storage, retrieval, and evaluation of information from document repositories, particularly textual information. The system assists users in finding the information they require but it does not explicitly return the answers to the questions. It informs the existence and location of documents that might consist of the required information</p><ul><li><strong>Part of Speech Tagging</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/414/1*GYxwIF4CIdB2hl0W2Gl4vw.png" /><figcaption>Part of Speech Tagging</figcaption></figure><p>Part-of-speech (POS) tagging is a popular Natural Language Processing process that refers to categorizing words in a text (corpus) in correspondence with a particular part of speech, depending on the definition of the word and its context</p><ul><li><strong>Language Detection and Machine Translation</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*CZGXjdZYiMCYfgJpWfZHyA.png" /><figcaption>Language Detection and Machine Translation</figcaption></figure><p>Machine translation is the process of translating a text in one natural language into another natural language using a computer system. Translating a document containing a single source language contents is easy but when the information in the source document is given in multilingual format then there is a need to identify the languages that are involved in such multilingual document. Language identification is the task in natural language processing that automatically identifies the natural language in which the content in the given document is written. Language identification is the fundamental and crucial step in many NLP applications.</p><ul><li><strong>Conversational Agents</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/295/1*0BUItDIok6pJWPp2Tq5gKA.png" /><figcaption>Conversational Agents</figcaption></figure><p>Conversational agents, also known as chatbots, are computer programs designed to simulate human text or verbal conversations</p><ul><li><strong>Knowledge Graph and QA Systems</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*vQn3eN3iHRH_akKJO8Yhng.jpeg" /><figcaption>Knowledge Graph and QA Systems</figcaption></figure><p>An automated question answering system applies a set of NLP techniques to analyze unstructured documents — from Wikipedia articles to social media newsfeed or medical records — by retrieving the needed information piece, analyzing it, and using the best part to answer the question</p><ul><li><strong>Text Summarization</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/328/1*kZNbydQ-t-vhQNu_SGU2HA.png" /><figcaption>Text Summarization</figcaption></figure><p>Text summarization is the technique for generating a concise and precise summary of voluminous texts while focusing on the sections that convey useful information, and without losing the overall meaning</p><ul><li><strong>Topic Modelling</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/269/1*uIJA1IsOF1j3_yN7zcKZjQ.png" /><figcaption>Topic Modelling</figcaption></figure><p>Topic Modelling is the task of using unsupervised learning to extract the main topics (represented as a set of words) that occur in a collection of documents.</p><ul><li><strong>Text Generation</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/429/1*dBx61Du2q5ZBi25lEf0tsQ.png" /><figcaption>Text Generation</figcaption></figure><p>Text generation is a subfield of natural language processing. It leverages knowledge in computational linguistics and artificial intelligence to automatically generate natural language texts, which can satisfy certain communicative requirements.</p><ul><li><strong>Spell Checking and Grammar Correction</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/299/1*b3cZbKZs_kcs9Q0blEYDTw.png" /><figcaption>Spell Checking and Grammar Correction</figcaption></figure><p>The spell checker compares every word typed with a list of thousands of correctly spelled, words and then uses algorithms to determine the correct spellings</p><ul><li><strong>Text Parsing</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/299/1*XT_1Hkq4lnApn4PMq-b_Sg.png" /><figcaption>Text Parsing</figcaption></figure><p>Text Parsing is the process of analyzing text made of a sequence of tokens to determine its grammatical structure concerning a given (more or less) formal grammar.</p><ul><li><strong>Speech to Text</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/298/1*w2ox_e2S3pxZ61o5pOnW3g.jpeg" /><figcaption>Speech to Text</figcaption></figure><p>Natural Language Processing speech to text is a profound application of Deep Learning which allows the machines to understand human language and read it with a motive to act and react, as usual, humans do.</p><p><strong><em>Approaches to NLP</em></strong></p><p><strong><em>Rule-Based — Heuristic Approaches</em></strong></p><p>Rule-based approaches are the oldest approaches to NLP. They are still used s because they are tried and true, and have been proven to work well. Rules applied to text can offer a lot of insight. Examples of rule-based NLP</p><ul><li>Regular Expressions</li><li>Wordnet</li><li>Open Mind Common Sense</li></ul><p><em>Advantages:</em></p><p>It offers high-performance cases when used specifically</p><p><em>Disadvantages:</em></p><p>The rule-based approach needs experts to create the Flexi rules and operate</p><p><strong><em>Machine Learning Methods</em></strong></p><p>Machine Learning (ML) approach is an algorithm-based system that aids the machine to learn and understand the language by using statistical methods. The system here does not require explicit programming, it starts analyzing the already interpreted data or the training set (annotated corpus) to develop its learnings, create its framework and its classifiers.</p><p><em>Algorithms Used:</em></p><ul><li>Naive Bayes</li><li>Logistic Regression</li><li>SVM</li><li>LDA</li><li>Hidden Markov Models</li></ul><p><em>Disadvantages:</em></p><ul><li>Sequential information of text data is lost when they are converted to numerical data.</li><li>Feature engineering has to be made before training.</li></ul><p><strong><em>Deep Learning Methods</em></strong></p><p>To overcome the disadvantages of Machine Learning models, like feature engineering and preserving sequential data, Deep learning models are used in NLP.</p><p><em>Architectures Used:</em></p><ul><li>RNN</li><li>LSTM</li><li>GRU/CNN</li><li>Transformers</li><li>Autoencoders</li></ul><p><strong><em>Major Natural Language Processing (NLP) Challenges</em></strong></p><p>NLP is a powerful tool with huge benefits, but there are still several Natural Language Processing limitations and problems:</p><ul><li>Contextual words and phrases and homonyms</li><li>Synonyms</li><li>Irony, sarcasm, and tonal diff</li><li>Ambiguity</li><li>Errors in text or speech</li><li>Colloquialisms and slang</li><li>Creativity</li><li>Diversity</li><li>Domain-specific language</li><li>Low-resource languages</li><li>Lack of research and development</li></ul><p><strong><em>Conclusion</em></strong></p><p>In this article, we have discussed an overview of Natural language processing, its real-world application, approaches, and challenges.</p><p>LinkedIn: <a href="https://www.linkedin.com/in/bhanumathiramesh/"><strong><em>Bhanumathi Ramesh</em></strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cd562e725873" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Must know Regression Matrics of Machine Learning]]></title>
            <link>https://medium.com/@bhanu0925/must-know-regression-matrics-of-machine-learning-41aba63b54f4?source=rss-4dfd629c989b------2</link>
            <guid isPermaLink="false">https://medium.com/p/41aba63b54f4</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[machinelearningalgorithms]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Bhanumathi Ramesh]]></dc:creator>
            <pubDate>Thu, 11 Nov 2021 09:32:03 GMT</pubDate>
            <atom:updated>2021-11-11T09:32:03.230Z</atom:updated>
            <content:encoded><![CDATA[<p><em>What’s measured improves…</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*GDweVH5O80kgHBGP0Pyi2Q.jpeg" /></figure><p>In this article, we will see the most widely used regression metrics. There are several metrics available in <a href="https://scikit-learn.org/stable/modules/classes.html#regression-metrics">scikit-learn</a> for regression, we will see a few of them here. We can understand these matrics in two different categories, they are,</p><ol><li>Metrics to measure the errors</li><li>Metrics to measure the proportion of variance explained</li></ol><p><strong><em>Error Metrics</em></strong></p><p>Three error metrics are commonly used for measuring the errors(also referred to as residuals) for evaluating and reporting the performance of a regression model; they are:</p><ol><li>Mean Absolute Error(MAE)</li><li>Mean Squared Error(MSE)</li><li>Root Mean Squared Error(RMSE)</li></ol><p>There are many other metrics for regression, although these are the most commonly used. You can see the full list of regression metrics supported by the scikit-learn Python machine learning library here:</p><p><a href="https://scikit-learn.org/stable/modules/classes.html#regression-metrics">Scikit-Learn API: Regression Metrics</a>.</p><p>The best fit line is obtained by minimizing the <em>residual</em>. Residual is the distance between the actual Y and the predicted Y, as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*d_xDmqu_Ej_t-BkqmrFvyQ.png" /><figcaption>Training data and predicted points</figcaption></figure><p><strong><em>Mean Absolute Error (MAE):</em></strong></p><p>MAE measures the average magnitude of the errors in a set of predictions, without considering their direction.</p><p>The formula is given by,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ll49lIuBZz5VHCWdow6Vsg.jpeg" /><figcaption>MAE</figcaption></figure><p>where,</p><p>Y: training data pint</p><p>Y-hat: predicted data point</p><p>n: total number of observations(rows)</p><p><strong><em>Advantages of MAE</em></strong></p><ol><li>It expresses average model prediction error in the units of the variable of interest. In other words, MAE is aligned with the scale of the target variable. For example, if the target variable is in the scale kilogram, MAE expressed will be in kilogram.</li><li>MAE is robust to outliers.</li></ol><p><strong><em>Disadvantages of MAE</em></strong></p><ol><li>Since MAE uses the modulus function, not a <a href="https://en.wikipedia.org/wiki/Differentiable_function"><em>differentiable function</em></a>, which is not differentiable at 0, optimizing algorithms like gradient descents can not be applied.</li></ol><p><strong><em>Mean Squared Error (MSE):</em></strong></p><p>MSE or mean squared deviation (MSD) of an estimator measures the average of the squares of the errors. The fact that MSE is almost always strictly positive (and not zero).</p><p>This is also referred to as <strong><em>L2 loss</em></strong> for error calculated for a single observation.</p><p>The formula is given by,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gRHLoig52uZp7LYieKZVrA.jpeg" /><figcaption>MSE</figcaption></figure><p><strong><em>Advantages of MSE:</em></strong></p><ol><li>Since the MSE is a square function it is differentiable at 0 and optimizing algorithms like gradient descents can be applied.</li><li>Units of the errors will be squared and difficult to interpret with target variables.</li></ol><p><strong><em>Disadvantages of MSE:</em></strong></p><ol><li>It is not robust to outliers.</li></ol><p><strong><em>Root Mean Squared Error(RMSE)</em></strong></p><p>Root mean squared error (RMSE) is <em>the square root of the mean of the square of all of the errors</em>. RMSE is a good measure of accuracy, but only to compare prediction errors of different models or model configurations for a particular variable and not between variables, as it is scale-dependent.</p><p>The formula is given by,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3hgiUC_cgUG-bGbcIuumrA.jpeg" /><figcaption>RMSE</figcaption></figure><p>The advantages and disadvantages are the same as MAE.</p><p><strong><em>Metrics to measure the proportion of variance explained</em></strong></p><p><strong><em>R-Squared score :</em></strong></p><p>R-squared is <strong>“</strong><em>a goodness-of-fit measure for linear regression models</em><strong>”</strong>. This statistic indicates the percentage of the variance in the dependent variable that the independent variables explain collectively. R-squared is also referred to as <strong><em>Coefficient of Determination.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*1NeNVHRMzQwZ67qzrjkN9Q.png" /><figcaption>R-Squared mathematical intuition</figcaption></figure><p>The formula for R-squared would be,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hd44l8kyW36XdHc6tXjTzg.jpeg" /></figure><p>R -Squared score ranges from (0–1). There are exception cases R squared can also go to -ve values, let us see the following cases.</p><p><strong>R- Squared = 0</strong>, this implies, the Regression line is passing through the mean line.</p><p><strong>R- Squared = 1</strong>, this implies, the Regression line is passing exactly through all the data points, which is impossible in practical scenarios.</p><p><strong>R- Squared = -ve</strong>, This happens when SSR is greater than SST, meaning, the Regression line produces more error than the mean line. One of the reasons for getting <em>-ve R-squared</em> values could be applying Linear Regression on highly nonlinear data.</p><p>Suppose if we have R-squared = 0.80, this will be read as , The amount of variance of {dependent variable} is explained by {independent variable/s}</p><p><strong><em>Disadvantages of R-Squared</em></strong></p><ol><li>R-squared score remains the same or increases upon adding the insignificant feature.</li><li>Does not provide information about feature importance.</li><li>It makes the model biased with insignificant features.</li></ol><p>To overcome mentioned disadvantages Adjusted R-squared was introduced.</p><p><strong><em>Adjusted R-squared score:</em></strong></p><p>Adjusted R-squared is a<strong> </strong>modified version of R-squared that has been adjusted for the number of predictors in the model.</p><p>The formula is given by,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jim1PMHn9KXF5Mnonc6ZrA.jpeg" /><figcaption>Adjusted R-Squared</figcaption></figure><p>where,</p><p>n: total number of observations</p><p>k: number of independent variables(predictors)</p><p><strong><em>Case1: Adding an insignificant column</em></strong></p><p>When an insignificant column is added, assuming R-Squared values remain constant, the denominator values decrease(n-1-k, decreases as k is increased). We can see in the below fig how the terms are getting affected with the addition of insignificant variables. Hence the Adjusted R-squared value always decreases upon adding insignificant variables.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l1yf6U5G5ZSGQP661la40w.jpeg" /></figure><p><strong><em>Case2: Adding a significant column</em></strong></p><p>When a significant column is added, the R-Squared value increases. We can see in the below fig how the terms are getting affected with the addition of significant variables. Hence the Adjusted R-squared value always increases upon adding significant variables.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BzRWZggv9-VW3ngDeUkKjw.jpeg" /></figure><p>The adjusted R-squared increases when the new term improves the model more than would be expected by chance. It decreases when a predictor improves the model by less than expected.</p><p><strong><em>Conclusion</em></strong></p><p>In this article, we discussed different metrics that are widely used while solving regression problems, Root Mean Squared Error(RMSE) is the error matric that is widely used in the field of deep learning as well as machine learning. R-Squared works well with simple linear regression but when it comes to multiple regression adjusted R-squared gives us a reliable score.</p><p>LinkedIn: <a href="https://www.linkedin.com/in/bhanumathiramesh/"><strong><em>Bhanumathi Ramesh</em></strong></a></p><p>Also See, <a href="https://medium.com/@bhanu0925/deriving-equation-for-simple-linear-regression-ols-method-44d4f7d38dc1"><em>Simple Linear Regression</em></a>, <a href="https://medium.com/@bhanu0925/deriving-normal-equation-for-multiple-linear-regression-85241965ee3b"><em>Multiple Linear Regression</em></a>, <a href="https://medium.com/@bhanu0925/gradient-descents-batch-stochastic-and-minibatch-a5e5186abeba">Gradient descents</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=41aba63b54f4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gradient Descents — Batch, Stochastic, and Minibatch]]></title>
            <link>https://medium.com/@bhanu0925/gradient-descents-batch-stochastic-and-minibatch-a5e5186abeba?source=rss-4dfd629c989b------2</link>
            <guid isPermaLink="false">https://medium.com/p/a5e5186abeba</guid>
            <category><![CDATA[machinelearningalgorithms]]></category>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Bhanumathi Ramesh]]></dc:creator>
            <pubDate>Sat, 06 Nov 2021 17:34:32 GMT</pubDate>
            <atom:updated>2021-11-06T17:48:34.799Z</atom:updated>
            <content:encoded><![CDATA[<h3>Gradient Descents — Batch, Stochastic, and Minibatch</h3><p><em>An act of moving downwards</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*9jPF2ObGrzzYrtm-C493pA.png" /><figcaption>Gradient Descents</figcaption></figure><p>In this article, we will see different types of Gradient descents and how they are different.</p><p><strong><em>What is Gradient Descent?</em></strong></p><p>In mathematics, the gradient is <strong>the measure of the steepness of a straight line</strong>. A gradient can be uphill in direction (from left to right) or downhill in direction (from right to left). Gradients can be positive or negative and do not need to be a whole number.</p><p>According to Wikipedia, gradient descent is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps into the opposite direction of the gradient of the function at the current point because this is the direction of steepest descent. Please find more on Wikipedia about <a href="https://en.wikipedia.org/wiki/Gradient_descent"><em>Gradient Descent</em></a></p><p><strong><em>Use of Gradient Descent:</em></strong></p><p>Gradient descent is majorly used in the following,</p><ol><li>Linear Regression</li><li>Logistic Regression</li><li>T- SNE</li><li>Deep learning(Backbone of deep learning)</li></ol><p><strong><em>Types of Gradient Descents:</em></strong></p><p>There are majorly three types of gradient descent that are widely used,</p><ol><li>Batch Gradient Descent</li><li>Stochastic Gradient Descent</li><li>Mini batch Gradient Descent</li></ol><p>To train a Linear Regression model, we have to learn some model parameters such as feature weights and bias terms(beta coefficients). An approach to do the same unlike the OLS method is <strong><em>Gradient Descent</em></strong> which is an iterative optimization algorithm capable of tweaking the model parameters by minimizing the cost function over the train data. It is a complete algorithm i.e it is guaranteed to find the global minimum (optimal solution) given there is enough time and the learning rate is not very high.</p><p><strong>Batch Gradient Descent:</strong> Batch Gradient Descent considers complete training data set for calculating gradients (model parameters) in each iteration. Batch Gradient descent is applied to algorithms that use convex cost functions.</p><p><strong>Stochastic Gradient Descent</strong>: Known as SGD is stochastic i.e it picks up a “random” instance of training data at each step and then computes the gradient. Stochastic Gradient descent is applied to algorithms that use non-convex cost functions.</p><p><strong><em>Mini Batch Gradient Descent: </em></strong>Mini-batch gradient descent is a<strong> </strong>variation of the gradient descent algorithm that splits the training dataset into small batches that are used to model coefficients. This is widely used in deep learning.</p><p><strong>Difference between Batch Gradient Descent and Stochastic Gradient Descent</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/969/1*5paFAwGI4M1CwyBsZefC_Q.png" /><figcaption>Difference between Batch and Stochastic gradient descents</figcaption></figure><p><strong><em>Conclusion:</em></strong></p><p>Gradient descent works in spaces of any number of dimensions, even in infinite-dimensional ones. The gradient descent can take many iterations to compute a local minimum with the required accuracy if the <a href="https://en.wikipedia.org/wiki/Curvature">curvature</a> in different directions is very different for the given function. Gradient descent also suffers from issues like choosing optimal learning rate, saddle point, and also fluctuation at global minima. In the upcoming post, we will see how to derive equations for batch, stochastic and mini-batch algorithms, and see how to overcome saddle point, fluctuation problems at global minima.</p><p>LinkedIn: <a href="https://www.linkedin.com/in/bhanumathiramesh/"><strong><em>Bhanumathi Ramesh</em></strong></a></p><p>Also See, <a href="https://medium.com/@bhanu0925/deriving-equation-for-simple-linear-regression-ols-method-44d4f7d38dc1"><em>Simple Linear Regression</em></a>, <a href="https://medium.com/@bhanu0925/deriving-normal-equation-for-multiple-linear-regression-85241965ee3b"><em>Multiple Linear Regression</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a5e5186abeba" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deriving Normal Equation for Multiple Linear Regression]]></title>
            <link>https://medium.com/@bhanu0925/deriving-normal-equation-for-multiple-linear-regression-85241965ee3b?source=rss-4dfd629c989b------2</link>
            <guid isPermaLink="false">https://medium.com/p/85241965ee3b</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[linear-algebra]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Bhanumathi Ramesh]]></dc:creator>
            <pubDate>Tue, 19 Oct 2021 15:17:53 GMT</pubDate>
            <atom:updated>2021-10-19T15:17:53.729Z</atom:updated>
            <content:encoded><![CDATA[<p>The mathematical formulation of Normal Equation</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zIkwmnaTDYiLxBTuyyA8vA.jpeg" /></figure><p>In this article, we will be seeing the steps to derive the normal equation for multiple linear regression.</p><p><strong><em>What is Multiple Linear Regression?</em></strong></p><p>Multiple Linear Regression is a supervised machine learning algorithm. This algorithm uses more than one independent variable to predict a target variable. The below table is an example of a multiple linear regression problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/442/1*B_gsf37e1So0drDMIo0ssA.png" /><figcaption>Table with 4 independent variables and 1 dependent variable</figcaption></figure><p>The above table has marks details and SAT score of 5 students. Each subject is a independent variable( as called predictors,explanatory variable or regressor (in regression problem)) and one dependent variable (also called label,target,response variable). Since we have more than one independent variable we have to find the <strong>coefficients </strong>for each independent variable. In my <a href="https://medium.com/@bhanu0925/deriving-equation-for-simple-linear-regression-ols-method-44d4f7d38dc1">Simple Linear Regression</a> article, we have discussed how to find the coefficients. Now we will extent simple linear to multiple linear and find the coefficients.</p><p><strong>Mathematical Derivation</strong></p><p>We know the equation of a line for simple linear regression is</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/241/1*Bsv3TGaU2dsxUFxOqsmeoA.png" /><figcaption>Equation of a line</figcaption></figure><p>Similarly, now for the multiple linear regression, the formula will be,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*cuxEeWYBk9PIqq3rVARZHw.png" /><figcaption>Equation of a plane with multiple independent variables</figcaption></figure><p>The above formula can be rewritten in matrix form as,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*BvOPDfcZuNxSJyn15MXfKg.png" /><figcaption>Matrix representation for n records</figcaption></figure><p>This can be further rewritten as,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/467/1*cT3ibH53ACbJJ3dqx0iVmw.png" /></figure><p>Now the above matrix can be represented as (all capital letters represents Matix in further equations)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/132/1*vZZPMfIp2hkYy8MSY99Fog.png" /><figcaption>matrix representation</figcaption></figure><p>we know, the error is given by (Actual — Predicted), So the error can be represented in matrix form as below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/339/1*MtUFk3_UA7ON4qlKiWk_Ug.png" /></figure><p>we know, for linear regression, for n observations, the <strong>error/cost function</strong> is given by,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/884/1*USz0_PPckSXD1WPjnQSILw.png" /><figcaption>Error/cost function</figcaption></figure><p>Lets, derive an equation for the error function to represent in matrix form</p><p>Expanding the equation 1, we can rewrite it as,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7c8PpR-GrbTKqjyyZSX6KQ.png" /><figcaption>Expanding the error function</figcaption></figure><p>The above equation can be rewritten in matrix form as,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/467/1*tsnO2GPGm_P70osso4AK2g.png" /><figcaption>Representing error function in matrix form</figcaption></figure><p>Hence, the Error/Cost function in matrix notation is given by,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/144/1*Wh4QqVNaonKfyLIzM5VpyQ.png" /><figcaption>Error Function in Matrix notation</figcaption></figure><p>Now the error equation becomes,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/356/1*7ewZlOQR_XZfmiwU-ycPSQ.png" /></figure><p>By Linear algebra,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/234/1*e7M1X8oRJR0_V_SsMeAf3w.png" /></figure><p>Substituting the values in equation 2, equation 2 can be rewritten as,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/418/1*oDW_Dfy2IqsdUo5U-3cRCQ.png" /></figure><p>Replacing, Yhat = Xβ in equation 3,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/548/1*cCSoH9ia_ijQotswRw1-wA.png" /></figure><p>To simplify equation 4, we have to prove the following term as equal</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/1*zyLG5tsnN4rKY7VyI9N0lw.png" /></figure><p>solving equation 5,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*2tmTFyAlcRw1VZfs5tm1zg.png" /></figure><p>after simplifying, equation 4 becomes,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*kK5AUxFJtuSnnmktIIaqGg.png" /></figure><p>Applying partial derivatives concerning β,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/474/1*M-Uvf4sNrrGIYbHB2muiKA.png" /></figure><p>By applying matrix differentiation on equation 8,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/287/1*EbMfi82bozIiTN71DLOv0w.png" /></figure><p>simplifying the equation further,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/294/1*Zr5tN7rrEF6JWTeb4QWLYA.png" /></figure><p>After simplifying we have got a β coefficient matrix.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/243/1*nMcGfO6X-3YCOk7Gu3xjrw.png" /></figure><p>In the above equation X represents X_train, Y represents y_train from the dataset. Plugging those values into the above equations will give us the β coefficient matrix. This matrix will have (independent variables + 1) different values. Suppose if your data set has 5 independent variables, your coefficients will be 6.</p><p>Predictions can be made using X_Test and β coefficient. That will be the dot product of X_test and β coefficient.</p><p><strong>Conclusion</strong></p><p>This normal equation is used in Sci-kit learn’s LinearRegression() which works well for small datasets. LinearRegression()’s computational complexity is too high since it involves calculating matrix inverse operation and complexity is o(n³). Please refer <a href="https://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations">complexity of matrix inverse operation</a>. To overcome these complexity issues, <a href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDRegressor.html">SGDRegressor</a>() method is used, which uses gradient descent to find the coefficients.</p><p>Please refer to the practical implementation of Normal Equation here — <a href="https://git.io/J6Wes"><strong>https://git.io/J6Wes</strong></a></p><p><strong>LinkedIn: </strong><a href="https://www.linkedin.com/in/bhanumathiramesh/"><strong>Bhanumathi Ramesh</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=85241965ee3b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deriving Equation for Simple Linear Regression - OLS Method]]></title>
            <link>https://medium.com/@bhanu0925/deriving-equation-for-simple-linear-regression-ols-method-44d4f7d38dc1?source=rss-4dfd629c989b------2</link>
            <guid isPermaLink="false">https://medium.com/p/44d4f7d38dc1</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[ordinary-least-square]]></category>
            <category><![CDATA[coefficient]]></category>
            <dc:creator><![CDATA[Bhanumathi Ramesh]]></dc:creator>
            <pubDate>Sun, 17 Oct 2021 19:13:31 GMT</pubDate>
            <atom:updated>2021-10-18T02:04:29.846Z</atom:updated>
            <content:encoded><![CDATA[<p>Deriving Equations was not this simple</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*ylFC9hwQrM_exbwLIlC9jQ.jpeg" /></figure><p>In this article, we will be deriving the equations for coefficients of simple linear regression by using the error terms and error function.</p><p><strong>What is Regression?</strong></p><p>Regression, in simple words, is a method of estimating a value based on another value. <br>For example, estimating height based on age.<br>- This method is used for,<br> 1. forecasting<br> 2. Finding out cause and effect relationship between values</p><p><strong>What is Linear Regression?</strong><br>Linear Regression is a supervised Machine learning algorithm. Linear regression is a type of regression analysis where there is a linear relationship between the independent(x) and dependent(y) variables. There can be positive Linear Relationship or Negative Linear Relationship</p><p><strong>The objective of Linear Regression :</strong><br>To fit a <strong>best-fit line</strong> in such a manner that the differences between the distance of the actual data points from the plotted curve/line are minimum</p><p><strong>Types of Linear Regression</strong><br>1 Simple Linear Regression<br>2 Multiple Linear Regression<br>3 Polynomial Linear Regression</p><p><strong>Simple Linear Regression</strong></p><p>It is an approach for predicting Y (dependent variable) on the basis of the single independent variable <br> It assumes that there is approximately a linear relationship between X and Y</p><p>this is given by <strong>Y ≈ β1X + β0</strong><br> where,<br> X is the independent variable<br> Y is the dependent variable<br> β1 is the slop (or coefficient or weight)<br> β0 is y-intercept when x = 0, (or offset)</p><p><strong>β0 and β1</strong> are two unknown constants that represent intercept and slop terms in the linear model.<br>This <strong>β0 and β1</strong> are known as model coefficients or <strong>model parameters</strong></p><p>Once we train the model with our training data to produce estimates β0 and β1 we can predict the future values on the basis of particular x value by computing <strong>yhat = β0hat + β1hat * x</strong></p><p><strong>Mathematical Intuition</strong><br><em>How to find the values of Coefficients (β0 and β1)</em></p><p>The coefficients can be found in two different ways<br><strong>Closed-form</strong><br>1. Uses direct formula — <a href="https://en.wikipedia.org/wiki/Closed-form_expression">Wikipedia</a><br>2. also called as <strong>Ordinary least square</strong> ( LinearRegression() in sci-kit learn uses OLS method by default)<br><strong>Non-Closed-form</strong> — uses differentiation <br> — Solved by Gradient decent (SGDRegressor() in sci-kit learn uses gradient descent)</p><p><strong><em>Why are we using Gradient Decent instead of the direct formula from OLS</em></strong><em>?</em><br> — When the dimensionality increases the complexity of calculation increases with the OLS formula. OLS is considered when the dataset is very small.</p><p><strong>Ordinary Least Square Method (Closed Form)</strong><br>OLS method tries to find the β0 and β1 which minimizes the sum of squared errors</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bPw3dgO42UV_9IphFbPI2w.jpeg" /><figcaption>Closed-form formulas from OLs method</figcaption></figure><p><em>Let&#39;s build this equation from scratch</em></p><p>Assuming that independent and dependent variables are linear in nature, we try to fit “<strong>best line of fit” </strong>that tries to pass through all the data points as close as possible to reduce the error (also called residuals)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DNJpsZ-gWiFGhr25YEpWrw.jpeg" /><figcaption>Residuals</figcaption></figure><p>Here d1,d2,d3,d4….dn represents errors(residuals)<br>then, Error can be written as</p><p>𝑑1+𝑑2+𝑑3+𝑑4+….+𝑑𝑛</p><p>Errors can be negative or positive values, this makes errors cancel out at some point. To overcome this issue and find the total errors, we square them</p><p><strong><em>Why modulus is not used to convert the errors to positive values?</em></strong></p><p>There are two reasons,<br>1. we want to penalize outliers<br>2. Mod can not be differentiable at the origin</p><p>now the Error term can be written as,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zRsVsGv-6J8IqbbKGq5qBw.jpeg" /><figcaption>Error Function</figcaption></figure><p>This E is now referred to as <strong>Error Function</strong> or <strong>Cost Function</strong>(You will see Error term is also represented by <strong>J</strong> in some books)</p><p>Decomposing the error term furthermore,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fhqBSfnsgdc9u3Qfo_PDWA.jpeg" /><figcaption>Error function of β0 and β1</figcaption></figure><p>Now we have the Error function decomposed, we have to now find the values of <strong>β0 </strong>and <strong>β1</strong> which minimizes the error.</p><p>From theory we know, y = f(x), meaning, y is a function of x, that is, a change in x will change the values of y.</p><p>Similarly, a change in β0 and β1 will change the values of error and finds the minimum errors.</p><p>From maths, we know,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lssXDS3iqjXq1COQqyoV6w.jpeg" /><figcaption>Derivatives and chain rule</figcaption></figure><p>To find the minimum value of the error function, we have to carry out two steps,<br>1. Find the partial derivatives of errors with respect to β0 and β1<br>2. set up the derivatives to 0 and find the critical points, where the slop of the tangent line becomes 0</p><p><em>Finding β0</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*hoN873XMh2jU92uQD5kJxg.jpeg" /></figure><p><em>Finding β1</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/984/1*DaQI-JKTDsYjK88hRsqQCA.jpeg" /></figure><p>The implementation and comparison of the derived formula and LinearRegression algorithm are presented in the below link.</p><p>Please find the example here — <a href="https://git.io/JK6sS"><strong>Github</strong></a></p><h3>Conclusion</h3><p>Deriving equations for finding coefficients of simple linear regression are presented in this article. Multiple Linear Regression and Polynomial Regression are out of the scope of this article and will be articulated in the coming articles.</p><p>LinkedIn Profile <a href="https://www.linkedin.com/in/bhanumathiramesh/">Bhanumathi Ramesh</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=44d4f7d38dc1" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>