<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Praful Mohanan on Medium]]></title>
        <description><![CDATA[Stories by Praful Mohanan on Medium]]></description>
        <link>https://medium.com/@praful.mohanan?source=rss-1660e518c831------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*rXiQcGnK3qHYrpLs23IU8A.jpeg</url>
            <title>Stories by Praful Mohanan on Medium</title>
            <link>https://medium.com/@praful.mohanan?source=rss-1660e518c831------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 15:54:51 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@praful.mohanan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Break into AI]]></title>
            <link>https://medium.com/dsc-dypcoe/break-into-ai-c74ab275ae16?source=rss-1660e518c831------2</link>
            <guid isPermaLink="false">https://medium.com/p/c74ab275ae16</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[mls]]></category>
            <dc:creator><![CDATA[Praful Mohanan]]></dc:creator>
            <pubDate>Thu, 17 Sep 2020 04:09:59 GMT</pubDate>
            <atom:updated>2021-07-14T04:51:59.656Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*orHvINL6keHXGNFv" /></figure><p>Hey there 👋🏼, so you are interested in entering in this field. You’ve come to the right place. Let me help you get started.</p><p>For starters, Who am I?<br>Well, the answer is I am you. Confused?<br>I am the same as you, only the difference is I am you skipped ahead in time.<br>I also was in the same place as you are now, a person who wants to enter into this field, but does not know where to start.<br>There is only one solution to this problem in all — <br>“‘<strong><em>There is not any one magic resource or way for you to start or learn it all</em></strong>’’<br>The point is to just <strong>start </strong><em>somewhere.</em></p><p><strong>Some may ask</strong>:<br>‘<em>If you are also showing us a </em><strong><em>way</em></strong><em>, Why are you writing this article?</em>’<br>The point is just like I said there is not one <strong>holy grail</strong> to get into this field, best method is to try multiple <em>ways </em>and see what works best for you. Just like everyone has different tastes for food, everyone has their <em>way </em>of doing things.</p><blockquote>Let’s consider for now that <br>‘<strong>This is the <em>way</em></strong>’.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/0*OONf7LL5ecn-frae.jpg" /><figcaption>*Insert random Mandalorian meme with quote*</figcaption></figure><h3>The Way</h3><p>1. First and foremost you should be absolutely clear of the technical jargons that you keep hearing — Artificial Intelligence, Data Science, Deep Learning, Machine Learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/369/1*_nT3VeOkgJm0QBi663PSpQ.png" /></figure><p>As it is visible from the diagram, multiple of those terms are subsets of each other.</p><p><strong>Artificial Intelligence</strong> — A set of methods which enables the machine to demonstrate machine intelligence. Eg. Knowledge Base<br>Input — If you have a car, you can go to the city. If you can go to the city, you can visit the fair.<br>Output — If you have a car, you can visit the fair.</p><p><strong>Machine Learning</strong> — A subset of AI which contains various algorithms which use mathematical models to learn patterns in data and infer results on unseen data. Eg. Logistic Regression — An algorithm which learns to classify between two classes. <br>Input — Data of students with marks<br>Output — Did the student Pass/Fail</p><p><strong>Deep Learning</strong> — A subset of Machine Learning which uses neural networks(A mathematical model inspired by how the brain works) to infer patterns from very large amounts of data.</p><p><strong>Data Science</strong> — A field which employs these methods, domain knowledge, statistics to make sense out of unstructured data to achieve a final goal.</p><p><strong>2.</strong> ’<em>How much Mathematics do I need?</em>’<br>You should be comfortable with basics of Linear Algebra and Calculus.<br>After that learn along the way, your answers are just a Google search away.<br>If you think you need to revisit, <a href="https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw">this</a> channel is good place to start.</p><p><strong>3.</strong> Let’s start with <em>Machine Learning</em> now. <br>We’ll be learning with <em>The Andrew Ng</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*-S3PTSfL_qVccGCJ" /></figure><p>He has a catchphrase — <br>‘<strong>Don’t worry about it if you don’t understand</strong>’.<br>Seriously, worrying is not going to help you figure out that the change in loss w.r.t filter is itself a convolution or know how converting a primal to a dual problem by Lagrangian method would help you compute the polynomial transformation without ever actually performing the transformation! I stressed on it so much because many a time, while learning you lose motivation to do so, seeing all the math. <br><em>But complex math is just simple math combined together and simple math is just simpler math clubbed together, just break it down, one piece at a time, Likewise</em>!</p><p>Anyway If you’re interested in learning Deep Learning, you should be quite familiar with the groundwork which is laid down in Machine Learning. Given that Deep Learning is the most exciting field in ML, most problems can be solved using ML techniques like <strong>Random Forests</strong> and <strong>Ensemble Learning</strong>.<br>Deep Learning is great for complex tasks like Computer Vision, Natural Language Processing, etc.</p><h4><strong>Andrew Ng Machine Learning </strong><a href="https://www.coursera.org/learn/machine-learning"><strong>Course</strong></a></h4><ul><li>The fundamentals are explained quite amazingly. The Professor has broke down all the concepts to small bits so they are easy to grasp.</li><li>The assignments in MATLAB are well tailored for each Week.</li></ul><p><strong>Some Gaps to Fill:</strong></p><ul><li>I had quite a hard time understanding Support Vector Machines, PCA and the math behind it. Opencourseware videos like <a href="https://youtu.be/_PwhiWxHK8o">these</a> are such a help.</li><li>Backpropagation is explained way better in his Deep Learning course.</li><li>Since the course is not in Python, you need to get familiar with the libraries, I have listed all the links at the end of the post.</li></ul><h4><strong>Deep Learning </strong><a href="https://www.coursera.org/specializations/deep-learning"><strong>Specialization</strong></a></h4><ul><li>Another masterpiece specialization by Andrew Ng and in Python</li><li>Course mainly focuses on Neural Networks Part that you have learned in the earlier course, It is a set of five course focusing on NN, Vision and NLP.</li><li>The courses touch on various research papers which have shaped Deep Learning to its Current State like YOLO, ResNet, Inception, etc.</li></ul><p><strong>Some Gaps to Fill:</strong></p><ul><li>Tensorflow and the Keras API is not very well introduced.</li><li>Course 5 Sequence Models(mainly LSTM) is not explained that well.</li></ul><h4><strong>Some Advice to Fill the Gaps:</strong></h4><ul><li>It’s not about completing the courses for the certificates or just for the sake of. The assignments are small and you can complete them very fast if you just want to fill in the blanks. Take time to implement the algorithms in the first course using just Python from scratch, this will help you gain better intuition regarding the algorithm.</li><li>Use the Coursera Discussion forums rigorously, they will answer all your questions.</li><li>You may have trouble with understanding BackProp, for a normal NN it is easy. CNN, RNN, LSTM are troublesome. Do the calculus. It will help you in understanding what is happening behind the scenes.</li><li>Get a book like the <a href="https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/">Hands-on</a> ML book which covers both ML and DL.<br> No, I am not affiliated with them in any way :). You will come across many algorithms which was not there in the first course such as Random Forests and even methods to evaluate your algorithm using various metrics.<br>Very useful book if you aspire to be a ML Practitioner.</li></ul><p>Last but not the least, Practise and Create. Don’t go on doing endless Courses.<br><strong><em>And Remember always maintain this ratio : Consumption &lt; Creation.</em></strong></p><p>Visit our <a href="https://github.com/DSC-DYPCOE/Learning-Resources/tree/master/Machine%20Learning">Repository</a> for more Resources drafted by Domain Leads.</p><p><strong>That’s all from my side Folks!</strong></p><h3>Thank you for reading</h3><p>I like explaining things <em>intuitively </em>and answering<em> “why”</em> questions.<br>If you liked my article, do 👏🏼.<br>Your appreciation inspires me to ✍🏼write more.<br>Check out my other articles:</p><p><a href="https://medium.com/dsc-dypcoe/bayes-rule-and-sentiment-analysis-24629eeae847">Bayes Rule and Sentiment Analysis</a></p><p><em>~Praful Mohanan<br>Connect with me on </em><a href="https://www.linkedin.com/in/praful-mohanan-26615873"><em>LinkedIn</em></a><em>, </em><a href="https://github.com/Praful932"><em>Github</em></a><em>.<br>If you need any help regarding ML, feel free to reach out!</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*-nSTuZIxiDiu4ukb.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c74ab275ae16" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dsc-dypcoe/break-into-ai-c74ab275ae16">Break into AI</a> was originally published in <a href="https://medium.com/dsc-dypcoe">GDSC DYPCOE</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bayes Rule and Sentiment Analysis]]></title>
            <link>https://medium.com/dsc-dypcoe/bayes-rule-and-sentiment-analysis-24629eeae847?source=rss-1660e518c831------2</link>
            <guid isPermaLink="false">https://medium.com/p/24629eeae847</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[sentiment-analysis]]></category>
            <dc:creator><![CDATA[Praful Mohanan]]></dc:creator>
            <pubDate>Mon, 08 Jun 2020 03:38:24 GMT</pubDate>
            <atom:updated>2020-06-08T03:38:24.164Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1tP7eVauHABOajOCiAeD1A.jpeg" /><figcaption>Photo Credits — Pexels</figcaption></figure><p>Exploring Natural Language Processing, the most fascinating thing that caught my eye was <strong>Bayes Rule</strong>.</p><blockquote><strong>Fun Fact </strong>: SS Central America which sank in 1857 carrying 20 tons of gold was found using the Bayesian Theory.</blockquote><p>In this article I explore the Bayes Rule First and how it is used to perform Sentiment Analysis followed with a Python code example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KDzmqkfpC4dIrOIAJxswaA.jpeg" /><figcaption>Bayes Rule</figcaption></figure><p><strong>Notation:<br>P(A and B) : </strong>Probability that Event A and Event B happens<strong><br>P(A|B)</strong> : Probability that Event A is going to happen given that B happens <strong>P(B|A)</strong> : Probability that Event B is going to happen given that A happens <strong>P(A)</strong> : Probability that Event A is going to happen<br><strong>P(B)</strong>: Probability that Event B is going to happen</p><p><strong>Rule of Multiplication:</strong><br>P(A and B) = P(A) P(B|A) = P(B)P(A|B)</p><p>So that’s a lot of notations. Let me break it down bit by bit. So here <strong>|</strong> symbol represents conditional probability. A conditional probability is the expression of how probable one event is given that some other event occurred.</p><p>For example: <br>What is the probability that I fall given that its slippery?<br><strong><em>P(I fall| Its slippery) = P(I falls and its slippery)/P(Its slippery)</em></strong></p><p>What is the probability that it rains given that its cloudy?<br><strong><em>P(It rains| Its cloudy) = P(It rains and its cloudy)/P(Its cloudy)</em></strong></p><blockquote>To think about this intuitively If i want to know the probability that it rains given that its cloudy, I want to consider <strong>all ways that it rains and its cloudy</strong> out of the <strong>only ways where its already cloudy</strong> because that is what we care about right!</blockquote><p>We know that <br><em>P(It rains and its cloudy) = P(Its cloudy and it rains)</em><br>So we can similarly say that <br><em>P(Its cloudy| It rains) = P(It rains </em><strong><em>and </em></strong><em>Its cloudy)/P(It rains). </em><br>Equate <em>P(Event A and Event B)</em> and we get the Bayes rule.</p><p><strong><em>P(It rains | Its cloudy) P(Its cloudy) = P(Its cloudy| It rains) P(It rains)</em></strong></p><p>Example : <br>Suppose I want to calculate the probability of rain in the afternoon. Information I have:<br>20% of days have rainy afternoons — <strong>P(It rains)</strong> = 0.20<br>30% of days have cloudy mornings — <strong>P(Its cloudy)</strong> = 0.30<br>70% rainy afternoons have cloudy morning — <strong>P(Its cloudy | It rains)</strong> = 0.70<br>Plugging in we get — <strong>P(It rains|Its cloudy) = 0.46</strong></p><p>So what does this mean? There’s 46% chance that it rains in the afternoon whenever the morning is cloudy. Interesting right? Earlier the probability was <strong>20%</strong> now it increased to <strong>46%</strong>. Why so?<strong><em> From the evidence we had, we calculated a new probability based on how accurate the evidence was!</em></strong></p><p>Now that you have a good grasp of Bayes theorem let’s use it to perform sentiment analysis on product reviews. <br>As far as the definition of <strong>Sentiment analysis</strong> goes, it is a type of analysis which interprets and classifies the emotions within text data using either statistical or linguistic techniques or both. For example, “<em>Didn’t like it</em>” is a negative review while “<em>Very useful</em>” is a positive review. We will be using a statistical technique called <strong>Bag-of-words </strong>model. Bag of words technique ignores the grammatical structure of the words while relying on appearance of words such as <strong>‘nice’, ‘great’, ‘bad’</strong> to classify it as a positive or negative sentiment.</p><p><strong>Notation</strong>:<br>S — Sentence<br>😀 — Positive Sentiment eg. S = I liked this product.<br>😐 — Negative Sentiment eg.S = Did not work as expected.<br>P(😀 | S) — Probability of Positive Sentiment given the Sentence<br>P(😐 | S) — Probability of Negative Sentiment given the Sentence</p><p>Consider a sentence for which we need to do sentiment analysis:<br><em>S = “Great product at affordable price”</em></p><p>We need to find P( 😀 | “Great Product at affordable price”) <br>According to the Bag-of-words model we will treat each sentence as a collection of words P( 😀 | “Great”, “Product”, “at”, “affordable”, “price ”)<br>This now means what is the probability that the Sentence is Positive given that the word “<strong>great</strong>” was in the sentence and “<strong>product</strong>” was in the sentence and the word ‘’<strong>at</strong>” was in the sentence.. likewise. This is where Bayes Rule comes in.</p><p>Notation — wᵢ : denotes iᵗʰ word for simplicity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PuZznCMf-XBIZD8Hhqds2w.jpeg" /><figcaption>Fig 1.1</figcaption></figure><p>Let’s walk through each of these steps:<br>1. According to Bayes theorem we can say that both of these terms are proportional to each other ignoring the denominator.<br><em>P(A|B) α P(B|A) P(A)</em></p><p>2. From Rule of multiplication — P(A|B) P(B) = P(A and B)</p><p>3. This part is where <strong>Naive </strong>of Naive Bayes comes in, normally it would have been P(A)P(A|B)P(A|B,C)… but the presence of one word does not affect the likelihood of another(Bag of words model) rather it is only affected by whether the word is positive or negative so <strong>P(A)P(B|A)P(C|A)</strong> likewise. The probability of being a positive or negative would not change if it is known that some word comes up less or more than some other word.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/965/1*_zZtKEaTa86wUGvYCqi0zw.png" /><figcaption>Fig 1.2 (From Fig 1.1)</figcaption></figure><p>Now we have shown that these both equations are proportional!<br>This is what we want given the sentence we need to calculate the probability that it is a positive sentiment.</p><p>Now Can we find the terms in Equation 2(Fig 1.2)? <strong>Yes</strong>.<br>We can calculate all the terms in the 2nd equation using:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*zp2Neb4Et8j6eLlM_ZgRXQ.jpeg" /><figcaption>Fig 1.3</figcaption></figure><p>Consider we have a labelled dataset containing both positive, negative reviews with 52% positive and 48% negative examples and some arbitrary word distribution as follows for this particular example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/564/1*LfriLubpWZpwGi_5tol2fw.jpeg" /><figcaption>Fig 1.4</figcaption></figure><p>For eg. <strong>0.3</strong> here denotes Probability that a sentence has the word “product” given that it is a positive example and how did we calculate that? <br>Refer Fig 1.3</p><p><em>S = “Great product at affordable price”<br></em>Now we can calculate P(😀| S) using the equation we found in Fig 1.2</p><p>Calculating we get:<br>P(😀 | S) = 0.52 * 0.8 * 0.3 * 0.1 * 0.7 * 0.2 = <strong>0.0017472</strong><br>This number in itself does not make sense let’s calculate P(😐| S) Similarly<br>P(😐 | S) = 0.48 * 0.05 * 0.7 * 0.5 * 0.4 * 0.66 = <strong>0.000336<br></strong>Normalizing:<br>P(😀|S) = 0.83 <br>P(😐|S) = 0.16</p><blockquote>83% The product review is a positive example. Pretty amazing right!</blockquote><h4>Now let’s look at some Code:</h4><p>We will be using the Naive Bayes Classifier from the nltk python package. <br>Our dataset :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/48dd9f3b283a9bbd5f597f41a5271692/href">https://medium.com/media/48dd9f3b283a9bbd5f597f41a5271692/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f668038be0ab97a39a153f9006fe38e6/href">https://medium.com/media/f668038be0ab97a39a153f9006fe38e6/href</a></iframe><p>The classifier expects training data in the form of list of tuples where each tuple is a feature(sentence) and a label, in this case there will be two labels — Positive and Negative. Here’s an example feature for the sentence <strong><em>“Product is very good”</em></strong>. The tuple will contain true for words in the sentence and false for words not in the sentence. So first we will need to generate these features for both Positive and Negative examples.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/87db2f88aee43acb65be4c19a56427ec/href">https://medium.com/media/87db2f88aee43acb65be4c19a56427ec/href</a></iframe><ol><li>Import necessary packages and extract both samples</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/950eb82312c966c766f0aff02e191868/href">https://medium.com/media/950eb82312c966c766f0aff02e191868/href</a></iframe><p>2. Get all the unique words from the dataset</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d0626e6124b040057c4c6064f7d4f616/href">https://medium.com/media/d0626e6124b040057c4c6064f7d4f616/href</a></iframe><p>3. Create features required for training and train using the classifier.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8337a14ba9f83884e00fbe5d06257332/href">https://medium.com/media/8337a14ba9f83884e00fbe5d06257332/href</a></iframe><p>4. Now our classifier is trained let’s give it the input from earlier:<br><em>S = “Great product at affordable price”<br></em>Just like our training data, the input sentence needs to converted into the proper form. After that we will use prob_classify function which returns the probability distribution for the respective labels.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/690078cc0171eacb7fcacf28f22f9009/href">https://medium.com/media/690078cc0171eacb7fcacf28f22f9009/href</a></iframe><pre>Positive: 0.9596<br>Negative: 0.0404</pre><p><strong>This shows us that it is a 😀 sentiment with 95% confidence!<br>Great Work! Now we have trained our very own sentiment classifier.</strong></p><p>Now you have an intuitive understanding of Bayes Theorem and also know how to use it to do sentiment analysis using the nltk package. You can use this concept similarly even to classify spam emails. Since we used a small sample data, <a href="https://data.world/datafiniti/consumer-reviews-of-amazon-products">here’ s</a> one Amazon review dataset which you can use to do sentiment analysis on.</p><h3>Thank you for reading</h3><p>I like explaining things <em>intuitively </em>and answering<em> “why”</em> questions. <br>If you liked my article, do 👏🏼.<br>Your appreciation inspires me to ✍🏼write more.</p><p><em>~Praful Mohanan<br>Connect with me on </em><a href="https://www.linkedin.com/in/praful-mohanan-26615873"><em>LinkedIn</em></a><em>, </em><a href="https://github.com/Praful932"><em>Github</em></a><em>.</em></p><p><strong>References:</strong></p><ul><li><a href="https://betterexplained.com/articles/an-intuitive-and-short-explanation-of-bayes-theorem/">Bayes Theorem</a></li><li><a href="https://cs50.harvard.edu/ai">HarvardX’s CS50-AI OpenCourseware</a>, <a href="http://cdn.cs50.net/ai/2020/spring/lectures/6/src6/sentiment/sentiment.py">Code Snippet</a></li><li><a href="https://the-gist.org/2016/09/man-vs-maths-and-a-ship-filled-with-gold/">Ship of Gold</a></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1018/1*eEBAui_alJUvE94u0b04Wg.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=24629eeae847" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dsc-dypcoe/bayes-rule-and-sentiment-analysis-24629eeae847">Bayes Rule and Sentiment Analysis</a> was originally published in <a href="https://medium.com/dsc-dypcoe">GDSC DYPCOE</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>