<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Joseph Roy on Medium]]></title>
        <description><![CDATA[Stories by Joseph Roy on Medium]]></description>
        <link>https://medium.com/@josephroy?source=rss-1742e5550527------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*9agBwnyYSP8Usqjj.</url>
            <title>Stories by Joseph Roy on Medium</title>
            <link>https://medium.com/@josephroy?source=rss-1742e5550527------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 09:25:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@josephroy/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Feature Engineering]]></title>
            <link>https://medium.com/@josephroy/feature-engineering-cfee4d665ae9?source=rss-1742e5550527------2</link>
            <guid isPermaLink="false">https://medium.com/p/cfee4d665ae9</guid>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Joseph Roy]]></dc:creator>
            <pubDate>Fri, 09 Aug 2019 07:10:02 GMT</pubDate>
            <atom:updated>2019-08-09T07:10:02.703Z</atom:updated>
            <content:encoded><![CDATA[<p>Feature Engineering</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*zfAg9Ny_jU8JSPqB" /></figure><p>Feature Engineering is the process of transforming raw data into features that is easier to interpret. It is the state of the art of combining art and science of deriving features from existing features to solve a business or scientific problem. Deriving important features from existing features is sometimes quite tricky and also requires domain knowledge.</p><p>For example : How does a FITBIT device count steps ?<br>Fitbit devices use a 3-axis accelormeter(lateral and vertical movements) to understand our motions. Accelerometer turns movement(acceleration) into digital measurements(data) when attached to the body. By analysing acceleration data, these devices provide information regarding distance travelled, calories burned, sleep quality etc.</p><p>So, we just saw how Fitbit devised a strategy to understand our motions. Let us assume they have taken 2-axis accelormeter(only lateral movements) for understanding our motions, in that case results might not be so suitable for our anlysis. The fact that Fitbit have used 3-axis accelormeter is an important<br>feature engineering technique which probably required brainstorming by different engineers/physicists to understand human motion.</p><p>There are various other methods such as Feature Scaling, Standard Normalization, Missing value handling which are also important feature engineering techniques.</p><p>Reference :<br>1. <a href="https://www.kdnuggets.com/2018/12/feature-engineering-explained.html">https://www.kdnuggets.com/2018/12/feature-engineering-explained.html</a><br>2. <a href="https://help.fitbit.com/articles/en_US/Help_article/1141">https://help.fitbit.com/articles/en_US/Help_article/1141</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cfee4d665ae9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Consumer Complaints(Financial) with pre-trained GLOVE]]></title>
            <link>https://medium.com/@josephroy/consumer-complaints-financial-cc9e9ba9ff9?source=rss-1742e5550527------2</link>
            <guid isPermaLink="false">https://medium.com/p/cc9e9ba9ff9</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[consumer-complaints]]></category>
            <dc:creator><![CDATA[Joseph Roy]]></dc:creator>
            <pubDate>Mon, 03 Sep 2018 08:51:37 GMT</pubDate>
            <atom:updated>2018-09-05T06:11:40.209Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/794/1*KWKCXvELFPdJWKQJK-AcLw.png" /></figure><p>Consumer Financial Protection Bureau(CFPB) regulates the offering and provision of consumer financial products or services under the federal consumer financial laws and educates and empowers consumers to make better informed financial decisions. CFPB receives lots of consumer complaints each day related to various financial products. For this project, I have used consumer complaints data and tried to predict the category of complaints from the consumer complaint narrative.</p><p>Website : <a href="https://www.consumerfinance.gov/data-research/consumer-complaints/search/?date_received_max=2018-01-09&amp;date_received_min=2017-12-31&amp;from=0&amp;has_narrative=true&amp;searchField=all&amp;searchText=&amp;size=25&amp;sort=created_date_desc">https://www.consumerfinance.gov/data-research/consumer-complaints/search/?date_received_max=2018-01-09&amp;date_received_min=2017-12-31&amp;from=0&amp;has_narrative=true&amp;searchField=all&amp;searchText=&amp;size=25&amp;sort=created_date_desc</a></p><p>Consumer data looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/857/1*1NAhLQAZIuTVjgtHR88dog.png" /></figure><p>Product categories:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/833/1*zzrIbhjEcDqxEmOCaiJ9dA.png" /></figure><p>We can see there are 18 financial products, across which CFPB received consumer complaints. I am interested to explore the ‘student loan’ data, which looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*THFJPlPcbCeIUZA2Fv3vNw.png" /></figure><p>Student loan data can be further categorized across 3 sub-products, mentioned below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/482/1*viwv3ou_ySDcVntpugmCAw.png" /></figure><p>The sub- products are considered as my target variable and ‘consumer complaints narrative’ are my features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/376/1*2dp1ghpElwn6Jj927Yhoew.png" /></figure><p>At first, I have cleaned the ‘consumer complaint narrative’ data by removing the punctuation, converting the data into lower case and lemmatizing the data.</p><h3>Data Preprocessing for ML</h3><p>This data has further been tokenized and vectorized using Tfidf Vectorizer where I have taken n-grams(1–2) as input. I used Logistic Regression and Multinomial NB for my predictions.</p><p>The cross- validation mean accuracy for Logistic Regression and Multinomial NB were 68.44% and 62.83 % respectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/833/1*jvsekZufR9hfTWjlkenN3w.png" /></figure><p>To improve the model further, I have used ‘GLOVE’ pre-trained embedding for the LSTM network. For experimenting purpose, both pre-trained embedding as well as embedding generated from my project has been taken into consideration.</p><h3>Pre-processing for Deep Learning</h3><p>For dealing with our training data, we need to know the corpus(total number of unique words in our vocabulary).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*vN3krs_3EK_LJ78qQV_GMA.png" /></figure><p>After finding the number of unique words, those words have been tokenized.</p><h4>Padding and Truncating data:</h4><p>The Recurrent Neural Network can take sequences of arbitrary length as input, but in order to use a whole batch of data, the sequences need to have the same length. But if we take the longest sentence we waste a lot of memory. So, we truncate longer sentences and pad shorter sentences.</p><p>First we count the number of tokens in all the sequences in the training data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/737/1*MaH7ixACEf3W9yOBF4ECSA.png" /></figure><p>The average number of tokens in a sequence is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*sksA2pXe5xixuOofqUsErg.png" /></figure><p>The max number of tokens we will allow is set to the average plus 2 standard deviations. It covers around 95% of the data-set.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/588/1*p1SS4qckjKyiD_FUiqrktg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*EXrNRU2tXX2vvbMHD1TOnA.png" /></figure><p>When padding or truncating the sequences that have a different length, we need to determine if we want to do this padding or truncating ‘pre’ or ‘post’. The choice of ‘pre’ or ‘post’ can be important because it determines whether we throw away the first or last part of a sequence when truncating, and it determines whether we add zeros to the beginning or end of the sequence when padding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/667/1*m761smRGBO2KUACw3xqocw.png" /></figure><p>The ‘target’ variable i.e. the sub-product has to be one-hot encoded. At first we do label encoding and then one-hot encoding so that each category gets equal importance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/623/1*pQzw9o5esB7hgEvyn6aUJw.png" /></figure><h4>Embedding</h4><p>We shall see if pretrained embeddings like GLOVE which are pretrained using billions of words could improve our accuracy score as compared to training our own embedding. We will compare the performance of models using these pretrained embeddings against the baseline model that doesn’t use any pretrained embeddings.</p><h4>How GLOVE works?</h4><p>GLOVE(“Global Vectors [for word representation]”) learns by constructing a co-occurrence matrix (words X context) that basically count how frequently a word appears in a context. Since it’s going to be a gigantic matrix, we factorize this matrix to achieve a lower-dimension representation. Here, we will use pre-trained word vectors which can be downloaded from the <a href="https://nlp.stanford.edu/projects/glove/">glove</a> website. There are different dimensions (50,100, 200, 300) vectors trained on wiki data. For this example, I have downloaded the 100-dimensional version of the model.</p><p>Here, we find the common words between our corpus and the pre-trained model. The embeddings of those words are taken from the pre-trained model, since they are trained for huge corpus.</p><p>With those embeddings, we construct our matrix which will eventually be used as input for our LSTM model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/845/1*2_6_4nOX27XqQnLMNg7lEw.png" /></figure><p>The embedding matrix looks like this. The weights are derived from the pre-trained model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*fUV6s5N1woBooHLJ9FsZKA.png" /></figure><p>The embed size is 100 which is because I have used 100 dimensional model. Our corpus size is 16119.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/627/1*G0XFO7k0rPaRrx9mW5jqvw.png" /></figure><p>With this embedding matrix we can construct the embedding layer which is the input for our LSTM model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/852/1*lraSWCvzSf06nNnXnGEpyw.png" /></figure><p>We are using LSTM bidirectional layer. But, how does it work?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*B5NHtY8_Y4we0DE4Y-acBA.png" /></figure><p>Imagine that the LSTM is split between 2 hidden states for each time step. As the sequence of words is being feed into the LSTM in a forward fashion, there’s another reverse sequence that is feeding to the different hidden state at the same time. You might noticed later at the model summary that the output dimension of LSTM layer has doubled to 120 because 60 dimensions are used for forward, and another 60 are used for reverse.</p><p>The greatest advantage in using Bidirectional LSTM is that when it runs backwards you preserve information from the future and using the two hidden states combined, you are able in any point in time to preserve information from both past and future.</p><p>Lets have a look at the model summary:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/594/1*a_KrFdgejqRWn3DCfKZFfQ.png" /></figure><p>To reduce the over fitting problem we have introduced Dropouts. The accuracy for this model on test data set is around 70.69 %.</p><h4>Baseline model</h4><p>Further, I wanted to experiment with our baseline model i.e. without using the pre-trained embeddings, rather using our text corpus for the embeddings. The accuracy of the LSTM model seems to be quite high(training accuracy) but there is a clear over fitting, since our validation accuracy is quite high compared to the training accuracy. The validation accuracy is around 70.93% which is quite similar to the pre-trained model.</p><h3>Conclusion:</h3><p>Although, I could not find any significant improvements by using pre-trained model in terms of accuracy. But, it would be interesting to see if the word2vec model or any other pre-trained model could give us better results. Please find the code for this work in my GitHub profile:</p><p><a href="https://github.com/joseph10081987/Machine-Learning_new/blob/master/complaints(1.2017-10.2018).ipynb">https://github.com/joseph10081987/Machine-Learning_new/blob/master/complaints(1.2017-10.2018).ipynb</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cc9e9ba9ff9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multi label Text Classification — Rotten Tomatoes]]></title>
            <link>https://medium.com/@josephroy/multi-label-text-classification-rotten-tomatoes-f998bf54f81?source=rss-1742e5550527------2</link>
            <guid isPermaLink="false">https://medium.com/p/f998bf54f81</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[sentiment-analysis]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[text-analysis]]></category>
            <dc:creator><![CDATA[Joseph Roy]]></dc:creator>
            <pubDate>Sun, 12 Aug 2018 11:03:27 GMT</pubDate>
            <atom:updated>2018-08-12T11:03:27.672Z</atom:updated>
            <content:encoded><![CDATA[<h4>How many times we go to a movie after looking at the review…for me review makes a huge difference</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HR6HpzDKrTwIWOwV.jpg" /></figure><p>Recently while I was exploring Kaggle, I found this interesting project on “Movie Review Sentiment Analysis ”. The task as mentioned by Kaggle: “Classify the sentiment of sentences from the Rotten Tomatoes data set”.</p><p>This is a multi-class classification problem, which simply means the data set have more than 2 classes(binary classifier). The five classes corresponding to sentiments:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/430/1*X_a9dz_LsmHcU7BUT773wQ.png" /></figure><p>The data set is quite challenging as the dataset is split into separate phrases and each one of them have a sentiment label. Moreover, obstacles like sentence negation, sarcasm, terseness, language ambiguity, and many others make this task very interesting for Natural Language Processing.</p><h3>Data Exploration</h3><p>Let us have a look at first few phrases of the training dataset:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/978/1*8EBK6pCh0OZ0xDuVsx5ALw.png" /><figcaption>Fig. Training Data</figcaption></figure><h3>Number of comments across categories</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/385/1*k1izU2bdptU7X31osYVBxA.png" /></figure><p>As we can see that the dataset is imbalanced, the dataset have to resampled so, that our model is not biased.</p><h3>Data Preprocessing</h3><p>At first, the data needs to be cleaned which means we remove the punctuation marks, convert the data to lower case and also lemmatize the data. Lemmatizing means to group together variant forms of the same word.</p><p>After cleaning the data, the ‘Phrase’ column has been converted into the ‘clean-review’ column.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*pYhIkKFaFIiWvNc6JghYlg.png" /></figure><h3>n-grams and Tfidf Vectorizer</h3><p>Since, we would like to differentiate between the words ‘good’ and ‘not good’ n-grams(1,2) is used, which would help to understand the context better as it would take into consideration both the words together, rather than a single word. Also, I have used Tfidf Vectorizer, which helps us to stop giving undue advantage to the stop words (since they are higher in frequency in a document) and gives more weightage to the contextual words.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1007/1*5ePEbxTAs7Npr2oReq_2DA.png" /></figure><h3>Machine Learning</h3><h4>Logistic Regression</h4><p>For fitting the model with Logistic Regression, I have used OnevsRest classifier. This strategy consists in fitting one classifier per class. For each classifier, the class is fitted against all the other classes. This strategy is used for multilabel learning, where a classifier is used to predict multiple labels for instance, by fitting on a 2-d matrix in which cell [i, j] is 1 if sample i has label j and 0 otherwise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/1*CSFqcQ_aCe8kye25J9IGQQ.png" /><figcaption>Logistic Regression</figcaption></figure><p>The accuracy for this model is 72.17%. I have also used Linear SVC and Multinomial Naive Bayes algorithms for which mean accuracy were 77.44% and 59.50% respectively.</p><p>Since, deep learning works well with text data, I have decided to further analyze the data sets using various deep learning techniques.</p><h3>Deep Learning</h3><h4>Splitting Training data</h4><p>The training data is split into training and validation data using sklearn train-test split library.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/991/1*Kku3Q1ulKplY4G-qxfYTaw.png" /></figure><h4>Data Preprocessing</h4><p>After splitting the data, total number of features have been calculated, which is the total number of unique words in the corpus.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1001/1*55otYmQTgZfWc11cFBcneA.png" /></figure><p>Once we got the maximum number of features, our next task is to find the maximum review length which is the total number of words in each review.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/991/1*kMBiQLFKbCMGwmY7M3URzw.png" /></figure><h3>Tokenizer</h3><p>A neural network cannot work directly on text-strings so we must convert it somehow. There are two steps in this conversion, the first step is called the “tokenizer” which converts words to integers and is done on the data-set before it is input to the neural network. The second step is an integrated part of the neural network itself and is called the “embedding”-layer, which is described further below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/890/1*HZXwQ1-e3y4wnsStBEFdDA.png" /><figcaption>Fig. Tokenizing texts to strings</figcaption></figure><h3>Sequence Padding</h3><p>Deep learning libraries assume a vectorized representation of your data. In the case of variable length sequence prediction problems, this requires that the data be transformed such that each sequence has the same length.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/1*seQakMDmwMtSeRc14pQzoA.png" /><figcaption>Fig. Sequence Padding</figcaption></figure><p>After tokenizing and sequence padding we are ready to build our deep learning model.</p><h3>Create the Recurrent Neural Network</h3><p>The first layer in the RNN is a so-called Embedding-layer which converts each integer-token into a vector of values. This is necessary because the integer-tokens may take on values between 0 and 10000 for a vocabulary of 10000 words. The RNN cannot work on values in such a wide range. The embedding-layer is trained as a part of the RNN and will learn to map words with similar semantic meanings to similar embedding-vectors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/772/1*TuJHxNQ0eg2zYgBZdyB_xg.png" /></figure><p>The maximum number of features which we calculated before has been taken as input for the embedding layer.</p><p>I have used 3 different variants of RNN and compared the accuracy.</p><p>LSTM(Long-Short-Term-Memory)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*Digg-teHPNcohnkfllW2ag.png" /><figcaption>Fig. Adding LSTM layers</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/1*FNbTD0yxOtt23kBEabfmHw.png" /><figcaption>Fig. Fitting the model</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*HkZezcV195B_wMNliv0UTw.png" /><figcaption>Fig. Predicting Classes</figcaption></figure><p>This model gave an accuracy of around 71.94 % . Other two models I have used were CNN (Convoluted Neural Network) and CNN+GRU. The accuracy were 80.17% and 78.21%.</p><p>To test the models further, new data has been taken into consideration.</p><h3>Model Testing</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/994/1*lCNNdGJs5uRcO6_fNc9Cjw.png" /><figcaption>Fig. New Data</figcaption></figure><p>This data is further processed (tokenize and sequence padding) for predicting with different RNN models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/611/1*bkGcSjyA6GG9FXeJ-PgM_Q.png" /><figcaption>Fig. Processing the new data</figcaption></figure><ol><li>LSTM</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*-RwZcQueUWSQH2DeWI6NIg.png" /><figcaption>Fig. Predicting using LSTM model</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/286/1*SsuL0zq0bd4vG1ig_3AFww.png" /><figcaption>Fig. Predictions for New Data</figcaption></figure><p>2. CNN</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/337/1*aflWxlTfKXeDss0dcHd3NA.png" /></figure><p>3. CNN+GRU</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/306/1*XRObsJQHPIbedewGp83MPQ.png" /></figure><p>The predictions seems quite similar for various models, only the 2nd model shows some variations. More or less the predictions seems to be right.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/430/1*X_a9dz_LsmHcU7BUT773wQ.png" /></figure><p>The full code for this post can be found on Github(<a href="https://github.com/joseph10081987/Machine-Learning_new/blob/master/Movie%20Review_DL.ipynb">https://github.com/joseph10081987/Machine-Learning_new/blob/master/Movie%20Review_DL.ipynb</a>).</p><p>I look forward to hearing any feedback or comment.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f998bf54f81" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When I decided to work on Sentiment Analysis, Amazon fine food review (Kaggle project) was quite…]]></title>
            <link>https://medium.com/@josephroy/when-i-decided-to-work-on-sentiment-analysis-amazon-fine-food-review-kaggle-project-was-quite-3575721a8849?source=rss-1742e5550527------2</link>
            <guid isPermaLink="false">https://medium.com/p/3575721a8849</guid>
            <category><![CDATA[review]]></category>
            <category><![CDATA[logistic-regression]]></category>
            <category><![CDATA[text-analysis]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[sentiment-analysis]]></category>
            <dc:creator><![CDATA[Joseph Roy]]></dc:creator>
            <pubDate>Thu, 02 Aug 2018 06:26:30 GMT</pubDate>
            <atom:updated>2018-08-02T06:26:30.221Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/523/1*bXDiOoCFTSJJdTQ7JbuijQ.png" /><figcaption>Amazon Food Review</figcaption></figure><p>When I decided to work on Sentiment Analysis, Amazon fine food review (Kaggle project) was quite interesting , as it gives us a good introduction to Text Analysis. I have downloaded the data set from Kaggle(.csv) file. The (.csv) file contains collection of text documents(568,454 food reviews) up to and including October 2012.</p><p>Our task is to predict the sentiment(positive or negative) of a reviewer’s score on a scale of 1 to 5, where 1 indicates the reviewer extremely dislikes the food he or she mentions in the review and 5 indicates the user likes the food a lot.</p><h3><strong>Background about the data</strong></h3><ol><li>To read the .csv file I have used the command:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1006/1*ZxDBLIb1Q6n4PSjmB6YLGw.png" /></figure><p>2. The data set consists of the following information:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1009/1*Gy0V2V2yfiBaFA-ndK7LCQ.png" /></figure><p>3. food_data.head() gave me the below mentioned information:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/990/1*6JXrj_N63tKQaHaWJVW6pQ.png" /><figcaption>Figure 1</figcaption></figure><p>While we can see there are many features in Figure 1, but we are not interested in all the features. So, I extracted the ‘Score’ and ‘Text’ column from the Figure 1. While I was curious to know the counts for each rating, I found that the rating 5 was quite high in number as compared to other number.</p><pre>5    363122<br>4     80655<br>1     52268<br>3     42640<br>2     29769</pre><p>Immediately, I realized that my data is imbalanced. Now, as a result when I train my algorithm with this data, there is a possibility of miss classification as my data is biased towards higher rating.</p><h3>Data Preprocessing</h3><p>So, my next task was to balance the data. For balancing an imbalanced data set we can mainly follow 3 approaches:</p><ol><li>Random under sampling: Random Under sampling aims to balance class distribution by randomly eliminating majority class examples. This is done until the majority and minority class instances are balanced out. In this approach, we reduce the data from higher class (data with 4 and 5 rating) to match the data with lower class(data with 1 and 2 rating).</li><li>Random over sampling: Over-Sampling increases the number of instances in the minority class by randomly replicating them in order to present a higher representation of the minority class in the sample.</li><li>Synthetic Data Generation: In simple words, instead of replicating and adding the observations from the minority class, it overcome imbalances by generates artificial data. It is also a type of oversampling technique. In regards to synthetic data generation, synthetic minority oversampling technique (SMOTE) is a powerful and widely used method.</li></ol><p>In this project, I used Random under sampling technique to balance the data set.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/1*RLUCbM4O_pe2X8uwxgj4AA.png" /></figure><p>Since, rating 3 does not make much of a difference, as in we cannot predict whether the user gave a positive or a negative sentiment, I decided to eliminate rating 3 from consideration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*yheTntlyqoSRZtNXDL6rkA.png" /></figure><h3>Splitting the data</h3><p>While my data set is now balanced my next task was to split the data into training and test set. My input is ‘Text’ and my output would be ‘Sentiment’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1002/1*23S0tRL4GdQrgiCqKxz8oQ.png" /></figure><h3>Count Vectorizer</h3><p>After splitting the data, I have used CountVectorizer() to convert our text documents to matrix of token counts. This configuration tokenize the strings and convert them to lower case and build a vocabulary of comma separated tokens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*jn8QKvJA4yEEw1on0zdO3w.png" /><figcaption>Figure 2</figcaption></figure><p>The data is further transformed to convert it into a matrix where our rows are the text documents and columns are the words. This data could further be interpreted to an array where the frequency of each word in a single text could be counted.</p><p>In short, CountVectorizer helps us to find the frequency of a particular word in a text document.</p><h3>TfidfVectorizer</h3><p>This data is further processed by applying Tfidf Vectorizer, which helps us to give more weight-age to important words which less important words for the case study would be given more weights.</p><p>Since, our code is based on counting the frequency of each word in the document, so if certain words like ‘the’, ‘if’ etc. which are present more frequently then words which are more important such as ‘buy’,’product’ etc. , which gives us the context.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1001/1*HkC4j51XXxkXrIk_PXarUg.png" /></figure><h3>n-grams</h3><p>I decided to make the model bit more interesting by fitting our model with ‘n-grams’. This would further improve our model for example it would help us to differentiate between ‘good’ and ‘not good’ as it would take both words together(for bi gram count pairs). Also, it would help us to work with more features. I have set the n-grams in the range of 1–2 which helps us to extract features for 1 and 2 grams.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/998/1*RQmHStwHgqYcbdHqSsmbsg.png" /></figure><h3>Fitting the model</h3><p>After the pre-processing part, the model has to be fitted. For fitting this model, I have decided to work with Logistic Regression and Multinomial Naive Bayes Algorithm. I would like to compare both the models.</p><h3>-&gt; Multinomial Naive Bayes Algorithm</h3><p>Text Processing works good with Multinomial NB Algorithm. So, I decided to apply Multinomial NB .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1007/1*iQFXN-MUy-BUAwwxeKZ61w.png" /></figure><p>After fitting the model with my training data, I have also predicted the accuracy of the model with ‘AUC’(Area under the curve). It gives us a score of 0.924 which is good.</p><h3>-&gt; Logistic Regression</h3><p>Since Logistic Regression works best with high dimensional sparse data, I have decided to fit my training data with Logistic Regression.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/997/1*HhNChnKDZP1CLz4qWe02XA.png" /></figure><p>The ‘AUC’ score (0.9428) seems to give better prediction for Logistic Regression than Multinomial NB.</p><h3>Testing the model</h3><p>Finally, I decided to test my model with random sentences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/1*jukQkId34LgIe74Ou41EzQ.png" /></figure><p>My model seems to correctly predict the sentiment.</p><h3>Conclusion</h3><p>As per my project, I found that Logistic Regression has given me better result compared to Multinomial NB algorithm. But, I would also like to mention that I have used under sampling technique which means that I have not worked with entire data set. The accuracy might further improve when the entire data set is taken into consideration. It would be interesting to work with the entire data set using Deep Learning techniques such as RNN which is beyond the scope of my current work.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3575721a8849" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>