<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Manu Raj Gehlot on Medium]]></title>
        <description><![CDATA[Stories by Manu Raj Gehlot on Medium]]></description>
        <link>https://medium.com/@rajgehlotmanu?source=rss-4ed55761bfd2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*BHtNLD1rcb8l0Ax7</url>
            <title>Stories by Manu Raj Gehlot on Medium</title>
            <link>https://medium.com/@rajgehlotmanu?source=rss-4ed55761bfd2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 03:21:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@rajgehlotmanu/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Natural Language Processing with Disaster Tweets]]></title>
            <link>https://medium.com/@rajgehlotmanu/natural-language-processing-with-disaster-tweets-e1ad2b68cbcf?source=rss-4ed55761bfd2------2</link>
            <guid isPermaLink="false">https://medium.com/p/e1ad2b68cbcf</guid>
            <dc:creator><![CDATA[Manu Raj Gehlot]]></dc:creator>
            <pubDate>Thu, 08 Sep 2022 08:49:00 GMT</pubDate>
            <atom:updated>2022-10-04T07:24:34.144Z</atom:updated>
            <content:encoded><![CDATA[<p>Twitter has become an important communication channel in times of emergency. But, it’s not always clear whether a person’s words are actually announcing a disaster as it is a social media language so it is not easy to identify them, as a result it is given as a Kaggle problem in which we have to build a machine learning model that predicts which Tweets are about real disasters and which one are not .hence in this case study we will cover the following Aspect</p><ol><li>Business Problem</li><li>Problem statement</li><li>Business Objective</li><li>Machine Learning Formulation</li><li>Mapping Real World problem to an ML problem</li><li>Metric Used</li><li>File Exploration</li><li>Existing Approaches and there improvement</li><li>Data Cleaning</li><li>Exploratory Data Analysis</li><li>First Cut Approach</li><li>Feature Engineering</li><li>Model Creation</li><li>Deployment</li><li>Future Works</li><li>Links</li><li>References</li></ol><p>BUSINESS PROBLEM</p><p>here we have access to a dataset of 10,000 tweets that were hand classified and three files have been provided train.csv, test.csv and sample_submission.csv where each sample in the train and test set has the following information: The text of a tweet, A keyword from that tweet (although this may be blank!) The location the tweet was sent from (may also be blank. Here we have to predict whether a given tweet is about a real disaster or not. If so, predict 1. If not, predict a 0 hence it is a binary classification problem here we will use f1 score which will give priority to accuracy and recall</p><p>PROBLEM STATEMENT</p><p>Given a Tweet predict whether it tells about a real disaster or not</p><p>BUSINESS OBJECTIVE AND CONSTRAINT</p><ol><li>There is no strict latency requirement but it should not take more than few seconds while predicting</li><li>It would be helpful if we know while a particular tweet is classified as related to real disaster</li></ol><p>MACHINE LEARNING FORMULATION</p><p>A.DATA</p><p>SOURCE:<br>https://www.kaggle.com/competitions/nlp-getting-started</p><p>PROVIDED BY:</p><p>company figure-eight and originally shared on their ‘Data For Everyone’.</p><p>DATA DESCRIPTION</p><p><strong>Files</strong></p><ol><li>train.csv — the training set</li><li>test.csv — the test set</li><li>sample_submission.csv — a sample submission file in the correct format</li></ol><p><strong>Columns</strong></p><ol><li>id — a unique identifier for each tweet</li><li>text — the text of the tweet</li><li>location — the location the tweet was sent from (may be blank)</li><li>keyword — a particular keyword from the tweet (may be blank)</li><li>target — in train.csv only, this denotes whether a tweet is about a real disaster (1) or not (0)</li></ol><p>MAPPING REAL WORLD PROBLEM TO ML PROBLEM</p><ol><li>Here our task is to predict tweets which corresponds to real disaster with the help of tweet and given features</li><li>It can be solved as a binary classification problem with 1 denoting tweet representing real disaster and 0 representing tweet does not representing real disaster</li></ol><p>METRIC USED</p><ol><li>F1 SCORE The F1-score combines the precision and recall of a classifier into a single metric by taking their harmonic mean</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*iL4pi8bMuH53uphEml6sqg.png" /></figure><p>.</p><p>EXISTING SOLUTION</p><p>1. <a href="https://www.kaggle.com/code/nabanitaroy/eda-and-tf-idf-prediction-model-with-80-accuracy">https://www.kaggle.com/code/nabanitaroy/eda-and-tf-idf-prediction-model-with-80-accuracy</a></p><p>in this case study EDA is performed and they checked for missing values,how data is distributed, distribution of keyword by distribution of target by keyword ,distribution of target by location and most frequently occuring words than performed text cleaning by removing,html links ,numbers,special character etc than for featurization did punctuation,keyword analysis etc and build features and printed word cloud</p><p>than converted features to vectors used count vectorizer and tfidf vectorizer and performed modelling by considering f1 score as metric and tried various model like decision tree,naïve bayes etc and found best f1 score for logistic regression</p><p>2. <a href="https://www.kaggle.com/code/mahmoudftolba/natural-language-processing-with-disaster-tweets">https://www.kaggle.com/code/mahmoudftolba/natural-language-processing-with-disaster-tweets</a></p><p>he also did data acquisition ,text cleaning,pre processing feature engineering as mostly same as previous but made changes with respect to how modelling was done as used BERT model</p><p>IMPROVEMENT TO EXISTING APPROACH</p><p>AS I Tried new models like Ridge Classifier ,ensemble techniques like Voting Classifier and Stacking Classifier along with I Hypertuned every model so that they perform good along with I tried LSTM model with some new features</p><p>DATA CLEANING</p><p>here first I checked for Null values and found that most of the null values are in Location column so I dropped it and very few Null values in Keyword Column were also dropped than generally text cleaning would be done before feature engineering but as out feature depends on uncleaned text we will be cleaning text after feature engineering and steps which are involved in text cleaning are simple like removing links ,numbers ,special keyword etc</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d4f1ee7f96bc10a05c526e677388b548/href">https://medium.com/media/d4f1ee7f96bc10a05c526e677388b548/href</a></iframe><p>EXPLORATORY DATA ANALYSIS</p><ol><li>To check how class are distributed I decided to plot bar plot. I found out that classes are unequally distributed with class 0 being the major class with 4323 points i.e. 57.24% and class 1 having 3229 points i.e 42.76%</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/733/1*PCGlqhpnaREt17hEviiM1Q.png" /></figure><p>2.I tried to find out what are the top 6 keywords which are most frequent used and so I plotted bar plot as a result what I identified top 6 words which are most frequent used are fatalities, armageddon, deluge, sinking, damage, harm</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/699/1*0l_mM-5QqwHPu3Gcp0-L2w.png" /></figure><p>3.I tried to found out the total number of characters in disaster and non disaster tweet as a result i plotted the Histplot and found out that highest character count for disaster tweet is 50 where as highest character count for non disaster tweet is also nearly 5o but more words have character count of 50 for non disaster tweet</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/714/1*rzhiDJteo1Zuv8gpUVBQFw.png" /></figure><p>4.Here I tried to find out the most common words which has been used for classification as disaster and non disaster tweet as a result I plotted the bar plot and found out that most common word for disaster tweet are fire, bomb, kill , news, disast and flood where as most common word for non disaster tweet is like, love, bodi, time, video, would</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*v3a6SvN3SlRDrRrukItQAw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*b3KriFz4OYdD2rdqbE8CnA.png" /></figure><p>5. Here I tried to find out that how many tweets carry mention in it that is ,name following @ as a result I plotted bar plot so i found out that there are 5545 tweets that does not carry any mentions and there are 1530 tweets which carry one mention and so on</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*t8TjbPYiItUDWxv5D5_KXg.png" /></figure><p>6. Here I tried to find out that in a single tweet how many sentence are there as a result I plotted bar plot and found out that maximum tweets carry one sentence only and 1907 tweets contain 2 sentence</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/726/1*bJLQw3bwUdf9wSB4AU3QKg.png" /></figure><p>7.Here I tried to find out the total number of hash(#) used in a tweet and as .a result I plotted bar plot for hash count vs total tweets having that count and found out that maximum tweets does not have # but there are 953 tweets that contain hash</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/669/1*9KFHE5S0RKDBJLW34WI1_Q.png" /></figure><p>8.Here I tried to found out that how many number of unique words are present in a tweet as a result I plotted the bar plot and I found out that at max tweets have 13 unique words</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*q3cvtB4A5zua5R901jG09g.png" /></figure><p>9. In figure one I tried to find out how many capital characters are in a tweet than for the same I plotted the bar plot and found out that most sentences have 1 or 2 capital character. In figure two I tried to find out how many punctuations are there in a tweet than for the same I plotted the bar plot and found out that most tweets have 4 or 3 punctuations</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/671/1*SqmLqkWQ6g_Kpd8PSuo5XQ.png" /></figure><p>10. I tried to find out the word length in disaster and non disaster tweet and as a result I plotted box plot for both the classes and found out that for disaster tweet total number of words are greater than equal to 6 and less than 10 for 50% of data and there is only one tweet with 20 words where as in non disaster tweet 50% of points have total number of words are greater than 5 and less than 8 and there are 3 points with 16,17,20 word length</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/424/1*QfOOqNbjlRxTfawDEV8tzw.png" /></figure><p>11. Here I tried to found out that how does number of sentence affects being classified as disaster tweet or non disaster tweet as a result i plotted this graph and than it can be identified that major points of both the classes have same count of sentence i.e. 1 but few points have 2 as a sentence count than it can be found that most points belong to negative class with 2,3 sentence count hence presence of only 2 sentence helps in identifying that majorly it belongs to negative class but overall it is not a much useful feature to tell whether a particular tweet belong to disaster class or not</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K7QEn-wp0o2sSLVuL48OUA.png" /></figure><p>12.Here i tried to found out that how does number of hash present in a tweet affects being classified as disaster tweet or non disaster tweet as a result i plotted this graph and than it can be identified that major points of both the classes have same count of has i.e. 0,1,2,3 majorly hence it is not a useful feature to tell whether a particular tweet belong to disaster class or not</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2rE0MDt7pcuw2iDjsYohsg.png" /></figure><p>13.Here I tried to found out that how does number of unique words present in a tweet affects being classified as disaster tweet or non disaster tweet as a result I plotted this graph and than it can be identified that major points of both the classes have same count of unique word</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iASoaY3a03YNjb8pOQZo9A.png" /></figure><p>14.in order to find overall most used words I plotted word cloud and found out fire,news,burn,flood,fatal,crash,attack,build,video,kill etc</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/345/1*7gs2N8YCdTfoIyhiG3dVQA.png" /></figure><p>FIRST CUT APPROACH</p><p>Did file exploration and Data cleaning by removing the columns which are not useful and also removed null values which are very few in number and than did feature engineering and tried very basic model like logistic regression and linear svm without hyper parameter tuning.</p><p>FEATURE ENGINEERING</p><p>Here I tried to find feature on uncleaned text and first feature which I used was total number of words used in tweet ,second feature which was used is total number of character used in a tweet and third feature which I used is how many capital character are there in a tweet and fourth feature which I used is total number of punctuations used in Tweet some other feature which I used are total number of hash count ,total mention count, total number of sentence used in a tweet along with average word length, average sentence length and average unique word ratio in a tweet</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0e14af3ec8cbfd7a2c82e7e7c6d7db78/href">https://medium.com/media/0e14af3ec8cbfd7a2c82e7e7c6d7db78/href</a></iframe><p>than I splitted the data by considering 33% of points as test data after that i normalized the numerical data to bring data on same scale and for keyword which is categorical feature I tried label encoder to convert into numerical value and for converting text to vector I used countvectorizer</p><p>MODEL CREATION</p><p>Logistic Regression</p><p>first model which I tried was Logistic Regression combined with Hyper parameter tuning with random search cv where cv is taking value 4 and the hyper parameter which is tuned is C which takes values 0.00,0.01,0.1,1,10 and solver just take a single value ‘newton-cg which gave train and test f1 score as 0.76 and 0.70 respectively</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fc8256fc06ff569a14767f1eb86d9e97/href">https://medium.com/media/fc8256fc06ff569a14767f1eb86d9e97/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/630/1*5hQJZMlXSJ_iXAn0HkKRMQ.png" /></figure><p>Second Model which I tried was Linear SVM combined with Hyper parameter tuning with random search cv where cv is taking value 4 and the hyper parameter which is tuned is C which takes values 0.00,0.01,0.1,1,10 and loss just take a single value squared hinge with penalty is being hyper tuned by taking values as l1,l2 which gave train f1 score as 0.74 and test f1 score as 0.69</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2e422ca971be56a98d5d31a960aa86b9/href">https://medium.com/media/2e422ca971be56a98d5d31a960aa86b9/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/705/1*3ROHV_iEILCk36HdNnfxfw.png" /></figure><p>Third model which I used is Ridge classifier which uses Ridge regression to create classifier by converting target class into +1 or -1 and the loss function which is used is MSE with l2 penalty if ridge regression prediction value is greater than 0 than point belongs to positive class else negative class gave train f1 score as 0.82 and test f1 score as 0.72 which is better than previous models</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b93b009dcb587ee22a550e4a3deddcd2/href">https://medium.com/media/b93b009dcb587ee22a550e4a3deddcd2/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*jYl21xngAEpmT9Ht-GkqEA.png" /></figure><p>fourth model which I used if Gaussian Naïve Bayes which is probabilistic classification algorithm based on applying Bayes Theorem with strong independence assumption which gave train f1 score as 0.60 and test f1 score as 0.55 which performing worst</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7d9931880a3d4a6c2c9d84c4e32c1bee/href">https://medium.com/media/7d9931880a3d4a6c2c9d84c4e32c1bee/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/664/1*r1Wdw68O5jcAddvGajqM3Q.png" /></figure><p>Fifth model which I used is XGB classifier combined with Hyper parameter tuning with random search cv where cv is taking value 4 and the hyper parameter which is tuned are learning rate,min_child_weight,max_depth,subsample with values being given below in XGB.py which gave train f1 score as 0.84 and test f1 score as 0.72</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/144383252683e8afd4045056494c144d/href">https://medium.com/media/144383252683e8afd4045056494c144d/href</a></iframe><p>Sixth model which I used is Random Forest classifier combined with Hyper parameter tuning with random search cv where cv is taking value 4 and the hyper parameter which is tuned along with values being given below in randomforest.py code gave train f1 score as 0.37 and test f1 score as 0.35 which shows that model is performing worst</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/09f00bb0830ad65a989b8a94ed17e4fb/href">https://medium.com/media/09f00bb0830ad65a989b8a94ed17e4fb/href</a></iframe><p>Seventh model which I used is Gradient Boosting classifier combined with Hyper parameter tuning with random search cv where cv is taking value 4 and the hyper parameter which is tuned along with values being given below in GradientBoosting.py code which gave train f1 score as 0.85 and test f1 score as 0.73 as a result it can be seen that this model has been performing worst</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/30e4e8ad86619abb1bcd95bf9155ab7b/href">https://medium.com/media/30e4e8ad86619abb1bcd95bf9155ab7b/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*6evi_s8G4kKq7dS4CsItrg.png" /></figure><p>eight model which I used is Voting classifier which is a ,machine learning model that trains on an ensemble of numerous models and predicts an output class based on there highest probability of chosen class as an output which gave train f1 score as 0.80 and test f1 score as 0.72</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/72f8c776fd33893b7df2072daff6783c/href">https://medium.com/media/72f8c776fd33893b7df2072daff6783c/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/637/1*HAOONCH3-AnL-0z1K4mMPA.png" /></figure><p>nineth model which I used is Stacking classifier which is an ensemble learning technique consist of two layer estimator where first layer consist of all the base line models which predicts the output and second layer consist of meta classifier which takes baseline model output as input and give final prediction which gave train f1 score as 0.80 and test f1 score as 0.72</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dc22569b421c14c6d374d22560842984/href">https://medium.com/media/dc22569b421c14c6d374d22560842984/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/610/1*GwERNS-_9mA-HiYh2JRUYw.png" /></figure><p>LSTM</p><p>LSTM stands for long short term memory is an advanced RNN which allows information to persist and it also overcomes the vanishing gradient problem faced by RNN so I used LSTM here due to its benefits and procedure which i followed is I used the tokenizer which was fitted on text train data as a result this helped in finding the vocab size and this tokenizer helped in converting text to sequence and on which further padding is done and on test data which is converted to sequence using train tokenizer on which padding is done than same procedure if followed for keyword also than all the text, keyword and numerical data is combined and on final vectors LSTM model is applied</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/49fe70ead926b797eb0961f60dfb4406/href">https://medium.com/media/49fe70ead926b797eb0961f60dfb4406/href</a></iframe><p>BERT</p><p>Bert stand for <strong>Bidirectional Encoder Representations from Transformers</strong> (<strong>BERT</strong>) is a transformer based machine learning technique for Natural Language Processing original English language Bert has two models Bert(Base) and Bert(Large) both models are pre-trained from unlabeled data extracted from Books corpus .here I created a Bert Model from tensor flow hub and than I did tokenization ,than i got the pretrained embedding vector for a given tweet from Bert and than using this embedding data applied neural network and classified the Tweet .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a3f04914f5edc8cf42d5324cbf4c4bc1/href">https://medium.com/media/a3f04914f5edc8cf42d5324cbf4c4bc1/href</a></iframe><p>MODEL COMPARISON</p><p>when compare all the models best model was LSTM model after that Gradient Boosting classifier and worst performing model is random forest</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/692/1*vzQNA_qlvoVzxwhIsQezJg.png" /></figure><p>FUTURE WORKS</p><p>various other NLP models can be tried along with ML models like lightBGM,h20 GBM,h20 random forest which can improve performance along with new features also can be devised</p><p>DEPLOYMENT</p><p>final model which I used for deployment is LSTM which is working good on train and test data compared to other models .along with this Streamlit library is used with pyngrok library to deploy model . for deploying I wrote app.py which is linked by ngrok with public url and while deploying it is taking individual points and and give final prediction</p><p>Video Link</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FQ7-kkWQGoYo%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DQ7-kkWQGoYo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FQ7-kkWQGoYo%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e9412609f7be264c63d7786f1a0c4c51/href">https://medium.com/media/e9412609f7be264c63d7786f1a0c4c51/href</a></iframe><p>SCREENSHOT OF WEB APP</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*HPu8LLJgSI6IRQvUXu_r2g.png" /></figure><p>REFERENCES</p><ol><li><a href="https://www.geeksforgeeks.org/stacking-in-machine-learning-2/">https://www.geeksforgeeks.org/stacking-in-machine-learning-2/</a></li><li><a href="https://www.appliedaicourse.com/">https://www.appliedaicourse.com/</a></li><li><a href="https://medium.com/">https://medium.com/</a></li><li><a href="https://streamlit.io/">https://streamlit.io/</a></li><li><a href="https://machinelearningmastery.com/clean-text-machine-learning-python/">https://machinelearningmastery.com/clean-text-machine-learning-python/</a></li><li><a href="https://en.wikipedia.org/wiki/Long_short-term_memory">https://en.wikipedia.org/wiki/Long_short-term_memory</a></li><li><a href="https://en.wikipedia.org/wiki/BERT_(language_model)">https://en.wikipedia.org/wiki/BERT_(language_model)</a></li></ol><p>LINKS</p><p>Kaggle dataset:</p><p><a href="https://www.kaggle.com/competitions/nlp-getting-started">https://www.kaggle.com/competitions/nlp-getting-started</a></p><p>LinkedIn profile :</p><p><a href="http://www.linkedin.com/in/manuraj-gehlot-143146219">www.linkedin.com/in/manuraj-gehlot-143146219</a></p><p>Github:</p><p><a href="https://github.com/manurajgehlot/CASE_STUDY_2.git">https://github.com/manurajgehlot/CASE_STUDY_2.git</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e1ad2b68cbcf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting Customer Satisfaction on Brazilian E-Commerce Olist Dataset]]></title>
            <link>https://medium.com/@rajgehlotmanu/predicting-customer-satisfaction-on-brazilian-e-commerce-olist-dataset-4a2a6019d5fe?source=rss-4ed55761bfd2------2</link>
            <guid isPermaLink="false">https://medium.com/p/4a2a6019d5fe</guid>
            <dc:creator><![CDATA[Manu Raj Gehlot]]></dc:creator>
            <pubDate>Thu, 01 Sep 2022 11:42:59 GMT</pubDate>
            <atom:updated>2022-09-01T11:42:59.067Z</atom:updated>
            <content:encoded><![CDATA[<p>Olist, the largest department store in Brazilian marketplaces. Olist connects small businesses from all over Brazil to channels without hassle and with a single contract. Those merchants are able to sell their products through the Olist Store and ship them directly to the customers using Olist logistics partners. See more on our website:<a href="https://www.olist.com/"> www.olist.com</a></p><p>After a customer purchases the product from OList Store a seller gets notified to fulfil that order. Once the customer receives the product, or the estimated delivery date is due, the customer gets a satisfaction survey by email where he can give a note for the purchase experience and write down some comments. Now here in this blog we will explore the problem of predicting the customer satisfaction after purchasing the products.</p><p>In this blog we will try to cover the following aspects</p><ol><li>Business problem</li><li>Problem statement</li><li>Business objective</li><li>Machine learning formulation</li><li>Mapping real world problem to ML problem</li><li>Metric used</li><li>File exploration</li><li>Existing Approaches and there improvement</li><li>Data cleaning and data exploration</li><li>First cut approach</li><li>Feature engineering</li><li>Model creation</li><li>Deployment</li><li>Future works</li><li>Links</li><li>References</li></ol><p>This is a Brazilian ecommerce public dataset of orders made at Olist Store. The dataset has information of 100k orders from 2016 to 2018 made at multiple marketplaces in Brazil. Its features allows viewing an order from multiple dimensions: from order status, price, payment and freight performance to customer location, product attributes and finally reviews written by customers. It also has geolocation dataset that relates Brazilian zip codes to latitude/longitude coordinates. This is real commercial data, it has been anonymized, and references to the companies and partners in the review text have been replaced with the names of Game of Thrones great houses.</p><p><strong>PROBLEM STATEMENT</strong></p><p>Given the customer data predict the customer satisfaction whether positive or negative i.e. whether he is satisfied with the purchase of products or not</p><p><strong>BUSINESS OBJECTIVE AND CONSTRAINT</strong></p><p>There is no strict latency requirement but it should not take very long time i.e. beyond few seconds and we have to make sure that both the classes are correctly classified</p><p><strong>MAPPING REAL WORLD PROBLEM TO AN ML PROBLEM</strong></p><ol><li>Here our main task is to predict review score given features like payment type, payment value, review comment message etc.</li><li>it can be solved as binary classification problem with giving 0 as negative review score and 1 as positive review score</li></ol><p><strong>METRIC USED</strong></p><p><strong>MICRO F1 SCORE: </strong>used micro f1 score as we have to take into consideration the frequency of occurrence of each class with focus on precision and recall of each class. micro f1 score computes a global average f1 score which is calculated by counting true positives (TP),False negatives(FN), and the False Positives(FP) across all classes and than put them in f1 score formula</p><p><strong>File Exploration:</strong></p><p>● Olist_customers_dataset.csv This dataset has information about the customer and its location. Use it to identify unique customers in the orders dataset and to find the orders delivery location.</p><p>● Olist_geolocation_dataset.csv This dataset has information Brazilian zip codes and its lat/lng coordinates.</p><p>● Olist_order_items_dataset.csv This dataset includes data about the items purchased within each order.</p><p>● Olist_order_payments_dataset.csv This dataset includes data about the orders payment options.</p><p>● Olist_order_reviews_dataset.csv This dataset includes data about the reviews made by the customers.</p><p>● Olist_orders_dataset.csv This is the core dataset. From each order you might find all other information.</p><p>● Olist_products_dataset.csv This dataset includes data about the products sold by Olist.</p><p>● Olist_sellers_dataset.csv This dataset includes data about the sellers that fulfilled orders made at Olist. Use it to find the seller location and to identify which seller fulfilled each product.</p><p>● Product_category_name_translation.csv Translates the product category name to English.</p><p>we have to read all this csv files and merge into a single data frame based on common keys which will help in retrieval of important information and removal less important information</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*byCWwwXKGJT2_HHa" /></figure><p><strong>EXISTING APPROACHES</strong></p><ol><li><a href="https://www.kaggle.com/code/henriqueosinski/nlp-of-reviews-with-binary-and-multiple-classes">https://www.kaggle.com/code/henriqueosinski/nlp-of-reviews-with-binary-and-multiple-classes</a> in this approach basic text preprocessing has been done and applied NLP on top of it to give prediction tried very basic model</li><li><a href="https://www.kaggle.com/code/andresionek/predicting-customer-satisfaction">https://www.kaggle.com/code/andresionek/predicting-customer-satisfaction</a> did EDA and extracted features and treated as a regression problem by trying just linear regression</li></ol><p><strong>IMPROVEMENT TO EXISTING APPROACH</strong></p><p>Tried new features like discount or not ,words given per review etc. to check whether they affect review score or not. Tried Most of the models with hyper parameter tuning like logistic regression ,linear SVM, XGB classifier, STACKING classifier</p><p><strong>DATA CLEANING</strong></p><p>Data cleaning is one of the most important part of Machine learning.it can decide the success and failure of model hence more time is invested in this stage it involves handling of missing data, removal of unwanted observations, fixing structural errors , Managing unwanted Outliers as a result, in various column we will check for null values and handle them and check for duplicate values and remove them and clean the text such that we will remove link ,numbers ,special character and other things except for text and cells of column having null values are filled with appropriate method and at last duplicate values are removed. To fill null values in cells of columns used median based imputation method for“product_description_lenght”,”price”,”payment_value”,”review_score”,”product_name_lenght”,”product_description_lenght”,”product_photos_qty”,”product_weight_g”,”product_height_cm”,”product_length_cm”,”product_width_cm” To fill null values in cells of columns in which value counts between first and second position having a huge difference as a result used the occurrence of first value to fill the columns like “payment_sequential”,”payment_type”,”payment_installments”,”product_category_name_english”,”order_item_id”,”seller_zip_code_prefix”,”seller_city”,”seller_state”, null value in review comment message are replaced with a text “no review given” in Portuguese and finally dropped the duplicates . wherever timestamp is empty filled with another relevant timestamp and done preprocessing on text data such that it removes hyperlinks, number special character, done stemming also and also dropped few columns where are not useful like “review_answer_timestamp”,”review_creation_date”,”review_id”,”review_comment_title”,”order_delivered_carrier_date”,”product_category_name”.Also made changes in review score so that it becomes binary classification problem as a result changed the value of review score greater than 3 as 1 (positive) and less than equal to 3 as 0(negative) and where ever date timestamp is given converted it to string to date time and than retrieved its date</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b1c3e68b9a539e43bb93e47932915729/href">https://medium.com/media/b1c3e68b9a539e43bb93e47932915729/href</a></iframe><p>TEXT CLEANING</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d331a0cfd03ecb6a9eeaaa4333e31650/href">https://medium.com/media/d331a0cfd03ecb6a9eeaaa4333e31650/href</a></iframe><p><strong>EXPLORATORY DATA ANALYSIS</strong></p><ol><li>To see distribution of number of data points based on review score .I plotted bar plot for review score and we can see in the bar plot that most of the data is of positive review so its a imbalanced data. We should be highly cautious while building model as it is heavily imbalanced data</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*VyNNaxCrhxteMsEX" /></figure><p>2.To check how much a particular customer used a particular payment method for payment I plotted a PIE-CHART for payment method and from visuals it is evident that around 76% of customer uses credit card to do the payment. There is a huge difference between % of people who uses credit card and % of people who uses baleno(voucher based payment method)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*4wDYP49VLpM7x36h" /></figure><p>3. tried to check that how does the payment value for products differ for both positive and negative review and as result plotted KDE plot from the above graph it can be seen than for positive review and negative review products most prices are less than 1000 both the plots are overlapping so it wont be much useful to predict the target class as review score</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/0*7H585XXFXSE7sFdM" /></figure><p>4. Tried to observe that people pattern of payment i.e. they prefer to pay in how many installments so it can be identified using bar plot for payment installment that most people prefer single installments</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*hAOKyfsGsw4nVUvp" /></figure><p>5.To check the order status of most of the products as most of the customer might be satisfied or dissatisfied due to particular order status i.e. if orders are not delivered as a result we can see that most of orders are Delivered</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*7IJ2o6BnL1zYd1KL" /></figure><p>6. To check does the customer payment method affects his satisfaction about the product plotted the bar plot between payment method and average review score given and it was found that people are more satisfied if they do payment with the debit card</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*HP6SB33BEGvYMOnw" /></figure><p>7. To check does product description, its name given and total number of photos given affect the review given by customer so I have plotted pair plot between all the variable.as they are completely mixed up so they are not much useful in predicting review score as there is not much separability and it can be seen that there are more positive review points.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*pkggkvTTY1qMklND" /></figure><p>8.To find out what is average rating of the product and which are the most rated product bar plot has been plotted between product and average rating.It has been found out that most rated product is cd_dvds_musicals which is around 4.7</p><p>9.By plotting bar plot for product name tried to find out products that were most preferred by customer and it was found out that bed_bath_table was most chosen product</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*-Y9MSy-OZwe7wIz7" /></figure><p>10.To check from which state most products are ordered I plotted bar plot and found out that they are from SP</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*Vknjp3ZjwJUJqhBp" /></figure><p>11. To find out which are the most expensive product, A bar plot has been plotted between product and average price It has been found out that most expensive product is computers which is around 1247 Brazilian real</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*fdz_bqFmHMoGxee4" /></figure><p>12. To Check does the final payment made by customer affects the review given by customer for the product I have plotted box plot between review_score and payment_value. It can be seen than most of the expensive products customer gives positive rating and few expensive as negative rating.It proves that customer thinks more expensive products are generally very good</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*AUhKhg1S3C4C063G" /></figure><p>13. To check does the name length and description length affects the review given by customer for a particular plot so I plotted kde plot by taking name length and description length on x axis and hue as review score from the plot it can be observed that negative review customer are more widely spread and both positive and negative review has more product name length between 25–60 and product description length 0–2000 majorly but it is not much useful as they are overlapping</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*_2cw5QYxjMQT77-s" /></figure><p>14.Plotted word cloud to check which words does occur most It can be seen that words NEG,OAO,PRODUT,REVEJ occurs the most</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/569/0*ILKSspXDV2vk1EIN" /></figure><p><strong>FIRST CUT APPROACH</strong></p><p>Did File exploration and Data cleaning by replacing null values, removing duplicates and removing the columns which are not useful than did in depth Exploratory Data analysis to identify which feature might be useful in predicting the target after that Extracted features basic features like on time or late, estimated delivery time etc. and tried just logistic regression without hyper parameter tuning</p><p><strong>FEATURE ENGINEERING</strong></p><p>1. f1 tells the estimated delivery that is days between approval date and estimated delivery date as the less the number of days for delivery the more the happy the customer is</p><p>2. f2 tells about in how many days the actual delivery was made the less it is the more happy the customer is</p><p>3. f3 tells about whether the customer was late or on time</p><p>4. f4 tells about that whether customer got some discount or not as the more the discount is the more happy the customer is</p><p>5. f5 tells whether for product to take approval does it take time or not .the lesser it is the more the happy customer is</p><p>6. f6 tells about compared to product price how much freight was imposed the costlier the freight value is the more faster and safer it would be and the more customer happy will be</p><p>7. f7 tells about the average price of product compared to other products</p><p>8. f8 tells about the day of week the purchase is made it might affect the review given as on particular day there might be some offer so that customer is happy</p><p>9. F9 tells about for particular product how many sellers are there</p><p>10. F10 tells about per order how many products are ordered</p><p>11. f11 is the day when purchase is made i.e. does a particular day affects review given by customer</p><p>12. f12 denotes words used per review</p><p>13. f13 denotes hour of the day when purchase is made</p><p>14. f14 denotes how many days it took to deliver order</p><p>15. f15 denotes week day when purchase is made</p><p>16. f16 denotes order purchase month</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/575a52996ef2fb2ba49ccd2db324ec85/href">https://medium.com/media/575a52996ef2fb2ba49ccd2db324ec85/href</a></iframe><p>Now Splitting of data has been done in the ratio of 70:30 as train and test and For the features which are numerical they have been normalized so that they come on same scale and for categorical variable they have been converted to vector using one hot encoding and for text it has been converted to vector using TFIDF vectorizer and finally horizontally stacked all the features</p><p><strong>Model</strong></p><p><strong>Logistic regression</strong></p><p>First model which had been used is LOGISTIC REGRESSION along with hyper parameter tuning and the hyper parameters which are tuned are penalty ,solver ,C(inverse of lambda) as result after tuning the best values of hyper parameter came out to be {‘solver’: ‘newton-cg’, ‘penalty’: ‘l2’, ‘C’: 10} and which gave train f1 score score as 0.929 and 0.921 as test f1 score.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/86469278cdda7d8215f03e3bd896f99b/href">https://medium.com/media/86469278cdda7d8215f03e3bd896f99b/href</a></iframe><p><strong>Linear SVM</strong></p><p>Second model which had been used is linear SVM along with hyper parameter tuning and the hyper parameters which are tuned are C taking values as [0.1,1,10,100],loss taking values as [‘hinge’,’squared_hinge’],penalty taking values as [‘l1’,’l2&#39;] so after doing hyper parameter tuning best values of hyper parameters came out to be {‘penalty’: ‘l2’, ‘loss’: ‘hinge’, ‘C’: 1} Which gave train_f1_score as 0.928 and test_f1_score as 0.922</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba991e7aab79b3f87a2757179d43f554/href">https://medium.com/media/ba991e7aab79b3f87a2757179d43f554/href</a></iframe><p><strong>RANDOM FOREST CLASSIFIER</strong></p><p>Third model which has been used is Random forest classifier along with hyper parameter tuning and the hyper parameters which are tuned are max_depth taking value as [3,6,10],min_samples_split taking value as [1,5,10] as a result after tuning best values of hyper parameter came out to be as min_samples_split is 10 and max_depth as 10 but these values gave train_f1_score is 0.876 and test_f1_score as 0.871 which is very low hence the model is not performing very good</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8d294b6c7083317cb61546fd3efe7d4d/href">https://medium.com/media/8d294b6c7083317cb61546fd3efe7d4d/href</a></iframe><p><strong>DECISION TREE</strong></p><p>Third model which has been used is Decision Tree classifier along with hyper parameter tuning and the hyper parameters which are tuned are criterion taking values as [‘gini’,’entropy’],max_depth taking values as [1,3,5],min_samples_leaf taking values as [1,2,3] and after hyper parameter tuning best values of hyperparameter values came out to be as ‘min_samples_leaf’: 3, ‘max_depth’: 5,’criterion’: ‘gini’ which gave train_f1_score as 0.892 and test_f1_score 0.891 which signifies that model has not been performing very good .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ff03c5cad7d0dbafce525ac42f310ee5/href">https://medium.com/media/ff03c5cad7d0dbafce525ac42f310ee5/href</a></iframe><p><strong>XGB CLASSIFIER</strong></p><p>Third model which has been used is XGB classifier with random search<strong> </strong>classifier along with hyper parameter tuning and the hyper parameters which are tuned are ‘learning_rate’ taking values as [0.01,0.1,1],’min_child_weight’ taing values as [1,3,5],‘Max_depth’ taking values as [3,5,7],’subsample’ taking values as [0.5,0.7] after which hyper parameter best values came out to be {‘subsample’: 0.5, ‘min_child_weight’: 5, ‘max_depth’: 7, ‘learning_rate’: 0.1} which gave train_f1_score 0.923 and test_f1_score 0.916 this model is good but not performing as good as logistic regression</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7f72f3c5790950b2a93dc17bb8d425ff/href">https://medium.com/media/7f72f3c5790950b2a93dc17bb8d425ff/href</a></iframe><p><strong>GRADIENT BOOSTING CLASSIFIER</strong></p><p>Third model which has been used is GRADIENT BOOSTING CLASSIFIER along with hyper parameter tuning and the hyper parameters which are tuned are learning_rate taking values as [0.01,0.1,1],min_samples_leaf taking values as [0.1,0.3,0.5],max_depth taking values as [1,3,5],Loss taking values as[‘log_loss’,’deviance’] after which hyper parameter best values came out to be as {‘min_samples_leaf’: 0.1, ‘max_depth’: 5, ‘loss’: ‘deviance’, ‘learning_rate’: 1} which resulted in train_f1_score came as 0.872 And test_f1_score 0.869</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d51462917aa7a651c84e35294c83680/href">https://medium.com/media/6d51462917aa7a651c84e35294c83680/href</a></iframe><p><strong>VOTING CLASSIFEIR</strong></p><p>A voting classifier is a machine learning estimator that trains various base models or estimators and predicts on the basis of aggregating the findings of each base estimator .it is an ensemble model hence I took base estimators as logistic regression, XGB classifier ,Linear SVM as result this method improved the performance of model which gave test_f1_score as 0.923 and train_f1_score as 0.930</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/707a407e6d9550b437a9f21b91f37fee/href">https://medium.com/media/707a407e6d9550b437a9f21b91f37fee/href</a></iframe><p><strong>STACKING CLASSIFIER</strong></p><p>Stacking is a way of ensembling classification models it consists of two-layer estimators. The first layer consists of all the baseline models that are used to predict the outputs on the test datasets. The second layer consists of Meta-Classifier which takes all the predictions of baseline models as an input and generate new predictions here I used base line model as logistic regression, XGB classifier ,linear SVM this improved the performance of model as a result it gave test_f1_score as 0.923 and train_f1_score 0.930</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/30c2b197a9ed95a5c6e528d3d1943ccd/href">https://medium.com/media/30c2b197a9ed95a5c6e528d3d1943ccd/href</a></iframe><p><strong>MODEL COMPARISON</strong></p><p>It can be seen that best model which came out to be are Voting Classifier and Stacking Classifier</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*UalqZShmjloDm8TT" /></figure><p><strong>FUTURE WORK</strong></p><p>It can be treated as Multiclass classification problem with multiclass log loss as metric and instead of 0 or 1 as class ,1,2,3,4,5 can be used as class</p><p><strong>DEPLOYMENT</strong></p><p>In Deployment best model has been used i.e. Stacking Classifier and Stream lit is used to deploy the model and while giving output all the files corresponding to one single point has been merged in a single excel file which is read at the time of execution and here I worked with 6 data points 2 positive and 2 negative and 3 false cases ,video and screenshot of the same is shown below</p><p>Video link :please open by right click on link than choose open link in new tab<a href="https://www.youtube.com/watch?v=tCw7TJvXTKQ"> https://www.youtube.com/watch?v=tCw7TJvXTKQ</a></p><p>SCREENSHOT OF WEB APP</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*v_ogfmBmoFBL4KVE" /></figure><p>Web APP</p><p><strong>REFERENCES</strong></p><ol><li><a href="https://www.appliedaicourse.com/">https://www.appliedaicourse.com/</a></li><li><a href="https://www.kaggle.com/">https://www.kaggle.com/</a></li><li><a href="https://www.geeksforgeeks.org/">https://www.geeksforgeeks.org/</a></li><li><a href="https://stackoverflow.com/questions/11331982/how-to-remove-any-url-within-a-string-in-python">https://stackoverflow.com/questions/11331982/how-to-remove-any-url-within-a-string-in-python</a></li><li><a href="https://streamlit.io/">https://streamlit.io/</a></li><li><a href="https://machinelearningmastery.com/clean-text-machine-learning-python/">https://machinelearningmastery.com/clean-text-machine-learning-python/</a></li><li><a href="https://machinelearningmastery.com/tour-of-ensemble-learning-algorithms/">https://machinelearningmastery.com/tour-of-ensemble-learning-algorithms/</a></li></ol><p><strong>LINKS</strong></p><p>Github Repository:</p><p><a href="https://github.com/manurajgehlot/case_study_1_review_sentiment_prediction">https://github.com/manurajgehlot/case_study_1_review_sentiment_prediction</a></p><p>LinkedIn profile:</p><p><a href="https://www.linkedin.com/in/manuraj-gehlot-143146219?lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_view_base_contact_details%3BT0XsDvg1TY%2BD8Qaw96JQOA%3D%3D">https://www.linkedin.com/in/manuraj-gehlot-143146219</a>/</p><p>Kaggle Dataset:</p><p><a href="https://www.kaggle.com/datasets/olistbr/brazilian-ecommerce/code">https://www.kaggle.com/datasets/olistbr/brazilian-ecommerce</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4a2a6019d5fe" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>