<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jibin Jose on Medium]]></title>
        <description><![CDATA[Stories by Jibin Jose on Medium]]></description>
        <link>https://medium.com/@jibinjose?source=rss-a03fbb3e091c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*KfMuC_LLpfZGL31D.</url>
            <title>Stories by Jibin Jose on Medium</title>
            <link>https://medium.com/@jibinjose?source=rss-a03fbb3e091c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 17:20:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jibinjose/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Predicting Discounts with Multi Armed Bandit Learning]]></title>
            <link>https://medium.com/@jibinjose/predicting-discounts-with-multi-armed-bandit-learning-ca89f0e30212?source=rss-a03fbb3e091c------2</link>
            <guid isPermaLink="false">https://medium.com/p/ca89f0e30212</guid>
            <category><![CDATA[machine-learing]]></category>
            <category><![CDATA[pricing]]></category>
            <category><![CDATA[ml-basics]]></category>
            <category><![CDATA[multi-armed-bandit]]></category>
            <dc:creator><![CDATA[Jibin Jose]]></dc:creator>
            <pubDate>Sun, 17 Apr 2022 07:41:42 GMT</pubDate>
            <atom:updated>2022-04-28T06:24:28.294Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong><em>Predicting Discounts with Multi Arm Bandit</em></strong></h3><p>Last 5 years I was part of a team which built systems to support ML. Though I didn’t have any ML background, I learned some of the ML concepts through regular discussions with my colleagues. It was easy for me to grasp these topics when someone explained to me in layman’s language. So, I thought of writing what I have learned in simple form so that it could be beneficial for anyone without a solid ML background.</p><p>Let me explain how we built an MAB system.</p><blockquote><strong><em>Problem </em></strong><em>: </em>What would be the optimal discounts we should give to a user to convince him/her to book a hotel room<em>.</em></blockquote><blockquote>We formed a team to solve the problem with the following objective.</blockquote><blockquote><strong>Objective</strong> : Increase the number of bookings by X with reasonable ROI(Y).</blockquote><p>As a first step, we started giving a 4% discount for some of our users to understand how discounts work. As expected, our booking went up. ROI was good but it was not sufficient to scale it for a larger audience. So we decided to build an ML system to optimize the discount for the ROI and increase the number of bookings.</p><p><em>ROI = Extra commission received / total discount given</em></p><ol><li><strong>A Simple classification model using static data.</strong></li></ol><p><em>Collect data for a period of time and train the model.</em></p><p>We decided to build our first model to identify the optimal discount. We needed data to train the model. To collect data, we randomly gave different discounts to users for a few weeks. Once we had enough data , we trained a classification model. As expected the model performed much better than the flat 4% discount and this helped us to scale the discounting campaigns.</p><p><strong>Challenge</strong>: Everything was good till we found degradation of ROI after a few months of model release. A detailed analysis showed that data used to train the model is not representative for the current market situation. Model got outdated very fast and we had to find a better way to train the model.</p><p>2.<strong> Continuous data collection and Frequent learning .</strong></p><p><em>Collect data continuously and retrain the model frequently.</em></p><p>To validate our hypothesis that retraining with newer data improves the model, we collected data once again and trained the model with the new data. Without any surprise, model started performing well. Frequent data collection was one way to address this issue. But it was not sustainable as we had to stop using models during data collection period, which was pretty bad for overall ROI.</p><p>As an alternative, we decided to collect data continuously and retrain the model daily. To achieve this, we split our users into 2 groups, where 90% of users got discount predicted by model and rest got a random discount. This approach improved the model performance considerably.</p><p><strong>Challenge</strong>: This method had a caveat. We used cookies to split the users and this had a potential danger of model getting a biased because of cookie refreshes. We tried different approaches to overcome this but decided to live with this as alternatives were very complex.</p><p><strong>3. Multi Arm Bandit learning.</strong></p><p><em>MAB is an online learning methodology where both data collection and training happens online. Data collection(exploration) and giving optimal discount(exploitation) is controlled by the system based on the performance of each discounts.</em></p><p>I will try to explain MAB with <strong>Zorro</strong>, an MAB based system we built to give discounts.</p><p>Zorro had mainly 3 components,</p><ol><li><strong>Discount prediction logic</strong>: Responsible for predicting discount. This module decides whether to do Exploration or Exploitation. Exploration is similar to continuous data collection, where different discounts are given to the users. Giving best possible discounts is considered as exploitation.</li><li><strong>Feedback systems</strong> : Responsible for collecting the feedback for different discounts. In our case, commission of booking and discount amount was sent as feedback.</li><li><strong>Performance assessment</strong>: This module calculated the performance of each discount for every segment. For example, if we use country as a feature, Zorro stores ROI and number of bookings for every country and discount combinations.</li></ol><p>Zorro started with exploration by giving discounts 0,3,6 or 9 to the users with equal distribution (known as Epsilon greedy strategy). Performance assessment component calculates the performance of different discount. Once it had enough data to identify the optimal discount, Zorro shifted to exploitation mode where the best discount were given to the user. Zorro also maintained a predefined percentage of traffic for the exploration.</p><p>Main difference of Zorro from the continuous exploration was,<strong><em> Zorro’s discount prediction logic was smart enough to use the best performing discount for exploitation</em></strong>. <strong><em>Based on the performance, Zorro shifted discounts between exploitation and exploration</em></strong>. This self adjusting system adapted to market changes very quickly and optimized the discount based on real time data.</p><p><strong>Challenges we faced</strong>:</p><ul><li>It was pretty complex to build an online learning system. We had to tackle many unforeseen challenges while developing the system.</li><li>It took many iteration for us to get it in a good shape. Tuning different parameters in the model was also difficult.</li><li>Another major drawback of the system was even a minor data quality issue impacted the model immediately. Retraining and redeploying the model was also a tedious task. To minimise the efforts, we backed up the model frequently and built a library to train the model offline from a given offset.</li></ul><p><em>All credit goes to </em><strong><em>Cintia Bruscantini </em></strong><em>who tirelessly explained me these concepts so many times and also built the system. Also thanks to</em><strong><em> Apoorv Anurag, Utkarsh Gupta </em></strong><em>for being partner in this journey.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ca89f0e30212" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>