<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yanqiao Wang on Medium]]></title>
        <description><![CDATA[Stories by Yanqiao Wang on Medium]]></description>
        <link>https://medium.com/@yanqiaowang_53574?source=rss-cc9b2fe35d7d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*sb7BAYeCdkcWT5lYrz2DZQ.jpeg</url>
            <title>Stories by Yanqiao Wang on Medium</title>
            <link>https://medium.com/@yanqiaowang_53574?source=rss-cc9b2fe35d7d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 16:22:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@yanqiaowang_53574/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A Review of Multi-Armed Bandits Applications at Lyft]]></title>
            <link>https://eng.lyft.com/a-review-of-multi-armed-bandits-applications-at-lyft-132550dd4df3?source=rss-cc9b2fe35d7d------2</link>
            <guid isPermaLink="false">https://medium.com/p/132550dd4df3</guid>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[marketplaces]]></category>
            <category><![CDATA[ridesharing]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[multi-armed-bandit]]></category>
            <dc:creator><![CDATA[Yanqiao Wang]]></dc:creator>
            <pubDate>Mon, 23 Jan 2023 18:28:29 GMT</pubDate>
            <atom:updated>2023-01-23T18:28:29.669Z</atom:updated>
            <content:encoded><![CDATA[<p><em>By </em><a href="https://www.linkedin.com/in/sara-smoot-a723b437/"><em>Sara Smoot</em></a><em>, </em><a href="https://www.linkedin.com/in/alexwc/"><em>Alex Contryman</em></a><em> and </em><a href="https://www.linkedin.com/in/yanqiao/"><em>Yanqiao Wang</em></a></p><p>Lyft hosts a dynamic marketplace connecting millions of people to a robust transportation network. In order to offer high value and quality service for both riders and drivers we need to make complex optimization decisions in near-real time. The environment can change quickly with traffic, events and weather, making these decisions even more challenging.</p><p>We have employed <a href="https://tor-lattimore.com/downloads/book/book.pdf">multi-arm bandits (MAB)</a> algorithms, a common machine learning method for decision making using long-term rewards, to improve our real-time decision making capability. MABs allow us to not only iterate at a faster cadence and lower cost, but also allow for dynamic user experiences and responsive marketplace systems. We will walk through some of our most impactful MAB applications in UI optimization and personalized messaging, concluding with applications in our marketplace algorithms.</p><p><strong>UI optimization</strong></p><p>At Lyft, portions of the website design are powered by MAB allowing designers to experiment with hundreds of combinations of text and images to find the most effective combination. For example, consider the landing page for riders on the Lyft website. We can experiment with many different combinations of the value proposition text and main image to find the best combination that inspires riders to take their first ride. To take this a step further, we can use information about the time-of-day or regional information to make the experience adaptive to the context of the visit. Perhaps in the morning, potential riders are more interested in commute options and on the weekends they would prefer to meet up with friends.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_Ptygg9JbpRT7ZzQ" /></figure><p>Using an integrated tool, we can experiment with many different combinations of value propositions and images with a simple interface. The tool connects designers and other web-based experiments to MAB algorithms such as NIG-Gaussian models with <a href="https://web.stanford.edu/~bvr/pubs/TS_Tutorial.pdf">Thompson Sampling</a> as well as other open source MAB libraries by providing an easy configuration experience.</p><p><strong>Communication Optimization</strong></p><p>At Lyft we communicate with our riders and drivers through a number of different channels including text, in-app notifications and emails. We use MAB algorithms to select the text of these messages to optimize for engagement. For example, if we want to inform our riders of a new product, we can utilize these communication channels while optimizing for the best wording dependent on the channel and time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/0*a8ec3Mv4vdcqkArF" /></figure><p>Two banner messages result in different rates of success, the model dynamically chooses the most successful.</p><p>Some channels, such as the in-app banner, have limited bandwidth — we only want to display one banner per ride. However, there may be multiple banners that we want to display to a particular rider. By using MAB algorithms to choose which banner to display, we can ensure that we are showing the most relevant message to the rider.</p><p><strong>Marketplace optimization</strong></p><p>In our marketplace we use marketing, dispatch, and pricing systems in order to maintain a balance between optimal service quality and value. We utilize the adaptability of MAB algorithms to adjust to rapidly changing environmental and market conditions.</p><p>One method we use to balance our marketplace on longer timescales is to strategically focus marketing spending on both drivers and riders when we anticipate imbalances. We use contextual and continuous MAB methods to optimize our budget allocations for incentive distributions.</p><p>On shorter timescales, one way in which we maintain service quality is to dynamically set the pickup window for the Wait and Save offer depending on context such as local ride demand and driver availability. We can choose the window that will provide optimal reliability and cost savings. For this application we employ contextual MAB methods with a reward function that jointly optimizes reliability and ride requests.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/387/0*NU7ey7dWdT6AW1Oz" /><figcaption><em>Wait &amp; Save pick-up window offer</em></figcaption></figure><p>Our dispatch and pricing systems have important parameters that need to be tuned to perform optimally. We dynamically tune some of these parameters with MAB algorithms so that they can quickly adjust to market conditions. In each application we define a reward function that allows us to balance between competing objectives. Price, for example, tends to be inversely related to the likelihood a rider will request a ride in the future. We can combine these two competing metrics in a single reward and allow the MAB to tune these parameters. For more information on our pricing system check out this <a href="https://eng.lyft.com/pricing-at-lyft-8a4022065f8b">blog post</a>.</p><p><strong>Where we go next</strong></p><p>MAB algorithms have had a substantial impact on making our marketplace more efficient. With the number of use cases increasing, we are incorporating these algorithms into our core Machine Learning platform to facilitate quicker adoption. We anticipate more impact as access to the MAB algorithms and libraries is expanded within Lyft.</p><p>If you find this work compelling, check out our <a href="https://www.lyft.com/careers#openings">job openings</a> and <a href="https://www.lyft.com/careers/early-talent">university programs</a> at Lyft and come work with us!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=132550dd4df3" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/a-review-of-multi-armed-bandits-applications-at-lyft-132550dd4df3">A Review of Multi-Armed Bandits Applications at Lyft</a> was originally published in <a href="https://eng.lyft.com/">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pricing at Lyft]]></title>
            <link>https://eng.lyft.com/pricing-at-lyft-8a4022065f8b?source=rss-cc9b2fe35d7d------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a4022065f8b</guid>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[pricing]]></category>
            <category><![CDATA[marketplaces]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[optimization]]></category>
            <dc:creator><![CDATA[Yanqiao Wang]]></dc:creator>
            <pubDate>Tue, 02 Aug 2022 16:45:36 GMT</pubDate>
            <atom:updated>2022-08-02T16:45:36.235Z</atom:updated>
            <content:encoded><![CDATA[<p><em>By </em><a href="https://www.linkedin.com/in/yanqiao/"><em>Yanqiao Wang</em></a></p><h3>Introduction</h3><p>Pricing forms the backbone of the Lyft’s marketplace system, which is core in the journey of improving people’s lives with the world’s best transportation. In general, the goal of price optimization is to find the price that balances supply and demand while covering the expenses necessary to provide an easy-to-use rideshare platform, where drivers have tremendous influence on the rider prices through their entry and exit on the platform. Specifically, we strive to find rideshare options and prices that are:</p><ul><li><strong>Reliable</strong>: balance consistent prices and pickup times across repeated usage;</li><li><strong>Fair</strong>: any two riders see the same price for the same ride intent with a clear explanation otherwise (e.g. one rider has a Lyft Pink membership);</li><li><strong>Competitive</strong>: efficient pricing to remain competitive compared to other transportation options.</li></ul><p>In this blog post, we discuss the rideshare pricing system architectural design, modeling approaches and potential future iterations of Lyft’s pricing engine to illustrate how we tackle price optimization to achieve Lyft’s long-term mission.</p><h3>System Design</h3><p>From the system design perspective, there are three layers of the stack:</p><ol><li><strong>Pricing Orchestration:</strong> a configurable engineering platform with fast iterations for precise execution to support algorithms and operations;</li><li><strong>Pricing Operations:</strong> a process and supporting toolkit to ensure prices are competitive;</li><li><strong>Algorithmic Pricing:</strong> a plug-in-play model library designed for consistent backtesting, experimentation, and productionization of automated data-driven decisions pursuing fair and reliable prices.</li></ol><p>All three layers work seamlessly together towards a unified goal for the long-term success:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nC9qCN87yHaScN5a" /></figure><p>We will introduce how each layer works in the following sections.</p><h3>Pricing Orchestration</h3><p>In pricing orchestration, we built a robust platform that ensures a rider’s price is stable as they consider their options and that ensures prices for similar rides are consistent between riders. The platform also encapsulates all of the business logic for determining the market price and generates a traceable history of each pricing decision.</p><p>In this platform, we leverage multiple technologies:</p><ul><li>Infra: Flink, <a href="https://eng.lyft.com/introducing-flyte-cloud-native-machine-learning-and-data-processing-platform-fb2bb3046a59">Flyte</a>, Spark, Redis, Dynamo, Hive;</li><li>Modeling: Tensorflow, Reinforcement learning, Neural Network, Linear regression, Convex Optimization;</li><li>Programming &amp; Serving: Python, Flask.</li></ul><p>The overall architecture design is as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QeAQewawDyipsoeH" /></figure><p>Each component has a distinct responsibility:</p><ul><li>API Layer: a set of endpoints that provide access to market signals and session-level information;</li><li>Cache Layer: provides price stability and consistency, as well as, reduce latency via user and route caching;</li><li>Calculation Layer: acts as an ingredient manager that fetches information from the cache layer (e.g. estimated travel time and distance, estimated tolls and venue fees) and executes the price calculation according to a list of step-by-step procedures for each rideshare option;</li><li>Storage: a GDPR/CCPA-compliant price quote store having highly reliable, efficient and low latency storage that records details of the price like: request, ingredients, line items and price quote ID.</li></ul><h3>Pricing Operations</h3><p>Pricing operations include strategic and tactical operations, which work harmoniously with pricing algorithms to drive both financials and growth.</p><p>Algorithmic pricing needs two types of guidance from pricing operations:</p><ul><li>Strategic Guidance: how to optimally consider trade off among multiple business goals (financial, growth and service levels);</li><li>Tactical Guidance: how to tackle special use cases (e.g. airport and venues) in the overall system.</li></ul><h3>Algorithmic Pricing</h3><p>We have built a large-scale real-time pricing algorithm by leveraging causal inference, machine learning and operations research methodologies. This design enables faster experimentation and system evolution, which is particularly crucial given the market volatility and marketplace dynamicity.</p><p>In general, the unified algorithmic pricing includes structural pricing and dynamic pricing with a near real-time feedback loop:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ed5KA1hPmN-d2lpN" /></figure><p>The structural pricing component serves as the foundation of pricing optimization at Lyft, in order to maximize the passenger long-term value taking ride level characteristics (location: origin and destination, estimated travel time and distance, timestamp and ride type) as inputs. Dynamic pricing aims for optimizing the entire rideshare network by looking at both riders and drivers to improve the throughput of rides. As soon as drivers sign on or off and riders check the Lyft app, the dynamic pricing algorithm starts counting and gathering input signals, and then trying to find a reasonable balance between demand and supply. The process is that drivers’ entry and exit onto the platform influence the pickup times, affecting service quality, such as cancels, lapses and driver utilization, which in turn impacts dynamic pricing’s throughput of rides by influencing riders’ conversion rate (check out this <a href="https://eng.lyft.com/dynamic-pricing-to-sustain-marketplace-balance-1d23a8d1be90">blog post</a> for more details).</p><p>To handle the dynamicity, we adopt an “offline training, online learning, and real-time evaluation” design principle, where we train the input signals generation model based on causal inference and machine learning in an offline setting, then simultaneously correct the potential bias of input signals, correct for market management inefficiencies, and optimize for the multi-objective goal using online reinforcement learning and off policy evaluation, in which one learns about a policy different from the policy that generates the data. Visually, the flow looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*V90jAVNgHvzFTDN8" /></figure><h3>Online Pricing with Reinforcement Learning</h3><p>As you can see, we handle the market volatility by building flexibility into algorithmic pricing and, specifically, online reinforcement learning enables the platform to support multi-objective, non-parametric decision making. Let’s dive in.</p><p>Pricing is a difficult problem because it is difficult to estimate how a price change today would impact the market, especially given the market volatility we have experienced over the last couple of years, and because even if one could estimate the demand response to price changes, the response is likely to be non-linear across the many different rideshare options which makes price optimization computationally challenging. Online pricing with reinforcement learning (RL) addresses these challenges by using continuous A/B tests on small price changes to identify the right price for the current marketplace dynamic conditions by processing a wide range of market and business inputs.</p><p>A general online reinforcement learning workflow would look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nHatri0ZY-YxqvVH" /></figure><p>We have successfully applied this technique to our algorithmic pricing component. As mentioned before, we use off-policy evaluation to warm-start prices and have for awhile, but what’s new with online reinforcement learning is that, in near real-time, we process transaction ride data as raw data to update the pricing policies (Selection &amp; Action) based on the realized pricing goals (Reward) and can be more robust to market instability (State) changes in the marketplace.</p><h3>What’s Next?</h3><p>To summarize, we have discussed pricing’s mission, reviewed different engineering, operations and algorithmic components on the system level, then briefly illustrated how online RL contributes to pricing. The next step would be to apply the hierarchical RL design for the long-term value maximization and system unification across and beyond marketplace optimization.</p><h3>Are You as Excited as We Are?</h3><p>Interested in operations research, reinforcement learning, causal inference, experiment design, applying science at scale and helping push the boundaries of data-driven decision-making in rideshare, or working at Lyft in general? Check out our <a href="https://www.lyft.com/careers#openings">job posting</a> or <a href="https://www.lyft.com/careers/university">university program</a> pages. If you find this blog interesting, don’t hesitate to reach out via <a href="/service/mailto:yanqiao@lyft.com">email</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a4022065f8b" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/pricing-at-lyft-8a4022065f8b">Pricing at Lyft</a> was originally published in <a href="https://eng.lyft.com/">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>