<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Andrew Scheuermann on Medium]]></title>
        <description><![CDATA[Stories by Andrew Scheuermann on Medium]]></description>
        <link>https://medium.com/@a15n?source=rss-ee2f91c9dd63------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*NI3JLWXkFfZhrpaOO81Kew.png</url>
            <title>Stories by Andrew Scheuermann on Medium</title>
            <link>https://medium.com/@a15n?source=rss-ee2f91c9dd63------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 05:59:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@a15n/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Creating Airbnb’s Page Performance Score]]></title>
            <link>https://medium.com/airbnb-engineering/creating-airbnbs-page-performance-score-5f664be0936?source=rss-ee2f91c9dd63------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f664be0936</guid>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[ios-app-development]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[Andrew Scheuermann]]></dc:creator>
            <pubDate>Thu, 18 Nov 2021 19:57:55 GMT</pubDate>
            <atom:updated>2021-11-18T20:57:29.336Z</atom:updated>
            <content:encoded><![CDATA[<p>Learn how Airbnb built the Page Performance Score, a 0–100 score that measures multiple performance metrics from real users on any platform.</p><p><a href="https://www.linkedin.com/in/scheuermann/">Andrew Scheuermann</a></p><figure><img alt="Two men playing guitar, one main playing an oboe." src="https://cdn-images-1.medium.com/max/1024/0*Mipmtj8Fk9XmyBMK" /></figure><p>Performance is important at Airbnb and part of our <a href="https://medium.com/airbnb-engineering/commitment-to-craft-e36d5a8efe2a">Commitment to Craft</a>. A fast experience is good for business and critical to our mission to “create a world where anyone can belong anywhere”.</p><p>Before we can create a fast experience we need to agree on what “fast” measures. Web, iOS, and Android each have different platform-specific performance metrics. For product engineers it can be challenging to understand which of these metrics to prioritize, and for management it’s difficult to compare platforms and keep progress reports succinct.</p><p>We’ve developed a new performance measurement system called the Page Performance Score that allows us to track multiple performance metrics from real customers across different platforms with ease. This post describes that system, and in the following weeks we’ll be publishing deep dives into the specifics for Web, iOS, and Android.</p><h3>Early Performance Measurement Efforts</h3><p>When Airbnb first started measuring performance, we used a single metric called “Time To Airbnb Interactive” (TTAI) that measured the time from page start to when content became visible and interactive. This approach had many positive outcomes. We built performance tracking architecture, fixed latency issues, and cultivated a company culture that valued performance.</p><p>However, TTAI also had shortcomings. Different platforms had different baselines and goals. Page comparisons were difficult because the “interactive” definition could change between similar pages. In some situations TTAI improved but engagement metrics did not. Most importantly, TTAI was a single metric and a single metric cannot capture the full spectrum of our customers’ performance expectations. Our definition of “fast” was incomplete and limited our overall performance efforts.</p><blockquote>A single metric cannot capture the full spectrum of our customers’ performance expectations.</blockquote><h3>Introducing the Page Performance Score</h3><p>We needed a nuanced view of performance while maintaining the simplicity of tracking a single number, so we created the Page Performance Score (PPS).</p><ul><li><strong>Page</strong>: The entire customer journey on Airbnb is divided into different pages.</li><li><strong>Performance</strong>: A page contains multiple performance metrics.</li><li><strong>Score</strong>: Every day, on each platform, we formulate a given page’s performance data into a 0–100 score.</li></ul><p>PPS allows us to combine multiple input metrics into an easily comparable score. PPS is a step-function improvement over our prior single-metric approach.</p><h3>The Metrics</h3><p>The metrics that we measure differ by platform, but the general approach of measuring multiple metrics and formulating a 0–100 score is the same. All of the metrics are <a href="https://web.dev/user-centric-performance-metrics/">user-centric</a> and fall into two general categories:</p><ol><li><strong>Initial Load Metrics</strong> measure the time from “page start” to content visible.</li><li><strong>Post Load Metrics</strong> measure page responsiveness after the initial load.</li></ol><figure><img alt="The Airbnb app opens, shows a loader, then the final meaningful page content." src="https://cdn-images-1.medium.com/max/600/0*BdTy3tjZIYtwtc2S" /><figcaption><em>The Airbnb homepage displays the loader and then meaningful content.</em></figcaption></figure><h4>Initial Load Metrics</h4><p><strong>Time To First Contentful Paint</strong> (Web) and <strong>Time To First Layout</strong> (Native) measure the time from “page start” until the first piece of content is visible, which is commonly a loader.</p><p><strong>Time To First Meaningful Paint</strong> (Web) and <strong>Time To Initial Load</strong> (Native) measure the time from “page start” until the meaningful content is displayed.</p><p>Initial Load Metrics are visualized on the left.</p><h4>Post Load Metrics</h4><p><a href="https://web.dev/fid/"><strong>First Input Delay</strong></a> (Web) measures the delay between user interaction and when the browser begins to respond. Delays of 50ms or longer are <a href="https://developer.mozilla.org/en-US/docs/Web/Performance/How_long_is_too_long">perceptible to the user</a>.</p><p><a href="https://web.dev/tbt/"><strong>Total Blocking Time</strong></a> (Web) and <strong>Thread Hangs</strong> (Native) cause the app to lag during layout, animations, and scrolling.</p><p><strong>Additional Load Time</strong> (Native) measures the average time that additional loaders are displayed within a page, such as during pagination.</p><p><strong>Rich Content Load Time</strong> (Native) measures the average time for images and videos to load.</p><p><a href="https://web.dev/cls/"><strong>Cumulative Layout Shift</strong></a> (Web) measures layout instability weighted by the size and distance of the element shift.</p><h3>The Formula</h3><p>After measuring the metrics we distill that information into a single number using the PPS Formula, which was forked from the <a href="https://web.dev/performance-scoring/">Lighthouse Formula</a>. For each metric we identified Good, Moderate, and Poor thresholds based on internal and <a href="https://web.dev/defining-core-web-vitals-thresholds/">industry data</a>. We created a scoring curve by assigning the Good range a score above 0.7, the Poor range below 0.5, and the Moderate range in between.</p><figure><img alt="A log normal curve with X values from 0 to 1, and Y values from 0 to 100,000." src="https://cdn-images-1.medium.com/max/976/0*6RlR2VE-YdrQVLqs" /><figcaption><em>A 10,000ms metric value would score ~0.9 in this example curve.</em></figcaption></figure><p>Every day we calculate a given page’s metric’s <a href="https://en.wikipedia.org/wiki/Truncated_mean">capped average</a> value from millions of real-user page loads. We map that capped average value against the metric’s curve to get a 0–1 score. We combine the metric scores into a composite PPS score by multiplying the metric scores by the metric weights. We chose the weights by examining our performance-focused A/B tests and ensuring that the weights are maximally aligned with Airbnb’s internal engagement metrics.</p><h4>Web Metric Weights</h4><figure><img alt="A percentage stacked bar chart with values TTFCP 35%, TTFMP 15%, FID 30%, TBT 15%, and CLS 5%." src="https://cdn-images-1.medium.com/max/1024/0*j1F-huoAevfFGkEF" /></figure><h4>Native Metric Weights</h4><figure><img alt="A percentage stacked bar chart with values TTFL 10%, TTIL 50%, TH 10%, ALT 15%, and RCLT 15%." src="https://cdn-images-1.medium.com/max/1024/0*jSbbMQYWVEIGUycR" /></figure><p>The resulting PPS formula can be expressed as….</p><p><em>PPS = curve(metric_1) * weight_1 + curve(metric_2) * weight_2 …</em></p><p>For example, on Web….</p><p><em>PPS = curve(TTFCP) * 35% + curve(TTFMP) * 15% + curve(FID) * 30% + curve(TBT) * 15% + curve(CLS) * 5%</em></p><h4>PPS Evolutions</h4><p>Migrating the company from a single metric to PPS was organizationally challenging. We had to train the company to stop viewing performance as a single seconds-based number, which is a paradigm shift that requires cross functional alignment. To help ease the transition we mapped the old TTAI ranges with the new PPS ranges.</p><figure><img alt="A table with the following values: Good Speed equals TTAI less than 3 seconds and also equals PPS greater than 70; Average Speed equals TTAI 3 to 5 seconds and also equals PPS 50 to 70; Slow Speed equals TTAI above 5 seconds and also equals PPS less than 50." src="https://cdn-images-1.medium.com/max/1024/0*eaCWefUEjUyvHvfH" /></figure><p>Once the company understood PPS, improving on it was comparatively easy. We simply add or replace metrics as our understanding of performance improves and the 0–100 score remains constant. PPS was designed to evolve. For example, in 2019 the Chrome team introduced <a href="https://web.dev/cls/">Cumulative Layout Shift</a>, which was a perfect candidate for Web PPS. It was a user-centric metric, had good browser coverage, and could be measured on direct and client-routed page loads. We instrumented the metric, validated the data, and then incorporated it into the next version of PPS. Easy!</p><h3>Weighted Average Score</h3><p>In addition to tracking individual pages’ PPS scores we track the entire organization’s overall performance progress by creating a Weighted Average Score (WAS). Consider these example PPS scores and traffic for three common pages:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eRUXs4sMKnOixViNn08zCA.png" /></figure><p><em>(73 * 5,000,000 + 84 * 20,000,000 + 75 * 10,000,000) / 35,000,000 = ~80</em></p><p>If these were the only pages at Airbnb our WAS would be ~80. Airbnb has hundreds of pages so a WAS helps us prioritize and proportionally weight the most high-traffic pages.</p><h3>Conclusion</h3><p>With PPS our engineers and data scientists now have a multitude of user-centric performance metrics to understand and improve their products. We can clearly compare the performance progress of different pages, different organizations, and even different platforms. PPS allows teams to set simple goals and determine which individual metrics to prioritize. PPS can evolve: metrics can be replaced, weights can change, targets can tighten, and yet the 0–100 score remains constant.</p><p>Changing our definition of “fast” has been well worth the effort. The company has evolved from viewing performance as a single metric to a 0–100 score that represents the rich, complex realities of performance. We have leveled up our performance measurement system and hope that you apply these learnings in your organization as well.</p><h3>Acknowledgments</h3><p>Thank you to the everyone who has helped build PPS over the years: <a href="https://www.linkedin.com/in/adityapunjani/">Aditya Punjani</a>, <a href="https://www.linkedin.com/in/alperkokmen/">Alper Kokmen</a>, <a href="https://www.linkedin.com/in/hdezninirola/">Antonio Niñirola</a>, <a href="https://www.linkedin.com/in/ben-weiher-123088122/">Ben Weiher</a>, <a href="https://www.linkedin.com/in/charlesx2013/">Charles Xue</a>, <a href="https://www.linkedin.com/in/egor-pakhomov-35179a3a/">Egor Pakhomov</a>, <a href="https://www.linkedin.com/in/eli-hart-54a4b975/">Eli Hart</a>, <a href="https://www.linkedin.com/in/elliotsachs/">Elliot Sachs</a>, <a href="https://www.linkedin.com/in/gabe-lyons-9a574543/">Gabe Lyons</a>, <a href="https://www.linkedin.com/in/guy-rittger-%E2%93%A5-1355b4/">Guy Rittger</a>, <a href="https://www.linkedin.com/in/jnvollmer/">Jean-Nicolas Vollmer</a>, <a href="https://www.linkedin.com/search/results/all/?keywords=joshua%20nelson%20%E2%9C%A8&amp;origin=RICH_QUERY_SUGGESTION&amp;position=0&amp;searchId=959d4aca-c80e-448a-b415-4a732ba7a84d&amp;sid=Rr6">Josh Nelson</a>, <a href="https://www.linkedin.com/in/joshpolsky/">Josh Polsky</a>, <a href="https://www.linkedin.com/in/lupinglin/">Luping Lin</a>, <a href="https://www.linkedin.com/in/markgiangreco/">Mark Giangreco</a>, <a href="https://www.linkedin.com/in/mattschreinerphd/">Matt Schreiner</a>, <a href="https://www.linkedin.com/in/nickbryanmiller/">Nick Miller</a>, <a href="https://www.linkedin.com/in/thenickreynolds/">Nick Reynolds</a>, <a href="https://www.linkedin.com/in/noahsmartin/">Noah Martin</a>, <a href="https://www.linkedin.com/in/xiaokangxin/">Xiaokang Xin</a>, and everyone else who helped along the way.</p><p><em>Interested in joining Airbnb? Check out these roles:</em></p><p><a href="https://grnh.se/feab9b481us"><em>Android Software Engineer, Guest Experience</em></a><em><br></em><a href="https://grnh.se/cbf480fa1us"><em>Senior iOS Software Engineer, Guest Experience</em></a><em><br></em><a href="https://grnh.se/23eb3d8b1us"><em>Senior Android Software Engineer, Guest Experience</em></a><em><br></em><a href="https://grnh.se/4092a3ba1us"><em>Staff iOS Software Engineer, Guest Experience</em></a><em><br></em><a href="https://grnh.se/a6f52fb91us"><em>Staff Android Software Engineer, Guest Experience</em></a><em><br></em><a href="https://grnh.se/0bcae9dd1us"><em>Senior Software Engineer, Guest Experience</em></a><em><br></em><a href="https://grnh.se/7d47f8ce1us"><em>Staff Fullstack Engineer, Guest Experience</em></a><em> <br></em><a href="https://grnh.se/e80a733a1us"><em>Senior Data Scientist — Analytics Engineering, Guest Experience</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f664be0936" width="1" height="1" alt=""><hr><p><a href="https://medium.com/airbnb-engineering/creating-airbnbs-page-performance-score-5f664be0936">Creating Airbnb’s Page Performance Score</a> was originally published in <a href="https://medium.com/airbnb-engineering">The Airbnb Tech Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Choosing the right bootcamp]]></title>
            <link>https://medium.com/@a15n/choosing-the-right-bootcamp-d59792c45b53?source=rss-ee2f91c9dd63------2</link>
            <guid isPermaLink="false">https://medium.com/p/d59792c45b53</guid>
            <dc:creator><![CDATA[Andrew Scheuermann]]></dc:creator>
            <pubDate>Wed, 27 Aug 2014 20:08:45 GMT</pubDate>
            <atom:updated>2016-01-15T17:57:25.941Z</atom:updated>
            <content:encoded><![CDATA[<h3>5 Things to Consider Before Deciding on a Dev Bootcamp</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eDH8bgJpImRjStdjaOM2IQ.jpeg" /></figure><p>Computer programming bootcamps are super hot right now, but which one is right for you?</p><p>In March of 2014 I decided to make a career shift from business development to software engineering, specifically web development. I had been teaching myself programming since January and I wasn’t learning fast enough. Frustrated, I began researching different bootcamps and was quickly overwhelmed.</p><p>Bootcamps are springing up everywhere! They each cater to a different demographic and have different teaching styles, curriculums, strengths, and weaknesses. It’s tough to make an informed decision with a limited understanding of the industry. Luckily for you I’m far enough into my career that my advice carries some weight and I’m new enough that I vividly remember the pain of switching careers and finding the right bootcamp. With that in mind, here are the five key points to consider when choosing a bootcamp.</p><ol><li>Pace: Some bootcamps proudly advertise that they will teach you for 12 hours a day, 6 days a week. I’m not a fan of this lifestyle to begin with but let me explain why its not a great idea in a bootcamp scenario. If you fall behind at all you will not have time to catch up! My bootcamp* offered 8 hours a day of structured learning. This left me plenty of time to catch up, dive deep, or spend time enjoying life. Pay attention to the pace!</li><li>Location: At least 90% of bootcamp graduates stay in the city where they attended their bootcamp. This means that going to NYC for a bootcamp and then moving to SF isn’t a great idea. I actually did this exact thing but I already had a strong network in SF. Go to a bootcamp where you plan to work after graduation.</li><li>Focus: The vast majority of bootcamps teach Ruby. However, I personally think that there is more of a future in Javascript. Have a look at <a href="https://twitter.com/dberkholz/status/395922559151009792/photo/1">this</a> and <a href="http://angelinsights.herokuapp.com/">this</a>. Also, be wary of a bootcamp that promise to teach you too much. You don’t want to leave your bootcamps as a “Jack of all trades, master of none”.</li><li><strong>Teachers</strong>: This one is huge! How proficient are your teachers? Are they industry professionals or recent bootcamp grads? Do they have experience teaching? How many hours a week will you have with the teachers compared to group work or free time? These are all important questions to consider. All teachers are not created equal.</li><li><strong>Difficulty</strong>: This one was <em>the</em> deciding factor for me. You can learn a lot about the difficult of a bootcamp from its application challenge. This challenge (or lack thereof) is indicative of the experience required by applicants, your fellow students’ knowledge, and the difficulty of the curriculum. If the application was too hard, don’t go. If the application was too easy, don’t go. If the application doesn’t exist, run! Find a school whose application process is difficult enough to challenge you, but not overwhelm you.</li></ol><p>In closing, attending a bootcamp is a great experience - if you choose a good one. I put a lot of thought into the process and was very pleased with my choice!</p><p>*Disclosure: I attended <a href="http://www.fullstackacademy.com/">Fullstack Academy</a> in NYC from April-July 2014.</p><p><em>If you liked this post you should hit the </em><strong><em>Recommend </em></strong><em>button below, which will mean more people see it. Thanks!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d59792c45b53" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>