<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Robert Jailall on Medium]]></title>
        <description><![CDATA[Stories by Robert Jailall on Medium]]></description>
        <link>https://medium.com/@imprisonevery1?source=rss-a8d3f71575dd------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*vSPSwXUqaDmq9E_3fBRdhA@2x.jpeg</url>
            <title>Stories by Robert Jailall on Medium</title>
            <link>https://medium.com/@imprisonevery1?source=rss-a8d3f71575dd------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 04:39:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@imprisonevery1/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to Stack Rank Your Organization]]></title>
            <link>https://medium.com/@imprisonevery1/how-to-stack-rank-your-organization-a5cb6d5c9149?source=rss-a8d3f71575dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/a5cb6d5c9149</guid>
            <category><![CDATA[business-simulation]]></category>
            <category><![CDATA[stackrank]]></category>
            <category><![CDATA[performance-reviews]]></category>
            <category><![CDATA[discrete-maths]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Robert Jailall]]></dc:creator>
            <pubDate>Mon, 13 Jan 2020 04:46:12 GMT</pubDate>
            <atom:updated>2020-05-12T22:44:57.058Z</atom:updated>
            <content:encoded><![CDATA[<h4>Performance Review Minutiae Series</h4><p><a href="https://en.wikipedia.org/wiki/Vitality_curve">Stack ranking</a> is terrible. But suppose your Human Resources department has decided to use stack ranking for your organization’s performance review. The way your organization implements stack ranking can range from bad to extremely bad — “bad” meaning the process is unfair in recognizing true employee performance. When you give employees lower ratings than they deserve, they quit. When you give employees higher ratings than they deserve, other employees see the disconnect between performance and reward and quit. This post assumes you’re interested in minimizing the number of people quitting due to inaccurate performance reviews.</p><p>So given that stack ranking sucks, how can your organization make the best of a bad situation?</p><p>“Stack ranking” in this post specifically means 1.) creating a sorted list of all employees and 2.) further assigning those employees to bins of preset sizes that represent performance review ratings. For example, a stack ranking that is rampant across Silicon Valley and one that we’ll examine in this post is as follows (<a href="https://en.wikipedia.org/wiki/Vitality_curve#Yahoo">this is from Yahoo</a>):</p><ul><li>Top 10% of employees get “Greatly Exceeds Expectations”</li><li>Next 25% get “Exceeds Expectations”</li><li>Next 50% get “Achieves Expectations”</li><li>Next 10% get “Occasionally Misses Expectations”</li><li>Bottom 5% get “Misses Expectations”</li></ul><p>Looking at this 10–25–50–10–5 distribution, a few questions immediately come to mind:</p><ol><li>What are the percentages a percentage of?</li><li>Where did this distribution of 10–25–50–10–5 come from?</li><li>Does this distribution change over time? (Especially as a result of applying it in successive performance reviews)</li></ol><h4>Question 1: What are the percentages a percentage of?</h4><p>In the best case, the percentages are of your entire organization, and your organization is large. In the worst case, the percentages are of the teams at the lowest level of the organization chart (typically 5–10 people, one manager).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/1*-YbV-HPNmHEPBP5rqGUG6Q.png" /><figcaption>Figure 1: Average results from Monte Carlo simulation of stack ranking.</figcaption></figure><p>Figure 1 shows the results from a Monte Carlo simulation of forcing teams of different sizes into the distribution above for a 200-person organization. It’s important to note that we generated the 200-person organization so that it indeed fit the desired 10–25–50–10–5 distribution. For each team size, we split the organization into teams of that size and randomly assigned team members from the organization.</p><p>For example, the first row with Team Size=5 means we tried to fit forty 5-person teams to the distribution above. This resulted in the poorest performance review accuracy, <strong>assigning almost half of the organization to the wrong bin!</strong></p><p>Why are the smaller team sizes the worst?</p><p>As it turns out, Computer Science graduates have studied all of the topics required to answer this question — discrete mathematics and statistics.</p><p>First<em>, discrete mathematics.</em></p><p>Trying to fit a small number of people into a similar number of bins becomes the <a href="https://en.wikipedia.org/wiki/Cutting_stock_problem">cutting stock problem</a>. For example, the distribution above calls for 25% of employees to fall into the “Exceed Expectations” bin. For a team of 10 though, we can only fit 2 employees into this bin, leaving an unused 5%. Since we can’t put half an employee in a bin, this means some employee in the 10-person team will be forced into an incorrect bin. We don’t have this cutting stock problem if we increase the team size such that individuals can fit into the bins without waste. For example, when team size is 100, we can fit exactly 25 employees into the 25% bin (and 10 into 10%, 5 into 5%, and so on).</p><p>&lt;Record scratch&gt;</p><p><em>Why are we talking about Computer Science esoterica?</em></p><p>Something’s wrong here. It’s pretty weird to implement stack ranking such that it’s highly susceptible to this cutting stock problem. Employee ratings, which are supposed to recognize performance, should not come down to a question of bin-packing efficiency. When we force small teams into the distribution, we aren’t fudging the math just a little bit — <strong>almost half of our ratings are wrong</strong>. Luckily, we can easily solve this problem by only forcing the distribution at large team sizes.</p><p>This leads us to our next discipline<em>, statistics</em>.</p><p>Another dubious assumption our stack ranking implementation makes is that all teams are created equal — if we have a population that conforms to our desired 10–25–50–10–5 distribution, then we allegedly should expect that our teams also conform to this distribution. I‘m assuming most will recognize this as a <a href="https://en.wikipedia.org/wiki/Sample_size_determination">sample size determination</a> problem — what’s the smallest number of samples we can take from a population and expect the <em>sample</em> distribution to match the <em>population</em> distribution. <strong>For a organization of 100–200, it’s certainly not 5 or 10 individuals</strong>!</p><p>Unless your organization’s teams were actually created to fit the distribution ahead of time (unlikely), then we should expect smaller and smaller teams to vary widely both from each other and the population’s distribution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2MOC_C8J-30o_RYTunuDhg.png" /><figcaption>Figure 2: Smaller team have wildly varying performance.</figcaption></figure><p>Figure 2 shows the average rating of teams of different sizes in a 64-person organization. We generated the 64-person organization the same way we did in the simulation above — matching the 10–25–50–10–5 distribution and randomly assigning individuals to teams. The number in each box is the average true rating of each team member, where each rating from lowest to highest (ME, OME, AE, EE, GEE) corresponds to an integer (0, 1, 2, 3, 4).</p><p>The average ratings of the organization’s teams are 2.3, but when we get down to the 8-person team, the weakest teams have an average rating of 1.6 and strongest team an average of 2.9 — <strong>the strongest team is almost twice as good as the weakest</strong>. In this set of runs, not a single 8-person team had the organization average of 2.3. As team size gets bigger, we finally see the variation between teams drop, and we see their ratings converge to the population average of 2.3.</p><p>Perhaps counterintuitively, this means it’s actually more fair to treat teams unequally during performance review if those teams are small. Some teams will simply perform a lot better than other teams and should get a greater proportion of the higher ratings.</p><h4><strong>Putting “the right thing” into Practice</strong></h4><p>The analysis above boils down to one thing: stack ranking should only apply at larger team sizes— let’s say 50.</p><p>However, if 50 represents a lot of teams and a lot of managers, then sorting 50 people via a group discussion becomes an unwieldy process (though as managers, we have few other responsibilities that are worth more effort). In practice, when all of an organization’s managers get together and face an unwieldy stack ranking process, they’ll employ shortcuts such as forced distributions and boundary-only comparisons. These make the performance review process faster but unfair.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NAia5TuMOfJgiBYGd0mdpg.png" /><figcaption>Figure 3: <a href="https://en.wikipedia.org/wiki/Merge_sort">Merge-sort</a> — sort and merge lists working from the bottom of the hierarchy to the top.</figcaption></figure><p>We can practice a fair and somewhat fast process by once again tapping Computer Science and using the <a href="https://en.wikipedia.org/wiki/Merge_sort">merge-sort algorithm</a>.</p><p>All of the managers at the lowest level of the hierarchy should pair up and sort the members of their two teams into one list. It’s much easier for two people to sit down together and discuss a subset of employees in depth (rather than 2 minutes per employee or not at all with the larger group). A more intimate discussion also makes it more likely that teams will suss out real differences between teams.</p><p>The first pairs of sorted lists should then be merged pairwise using the same process to create larger sorted lists. We should expect the successive merges to be easier than the first merges because managers will have practiced how to rank employees. Managers can continue merge-sorting until they reach a sorted list size that is not subject to problems described in this post.</p><p>We must not forget that this post is only about making the best out of a bad situation — <a href="https://en.wikipedia.org/wiki/Vitality_curve#Criticism">stack ranking employees has well-known downsides</a> and creates perverse incentives that push organizations into dysfunction. It’s a way for organizations who don’t trust their middle management to remove responsibilities, limiting the damage its middle management can do by promoting employees overzealously. In an ideal world, if an organization cannot trust its middle managers, it should train them or get new middle managers who can handle the responsibility of making sure employee compensation reflects the market rate.</p><h4>Question 2: Where did this distribution of 10–25–50–10–5 come from?</h4><p>Stay tuned for another exciting chapter in the Performance Review Minutiae Series to answer this question.</p><h4>Question 3: Does this distribution change over time? (Especially as a result of applying it in successive performance reviews)</h4><p>Stay tuned for another exciting chapter in the Performance Review Minutiae Series to answer this question.</p><p>The code used to generate the data and plots in this post are <a href="https://github.com/robjailall/stack-ranking">available on GitHub</a>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a5cb6d5c9149" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Most Productive Day]]></title>
            <link>https://medium.com/@imprisonevery1/a-most-productive-day-1a9a6d67e3b5?source=rss-a8d3f71575dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/1a9a6d67e3b5</guid>
            <category><![CDATA[engineering-mangement]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[project-management]]></category>
            <dc:creator><![CDATA[Robert Jailall]]></dc:creator>
            <pubDate>Mon, 25 Feb 2019 03:11:42 GMT</pubDate>
            <atom:updated>2019-02-25T05:13:20.438Z</atom:updated>
            <content:encoded><![CDATA[<h4>Working backwards to create the ideal engineer day</h4><p>I use this exercise to figure out what I as a manager should do to make my teams better: describe the most productive day for an engineer and work backwards to make it true. The answer should be “yes” to all of these questions below. If not, I get my teams to take the most direct action to flip the “noes” to “yeses.”</p><blockquote><strong><em>This exercise tailors responsibilities, roles, processes, and behaviors to each team, respecting the talents and experiences of team members.</em></strong></blockquote><blockquote><strong><em>It’s a joy to work on these teams.</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*4pyy3v4eLRh2gJh9Vh3Qbg.png" /></figure><p>Here’s what I remember (or perhaps imagined) was my best engineer day:</p><h3>Do I agree with what is being built?</h3><ul><li>Do we understand the product objectives, and are they clearly stated and documented?</li><li>Do we know how our product helps the bottom line?</li><li>Do I believe I am building a high-quality product?</li><li>Did I get a chance to influence early product decisions?</li><li>Do deadlines correspond with the task breakdown?</li></ul><h3>Do I know what to build?</h3><ul><li>Are there product specs? Are exceptions documented? Are there designs or mock-ups?</li><li>Is there one, up-to-date, source of truth? Are changes documented quickly?</li></ul><h3>Do I know how to build it?</h3><ul><li>Is there an example of what to build or have I built this before? If not, can someone create an example or pair with me?</li><li>Is there a well-written, up-to-date technical specification? Are changes quickly documented?</li><li>Are the objectives of the technical design clearly stated? Are all interactions and data formats documented? Do we know all the code bases that have to change? Are third-party dependencies handled? Are all libraries selected? Is the algorithm complete? Are databases prepared?</li><li>Have we rationally discussed and agreed to the technical decisions above?</li><li>Are the broken-down tasks meaningful, atomic, well-described, and up-to-date?</li><li>Do we know what the deadlines are? Have we resolved conflicting deadlines?</li></ul><h3>Is writing code easy?</h3><ul><li>Am I distracted by fires or bugs?</li><li>Am I overworked?</li><li>Am I distracted by meetings?</li><li>Is my work environment conducive to <a href="https://en.wikipedia.org/wiki/Flow_(psychology)">flow</a>?</li><li>Are code reviews done in a reasonable amount of time? Are review comments effective in improving design and catching bugs? Is there a source of truth for review standards that everyone follows?</li><li>Do review disagreements get settled once-and-for-all via retrospectives? Do we then update our review standards?</li></ul><h3>Is shipping code easy?</h3><ul><li>Can I easily deploy? Is the process documented? Do we depend on the deployments of other teams?</li><li>Do I smoke-test my own changes? Can I debug problems?</li><li>Do automated tests have good coverage, and do they run quickly on dev boxes and continuous integration boxes?</li><li>Do we know what the quality assurance (QA) team is testing? Is the QA team testing against a spec? Is the QA script up-to-date? Does the QA team have enough time to test before releases?</li><li>Are bug tickets appropriately detailed?</li><li>Are release environment states known? Are the release environments well-maintained?</li><li>Do we retro all hotfixes? And record common problems in a playbook?</li></ul><h3>Is team culture healthy?</h3><ul><li>Do I like my teammates? Do I feel comfortable asking for help, raising problems and delivering bad news?</li><li>Do I pitch in to solve problems?</li><li>Are we retro-ing bugs, missed commitments and process problems? Do we make changes to prevent problems from recurring? Do we follow up on these changes?</li><li>Do we know how to work well with teams we depend on (understand their release processes, schedules, personnel)? Do we clearly and quickly communicate changes in our agreements with other teams?</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1a9a6d67e3b5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[It doesn’t work: Story points, planning poker, software prediction]]></title>
            <link>https://medium.com/@imprisonevery1/it-doesnt-work-story-points-planning-poker-software-prediction-2bfaefaf59ea?source=rss-a8d3f71575dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/2bfaefaf59ea</guid>
            <category><![CDATA[planning-poker]]></category>
            <category><![CDATA[agile]]></category>
            <category><![CDATA[planning]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[estimations]]></category>
            <dc:creator><![CDATA[Robert Jailall]]></dc:creator>
            <pubDate>Mon, 02 Apr 2018 03:37:27 GMT</pubDate>
            <atom:updated>2019-07-19T16:08:32.502Z</atom:updated>
            <content:encoded><![CDATA[<h4>But we still do it….</h4><p>When I first became a manager, Scrum was a popular thing. My company sent me to an expensive, week-long official ScrumMaster certification course, taught by the co-inventor of Scrum, where I learned about estimating projects using story points and planning poker (and a lot about Post-It notes)*. I was excited to have something manager-y to do, and I brought these estimation processes to my team.</p><p>They hated it.</p><p>It turns out that story points and planning poker are both mathematically and practically poor ways to predict, and software prediction itself is boring management work that doesn’t help engineers build faster or better.</p><p>We do software prediction because</p><ul><li>We want to know when a team will complete a given list of tasks.</li><li>We want to know what set of tasks to select to fit into a deadline.</li></ul><p>The typical story point+planning poker I’ve done at all the places I’ve worked doesn’t do any of these well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xE-LrcfhgPBmknuNuBKJMQ.jpeg" /><figcaption>Radars do a lot of prediction really quickly</figcaption></figure><p><strong>Statistical Prediction</strong></p><p>Software prediction reminds me of radars. I happened to work at a company that designed radars, and statistical prediction of target motion is fundamental to how radars work. Radars optimally predict where targets are going and direct their beams to stay pointed at the target.</p><p>They do this thusly:</p><ol><li>Given the current statistical model of the target’s state (position, velocity, acceleration), make a prediction of where the target will be.</li><li>Send radar beams to the predicted location and search for the target.</li><li>Once found, measure the error between the prediction and where the target was found.</li><li>Update statistical model of the target to account for prediction error.</li><li>Repeat (this all happens automatically in a fraction of a second).</li></ol><p><strong>Story Point And Planning Poker Prediction</strong></p><p>Story point prediction does not work this way:</p><ol><li>We model how long tasks will take with complexity. Teams use planning poker to vote on task complexity.</li><li>Most people use the naive prediction: the team will produce the amount of story points it produced in the last sprint. We even call this <em>velocity</em>, too.</li><li>When the predictions turn out to be wrong, maybe the team adjusts how they model task complexity. Maybe they don’t. Many teams don’t even examine how much longer or shorter each task took, so they don’t know what the error in the complexity model is.</li><li>Repeat.</li></ol><p>What teams repeat are meetings where they count up story points that are rarely the predicted value, do planning poker for new tickets, and select the work they were going to work on anyway.</p><p><strong>Why doesn’t this work?</strong></p><ul><li>Compared to radars, the typical software prediction process doesn’t include any rigorous mechanism for correcting error in the complexity model.</li><li>You actually can’t add story points! Eight 1-point tasks take much less time than one 8-point task. You have to convert points to hours to add them up and compare with previous sprints. Point totals across sprints are not comparable because of this basic math problem. (Coincidentally, you can’t add up radar returns in decibels because they are on a logarithmic scale, so radars do a similar transformation.)</li><li>Task complexity is not a good way to model team productivity because things that have nothing to do with the task often determine productivity: work distractions, fires, meetings, personal life, health, etc.</li><li>A bad sensor or a randomly moving target will always yield bad predictions. If, say, 50% of the error in predictions comes from things the team can’t control, predictions will always be wrong, especially for small task lists such as sprints.</li><li>This type of planning assumes a platonic employee will complete a task. Many times, we know who will do a task. Instead of taking advantage of the information, we throw it away and do planning poker. The person doing the task will predict how long he or she will take much better than planning poker.</li><li>Psyche. Engineers lie to themselves about how much work they can do all the time. Call them a bad sensor.</li><li>In particularly boring planning pokers (all of them after the first, in my experience), people just throw up the average number for most tasks. Which means the manager could have come up with the same sprint estimate by taking the average task size and multiplying it by the size of the task list anyway without even having a meeting to get the team to do this. I challenge all managers to do this and compare with the team’s prediction.</li></ul><p><strong>What would I do instead?</strong></p><p>Predicting how much work the team can do in a sprint is not particularly valuable to me as a manager or the engineer because it doesn’t change decisions. Everyone just works on a priority queue. Predicting how much the team can do in a quarter <em>is</em> useful for things such as headcount planning, release planning, marketing, etc. Thanks to the <a href="https://en.wikipedia.org/wiki/Central_limit_theorem">central limit theorem</a>, you can make reliable long term predictions by using average task size. You can also just compare proposed projects to previous projects and assume they will take the same amount of time.</p><p>The team <em>should</em> produce a broken down task list. They should not have to do the chore of estimating. The engineering or project manager should use their knowledge of who is doing the task, engineer skills, and past performance to make a prediction. They should mitigate engineer optimism about their productivity. They should also appreciate whether the engineer has experience doing the task.</p><p>In fact, I consider engineer productivity a bimodal model of “things engineers have done before” and “things engineers have not done before.” The latter takes an order of magnitude longer. Managers need to make sure engineers have the right balance of these two types of tasks — familiar tasks to keep the team shipping and unfamiliar tasks to keep the team learning. Learning is the lasting way to increase team productivity.</p><p>That focus on increasing real team productivity is what gets lost when you take up valuable (and hopefully limited) team meeting time to do an administrative chore. This is why I’m not too interested in improving the prediction process above (e.g. linearize story points). As with almost any skill, engineers get better at software development through practice, study, objective critique, and deliberate change. This may look like <a href="https://en.wikipedia.org/wiki/Pomodoro_Technique">pomodoro</a>, code reviews, retrospectives that produce remediations, book/video/paper discussion, spec writing, and team building. Most of the team’s meeting should focus on whether they are doing the right things to get faster, not assigning a number to their speed. Leave that boring stuff to managers, and leave the building to engineers.</p><ul><li>Update 2018/10/22: My Scrum training class was taught by <a href="https://en.wikipedia.org/wiki/Jeff_Sutherland">Jeff Sutherland</a>, co-inventor of Scrum.</li><li><a href="http://deathtoagile.com/">http://deathtoagile.com</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2bfaefaf59ea" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Super Boring Secrets of the Performance Review]]></title>
            <link>https://medium.com/@imprisonevery1/super-boring-secrets-of-the-performance-review-5e09ec6f350b?source=rss-a8d3f71575dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/5e09ec6f350b</guid>
            <category><![CDATA[human-resources]]></category>
            <category><![CDATA[management]]></category>
            <category><![CDATA[salary]]></category>
            <category><![CDATA[performance-reviews]]></category>
            <dc:creator><![CDATA[Robert Jailall]]></dc:creator>
            <pubDate>Fri, 09 Mar 2018 20:49:51 GMT</pubDate>
            <atom:updated>2020-03-10T05:26:09.690Z</atom:updated>
            <content:encoded><![CDATA[<h4>It‘s the Thanksgiving dinner you go to every year even though you know it will end in yelling and stress drinking</h4><p>All fundamentally ethical organizations must regularly adjust employee pay to match market rates. Most do this with“performance reviews” — a theatrical set of rituals on top of a fundamentally financial process. As an individual contributor and first-time manager, I stressed — Why are we subjecting ourselves to this? What does it all mean? Am I being punished?? Aaaaargh…</p><p>This is a guide to performance reviews, their pitfalls, and how employees and managers can optimize their outcomes. This post endorses no part of the process.</p><p><em>Performance review</em>, at its core, allocates raise budgets. As a side effect, it gives feedback (poorly). It does little to make teams better.</p><p>Raise budgets are finite, so performance review is zero sum. Individuals optimize their own outcomes at the expense of others’. (If your budgets aren’t finite, congratulations — you just saved 7 minutes from not having to read this post.) Performance review incentivizes no person and includes no process that strongly optimizes raises globally— or in other words, tries to give everyone a fair raise.</p><p>All of my employers (two managed by Google alums) have performed performance reviews similarly. The exciting <a href="https://hbr.org/2016/10/the-performance-management-revolution">history</a> of human resources may explain this similarity. Why no variety? I suspect it’s rare to find an executive or HR department that has the job security, energy, and focus to experiment with non-standard processes. The incentive to <a href="https://www.goodreads.com/quotes/192407-failing-conventionally-is-the-route-to-go-as-a-group">fail conventionally prevails</a>.</p><h4>Performance Review Rituals</h4><p>Levels, level guides, goals, ratings, calibration, reviews, promo committee, raise actions are the performance review rituals. Companies assemble these rituals according to tastes, the same way you assemble an outfit. Perhaps, managers and HR have lobbied to include their favorite implementations. I’ve not experienced a particularly inspired configuration of any of these rituals. These rituals are subject to many pitfalls:</p><ul><li><em>Connections </em>— Reputation, hearsay, and personal relationships determine outcomes (favorably or unfavorably)</li><li><em>Busywork </em>— The ritual requires a lot of work but no one actually uses the work to make decisions</li><li><em>Anchoring </em>— Initial perceptions determine outcomes rather than ensuing discussion</li><li><em>Endurance</em> — Decisions depend on the physical energy of decision-makers</li><li><em>Writing </em>— Decisions depend on poor write-ups</li><li><em>Bargaining</em> — Skill in negotiating determines outcomes</li><li><em>Opaque</em> — The decision-making process is hidden (even to managers)</li><li><em>Side-job</em> — The person performing the task doesn’t have the incentive or focus to be good at the task</li></ul><p><strong>Levels/Pay Bands<br></strong><em>Input: Industry salary surveys and data, offer history<br>Output: Pay bands, a spreadsheet with current employee levels (position in pay band)<br>Pitfalls: Opaque</em></p><p>Levels map to pay bands. Companies may hide pay bands from employees (though <a href="https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201720180AB168">a 2018 California law</a> requires companies to share with applicants the pay scale for a job, if asked). Savvy managers should have broad knowledge of who is in what pay band. Over time, managers should set up employees so they sail through peer comparisons and, thus, pay bands. It may be convenient for managers to bring new employees in at a high level because it’s easier to fix a high level (small raise) than a low one (promotion).</p><p><strong>Level Guide<br></strong><em>Input: Levels, manager/HR background knowledge<br>Output: A table<br>Pitfalls: Busywork, writing</em></p><p>Level guides abstract pay bands from employees. They are largely unnecessary for performance review. Managers write these guides using language that is so ambiguous and confusing they are not useful for comparing employees. Often when you start using them, you end up talking more about the guide.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w_9J0vRLE6jHNIYFeQ77hQ.png" /><figcaption>What you delivered and what/who you lead will always win out in a comparison of two employees. Your actual team cares about communication and teamwork, but the raise process does not.</figcaption></figure><p>I tell my employees the true way to know where you stand is assess how much risk the organization is taking with the responsibilities it has given you. If your responsibilities are consequential/inconsequential to the fortunes of the business, your level should reflect that. The table doesn’t tell managers this, so managers don’t use it.</p><p><strong>Goals/Objectives/OKRs<br></strong><em>Input: Employee interpretation of level guide<br>Output: Goals for the next review period<br>Pitfalls: Busywork, writing</em></p><p>If goals do tie to raises, they are an agreement that is less binding than a pinky swear between manager and employee on how to get the next raise . If goals don’t tie to raises, they are part of reviews (discussed below). None of my employers tied raises to goals, so everyone just treated goals as another chore.</p><p><strong>Ratings</strong> <br><em>Input: Manager background knowledge, reviews<br>Output: A spreadsheet of proposed ratings for all employees<br>Pitfalls: Opaque, connections</em></p><p>One through five. Always one through five. Ratings represent how much your level should change, but none of my employers ever advertised by how much. Does “greatly exceeds” move me to the next pay band? <em>This is the information everyone really cares about</em>. Some terrible companies ration these ratings (on top of the already-rationed raise budget).</p><p><strong>Reviews<br></strong><em>Input: Background knowledge of managers, peers, underlings, and yourself<br>Output: Written collateral on all employees<br>Pitfalls: Writing, connections, busywork, side-job</em></p><p>Reviews support the ratings of employees who decision-makers don’t know well. These unknown employees compete for ratings after well-known employees easily get theirs.</p><p>Review quality makes a big difference in outcomes. This is where that annoying writing class you took in university matters. Savvy managers must prevent and correct poorly-written reviews. Other managers may seize on the slightest criticism or even absence of praise to distinguish unknown employees. Only include negative feedback to prevent someone from getting a raise or promotion. Negative feedback is appropriate for genuine coaching venues, not the performance review.</p><p><strong>Calibration<br></strong><em>Input: Managers’ background knowledge, a spreadsheet of proposed ratings for all employees, reviews<br>Output: A spreadsheet of final ratings for all employees, promotion proposals<br>Pitfalls: Anchoring, connections, bargaining, endurance, writing</em></p><p>Calibration is where all the cloak and dagger stuff happens. Managers meet to theoretically ensure their proposed ratings mean the same thing. But falling short of that, calibration ends up being a high-leverage moment where managers use a rating to recognize or discount an employee’s year of work. Calibration is often grueling. It’s imprecise. It’s subject to human failings. It’s subject to meeting-management incompetence.</p><p>The pitfalls I’ve seen dominate these meetings are anchoring, bargaining, and endurance.</p><p>For example, sometimes managers underrate their employees. Because this is not Lake Wobegon, they assume all their employees can’t be above average. But in a group setting, managers shy away from significantly arguing down ratings for a variety of reasons: <em>quid pro quo</em>, not wanting to create a beef, exhaustion. Thus, savvy managers propose high ratings and, if necessary, negotiate down to palatable ratings. For disciplines unfamiliar to most managers (such as design or QA), the group accepts ratings proposals without quarrel. Highly-visible or or well-reputed employees breeze through calibration. Employees discussed at the end of meetings also breeze through (for better or worse) because managers simply don’t have the energy to argue anymore.</p><p><strong>Promotion Committee</strong> <br><em>Input: Promotion proposals from managers, reviews, committee background knowledge<br>Output: Promotion denials<br>Pitfalls: Anchoring, connections, writing, endurance, side-job</em></p><p>Calibration is susceptible to the problem of managers overzealously advocating for their reports — a problem no report has ever complained about in the history of the corporate world. Nevertheless, employers created the promotion committee bureaucracy to solve this “problem.” (Curiously, no corresponding bureaucracy deals with underzealous managers.)</p><p>The committee’s purpose is to reject promotion proposals. Committee members don’t have the same bias as managers, but they also don’t have the same quantity and quality of information that comes from regular 1–1s, manager meetings, and daily interaction. Only employees well-known to committee members will sail through.</p><p>Savvy managers will learn who’s on the committee (If it’s a secret. Spoiler: it’s never a secret) and make sure their reports’ tasks get them good face time with committee members. I’ve been friends with committee members. They take their responsibility seriously and have the best intentions, but they don’t have the time or focus to become excellent at reviewing performance. It can be no more than a hobby. (If only there were some class of employee who got paid full-time to specialize in this…)</p><p><strong>Raise Action<br></strong><em>Input: Current levels, final ratings, approved promotions, raise budget, extra-performance-review factors, HR/VP/manager background knowledge<br>Output: A spreadsheet with raises for all employees<br>Pitfalls: Opaque, anchoring, connections</em></p><p>I’ve just written a bunch of words above on rituals that produce a spreadsheet of <em>recommendations. </em>Here, a final editor (VP, director, HR, sometimes managers) writes dollars amounts next to names on that spreadsheet.<em> </em>They may consider their own impressions of employees, team membership, job function, unpublicized compensation goals, <a href="https://github.com/robjailall/raise_calculator">allocation formulary</a>, flight risk, attitude towards proactively paying market rates, etc. Changes at this step outweigh preceding decisions.</p><p>This begs the question — what were those dozens of hours spent on preceding steps for? (“Dance, monkey, dance…”)</p><h4>Random Closing Thoughts</h4><ul><li>The true steps of allocating a raise budget are clear — figure out the market rate for responsibilities, figure out what responsibilities employees actually have, optimally allocate dollars to pay employees their market rates. No part of the typical performance review is well-tailored to do any of these steps.</li><li>I tell employees expect raises to be a function of, in order:<br>1. Financial performance of the company<br>2. Expected employee responsibility<br>3. Market rate of that responsibility<br>4. Current pay for each employee<br>5. Performance in review period</li><li>Employees care when performance review is wrong about them, but organizations only care that the error rate is low enough to keep the organization functioning. What’s the organization’s incentive to drive an error rate from X% to Y%?</li><li>Errors can be team-killing. They create jealousy and purposeless competition, and they disgruntle employees. These feelings lead to bad behaviors that drag on team performance.</li><li>Performance review is not the only way to determine pay, and in fact, most organizations have a second process for this — hiring. Creating a job offer is more standardized and often less bureaucratic than raising pay. Offers consider more matter-of-fact things: years of experience, education, job description, competing offers, etc.</li><li>For some reason (blame <em>Finance</em>), we cannot apply the new hire budget to raises. It’s much easier to hire a new employee at level III for an extra $20,000 than to give an existing employee at level III a $20,000 raise.</li><li>Something else also bypasses the raise process — entering the job market. You’ll find out your market rate. If the organization really values you, you’ll magically get a pay raise without performance review theatrics. Even if you don’t get a raise, you might find a better place to spend your working years. Organizations without scruples penalize employees who ask for their market rate.</li></ul><h4>Further Reading</h4><ul><li>Do people read these?</li><li><a href="https://hbr.org/2016/10/the-performance-management-revolution">The Performance Management Revolution</a> — Deloitte tries something interesting</li><li><a href="https://github.com/robjailall/raise_calculator">Raise Calculator</a> — My script for optimally allocating raise dollars given new employee levels and a raise budget</li><li><a href="https://mtlynch.io/why-i-quit-google/">Why I Quit Google</a> — Found this <a href="https://medium.learningbyshipping.com/performance-of-performance-reviews-cd7494eeaef8">here</a>. This guy was not in love with the Google promotion committee</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5e09ec6f350b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>