<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Roy Williams on Medium]]></title>
        <description><![CDATA[Stories by Roy Williams on Medium]]></description>
        <link>https://medium.com/@rowillia?source=rss-83038993d868------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*Fabx54ody87gEWSG.jpg</url>
            <title>Stories by Roy Williams on Medium</title>
            <link>https://medium.com/@rowillia?source=rss-83038993d868------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 18:50:28 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@rowillia/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Gevent Part 3: Performance]]></title>
            <link>https://eng.lyft.com/gevent-part-3-performance-e64303fa102b?source=rss-83038993d868------2</link>
            <guid isPermaLink="false">https://medium.com/p/e64303fa102b</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[asynchronous]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[performance]]></category>
            <dc:creator><![CDATA[Roy Williams]]></dc:creator>
            <pubDate>Fri, 16 Oct 2020 14:08:06 GMT</pubDate>
            <atom:updated>2020-10-16T20:07:51.124Z</atom:updated>
            <content:encoded><![CDATA[<h3>Gevent Performance (Part 3 of 4)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ADBjaENCxhBhyq3l.jpg" /></figure><h3>Overview</h3><p>Gevent’s sweet spot for performance is network-bound workloads. Coroutines allow us to efficiently interleave other CPU work while waiting on the network instead of just waiting for results.</p><p>As we discussed in <a href="https://eng.lyft.com/what-the-heck-is-gevent-4e87db98a8">part 1</a>: gevent uses coroutines for multitasking and coroutines are <em>not</em> threads. They run completely in user space, from the kernel’s perspective there is still a single task. Coroutines rely on user level libraries for scheduling via an event loop — in gevent’s case the default is libev. Coroutines are significantly lighter weight than real threads so a process can run significantly more of them, and switching between coroutines can also be much faster as there&#39;s no requirement to thunk into kernel space.</p><p>In exchange for being higher speed and lighter weight, coroutines require more thought and work to be done in user space. Since coroutines are not threads, there aren’t any interrupts as with real threads. If an OS thread is hogging too many resources the OS will interrupt it for another task to run, keeping things as fair as possible.</p><p>At the end of the day the machine only has so many CPUs though and we have to deal with those real physical limitations — coroutines are not useful for parallelizing CPU-bound work, they’ll probably make things worse, but are incredibly useful for parallelizing network-bound work.</p><h3>Effective use of gevent</h3><p>Let’s imagine we have an endpoint that needs to fetch 3 resources, does some computation, and returns. The simplest way to write this would be to fetch all 3 resources serially, compute the result, and return:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4c0b128b1a3ac85511875ec31a340088/href">https://medium.com/media/4c0b128b1a3ac85511875ec31a340088/href</a></iframe><pre>6</pre><p>Great! This works as expected. Let’s see how long it takes:</p><pre>%timeit compute_thing()</pre><pre>30.7 ms ± 39 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)</pre><p>Hmm — not bad, but it can probably be faster. This workload appears to be totally dominated by network time, so it’s probably a good candidate to be sped up with coroutines!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/46c91655268a6d3930583272ea778a89/href">https://medium.com/media/46c91655268a6d3930583272ea778a89/href</a></iframe><pre>%timeit compute_thing_parallel()</pre><pre>10.3 ms ± 24.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)</pre><h3>Mixing CPU and I/O</h3><p>Few workloads are as clean as above though — usually our servers do some amount of compute, some amount of I/O, even if that compute is just serialization/deserialization of requests. Let’s create a benchmark where we vary the amount of CPU work, Network-bound work, and the number of coroutines we have running in parallel to simulate concurrent requests. This benchmark does half of its CPU work, then makes a network call, then does the remaining CPU work. You’ll find the code for this benchmark in the last cell of the Jupyter Notebook attached.</p><p>First, let’s run a workload that does 5ms of CPU work and 55ms of networking, with 1 greenlet as a start:</p><pre>5ms CPU/55ms Network per request (500 requests with 1 green threads)<br>	Throughput: 16.49 rps (1.00X Speedup)<br>	CPU Utilization: 8.24%<br>	p50: 60.53ms<br>	p95: 60.60ms<br>	p99: 60.62ms</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*H8y3_HMm7cd7ux2lJlOo-g.png" /><figcaption>Response Time Histogram: 1 green thread</figcaption></figure><p>So far so good — our RPS is roughly what we would expect from a 60ms workload and we have a really tight bounding of response times. The only downside is that utilization number — we’re only using 8.24% of our CPU! Let’s throw more green threads at this task since we’re <em>mostly</em> dominated by I/O:</p><pre>5ms CPU/55ms Network per request (500 requests with 5 green threads)<br>	Throughput: 81.61 rps (4.94X Speedup)<br>	CPU Utilization: 40.80%<br>	p50: 60.88ms<br>	p95: 61.38ms<br>	p99: 64.57ms</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*r9B98VE8CBZIqwrM47MQgQ.png" /><figcaption>Response Time Histogram: 1 green threads</figcaption></figure><p>Amazing! A nearly linear speedup with the number of green threads, and now we’re using 41% of our CPU instead of 8%! Well if 5 green threads was good, 50 must surely be better, right?</p><pre>5ms CPU/55ms Network per request(500 requests with 50 green threads)<br>	Throughput: 192.46 rps (11.60X Speedup)<br>	CPU Utilization: 96.23%<br>	p50: 182.12ms<br>	p95: 182.52ms<br>	p99: 182.77ms</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*GVjdngDABVpaUw1exTjIRQ.png" /><figcaption>Response Time Histogram: 50 green threads</figcaption></figure><p>Uh Oh….while our throughput is 11.6x higher than the baseline (a number tantalizingly close to 12, or 1/(1-(55/60)) but more on that later) our response times have skyrocketed! How can it be the case that the p50 time for a workload that takes 60ms is 188ms? Probably the GIL or garbage collector, right? Let&#39;s try to debug - let&#39;s dump who was doing what during our longest request. We&#39;ll use a different mix to more simply show the problem (+ indicates the start of work on a request, -indicates that work finishing)</p><pre>50ms CPU/150ms Network per request (500 requests with 5 green threads)<br>	Throughput: 19.49 rps (3.91X Speedup)<br>	CPU Utilization: 97.46%<br>	p50: 230.27ms<br>	p95: 255.59ms<br>	p99: 255.89ms<br>=========================<br>Longest Request:<br>	[0.763] Request 3: + 25.00ms CPU time<br>	[0.788] Request 3: + 150.00ms Network time<br>	[0.788] Request 4: + 25.00ms CPU time<br>	[0.813] Request 4: + 150.00ms Network time<br>	[0.863] Request 0: - 150.00ms Network time took 150.72ms<br>	[0.863] Request 0: + 25.00ms CPU time<br>	[0.889] Request 5: + 25.00ms CPU time<br>	[0.914] Request 5: + 150.00ms Network time<br>	[0.915] Request 1: - 150.00ms Network time took 177.05ms<br>	[0.915] Request 1: + 25.00ms CPU time<br>	[0.940] Request 2: - 150.00ms Network time took 177.02ms<br>	[0.940] Request 2: + 25.00ms CPU time<br>	[0.965] Request 6: + 25.00ms CPU time<br>	[0.990] Request 6: + 150.00ms Network time<br>	[0.990] Request 7: + 25.00ms CPU time<br>	[1.015] Request 7: + 150.00ms Network time<br>	[1.016] Request 3: - 150.00ms Network time took 228.47ms<br>	[1.016] Request 3: + 25.00ms CPU time</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*N0A-iUfF0WLvB0oVQ7oQ5A.png" /></figure><p>Interesting — when running with 5 green threads we see that the issue is for the longest request (Request 3) 150ms of network time “took” 228.47ms. Looking at the details we see that there was 25ms of CPU time scheduled for 8 other requests — helping us blow past our 150 network time. The first 6 times CPU time was scheduled it was OK as our request wasn’t done…but every additional time CPU work was scheduled we were starved. The issue is as the number of concurrent requests increases the likelihood more work gets scheduled than can be effectively be parallelized increases.</p><p>Let’s see what this looks like in the limit by plotting the throughput speedup vs. # Green Threads and Response Time vs. # of Green Threads:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/382/1*N8mnHVHEnidnGqu9UqRFhA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*bLMHo3QZOH0aRf7Dh9JWvg.png" /></figure><p>We see that the speedup asymptotically approaches 12, but after ~11 concurrent requests p50 response times start to go up. Let’s revisit that 1/(1-(55/60)) number - this come from <a href="https://en.wikipedia.org/wiki/Amdahl%27s_law">Amdahl&#39;s Law</a> which states the maximum speedup of a task by parallelization is 1/(1-p) where p = {Parallelizable Ratio of Workload}. In our example here 55ms out of 60ms can be parallelized, so p = 0.92</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eWqeUPTx_81RlhQ1" /></figure><p>Our use case here is <em>slightly</em> different than Amdahl’s law — in the case of Amdahl’s Law we throw more machines at a problem to incrementally reduce the cost of the parallelizable portion. With gevent and cooperative multitasking after 1/1-p workers the amount of time the parallelizable portion takes goes to zero as we can run all our CPU work while the network tasks are running.</p><h3>Putting it into practice</h3><p>Even though we focused on Python and Gevent, these issues aren’t limited to gevent — they would be just as applicable to NodeJS, Asyncio, or any other cooperative multitasking system — if a cooperative system attempts to handle more simultaneous requests than it can response times will degrade. For putting this into practice, however, we will focus on gevent and gunicorn.</p><p>In practice, this means we would like to minimize the number of concurrent requests a given process is handling — any processes handling too many concurrent requests will degrade response times. With this in mind, there’s a few things to be aware of:</p><ol><li><a href="https://idea.popcount.org/2017-02-20-epoll-is-fundamentally-broken-12/">epoll unfairly routes requests</a>, favoring higher PIDs. This can quickly lead to a situation where one process is handling too many requests, running super hot while the others are barely scheduled. This is exacerbated by a <a href="https://github.com/benoitc/gunicorn/pull/2266">bug in gunicorn</a> that causes a worker to accept as many connections as it can upon waking up. Pulling in this fix leads to more fair routing.</li><li>Setting SO_REUSEPORT on the listening socket also <a href="https://lwn.net/Articles/542629/">improves fairness</a>. This flipped from always being on <a href="https://github.com/benoitc/gunicorn/pull/1669">to being opt-in in gunicorn 19.8</a>. Services should consider manually specifying --reuse-port in their gunicorn configuration.</li><li>Gunicorn allows services to specify the maximum number of concurrent requests a worker will services with the <a href="https://docs.gunicorn.org/en/stable/settings.html#worker-connections">— worker-connections</a> flag. This defaults to 1000 which is almost certainly too high for most services (for example, services would have to do 1ms of CPU work, 1s of network bound work per request for this to be correct). Consider tuning this down to a more appropriate number using tools like tracing to determine how long the service is blocked on network vs. consuming CPU. A more proper number is {network_time}/{cpu_time} - for example if a service does 5ms of CPU work and 55ms of waiting on network for a typical request it should likely tune --worker-connections=12 - anything higher and response times will degrade.</li><li>Optimize CPU time — Reducing CPU increases the {network_time}/{cpu_time} naturally allowing the service to handle more concurrent requests.</li><li>Reduce how often critical code yields — Yielding more often to the event loop will reduce predictability of responses since there are now more opportunities for other requests to run and slow down the running request. Below we run the same benchmark, except instead of 1 55ms network call per request there&#39;s 5 11ms network calls per request. We see response times degrade faster as the number of green threads grow.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/382/1*7TNmU0BLN2993N9ojKYvdQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*i3egDdh30kmu_qvDR42W3Q.png" /></figure><p>Note that all of the above optimizations really only help services that are running somewhat hot. If each host is only ever serving a single request at a time this won’t help — concurrency isn’t an issue in that case. That said, the issues mentioned above are important blockers to running hotter — if a service saw response times degrade at higher CPU try making the above optimizations and try running hotter again.</p><p>Once again, you can download this blog post as a Jupyter Notebook! I encourage you to play with the benchmarks to help understand how performance degrades as you try to do more work in greenlets.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6f01ec58c4b2030d8b6a8b412a865fb0/href">https://medium.com/media/6f01ec58c4b2030d8b6a8b412a865fb0/href</a></iframe><p><em>Lyft is hiring! If you’re interested in improving peoples lives with the world’s best transportation, </em><a href="https://www.lyft.com/careers"><em>join us</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e64303fa102b" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/gevent-part-3-performance-e64303fa102b">Gevent Part 3: Performance</a> was originally published in <a href="https://eng.lyft.com/">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gevent Part 2: Correctness]]></title>
            <link>https://eng.lyft.com/gevent-part-2-correctness-22e3b7998382?source=rss-83038993d868------2</link>
            <guid isPermaLink="false">https://medium.com/p/22e3b7998382</guid>
            <category><![CDATA[asynchronous]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[bugs]]></category>
            <dc:creator><![CDATA[Roy Williams]]></dc:creator>
            <pubDate>Thu, 15 Oct 2020 20:42:25 GMT</pubDate>
            <atom:updated>2020-10-15T20:42:25.732Z</atom:updated>
            <content:encoded><![CDATA[<h3>Gevent Correctness (Part 2 of 4)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xDCTODeH_jGDFKDC.png" /></figure><p>As we talked about in <a href="https://eng.lyft.com/what-the-heck-is-gevent-4e87db98a8">part 1</a>, gevent makes all code magically cooperative, but not all code was written to be cooperative.</p><p>This can lead to <em>really</em> subtle problems in production that are incredibly hard to debug — code that works in tests but breaks under load. Take the code below — let’s imagine we have an in-memory database of account balances and we send money in a circle. At the end, there should be no difference in balances — everyone should have $100.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/698b7abe9d4c97c06edcf9e63ce14a76/href">https://medium.com/media/698b7abe9d4c97c06edcf9e63ce14a76/href</a></iframe><pre>Tests passed!<br>{&#39;roy&#39;: 100, &#39;jack&#39;: 100, &#39;nick&#39;: 100}</pre><p>So far, so good. We have a unit test that passes and even after 100 concurrent requests we still have everyone at the correct balance. Now imagine we want to introduce some observability and track how much money is moving around. We want to maximize utilization of our service so our stats library uses gevent. compute_new_balances seems like the correct place to instrument, so we add a stats call. We make the change and then run our unit test again to ensure it&#39;s safe.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/577f0d2675459a8509fdaeb030b5ec78/href">https://medium.com/media/577f0d2675459a8509fdaeb030b5ec78/href</a></iframe><pre>Tests passed!</pre><p>Perfect! We introduced a stats change, tests passed, how dangerous can it be? Let’s run this bad boy in production…</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/76a5999c58ce66c477114133dda31810/href">https://medium.com/media/76a5999c58ce66c477114133dda31810/href</a></iframe><pre>{&#39;roy&#39;: 310, &#39;jack&#39;: 310, &#39;nick&#39;: 110}</pre><p>Uh oh….everyone’s supposed to have the same balance at the end of this but now everyone’s wildly off. Our customers are going to be really disappointed (or really happy!). How did this happen? Our Bank code was <em>proven to be </em>correct with unit tests. The core of the issue is our Bank code was written assuming serial, uninterrupted execution - send_money assumed the underlying database wasn&#39;t modified during between when we read the value and when we wrote it. This assumption was correct until a stats call was introduced - now a change in code I was calling into broke that assumption.</p><p>This is the crux of <em>why</em> gevent causes bugs — magically interleaving code that wasn’t meant to be interleaved causes bugs. This isn’t unique to Python — this is a super common problem in threaded languages, for example <a href="https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html">Java’s HashMap</a> is not thread safe.</p><h3>Mitigating correctness problems.</h3><p>When faced with problems like this frequently the desire is to reach for locks — rarely is that the correct solution. Locks lead to even more performance and correctness problems and should be used as a last resort.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/806/0*NrzE2P7RgY-axNfd.png" /></figure><p>One thing <a href="https://slack.engineering/taking-php-seriously-cf7a60065329">PHP got right</a> was “no shared state” — it’s <em>nearly</em> impossible to accidentally share state between requests like this. The first step of mitigation is to avoid shared state whenever possible. In the above example — we should either create a new Bank per request or use an external database. Each request should be <em>stateless</em>.</p><p>If that’s not possible we should ensure that all state updates are consistent. In this case, the core of the problem is the assumption the read and write are consistent and the underlying db did not change in between. A more correct way would be to accumulate transactions in a log, replaying them when we request the database. Appending to the list is safe as we are never updating the underlying database.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/22848422073814551e8b8929d5f5b3b4/href">https://medium.com/media/22848422073814551e8b8929d5f5b3b4/href</a></iframe><pre>{&#39;roy&#39;: 100, &#39;jack&#39;: 100, &#39;nick&#39;: 100}</pre><p>If avoiding shared state isn’t possible, we should identify our critical sections and <em>ensure</em> no switching occurs when we are reading from/writing to this shared state. gevent has a utility <a href="http://www.gevent.org/api/gevent.util.html#gevent.util.assert_switches">gevent.util.assert_switches</a> which does the opposite of what we want, but can be coerced into what we need:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2f4b7e1b4375dde2b5ede6740843e232/href">https://medium.com/media/2f4b7e1b4375dde2b5ede6740843e232/href</a></iframe><pre>---------------------------------------------------------------------------</pre><pre>Exception                                 Traceback (most recent call last)</pre><pre>&lt;ipython-input-9-767ee5373c1f&gt; in &lt;module&gt;<br>     13     print(&#39;Tests passed!&#39;)<br>     14 <br>---&gt; 15 test_send_around_money()<br></pre><pre>&lt;ipython-input-9-767ee5373c1f&gt; in test_send_around_money()<br>      9        pass<br>     10     else:<br>---&gt; 11         raise Exception(&#39;Unexpected contextswitch!&#39;)<br>     12     assert b.db == {&#39;roy&#39;: 100, &#39;jack&#39;: 100, &#39;nick&#39;: 100}<br>     13     print(&#39;Tests passed!&#39;)<br></pre><pre>Exception: Unexpected contextswitch!</pre><h3>Conclusion</h3><p>gevent has all of the correctness downsides of threads - code is hard to reason about as engineers have to consider all possible inter-leavings. It&#39;s easy to write bugs that don&#39;t reproduce in unit tests and bite us in production. gevent is incredibly powerful, but we have to be aware of these downsides and avoid them if at all possible.</p><p>PHP has millions of problems, but shared state isn’t one of them!</p><p>If you want to play around with this code yourself, you can download this as a <a href="https://jupyter.org/">Jupyter Notebook</a>! Play around with code and see the problems for yourself!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/83a269268074eb93d7062ade4ebcc0a8/href">https://medium.com/media/83a269268074eb93d7062ade4ebcc0a8/href</a></iframe><p>This post is heavily inspired by <a href="https://glyph.twistedmatrix.com/2014/02/unyielding.html">Unyielding by Glyph</a></p><p><em>Lyft is hiring! If you’re interested in improving peoples lives with the world’s best transportation, </em><a href="https://www.lyft.com/careers"><em>join us</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22e3b7998382" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/gevent-part-2-correctness-22e3b7998382">Gevent Part 2: Correctness</a> was originally published in <a href="https://eng.lyft.com/">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What the heck is gevent?]]></title>
            <link>https://eng.lyft.com/what-the-heck-is-gevent-4e87db98a8?source=rss-83038993d868------2</link>
            <guid isPermaLink="false">https://medium.com/p/4e87db98a8</guid>
            <category><![CDATA[gevent]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[asynchronous]]></category>
            <category><![CDATA[infrastructure]]></category>
            <dc:creator><![CDATA[Roy Williams]]></dc:creator>
            <pubDate>Wed, 14 Oct 2020 16:55:10 GMT</pubDate>
            <atom:updated>2020-10-16T17:12:50.976Z</atom:updated>
            <content:encoded><![CDATA[<h3>What the heck is gevent? (Part 1 of 4)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bcMNe0CwPf4DsKrP.png" /></figure><h3>Overview</h3><p>gevent is a <a href="https://en.wikipedia.org/wiki/Coroutine">coroutine-based</a> <a href="https://en.wikipedia.org/wiki/Cooperative_multitasking">cooperative multitasking</a> python framework that relies on <a href="https://en.wikipedia.org/wiki/Monkey_patch">monkey patching</a> to make all code cooperative. Gevent actually draws its lineage from Eve Online which was implemented using <a href="https://en.wikipedia.org/wiki/Stackless_Python">Stackless Python</a> which eventually evolved into <a href="https://pypi.org/project/eventlet/">eventlet</a> which inspired gevent. Just like Eve Online, gevent is full of skullduggery and confounds outsiders.</p><p>There’s a lot in there, so let’s break it down.</p><h3>Coroutine</h3><p>Coroutines split up programs into <em>blocking</em> and <em>non-blocking</em> work. Frontend developers are familiar with <a href="http://callbackhell.com/">callback hell </a>— each of those callbacks is a coroutine.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0bd38d1c9a0aa4b92fb5b02eba9e18c4/href">https://medium.com/media/0bd38d1c9a0aa4b92fb5b02eba9e18c4/href</a></iframe><pre>doing some work<br>done first<br>done second</pre><p>In the above example, the functions are coroutines - they do some work, call a blocking function with a callback for what to do when that blocking function finishes. Importantly coroutines have to explicitly yield control for another coroutine to run which allows us to more easily reason about where we have to be concerned about interleaving (more on this later). coroutines are <em>not</em> threads! A coroutine is also known as a greenlet in gevent.</p><h3>Cooperative Multitasking</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*d3Aa5W1SuBAPcuwk.png" /></figure><p>Cooperative Multitasking is a method used to enable a computer to handle more concurrent work than there are workers (e.g. cores on your computer). Cooperative Multitasking dates back to the dawn of timesharing — <a href="https://retrocomputing.stackexchange.com/a/796">Windows 3.1</a> relied on cooperative multitasking to allow multiple applications to run at the same time on a computer with 1 core.<strong> The main downside, then and now, is cooperative systems rely on tasks to…well..cooperate.</strong> In a <a href="https://en.wikipedia.org/wiki/Preemption_(computing)#PREEMPTIVE">preemptive multitasking</a> system like modern operating systems, the OS understands priorities and how much time each task has run for and can interrupt long running process to allow another to run. No such <a href="https://en.wikipedia.org/wiki/Scheduling_(computing)#Scheduling_disciplines">fairness</a> mechanism exists for a cooperative system.</p><h3>Monkey Patching</h3><p>Monkey patching is a technique for modifying behavior at runtime for the standard library or third party libraries. Monkey patching is generally considered a bad practice as it can lead to hard-to-understand bugs and requires the monkey patches to be updated with the underlying library being patched. In the case of gevent — monkey patching has to be the absolute <em>first</em> thing a process does — otherwise libraries will get a handle to the real, blocking implementations.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d5772455dd131ee4486fa37f682fa206/href">https://medium.com/media/d5772455dd131ee4486fa37f682fa206/href</a></iframe><pre>Should have slept 10 seconds</pre><h3>Bringing it all together</h3><p>Below we have a full example using gevent - we first monkey patch the standard library which then magically makes time.sleep <em>cooperative</em> - instead of blocking the CPU it yields control for at least the sleep time. No other code changes were required. gevent avoids callback hell by monkey patching the standard library, automatically creating coroutines - effectively resumption of work at the yield point. A useful corollary is <a href="https://devblogs.microsoft.com/premier-developer/dissecting-the-async-methods-in-c/">how C# generates async/await code</a> - it effectively breaks the function up every time there&#39;s an await.</p><p>We’ll run this code using gevent.spawn to simulate concurrent requests.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/09be83fa460ed6801fd6291410307570/href">https://medium.com/media/09be83fa460ed6801fd6291410307570/href</a></iframe><pre>0: Doing first step<br>0: Doing second step<br>1: Doing first step<br>1: Doing second step<br>0: Doing third step<br>1: Doing third step</pre><p>These coroutines are cooperative - notice we <em>never</em> see anything else run between first step and second step because control was never yielded.</p><h3>Gevent — the good parts</h3><p>gevent isn&#39;t all bad. Coroutines are substantially lighter weight than threads or processes - we can run tens of thousands of greenlets on a host that could run hundreds of threads. gevent works with almost every third party library with no code changes - <a href="https://twistedmatrix.com/trac/">twisted</a>, <a href="https://www.tornadoweb.org/en/stable/">tornado</a>, even <a href="https://docs.python.org/3/library/asyncio.html">asyncio</a> all require using libraries that play nicely with their event loops for any blocking I/O - in contrast gevent works with <a href="https://github.com/psf/requests">requests</a> out of the gate. gevent promises &quot;Add these two lines of code to your project and it&#39;s magically higher throughput&quot; and it mostly delivers.</p><h3>Problems with Gevent</h3><p>Unfortunately it’s not all sunshine and rainbows with gevent. gevent can lead to real correctness problems by interleaving code in unexpected ways. If a library is using native code we can still face blocking. Lastly while gevent optimizes for increasing throughput, it can lead to real latency problems.</p><p>But this post is already getting quite lengthy — so we’ll go over correctness problems and how to avoid them in part 2 and responsiveness issues in part 3.</p><p>If you want to play around with this code yourself, you can download this as a <a href="https://jupyter.org/">Jupyter Notebook</a>!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/35592739797f5f5362878af50e15061d/href">https://medium.com/media/35592739797f5f5362878af50e15061d/href</a></iframe><p><em>This is part of a 4-part series. We suggest you also check out </em><a href="https://eng.lyft.com/gevent-part-2-correctness-22e3b7998382"><em>Part 2: Correctness</em></a><em>, </em><a href="https://eng.lyft.com/gevent-part-3-performance-e64303fa102b"><em>Part 3: Performance</em></a><em>, and </em><a href="https://eng.lyft.com/applying-gevent-learnings-to-deliver-value-to-users-part-4-of-4-36ad932deea8"><em>Part 4: Applying Learnings to Deliver Value to Users</em></a><em>.</em></p><p><em>Lyft is hiring! If you’re interested in improving peoples lives with the world’s best transportation, </em><a href="https://www.lyft.com/careers"><em>join us</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4e87db98a8" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/what-the-heck-is-gevent-4e87db98a8">What the heck is gevent?</a> was originally published in <a href="https://eng.lyft.com/">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hashing and Equality in Python]]></title>
            <link>https://eng.lyft.com/hashing-and-equality-in-python-2ea8c738fb9d?source=rss-83038993d868------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ea8c738fb9d</guid>
            <category><![CDATA[hashing]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[lyft]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[programming-languages]]></category>
            <dc:creator><![CDATA[Roy Williams]]></dc:creator>
            <pubDate>Wed, 15 Apr 2020 17:55:58 GMT</pubDate>
            <atom:updated>2020-04-15T17:55:58.199Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*CIiQ-SI2r6bxdARy.png" /><figcaption>Things get wacky when we’re confused as to what’s what.</figcaption></figure><h3>TL;DR</h3><p>Don’t override __hash__ and __eq__ to force objects to hashable. Use immutable objects instead.</p><h3>Overview</h3><p>Dictionaries and Sets are some of the most common data structures, used heavily for their O(1) lookup times. This O(1) look is enabled by <a href="https://en.wikipedia.org/wiki/Hash_function"><em>hash functions</em></a> which have the following properties:</p><ul><li>If a == b then hash(a) == hash(b)</li><li>If hash(a) == hash(b), then a <em>might</em> equal b</li><li>If hash(a) != hash(b), then a != b</li></ul><p>Dictionaries and sets, regardless of language, use these assumptions to enable fast O(1) lookups. This post will focus on Python, but the lessons are applicable regardless of language.</p><h3>Quick Tour of Dictionaries</h3><p>Python dictionaries are quite advanced and optimized — for the details see this <a href="https://www.youtube.com/watch?v=npw4s1QTmPg">great talk by Raymond Hettinger</a>, but to understand why these properties are important we will go over how they work at a high level.</p><h4>Storing an object:</h4><ol><li>Call __hash__ on the key to compute the hash of the key. If the key is not hashable raise a TypeError</li><li>Store (hash_value, key, value) in an array at location hash_value % len(array).</li><li>If the array requires resizing, re-use the previously computed hash_values to re-insert all previously stored values.</li></ol><h4>Retrieving an object by key</h4><ol><li>Call __hash__ on the key to compute the hash of the key. If the key is not hashable raise a TypeError</li><li>Look in hash(key) % len(array)for an entry with a matching hash_value. If one exists — check for equality, first by identity, then by calling __eq__.</li></ol><p>To have functionally correct dictionaries, a few things follow from this:</p><ol><li>__eq__ and __hash__ <em>must</em> agree - equal objects must have equal hashes.</li><li>__hash__ <em>must</em> never change. The hash of an object is never re-computed once it is inserted.</li><li>Objects that implement logical equality (e.g. implement __eq__) must be immutable to be hashable. If an object has logical equality, updating that object would change its hash, violating rule 2.dict, list, set are all inherently mutable and therefore unhashable. str, bytes, frozenset, and tuple are immutable and therefore hashable.</li></ol><h3>Diving into the details</h3><p>Let’s first dive into how objects work if we <em>don’t</em> implement __hash__ and __eq__.</p><p>By default, equality and hashes are based on the <a href="https://docs.python.org/3/library/functions.html#id">identity</a> of an object. The current implementation is id &gt;&gt; 4 (we should never rely on this implementation detail).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ed845467dea386600d6c9b566f7cbfdd/href">https://medium.com/media/ed845467dea386600d6c9b566f7cbfdd/href</a></iframe><pre>Hash of x: 8729258887714<br>id of x: 139668142203424<br>id of x &gt;&gt; 4: 8729258887714<br>Is x equal to itself? True</pre><p>Dictionaries and sets assume that if an object has an equivalent identity to an object in a set or dict the objects are equal (e.g. we assume an object is always equal to itself). This is an important optimization since __eq__ functions can be expensive.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/167ab011ec9bf0453cb33b887d448a3b/href">https://medium.com/media/167ab011ec9bf0453cb33b887d448a3b/href</a></iframe><pre>Is never_equal in a set of itself? True</pre><p>__hash__ is only called <em>once</em> when an object is inserted into a dict or a set. Note we never see the hash for a or b called again even though we&#39;ve forced the set to resize by inserted 10,000 more items and we&#39;ve retrieved a logically identical object.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f8e1038d01429ea80c062078f3c58081/href">https://medium.com/media/f8e1038d01429ea80c062078f3c58081/href</a></iframe><pre>Calling hash for a: 42<br>Calling hash for b: 37<br>Calling hash for c: 42<br>True</pre><p>This is why it is absolutely critical the __hash__ of an object <em>never</em> change over its lifetime. This works out of the gate for object identity (if id(a) == id(b) we know a == b).</p><p>Should the __hash__ change we can get into a situation where dictionaries and sets no longer work and have extremely hard-to-find bugs.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/da3e55c67a8af6312318c7e4d6b8e3db/href">https://medium.com/media/da3e55c67a8af6312318c7e4d6b8e3db/href</a></iframe><pre>Object is in list: True<br>Object is in set: True<br>Logically equivalent object is in list: True<br>Logically equivalent object is in set: True</pre><p>So far — so good. We’ve outsmarted Python and really forced that dictionary it didn’t want to hash to be hashable! But what happens if we mutate d - the underlying dictionary</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/569283be42e77730eda05ce36d549d08/href">https://medium.com/media/569283be42e77730eda05ce36d549d08/href</a></iframe><pre>Object is in list: True<br>Object is in set: False<br>Logically equivalent object is in list: True<br>Logically equivalent object is in set: False</pre><p><strong>Wat?</strong> I thought an object was always equal to itself? Why is x in the list but not in the set which contains the exact same objects? Same for a logically equivalent object?</p><p>Recall — <em>all</em> of the optimizations that enable dictionaries and sets to be super fast with O(1) lookups <em>require</em> hashes to never change. By fighting against the language we&#39;ve introduced the opportunity for really subtle, confounding bugs.</p><h3>OK, I get it, what should I do instead?</h3><p>I’m glad you asked!! If we <em>really</em> want these objects to be hashable we can achieve this with immutability!</p><p>There’s a few built-ins we can leverage, namely tuple and frozenset, and some third-party-ish libraries we can use - immutables and attrs. While both of these are third party libraries, both have strong analogs in the Python standard library (immutables may be part of the standard library in 3.9 as it&#39;s used as part of <a href="https://docs.python.org/3/library/contextvars.html#module-contextvars">asyncio contextvars</a> and attrs had been mostly duplicated in the standard library with <a href="https://docs.python.org/3/library/dataclasses.html">dataclass</a>).</p><h4>Builtins</h4><p>If we have a list we want to hash, we can convert it to a tuple to ensure it&#39;s immutable:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cba7bc49fe52bbe24fb0a9a7c0c45af3/href">https://medium.com/media/cba7bc49fe52bbe24fb0a9a7c0c45af3/href</a></iframe><pre>Exception when trying to hash a list! unhashable type: &#39;list&#39;<br>hash(tuple(x)): 2528502973977326415</pre><p>If we have a set we want to hash, we can convert it to a frozenset to ensure it&#39;s immutable while still maintaining O(1) lookups:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4e6424fdcd255caa6ef2ee706b20a7fc/href">https://medium.com/media/4e6424fdcd255caa6ef2ee706b20a7fc/href</a></iframe><pre>Exception when trying to hash a set! unhashable type: &#39;set&#39;<br>hash(frozenset(x)): -272375401224217160</pre><h4>Classes and Dictionaries</h4><p>We’re going to use two high quality third party libraries, <a href="https://www.attrs.org/en/stable/">attrs</a> and <a href="https://github.com/MagicStack/immutables">immutables</a></p><p>attrs supports frozen objects which can be treated as immutable and therefore are hashable. As bonus points it gets rid of a <em>ton</em> of boiler-plate code!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/622746963d0793e639aeb0ed338dde32/href">https://medium.com/media/622746963d0793e639aeb0ed338dde32/href</a></iframe><pre>Created an ImmutableThing: ImmutableThing(x=42, y=&#39;hello world&#39;, computed_field=(&#39;hello&#39;, &#39;world&#39;))<br>Caught an error trying to assign to ImmutableThing! &lt;class &#39;attr.exceptions.FrozenInstanceError&#39;&gt;<br>hash(thing): -7021629421173658791<br>ImmutableThing respects logical equality: True<br>ImmutableThing as a dictionary key: {&#39;thing&#39;: &#39;thing is a key&#39;}</pre><p>For dict we&#39;re going to use immutables.Map. immutables.Map is an immutable Map, and therefore Hashable. Any mutations create a <em>new</em> map all together, but leaves the underlying map unchanged. It&#39;s significantly higher performance than deepcopy.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c37ec56bdb3f66a76ad186dda467e53e/href">https://medium.com/media/c37ec56bdb3f66a76ad186dda467e53e/href</a></iframe><pre>Exception when trying to hash a dict! unhashable type: &#39;dict&#39;<br>hash(m): -4868096002692125985<br>Map as a dictionary key: {&#39;m&#39;: &#39;map is a key!&#39;}<br>&#39;immutables._map.Map&#39; object does not support item assignment<br>m after assignment: &lt;immutables.Map({&#39;c&#39;: 3, &#39;b&#39;: 2, &#39;a&#39;: 1}) at 0x7f0705d0d1b0&gt;<br>m_prime after assignment: &lt;immutables.Map({&#39;c&#39;: 3, &#39;b&#39;: 2, &#39;a&#39;: 42}) at 0x7f0705d0d5e8&gt;</pre><h3>Conclusion</h3><p>Fighting against the language to try to force objects to be hashable is both slow and unsafe — wasting money, hurting response times, and damaging reliability. We’ve gone over some of the tools you can use here to make objects hashable when you need to while preserving correctness and performance. We used Python as an example here, these problems and contracts aren’t limited to Python — they’re fundamental to how data structures like hash maps or hash sets work. This blog post is <strong><em>also </em></strong>available as a <a href="https://gist.github.com/rowillia/05a7cf41ba4c346e7f4bd31a2025165d">Jupyter notebook</a> you can download and play with the examples yourself!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0de984f52753eab3426eb2bc2827c15d/href">https://medium.com/media/0de984f52753eab3426eb2bc2827c15d/href</a></iframe><p>If you find yourself fighting against the language stop and think if this is really the correct thing to be doing.</p><p><em>If you’re interested in joining the team at Lyft, we’re hiring! Feel free to check out our </em><a href="https://www.lyft.com/careers"><em>careers page</em></a><em> for more info.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ea8c738fb9d" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/hashing-and-equality-in-python-2ea8c738fb9d">Hashing and Equality in Python</a> was originally published in <a href="https://eng.lyft.com/">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tautology Tests]]></title>
            <link>https://medium.com/@rowillia/tautology-tests-7dabd81ade30?source=rss-83038993d868------2</link>
            <guid isPermaLink="false">https://medium.com/p/7dabd81ade30</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Roy Williams]]></dc:creator>
            <pubDate>Fri, 11 Aug 2017 15:01:01 GMT</pubDate>
            <atom:updated>2017-08-13T02:22:07.479Z</atom:updated>
            <content:encoded><![CDATA[<p>Tests are an incredibly important part to producing quality software quickly, but, as with all things in life, can cause more harm than good when used incorrectly. Consider the following overly simple function and test. In this case, the author wants to insulate their tests from external dependencies, so they rely on mocks.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4ae3baf4ea5e7f744c1f55ab062ca935/href">https://medium.com/media/4ae3baf4ea5e7f744c1f55ab062ca935/href</a></iframe><p>Looks great! Fully tested, 4 assertions to ensure the code executed as expected. The tests even pass!</p><pre>$ python3.6 -m pytest test_simple.py<br>========= test session starts =========<br>itemstest_simple.py .<br>======= 1 passed in 0.03 seconds ======</pre><p>Of course, the problem is this code is wrong. <em>md5</em> only accepts <em>bytes</em> not <em>str </em>(See <a href="http://eli.thegreenplace.net/2012/01/30/the-bytesstr-dichotomy-in-python-3">this blog post</a> for an explanation of how bytes and strings changed in Python 3). The test case had little value; it really only tested string formatting, granting us a false sense of security. We thought we had written our code correctly and we had test cases to prove it!</p><p>Thankfully <a href="http://mypy-lang.org/">mypy</a> catches these issues:</p><pre>$ mypy test_simple.py<br>test_simple.py:6: error: Argument 1 to “md5” has incompatible type “str”; expected “Union[bytes, bytearray, memoryview]”<br>test_simple.py:8: error: Argument 1 to “update” of “_Hash” has incompatible type “str”; expected “Union[bytes, bytearray, memoryview]”</pre><p>OK great, we fix the underlying code to first encode our strings as bytes:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/46182d08edd4ebd7c3c1f3adb01a04b6/href">https://medium.com/media/46182d08edd4ebd7c3c1f3adb01a04b6/href</a></iframe><p>Our code now works, but we still have a problem. Let’s say someone else comes through and simplifies our code even further to just a couple of lines:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a28231a60e94f1871d49742d981609e7/href">https://medium.com/media/a28231a60e94f1871d49742d981609e7/href</a></iframe><p>This code is functionally identical to the original code. For the same set of inputs it will always return the same output. Even so, our tests fail:</p><pre>E AssertionError: Expected call: md5(b’hello’)<br>E Actual call: md5(b’helloworld’)</pre><p>There’s clearly a problem with this simple test. It’s both been subject to <a href="https://www.khanacademy.org/math/statistics-probability/significance-tests-one-sample/idea-of-significance-tests/v/type-1-errors">Type I Error</a> (failed when the underlying production code was fine) <em>and</em> to <a href="https://en.wikipedia.org/wiki/Type_I_and_type_II_errors#Type_II_error">Type II Error</a> (passed when the underlying production code was broken). In an ideal world tests would fail if and only if the underlying code was broken. In an even more perfect world if our tests passed we would have 100% confidence our production code was correct. While neither of these ideals is achievable, they are paragons worth pursuing.</p><p>I call these kinds of tests “<strong><em>Tautology Tests</em></strong>”. They assert that the code is correct by ensuring the code executes as written which, of course, assumes the way it is written is correct.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/0*JrEwUp-Zf8v0DQRJ." /><figcaption>Comic by xkcd, licensed under CC-BY-NC 2.5, available <a href="https://xkcd.com/703/">https://xkcd.com/703/</a></figcaption></figure><p>I argue that Tautology Tests are a net negative for your code for a few reasons:</p><ol><li><strong>Tautology Tests give engineers a false sense of security that their code is correct.</strong> They might look at the high code coverage and feel good about their project. Others coming into the code base will feel confident pushing changes as long as tests pass even though those tests aren’t really testing anything.</li><li><strong>Tautology Tests effectively freeze the implementation rather than test that the code behaves as desired.</strong> Whenever implementation details change, the tests must be updated to reflect the new implementation, instead of changing tests when the expected output changes. In turn, this trains engineers to correct the tests when they fail — instead of investigating <em>why</em> the test failed. When this happens tests reduce themselves to burdens, losing their original purpose as tools for preventing bugs from getting into production.</li><li><strong>Static Analysis tools are capable of finding blatant errors in your code like typos that would otherwise be caught by a Tautology Test.</strong> Especially in the realm of dynamic languages, static analysis tools have improved significantly over the past 5 years, be it <a href="http://mypy-lang.org/">mypy</a> in Python, <a href="http://hacklang.org/">hack</a> in PHP, or <a href="https://www.typescriptlang.org/">TypeScript</a> in JavaScript. These tools are frequently more appropriate for finding typos and provide additional value to other engineers by making the code easier to understand and navigate.</li></ol><p>In short, Tautology Tests frequently miss real issues, encourage the bad habit of blindly fixing tests, and cost substantially more to maintain than the value they provide.</p><p>Now consider if we rewrite the test to just test the expected output:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2bba30ccacd9b77d8164656342067342/href">https://medium.com/media/2bba30ccacd9b77d8164656342067342/href</a></iframe><p>Now my test doesn’t care about the internal details of get_key and will break only if get_key returns an incorrect value. I can change the internals of get_key as I see fit without having to update tests (unless I change the public behavior). My test is also succinct and easier to understand.</p><p>While this is a contrived example, it’s easy to find examples in real code that — for example — assume the output of an external service matches the implementation’s expectations just to increase code coverage.</p><h3>How to Find Tautology Tests</h3><ol><li><strong>Tests that get updated much more frequently than the code they’re testing when they fail.</strong> Every time this happens it’s part of the cost we pay for test coverage. If that cost starts to exceed the value derived from the test it’s a strong hint that our test code is too tightly coupled to the implementation. A related problem: a small production code change requires updating a much larger number of tests.</li><li><strong>Test code that’s impossible to edit without looking at the implementation.</strong> If it’s impossible to modify a test without also looking at the implementation then there is a strong chance you’ve got a Tautology Test. Google’s <a href="https://testing.googleblog.com/2013/05/testing-on-toilet-dont-overuse-mocks.html">Don’t Overuse Mocks</a> “Testing on the Toilet” had an all too familiar example of this — you could re-write the implementation from the test:</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/37598fad28d31bd3875b13c8572751a7/href">https://medium.com/media/37598fad28d31bd3875b13c8572751a7/href</a></iframe><h3>How to fix Tautology Tests</h3><ul><li><strong>Keep I/O separate from logic</strong>. I/O is one of the most frequent reasons engineers have to reach for mocks. I/O is super important…without I/O all we can do is spin CPU cycles and heat up our computers…but it should be pushed to the peripheries of your code instead of being interleaved with logic. The <a href="https://sans-io.readthedocs.io/how-to-sans-io.html">Sans-I/O</a> working group in the Python Community has some great documentation on this topic and Cory Benfield covered this well at PyCon 2016 in his talk <a href="https://www.youtube.com/watch?v=7cC3_jGwl_U">Building Protocol Libraries The Right Way</a>.</li><li><strong>Avoid mocking in memory objects</strong>. If you find yourself mocking out dependencies that exist entirely within the confines of memory, there should be some very good reason for using a mock — maybe the underlying function is non-deterministic or takes too long to execute. Using real objects improve the value of the tests by testing more interactions per test case.<br><br>Even then, there should be some tests that ensure our code is using that dependency in the correct way — like a test that ensures the output is within some expected range or matches. As an example below, we have a test that ensures our code works if <em>randint</em> returns a specific value and a test that ensures we’re calling <em>randint</em> correctly:</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/57bb857d1b3dad480e052cb45d2be03f/href">https://medium.com/media/57bb857d1b3dad480e052cb45d2be03f/href</a></iframe><ul><li><strong>Use fixture data</strong>. If the dependency being mocked is an external service, consider creating a common set of fakes or using a mock server to provide fixture data. Centralizing the implementation of the fake allows for careful emulation of the real implementation’s behavior and minimizes how much test code has to change if the underlying implementation changes.</li><li><strong>Don’t be afraid to leave some code uncovered!</strong> Given the choice of testing some code well and no tests the answer is clearly to test it well. Given the choice of a Tautology Test and no test it’s less clear. Hopefully I’ve convinced you that Tautology Tests are a net negative and leaving some code untested is a signal to future developers as to the current state of the world. They can chose to exercise caution when modifying that section of code or — preferably–use some of the techniques above to add proper tests.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3C6B6G2IXMNSzB8Y." /></figure><blockquote>It’s better to leave a line of code untested than to give the illusion that it is well tested.</blockquote><p>Keep an eye out for Tautology Tests during code review as well. Ask yourself what this test is actually testing and not just “does it cover some lines of code?”</p><p>Remember, Tautology Tests are bad because they are not good.</p><h3>Further Reading</h3><ul><li><a href="https://martinfowler.com/articles/mocksArentStubs.html">Mocks Aren’t Stubs</a></li><li><a href="https://testing.googleblog.com/2013/05/testing-on-toilet-dont-overuse-mocks.html">Don’t Overuse Mocks</a></li><li><a href="https://teamgaslight.com/blog/testing-behavior-vs-testing-implementation">Testing Behavior vs. Testing Implementation</a></li></ul><p><em>Thanks to Kent Beck, Simon Stewart, Ben Hamilton, and Josh Cincinnati for feedback!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7dabd81ade30" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Toasted Marshmallow — Marshmallow but 15X Faster.]]></title>
            <link>https://eng.lyft.com/toasted-marshmallow-marshmallow-but-15x-faster-17bdcf34c760?source=rss-83038993d868------2</link>
            <guid isPermaLink="false">https://medium.com/p/17bdcf34c760</guid>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Roy Williams]]></dc:creator>
            <pubDate>Wed, 09 Aug 2017 15:01:01 GMT</pubDate>
            <atom:updated>2022-10-24T23:07:13.492Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/290/1*Hi5qxR_vjl0Dtx-Q2MuPPA.png" /><figcaption>Lyft Loves Marshmallows!</figcaption></figure><p><a href="https://marshmallow.readthedocs.io/en/latest/">Marshmallow</a> is a fantastic ODM that we leverage extensively at Lyft. It has an extensible, easy-to-use API and is framework agnostic. We use it to validate incoming requests, to deserialize data from Mongo or Dynamo, and to serialize outgoing requests. Unfortunately, we ran into a sticky problem with Marshmallow. <a href="https://voidfiles.github.io/python-serialization-benchmark/">Benchmarks</a> suggest it’s one of the slowest serialization frameworks for Python. This put us in a tough spot…we love the powerful, easy-to-use API, but we wanted better performance than Marshmallow could offer. We huddled up and consulted with our trusted confidant, who had an incredible suggestion:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/0*byEkJLIcnMEooCC3." /></figure><p>To understand how both are possible, one must first understand how Marshmallow works. Marshmallow is slow because it’s effectively implementing an interpreter in an interpreter. When you serialize an object through Marshmallow it iterates through all the fields it knows about, then does a ton of reflection to extract the field from the input object. In practice, this is hugely wasteful. We are serializing the same object over and over again and it generally complies with the provided schema.</p><h3>Let’s Heat Things Up with Toasted Marshmallow</h3><p>With Toasted Marshmallow we decided to take an approach where we generate a serialization method at runtime that <strong>assumes</strong> the incoming object complies with the schema, falling back to the original Marshmallow code if it doesn’t.</p><p>As a concrete example, say we had the following Schema:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a5cf915a5df9d0fd81ce8a97b426c719/href">https://medium.com/media/a5cf915a5df9d0fd81ce8a97b426c719/href</a></iframe><p>The generated serialization method would be:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cd2fbb07ca1523f369ba89dc9b1e5602/href">https://medium.com/media/cd2fbb07ca1523f369ba89dc9b1e5602/href</a></iframe><p>Toasted Marshmallow will first use this method, and — should any exceptions get thrown or the object fail to parse — fall back to the original Marshmallow code. This approach is <strong>15–20x faster</strong> than reflection alone.</p><h3>Results, Hot Off The Press</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RIanWnzPlgUxiWjN." /></figure><p>In addition to reducing CPU consumption of our hosts using Marshmallow, rolling out Toasted Marshmallow resulted in nearly a 50% drop in our p95 response times!</p><h3>Getting Started</h3><p>Toasted Marshmallow is a drop in replacement for Marshmallow, there should be no difference in your code. Simply replace your dependency on marshmallow==3.0.0b2 with toastedmarshmallow==0.1.0 and set the environment variable MARSHMALLOW_SCHEMA_DEFAULT_JIT to toastedmarshmallow.Jit(you can also <a href="https://github.com/lyft/toasted-marshmallow#enabling-toasted-marshmallow">optimize individual schemas</a>). Toasted Marshmallow includes a <a href="https://github.com/marshmallow-code/marshmallow/pull/629/files">slightly modified version</a> of Marshmallow to add some necessary hooks for calling into the generated code first and regenerating the code when needed. We structured the code to minimize the modifications needed in Marshmallow making it easier to track upstream in the future.</p><p>We treat any differences as a bug, so if you have any code that worked with Marshmallow but fails with Toasted Marshmallow please <a href="https://github.com/lyft/toasted-marshmallow/issues">file an issue</a>.</p><p>Check out <a href="https://github.com/lyft/toasted-marshmallow">Toasted Marshmallow</a> today, and add s’more speed to your code!</p><blockquote>Interested in open source work while having a big impact on the future of transportation? Lyft is hiring! Apply <a href="https://www.lyft.com/careers">through our application system</a>, or drop me a note at <a href="/service/mailto:rwilliams@lyft.com">rwilliams@lyft.com</a>.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17bdcf34c760" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/toasted-marshmallow-marshmallow-but-15x-faster-17bdcf34c760">Toasted Marshmallow — Marshmallow but 15X Faster.</a> was originally published in <a href="https://eng.lyft.com/">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automated Style Review with Linty Fresh]]></title>
            <link>https://eng.lyft.com/automated-style-review-with-linty-fresh-17c6cd9d2c3b?source=rss-83038993d868------2</link>
            <guid isPermaLink="false">https://medium.com/p/17c6cd9d2c3b</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[lint]]></category>
            <category><![CDATA[code-review]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[engineering]]></category>
            <dc:creator><![CDATA[Roy Williams]]></dc:creator>
            <pubDate>Tue, 22 Dec 2015 23:33:42 GMT</pubDate>
            <atom:updated>2015-12-22T23:37:47.354Z</atom:updated>
            <content:encoded><![CDATA[<p>Lyft is happy to announce the open sourcing of <a href="https://github.com/lyft/linty_fresh">Linty Fresh</a>, a tool for integrating in Linters/Static Analysis tools into your code review process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/711/0*3-lvN1hHGKN9v4zj." /></figure><p>At Lyft we have a very fast moving, rapidly growing engineering team. We ship our mobile apps once a week and we use continuous deployment for our backend services. We want to ship high quality code as fast as possible to improve our service for our users. We also want to ensure that new teams members can quickly feel productive and ramped up on our code. Code review has been a fantastic tool for both needs.</p><p>Code review has been <a href="http://blog.codinghorror.com/code-reviews-just-do-it/">shown</a> to be one of the most effective ways to reduce the number of bugs that get to production, like dereferencing a null pointer. Perhaps more importantly, it’s also an incredibly effective way to <a href="http://research.microsoft.com/apps/pubs/default.aspx?id=180283">spread knowledge</a> around an organization and to ensure multiple people in an organization understand the code that’s being shipped to production.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FbLHL75H_VEM%3Ffeature%3Doembed&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DbLHL75H_VEM%26feature%3Dyoutu.be&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FbLHL75H_VEM%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/bd05245f3e1bff236f9928259c120d24/href">https://medium.com/media/bd05245f3e1bff236f9928259c120d24/href</a></iframe><p>Even though code review is immensely valuable, it’s also not free. Engineers have to take time out of their day from writing features to review their peers code, so we wanted to focus on making this as efficient as possible. In the Microsoft study cited above, the most common kind of comment was “Code Improvements”, generally around code style or formatting. Engineers didn’t find those comments to be highly useful and found them to be nit-picky at times. This can lead to inter-personal conflict between engineers over <a href="https://en.wikipedia.org/wiki/Parkinson%27s_law_of_triviality">bike shedding</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OZhe8TfRqdSN8BPh8E2Euw.jpeg" /><figcaption>That’s a great Bike Shed, but it should be Blue.</figcaption></figure><p>Coding style is still important! Engineers <strong>really</strong> care about their style. The tabs vs. spaces holy war has raged as long as there have been text editors. Having a uniform style increases <a href="http://synesthesiam.com/posts/what-makes-code-hard-to-understand.html">comprehension</a> of code, and engineers spend significantly more time reading code or modifying existing code than writing new code. Clean codebases are less likely to rot due to broken windows. In order to maximize the effectiveness of code review however, we want engineers focused on higher level design and architecture and not being human linters. Leave the linting to the robots and let humans focus on high level design and architecture questions without getting burnt out on style problems.</p><blockquote>Linty Fresh is a tool that allows us to offload the task of checking things that machines can check to CI, but brings that information back into the context of code review.</blockquote><p>Linty Fresh is a tool that allows us to offload the task of checking things that machines can check to CI, but brings that information back into the context of code review. Running lint in CI ensures that our code base remains clean. Bringing those comments into the context of a code review reduces the amount of time engineers have to peck through hard to read log files and gives other engineers the assurance that style will be checked by a machine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*k5vhpqHsU_quy_KF." /><figcaption>Digging through a 1000+ line log file to find lint errors. Not very fun!</figcaption></figure><p>We built Linty Fresh to be extremely easy to integrate. Lyft uses a combination of in house Jenkins with some third party CI providers for mobile builds and tests. We wanted integration to be as easy as `pipe` without having to set up any backend services. Solutions like SonarQube or Hound perform similar tasks, but require standing up separate services to handle reporting and tracking.</p><p>To give Linty Fresh a try, check out <a href="https://github.com/lyft/linty_fresh">https://github.com/lyft/linty_fresh</a> and keep your codebase clean with the power of <strong>LINT.</strong></p><p>Linty Fresh uses the <a href="https://github.com/lyft/linty_fresh/blob/master/LICENSE">Apache 2 License</a>.</p><blockquote>If you’re interested in building open source developer tools, Lyft is hiring! Shoot me a message on <a href="https://twitter.com/roywilliamsiii"><strong>Twitter</strong></a> or email me at <strong>rwilliams@lyft.com</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17c6cd9d2c3b" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/automated-style-review-with-linty-fresh-17c6cd9d2c3b">Automated Style Review with Linty Fresh</a> was originally published in <a href="https://eng.lyft.com/">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>