<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Roberto Vitillo on Medium]]></title>
        <description><![CDATA[Stories by Roberto Vitillo on Medium]]></description>
        <link>https://medium.com/@robertovitillo?source=rss-648bcffcbfa6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*QPK0j9N9w4-mawshTD5HVg.jpeg</url>
            <title>Stories by Roberto Vitillo on Medium</title>
            <link>https://medium.com/@robertovitillo?source=rss-648bcffcbfa6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 03:24:19 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@robertovitillo/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What every developer should know about database consistency]]></title>
            <link>https://medium.com/swlh/what-every-developer-should-know-about-database-consistency-cff3183913cb?source=rss-648bcffcbfa6------2</link>
            <guid isPermaLink="false">https://medium.com/p/cff3183913cb</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <dc:creator><![CDATA[Roberto Vitillo]]></dc:creator>
            <pubDate>Sun, 29 Nov 2020 08:07:59 GMT</pubDate>
            <atom:updated>2021-02-03T17:42:11.649Z</atom:updated>
            <content:encoded><![CDATA[<h3>What Every Developer Should Know About Database Consistency</h3><p>Imagine assigning some value to a variable, reading it back immediately after, and finding out that somehow the write had no effect at all — madness!</p><p>This is precisely what can happen when you are using a distributed data store with weak consistency guarantees. “But wait, aren’t databases supposed to take care of consistency issues for me?” I hear you ask. Whether an update becomes visible sooner rather than later depends on the guarantees offered by the database.</p><p>Some databases come with counter-intuitive consistency guarantees to provide high availability and performance. Others have knobs that allow you to chose whether you want better performance or stronger guarantees, like <a href="https://docs.microsoft.com/en-us/azure/cosmos-db/consistency-levels">Azure’s Cosmos DB</a> and <a href="https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/dml/dmlConfigConsistency.html">Cassandra</a>. Because of that, you need to know what the trade-offs are.</p><h3>Anatomy of a database request</h3><p>Let’s take a look at what happens when you send a request to a database. In an ideal world, your request executes instantaneously:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/0*Rdjr2i_2302WLGnU.png" /></figure><p>But we don’t live an ideal world — your request needs to reach the data store, which then needs to process the request and finally send back a response to you. All these actions take time and are not instantaneous:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/0*tFX-sWg1AwnIbPyD.png" /></figure><p>The best guarantee a database can provide is that the request executes somewhere between its invocation and completion time. You might think that this doesn’t look like a big deal — after all, it’s what you are used to when writing single-threaded applications — if you assign 1 to x and read its value right after, you expect to find 1 in there, assuming there is no other thread writing to the same variable. But, once you start dealing with data stores that replicate their state on multiple machines for high availability and scalability, all bets are off. To understand why that’s the case, we will explore the trade-offs a system designer has to make to implement reads in a simplified model of a distributed database.</p><p>Suppose we have a distributed key-value store, which is composed of a set of replicas. The replicas elect a leader among themselves, which is the only node that can accept writes. When the leader receives a write request, it broadcasts it asynchronously to the other replicas. Although all replicas receive the same updates in the same order, they do so at different times.</p><p>You are tasked to come up with a strategy to handle read requests — how would you go about it? Well, a read can potentially be served by the leader or a replica. If all reads were to go through the leader, the throughput would be limited by what a single node can handle. Alternatively, any replica could serve any read request — that would definitely scale, but then two clients, or observers, could have a different view of the system’s state, as replicas can lag behind the leader and between them.</p><p>Intuitively, there is a trade-off between how consistent the observers’ views of the system are, and the system’s performance and availability. To understand this relationship, we need to define precisely what we mean by consistency. We will do so with the help of <a href="https://jepsen.io/consistency">consistency models</a>, which formally define the possible views of the system’s state observers can experience.</p><h3>Strong Consistency</h3><p>If clients send writes and reads exclusively to the leader, then every request appears to take place atomically at a very specific point in time as if there was a single copy of the data. No matter how many replicas there are or how far behind they are lagging, as long as the clients always query the leader directly, from their point of view there is a single copy of data.</p><p>Because a request is not served instantaneously, and there is a single node serving it, the request executes somewhere between its invocation and completion time. Another way to think about it is that once a request completes, it’s side-effects are visible to all observers:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/0*1JJoCAWN91lm64vT.png" /></figure><p>Since a request becomes visible to all other participants between its invocation and completion time, there is a real-time guarantee that must be enforced — this guarantee is formalized by a consistency model called <a href="https://jepsen.io/consistency/models/linearizable">linearizability</a>, or strong consistency. Linearizability is the strongest consistency guarantee a system can provide for single-object requests.</p><p>What if the client sends a read request to the leader, but by the time the request gets there, the server that received the request thinks it’s still the leader, but it actually was deposed? If the ex-leader was to process the request, the system would no longer be strongly consistent. To guard against this case, the presumed leader first needs to contact a majority of the replicas to confirm whether it still is the leader. Only then is it allowed to execute the request and send back the response to the client. This considerably increases the time required to serve a read.</p><h3>Sequential Consistency</h3><p>So far, we have discussed serializing all reads through the leader. But doing so creates a single chokepoint, which limits the system’s throughput. On top of that, the leader needs to contact a majority of replicas to handle a read. To increase the read performance, we could allow the replicas to handle requests as well.</p><p>Even though a replica can lag behind the leader, it will always receive new updates in the same order as the leader. If a client A only ever queries replica 1, and client B only ever queries replica 2, the two clients will see the state evolving at different times, as replicas are not entirely in sync:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/0*57BQzW2AgG8pvVxO.png" /></figure><p>The consistency model in which operations occur in the same order for all observers, but doesn’t provide any real-time guarantee about when an operation’s side-effect becomes visible to the observers, is called <a href="https://jepsen.io/consistency/models/sequential">sequential consistency</a>. The lack of real-time guarantees is what differentiates sequential consistency with linearizability.</p><p>A simple application of this model is a producer/consumer system synchronized with a queue — a producer node writes items to the queue, which a consumer reads. The producer and the consumer see the items in the same order, but the consumer lags behind the producer.</p><h3>Eventual Consistency</h3><p>Although we managed to increase the read throughput, we had to pin clients to replicas — what if a replica goes down? We could increase the availability of the store by allowing a client to query any replica. But, this comes at a steep price in terms of consistency. Say there are two replicas 1 and 2, where replica 2 lags behind replica 1. If a client queries replica 1 and right after replica 2, it will see a state from the past, which can be very confusing. The only guarantee a client has is that eventually, all replicas will converge to the final state if the writes to the system stop. This consistency model is called eventual consistency.</p><p>It’s challenging to build applications on top of an eventually consistent data store because the behavior is different from the one you are used to when writing single-threaded applications. Subtle bugs can creep up that are hard to debug and to reproduce. Yet, in eventual consistency’s defense, not all applications require linearizability. You need to make the conscious choice whether the guarantees offered by your data store, or lack thereof, satisfy your application’s requirements. An eventually consistent store is perfectly fine if you want to keep track of the number of users visiting your website, as it doesn’t really matter if a read returns a number that is slightly out of date. But for a payment processor, you definitely want strong consistency.</p><h3>PACELC Theorem</h3><p>There are more <a href="https://jepsen.io/consistency">consistency models</a> than the ones presented in this post. Still, the main intuition behind them is the same: the stronger the consistency guarantees are, the higher the latency of individual operations is, and the less available the store becomes when failures happen. This relationship is formalized by the <a href="https://en.wikipedia.org/wiki/PACELC_theorem">PACELC theorem</a>. It states that in case of network partitioning (P) in a distributed computer system, one has to choose between availability (A) and consistency (C), but else (E), even when the system is running normally in the absence of partitions, one has to choose between latency (L) and consistency (C).</p><p>In my <a href="http://understandingdistributed.systems/">distributed systems book</a>, I explore other tunable trade-offs data stores make to guarantee high availability and performance, like the isolation guarantees that prevent a group of operations within a transaction from interfering with other concurrently running transactions.</p><p><em>Originally published at </em><a href="https://robertovitillo.com/what-every-developer-should-know-about-database-consistency/"><em>https://robertovitillo.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cff3183913cb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/what-every-developer-should-know-about-database-consistency-cff3183913cb">What every developer should know about database consistency</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Conduct a System Design Interview]]></title>
            <link>https://levelup.gitconnected.com/how-to-conduct-a-system-design-interview-9031a261c08e?source=rss-648bcffcbfa6------2</link>
            <guid isPermaLink="false">https://medium.com/p/9031a261c08e</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[interview]]></category>
            <dc:creator><![CDATA[Roberto Vitillo]]></dc:creator>
            <pubDate>Mon, 24 Aug 2020 11:53:08 GMT</pubDate>
            <atom:updated>2021-02-03T17:46:16.048Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to conduct a System Design Interview</h3><p>The system design interview is a great way to assess the seniority of a candidate in an interview. You can find a lot on the Internet on how to prepare for the design interview or which system design interview questions to expect, but very little on how to conduct an actual interview.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*WLD_A4bCxiJ3THP7JzCk3A@2x.jpeg" /></figure><p>In this post, I will describe my experience interviewing senior candidates for distributed systems roles. If you stumbled on this piece because you want to know how to ace the design interview, you should keep reading — a good way to prepare for the interview is to know how interviewers think about them.</p><p>A word of advice before I continue: this type of interview is best suited for interviewing engineers who have some experience working in the industry. Engineers straight out of school, or with few years of experience, don’t have the skills necessary to ace this interview yet.</p><h3>Pick a system you are familiar with</h3><p>A system design interview entails asking a candidate to design a system that solves a specific problem. The interview, which typically takes about an hour, is mostly an open-ended conversation — the goal is to get a sense of the candidate’s experience and problem-solving skills while working with them as they were your coworker.</p><p>You want to pick a system you worked on in the past, or at the very least, know well. Ideally, it should be something relevant to what your team or your company does so that the candidate gets the chance to see the kind of projects they will be working on if they join your team. <strong>Don’t forget that it’s not just you evaluating them — they are also evaluating you</strong>.</p><p>Another reason to pick a system you know well is that you have spent a considerable amount of time internalizing its solution space, scalability limitations, tradeoffs, and failure modes. It’s not only easier for you to prepare for the interview, but it also gives you the ability to compare their solution to what you, or your team, came up with. And who knows, they might even give you some insights that you missed — I call this a win-win.</p><p><strong>Whatever you do, don’t ask them how to design a Twitter clone</strong> (unless you work at Twitter), or any other typical design interview question. When you are in the hotel booking business and ask to design a Twitter clone during the interview, you are telling the candidate that there is nothing interesting going on where you work. It’s a big red flag for a senior engineer, and rightly so.</p><p>“But what if I don’t have a Twitter-scale problem?” — I hear you ask. That’s perfectly fine, you don’t need to have millions of users to reason about scalability — plus, it’s just one of the many challenges to solve when designing a distributed system.</p><h3>Start with a small scale design</h3><p>Spend the first minutes talking to the candidate to get a sense of their past experience, their strengths, and their weaknesses. Then, pick a design question you prepared before the interview and present it to the candidate. <strong>Draw a basic sketch</strong> of the system you want them to architect on the whiteboard — this will help make them feel more relaxed as empty whiteboards can make people nervous.</p><p>When presenting the system they should design, start with a small scale and make sure the candidate is on the same page — there will be time later to dig into scalability. Also, <strong>don’t hash out all the requirements in detail</strong>, as you want the candidate to ask questions to see whether they can get a clear picture of what they have been asked to design before jumping into the solution.</p><p>Finally, ask them how they would design the system. But don’t just sit silently in a corner, <strong>talk to the candidate, and try to understand why they are making the decisions they are</strong> and how they are approaching the problem. At this stage, there is no need to deep dive into a specific component of the design or technology — go broad and give the candidate space to hash out a complete architecture.</p><p>If you feel there are too many moving parts, ask whether there is a way to remove specific components while not affecting the overall solution. Also, ask the candidate to justify the tradeoffs they are making — for example, why did they pick a NoSQL store rather than SQL one?</p><p>By the end of the first 30 minutes, they should have a simple design sketched out on the whiteboard.</p><h3>Crank up the scale</h3><p>Now that the candidate has an initial design, crank up the scale. Increase the number of requests per second, the volume of the data ingested, or anything else relevant to your specific problem. <strong>No matter how the initial design looks like, it’s bound to hit a brick wall as you increase the load it’s under.</strong></p><p>Ask them what they think will break as the scale increases and how they would address it. It’s a great sign if the candidate can point out brick walls on their own. Once they have identified the breaking points, work with the candidate to eliminate them. Start with the one that has the highest likelihood of being hit first, as the others might become less relevant once the design evolves. This stage is very much iterative, and as the design changes, so do the tradeoffs — discuss what those are and whether they make sense.</p><p><strong>Point out failure modes they missed</strong> and ask them what would happen if this component, or that link, was to go down. You can tell a lot about a candidate’s experience by how quickly they can spot failure modes, like single points of failure, and come up with elegant ways to solve them. Because you are asking about a system you have experience with, you can just recollect what the most typical issues were in the past and how you mitigated them.</p><p>By the end of the first 50 minutes, you should have discussed at length scaling and reliability issues. If there is time for it, <strong>pick a slice of the design and drill down into it</strong>. For example, if the candidate’s design requires replicating data, talk about consistency guarantees, and how to enforce them in detail.</p><p>This is akin to the question: “tell me what happens when you type example.com in your browser,” but tailored to the design at hand. If you are hiring a senior systems engineer, you expect them to have a deep understanding of the basics, like TCP, paging, consistency models, etc. This knowledge is critical when debugging, and optimizing, complex systems.</p><p>By the end of the interview, you should have a good idea of the candidate’s strengths and weaknesses and how they approach problems that don’t have a single best answer. A senior candidate should be able to quickly go through the first part of the interview and have no issues scaling up the system and addressing failure modes in the second part. They should have a good explanation of every decision they took and be able to list the tradeoffs of their design. <strong>When interviewing strong candidates, you should leave the interview feeling like you learned something from them</strong>.</p><h3>Final words</h3><p>The system design interview is a great way to get a feeling of a senior’s candidate’s problem-solving skills. Connecting boxes with arrows is just one part of it, though — and not the most challenging one. The tricky part is understanding the requirements, failure modes, and tradeoffs. This type of interview also encourages lively discussions, which gives you a glimpse into the candidate’s soft skills. Can they explain things clearly? Do they receive constructive feedback well?</p><p>If you are interested to learn more about designing distributed systems, not just in the context of interviews, check out <a href="https://understandingdistributed.systems/">my book</a>. It talks at length about network fundamentals, the theory of distributed systems, architectural patterns of scalable systems, resiliency patterns, and operational best practices on how to maintain large-scale systems.</p><p><em>Originally published at </em><a href="https://robertovitillo.net/how-to-conduct-a-system-design-interview/"><em>https://robertovitillo.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9031a261c08e" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/how-to-conduct-a-system-design-interview-9031a261c08e">How to Conduct a System Design Interview</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Back of the envelope estimation hacks]]></title>
            <link>https://levelup.gitconnected.com/back-of-the-envelope-estimation-hacks-4de943cb939d?source=rss-648bcffcbfa6------2</link>
            <guid isPermaLink="false">https://medium.com/p/4de943cb939d</guid>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Roberto Vitillo]]></dc:creator>
            <pubDate>Sat, 01 Aug 2020 10:03:12 GMT</pubDate>
            <atom:updated>2021-02-03T17:45:08.926Z</atom:updated>
            <content:encoded><![CDATA[<p>There are two types of engineers, the ones that can quickly do estimates and the ones that can’t. Are these people just smarter, or is there more to it?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/0*ylH3YcvxebW6sWsU.jpg" /><figcaption><a href="https://en.wikipedia.org/wiki/Fermi_problem"><em>Enrico Fermi</em></a><em> was a master at estimation</em></figcaption></figure><p>Back of the envelope estimation is a skill that can be learned with practice. It becomes easier once you get familiar with some tricks of the trade.</p><h3>Know your numbers</h3><p>How fast can you read data from the disk? How quickly from the network? You should be familiar with ballpark performance figures of your components. What’s important are not the exact numbers per se but their relative differences in terms of orders of magnitude.</p><h3>Approximate with powers</h3><p>The goal of estimation is not to get a correct answer, but to get one in the right ballpark. Using powers of 2 or 10 makes multiplications very easy as in logarithmic space, multiplications become additions.</p><p>For example, let’s say you have 800K users watching videos in UHD at 12 Mbps. A machine in your Content Delivery Network can egress at 1 Gbps. How many machines do you need?</p><p>Approximating 800K with 10⁶, and 12 Mbps with 10⁷ Mbps yields:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/349/1*-vstTE3GGejGOh0Wug-rMg.png" /></figure><p>Summing 6 to 7 and subtracting 9 from it is much easier than trying to get to the exact answer:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/216/1*lwmSaz6GauI0YVj_oU0n3g.png" /></figure><p>Does the difference really matter? Not really; the precise answer is not exact anyway. Users vary over time, the network bandwidth is not constant, etc.</p><h3>The Rule of 72</h3><p><a href="https://web.stanford.edu/class/ee204/TheRuleof72.html">The rule of 72</a> is a method to estimate how long it will take for a quantity to double if it grows at a certain percentage rate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/112/1*GyXPDg-5QVmFeqCUEPpkzg.png" /></figure><p>For example, let’s say the traffic to your web service is increasing by approximately 10% weekly, how long will it take to double?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/149/1*mR1mzVsC165TYZc_W5wS9Q.png" /></figure><p>It will take approximately 7 weeks for the traffic to double.</p><p>If you combine the rule of 72 with the powers of two, then you can quickly find out how long it will take for a quantity to increase by several orders of magnitudes.</p><h3>Little’s Law</h3><p>A queue can be modeled with three parameters:</p><ul><li>the average rate at which new items arrive, <em>λ</em>,</li><li>the average time an item spends in the queue, <em>W </em>,</li><li>and the average number of items in the queue, <em>L</em>.</li></ul><p>Lots of things can be modeled as queues. A web service can be seen as a queue, for example. The request rate is the rate at which new items arrive. The time it takes for a request to be processed is the time an item spends in the queue. Finally, the number of concurrently processed requests is the number of items in the queue.</p><p>Wouldn’t it be great if you could derive one of the three parameters from the other two? It turns out there is a law that relates these three quantities to one another! It’s called <a href="https://en.wikipedia.org/wiki/Little%27s_law">Little’s Law</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/95/1*mcLdWUHUgVBWvbesmbLefQ.png" /></figure><p>What it says is that the average number of items in the queue equals the average rate at which new items arrive, multiplied by the average time an item spends in the queue.</p><p>Let’s try it out. Let’s say you have a service that takes on average 100 ms to process a request. It’s currently receiving about two million requests per second (rps). How many requests are being processed concurrently?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/301/1*JSLrcsY96IlCe2e2E5gsOg.png" /></figure><p>The service is processing 200K requests concurrently. If each request is CPU heavy and requires a thread, then we will need about 200K threads. If we are using 8 core machines, then to keep up, we will need about 25K machines.</p><h3>Remember this</h3><p>Estimation is a vital skill for an engineer. It’s something you can get better at by practicing and using the hacks I have presented in this post:</p><ul><li>Get familiar with your components’ performance numbers.</li><li>Approximate with powers of 2 or 10 to transform multiplications into additions.</li><li>Use the rule of 72 to find out how long it takes for a quantity to double given its growth rate.</li><li>Model your systems as queues and leverage Little’s Law.</li></ul><p>Check out my <a href="http://understandingdistributed.systems/">distributed systems book</a> if you want to learn more about designing large scale systems.</p><p><em>Originally published at </em><a href="https://robertovitillo.com/back-of-the-envelope-estimation-hacks/"><em>https://robertovitillo.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4de943cb939d" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/back-of-the-envelope-estimation-hacks-4de943cb939d">Back of the envelope estimation hacks</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Every Developer Should Know About TCP]]></title>
            <link>https://medium.com/swlh/what-every-developer-should-know-about-tcp-755ece34f13b?source=rss-648bcffcbfa6------2</link>
            <guid isPermaLink="false">https://medium.com/p/755ece34f13b</guid>
            <category><![CDATA[development]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[networking]]></category>
            <dc:creator><![CDATA[Roberto Vitillo]]></dc:creator>
            <pubDate>Thu, 23 Jul 2020 10:23:10 GMT</pubDate>
            <atom:updated>2021-02-03T17:44:06.720Z</atom:updated>
            <content:encoded><![CDATA[<h3>What every developer should know about TCP</h3><p>Why do you need to place your servers geographically close to your users? One of the reasons is to achieve lower latencies. That makes a lot of sense when you are sending short bursts of data that should be delivered as quickly as possible. But what about large files, such as videos? Surely there is a latency penalty for receiving the first byte, but shouldn’t it be smooth sailing after that?</p><p>A common misconception when sending data over TCP, like with HTTP, is that bandwidth is independent of latency. But, with TCP, bandwidth is a function of latency and time. Let’s see how.</p><h3>Handshake</h3><p>Before a client can start sending data to a server, it needs to perform a handshake for TCP and another one for TLS.</p><p>TCP uses a three-way handshake to create a new connection.</p><ol><li>The sender picks a randomly generated sequence number “x” and sends a SYN packet to the receiver.</li><li>The receiver increments “x,” chooses a randomly generated sequence number “y” and sends back a SYN/ACK packet.</li><li>The sender increments both sequence numbers and replies with an ACK packet and the first bytes of application data.</li></ol><p>The sequence numbers are used by TCP to ensure the data is delivered in order and without holes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/0*cIRg40Fh_yf0v5vv.png" /></figure><p>The handshake introduces a full round-trip, which depends on the latency of the underlying network. The <a href="https://www.cloudflare.com/learning/ssl/what-happens-in-a-tls-handshake/">TLS handshake</a> also requires up to two round trips. Until the TLS connection is open, no application data can be sent, which means your bandwidth is for all intent and purposes zero until then. The lower the round trip time is, the faster the connection can be established.</p><h3>Flow Control</h3><p>Flow control is a backoff mechanism implemented to prevent the sender from overwhelming the receiver.</p><p>The receiver stores incoming TCP packets waiting to be processed by the application into a receive buffer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/0*__7-aCYvUEJhKyBL.png" /></figure><p>The receiver also communicates back to the sender the size of the buffer whenever it acknowledges a packet. The sender, if it’s respecting the protocol, avoids sending more data that can fit in the receiver’s buffer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/0*OYzxC99fkUwI0FSu.png" /></figure><p>This mechanism is not too dissimilar to <a href="https://www.youtube.com/watch?v=epM2jxi3Yi0">rate-limiting</a> at the application level. But, rather than rate limiting on an API key or IP address, TCP is rate-limiting on a connection level.</p><p>The lower the round-trip time (RTT) between the sender and the receiver is, the faster the sender adapts its outgoing bandwidth to the receiver’s capacity.</p><h3>Congestion Control</h3><p>TCP not only guards against overwhelming the receiver but also against flooding the underlying network.</p><p>How can the sender find out what the available bandwidth of the underlying network is? The only way to estimate it is empirically through measurements.</p><p>The idea is that the sender maintains a so-called “congestion window.” The window represents the total number of outstanding packets that can be sent without waiting for an acknowledgment from the other side. The size of the receiver window limits the maximum size of the congestion window. The smaller the congestion window is, the fewer bytes can be in flight at any given time, and the less bandwidth is utilized.</p><p>When a new connection is established, the size of the congestion window is set to a system default. Then, for every packet acknowledged, the window increases its size exponentially. This means that we can’t use the full capacity of the network right after a connection is established. Once again, the lower the round trip time is, the quicker the sender can start utilizing the underlying network’s bandwidth.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/0*H5VOn3mpo_kjTvYN.png" /></figure><p>What happens if a packet lost? When the sender detects a missed acknowledgment through a timeout, a mechanism called “congestion avoidance” kicks in, and the congestion window size is reduced. From there onwards, <a href="https://en.wikipedia.org/wiki/CUBIC_TCP">time increases the window size</a> by a certain amount, and timeouts decrease it by another.</p><p>As mentioned, the size of the congestion window defines the maximum number of bits that can be sent without waiting for an acknowledgment. The sender needs to wait for a full round trip to get an acknowledgment. So, by dividing the size of the congestion window by the round trip time, we get the maximum theoretical bandwidth:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/336/1*ulYquDblBgIQI4IYnjjvWg.png" /></figure><p>This simple <a href="https://en.m.wikipedia.org/wiki/Bandwidth-delay_product">equation</a> shows that bandwidth is a function of latency. TCP will try very hard to optimize the window size since it can’t do anything about the round trip time. But, that doesn’t always yield the optimal configuration.</p><p>To summarize, congestion control is an adaptive mechanism used to infer the underlying bandwidth and congestion of the network. A similar pattern can be applied at the application level as well. Think of what happens when you watch a movie on Netflix. It starts blurry; then, it stabilizes to something reasonable until there is a hiccup, and the quality worsens again. This mechanism applied to video streaming is called <a href="https://en.wikipedia.org/wiki/Adaptive_bitrate_streaming">adaptive bitrate streaming</a>.</p><h3>Remember this</h3><p>If you are using HTTP, then you are at the mercy of the underlying protocols. You can’t get the best performance if you don’t know how the sausage is made.</p><p>Bursty requests suffer from the cold start penalty. Multiple round trips can be required to send the first byte with TCP and TLS handshakes. And due to the way congestion control works, the lower the round trip time is, the better the underlying network’s bandwidth is utilized.</p><p>Entire <a href="https://www.amazon.co.uk/TCP-Illustrated-Protocols-Addison-Wesley-Professional-ebook/dp/B00666M52S/">books</a> have been written on the subject, and there is a lot you can do to squeeze every ounce of bandwidth. But if you have to remember one thing about TCP then let it be this:</p><p>You can’t send data faster than the speed of light, but what you can do is put your servers closer to the clients and re-use connections to avoid the cold start penalty.</p><p>If you want to learn more about how the network affects distributed systems, check out my <a href="http://understandingdistributed.systems/">book</a>.</p><p><em>Originally published at </em><a href="https://robertovitillo.com/what-every-developer-should-know-about-tcp/"><em>https://robertovitillo.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=755ece34f13b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/what-every-developer-should-know-about-tcp-755ece34f13b">What Every Developer Should Know About TCP</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>