<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Samir Kattel on Medium]]></title>
        <description><![CDATA[Stories by Samir Kattel on Medium]]></description>
        <link>https://medium.com/@probablysamir?source=rss-2b9c9c442e1b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ryWKJKRd_jtFRk2KxuQHqA.jpeg</url>
            <title>Stories by Samir Kattel on Medium</title>
            <link>https://medium.com/@probablysamir?source=rss-2b9c9c442e1b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 05:04:24 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@probablysamir/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Order flow remembers. Prices don’t.]]></title>
            <link>https://medium.com/@probablysamir/order-flow-remembers-prices-dont-bc69beecc3fe?source=rss-2b9c9c442e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc69beecc3fe</guid>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Samir Kattel]]></dc:creator>
            <pubDate>Wed, 11 Nov 2026 07:11:03 GMT</pubDate>
            <atom:updated>2026-11-11T07:20:26.907Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Checking a 2025 econophysics paper against a week of NASDAQ data</em></p><p>You can predict the next trade on NASDAQ. Not the price, the trade. If the last market order in a stock was a buy, the next one is a buy far more often than a coin flip would allow.</p><p>So why can’t you predict the price?</p><p>That question has bugged market microstructure people for twenty years, and a paper from February 2025 (revised in November 2026) claims to settle it. I wanted to see whether its predictions survive contact with real US data, so I rebuilt every NASDAQ order book for a full week and checked. Three things held. One didn’t, and the reason it didn’t is more interesting than the result.</p><h3>The puzzle</h3><p>Big funds can’t buy a million shares at once. It would eat through the whole order book and they’d pay a terrible average price. So they slice the order into thousands of small ones and drip them into the market over hours or days. In the literature the big order is called a metaorder and the slices are child orders.</p><p>Every child order of a buy metaorder is a buy. So if you look at the stream of market order signs (+1 for a buy, −1 for a sell), you see long streaks. The autocorrelation of the signs decays very slowly, like a power law:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/1*9LFte91JS5lfjb_VabC_xg.png" /></figure><p>That’s not a small effect. With γ below 1 the autocorrelations don’t even add up to a finite number. The order flow has genuinely long memory.</p><p>Now the problem. Every buy pushes the price up a bit. If buys keep following buys, the price should keep drifting up, and you should be able to make money just by following the flow. But prices don’t do that. Returns are basically uncorrelated and the variance of price changes grows linearly with time, which is the textbook definition of a random walk.</p><p>Predictable flow, unpredictable price. Something has to give.</p><h3>What the paper says</h3><p>The paper is Sato, Fujiwara and Kanazawa, <em>Why financial prices exhibit Brownian motion despite predictable order flow</em> (arXiv:2502.17906). The key ingredient is something that’s been measured for years: the square-root law of price impact. Buy Q shares and the price moves by roughly √Q, not Q. Double the order and the impact goes up about 1.4×, not 2×.</p><p>The older explanation of the puzzle needed the impact of each trade to fade away at exactly the right speed to cancel the memory in the flow. It works, but it needs a suspiciously precise coincidence.</p><p>The authors take the standard order-splitting model (Lillo, Mike and Farmer), let impact grow as Qᵟ instead of linearly, and solve it exactly. The answer is a clean rule:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/352/1*mBkAGcQPOGRDHU_zOhLJ3w.png" /></figure><p>where α describes how heavy-tailed metaorder sizes are, somewhere between 1 and 2. The same model ties α to the sign memory: γ = α − 1.</p><p>Plug in square-root impact, δ = 1/2, and 2δ = 1. Since α is always above 1, the condition never holds. Prices diffuse no matter how much memory the flow has. Concave impact means even a giant metaorder only moves the price a bounded amount, every metaorder’s impact has finite variance, and the central limit theorem does the rest.</p><p>It also predicts that returns only have short memory, decaying like τ⁻ᶿ with θ = α − 2δ + 1. Which gave me an idea: if I can measure γ and θ from public data, I can solve for δ without ever seeing who traded. More on how that went later.</p><h3>Why NASDAQ ITCH</h3><p>The paper’s evidence comes from the Tokyo Stock Exchange, using a research dataset where trades are tagged by account. I don’t have that. What I do have is free: NASDAQ publishes full TotalView-ITCH 5.0 days as sample files. Every add, cancel, replace and execution, timestamped to the nanosecond.</p><p>ITCH has one big advantage over the usual trade-and-quote data. When an order executes, the feed tells you which resting order got hit, and that order’s side. So you know the aggressor’s side exactly. With trade-and-quote data you have to guess it from where the price printed relative to the quote.</p><p>I wrote the pipeline in Go. It replays the feed, rebuilds every stock’s book from the add, replace, delete, cancel and execute messages, and writes one record per market order: timestamp, stock, sign, shares, and the best bid and ask just before it hit.</p><p>One detail matters more than it sounds. A single aggressive order that sweeps several resting orders shows up as several executions with the same timestamp. On 9 December at 12:30:25.377728978, one TSLA buy filled 1,435 resting sell orders at exactly $450.00. Same nanosecond, 41,180 shares. Count those as 1,435 buys and your sign memory is mostly measuring your own bookkeeping. So fills with the same stock, same nanosecond and same side get merged into one market order.</p><p>I skipped hidden executions and cross prints and kept only executions against displayed orders, where the resting side is known. Over 8 to 12 December 2025 that’s about 43.6 million market orders. The book rebuild never once hit an order it didn’t know about, which is a nice sanity check that the replay is right.</p><p>For the analysis I kept regular hours only, worked in trade time (τ counts market orders, not seconds), and used the 180 stocks with at least 5,000 market orders on every one of the five days.</p><h3>Result 1: order flow remembers</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iD8jEBUr7b_onwXXGPJOQQ.png" /></figure><p>The black line is the sign autocorrelation averaged across the 180 stocks. At lag 1 it’s 0.38. At lag 10 it’s 0.032, at lag 100 it’s 0.009, and it’s still clearly above the noise floor at 1,000 market orders.</p><p>And it’s not a few stocks dragging the average. At lag 100 the sign autocorrelation is positive in 89% of them.</p><h3>Result 2: prices don’t</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*233JrI6L1737xtMfGDrt4w.png" /></figure><p>For a random walk, variance divided by the lag is flat. That’s the whole test. Across five different fit windows the median exponent of variance against lag comes out between 0.98 and 1.01. It’s a random walk.</p><p>The red squares in the first chart are the return autocorrelations. At lag 10 it’s 0.002, about 16 times smaller than the sign memory at the same lag. It’s small but real at short lags (with 180 stocks the standard error is about 0.0006), and beyond roughly 20 market orders it sinks into the noise and flips sign at random.</p><p>Here’s a number that makes the paper’s point better than any sentence. From the memory tail I measured, α comes out around 1.36. If impact were linear (δ = 1), the model says variance would grow much faster than the lag. Prices would trend, hard. Here’s that against what I measured:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*v3-1D4xUNUtOOFJXNYCb7g.png" /></figure><h3>Where it broke, part 1: γ isn’t one number</h3><p>The paper’s prediction is a single power law. The data has two regimes.</p><p>Over the first ten or so market orders the memory falls off fast. Fit only short lags and you get γ around 0.96. After that it settles into a much slower tail, and that’s the long memory the paper is about: the per-stock median over lags 10 to 1000 is γ ≈ 0.36.</p><p>I wasn’t sure at first whether that bend was real or just noise at long lags, so I watched how the tail estimate moved as I added days. One day gave 0.23. Two days, 0.24. Four days, 0.32. Five days, 0.36. Noise doesn’t converge like that. The fast part at the start is probably sweeps and immediate reactions that no metaorder model is trying to describe.</p><p>One thing to be careful with if you quote this: the dashed line in the chart is fitted to the average curve across stocks, which gives 0.47. The median of the per-stock fits is 0.36. The average curve gets pulled up by the stocks with the strongest memory. Both are real numbers, they just answer different questions.</p><h3>Where it broke, part 2: you can’t back out δ</h3><p>This is the part I find most interesting, because it’s the part that didn’t work.</p><p>The plan was simple. Measure γ from the signs, measure θ from the returns, and solve for δ:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/1*lNt6zTwgWqj66iB-AsykrA.png" /></figure><p>The square-root law, recovered from anonymous data.</p><p>It gave δ between 1.0 and 1.2 depending on the sample. That would mean impact is linear or worse, which contradicts the very prices I’d just shown are diffusive. The problem is θ. To fit a power-law decay you need the curve to stay measurably above zero for a few decades of lag, and the return autocorrelation is gone after about 20 market orders. The fit is just reading noise.</p><p>Would more data fix it? A bit. Going from one day to five shrank the spread of the return autocorrelation across stocks from 0.021 to 0.0075 at lag 10. But the signal itself is around 0.002 and dies fast, so you’d need a lot more than five days to pin a decay exponent on it, probably more than any public sample gives you. The paper’s own conclusion is the reason: square-root impact is exactly what keeps price memory short. The mechanism that makes the theory work is the same one that hides the evidence.</p><h3>Why the US is harder than Tokyo</h3><p>There’s another reason my numbers look weaker than the Tokyo ones, and it has nothing to do with statistics.</p><p>Tokyo trading happens mostly on one exchange. US trading is split across more than a dozen exchanges plus dark pools and wholesalers. A fund working a metaorder in TSLA sends child orders to many of them. NASDAQ only sees its slice.</p><p>So the sign series I measured is a thinned version of the real flow. A run of twenty buys might show up on NASDAQ as six, interleaved with sells from other people that happened to land there. That dilutes the memory. It’s honestly surprising how much survives.</p><p>Side finding along the same lines: ETFs forget much faster than single stocks. SPY and QQQ have a lag-1 sign autocorrelation around 0.19, against 0.46 for TSLA, and it’s close to zero by about 14 orders. I don’t have a clean explanation for that yet.</p><h3>What this doesn’t show</h3><ul><li>I never measured the square-root law itself. That needs account IDs to link child orders to their metaorder, and ITCH is anonymous. I tested its consequences, not the law.</li><li>Five days. Per-stock tails beyond about 100 lags are close to the noise floor.</li><li>One venue, for the reason above.</li></ul><h3>So</h3><p>The paper’s main claim holds on a week of NASDAQ: the flow is predictable, the price isn’t, and the gap is huge. The single power law doesn’t, quite. And the neat trick of recovering the impact exponent from public data doesn’t work at all, for a reason the paper itself explains.</p><p>You can predict the next trade. The market already priced it in.</p><p>Code, data instructions and every number in this post: <a href="https://github.com/probablysamir/market-notes/tree/v1-sato/projects/2026%E2%80%9311-sato-replication">https://github.com/probablysamir/market-notes/tree/v1-sato/projects/2026–11-sato-replication</a> . The ITCH sample files are on NASDAQ’s site if you want to run it yourself.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc69beecc3fe" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Parsing NASDAQ ITCH on an FPGA]]></title>
            <link>https://medium.com/@probablysamir/parsing-nasdaq-itch-on-an-fpga-421dac8787ed?source=rss-2b9c9c442e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/421dac8787ed</guid>
            <category><![CDATA[system-verilog]]></category>
            <category><![CDATA[algorithmic-trading]]></category>
            <category><![CDATA[fpga]]></category>
            <category><![CDATA[hft]]></category>
            <category><![CDATA[hardware]]></category>
            <dc:creator><![CDATA[Samir Kattel]]></dc:creator>
            <pubDate>Fri, 07 Aug 2026 18:31:58 GMT</pubDate>
            <atom:updated>2026-08-07T18:31:58.306Z</atom:updated>
            <content:encoded><![CDATA[<p>Your computer is too slow to read the stock market. No, I don’t mean the internet. It’s your computer itself.</p><p>By the time you finished that sentence, NASDAQ announced a few million things.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*DR19MEyyR4w6bNQl.gif" /><figcaption>Ticker data coming in too quick</figcaption></figure><p>Before diving further let me simplify the jargons in the title for you.</p><p>For simplicity:</p><p><strong>NASDAQ: </strong>The first fully electronic American exchange</p><p><strong>ITCH: </strong>The format NASDAQ uses to “broadcast” order book changes. More on this later</p><p><strong>FPGA: </strong>Reconfigurable hardware that sits where your network card would. The feed lands directly on the chip and is decoded as the bytes arrive, so nothing ever travels into memory and up through the operating system to be understood.</p><h3>Introduction</h3><p>Market data feeds are one of the few places where a few hundred nanoseconds decide whether your trade wins or loses.</p><p>If you’re working on a market-making strategy, especially in HFT, you can buy the fastest machine on the market and still lose, because the delay happens before the computing even starts. A packet crosses PCIe, gets copied to memory, climbs the kernel’s network stack and only then reaches your code. Did you notice that we haven’t read a single byte yet?</p><p>A software parser sitting behind the kernel network stack is working in microseconds. Every one of those microseconds is a trade you didn’t get.</p><p>Also, the software being slower is not the only issue. It’s “unpredictably” slower. CPU latency drifts by microseconds in different trades. Sometimes a page fault, or a scheduler preemption, or sometimes a cache miss at the wrong moment or all of them at once. These are beyond your control. You can build around a delay you can count on. You can’t build around one you can’t.</p><h3><strong>ITCH is for listening, not for trading</strong></h3><p>Let’s talk about ITCH first before heading into implementation. ITCH is strictly an “outbound” protocol. It’s used by NASDAQ to broadcast market activity to you. You cannot place an order using the same protocol.</p><p>For sending orders, there’s a separate inbound path: OUCH which is NASDAQ’s native low-latency execution protocol, or FIX for more general use case (FIX is supported by multiple other exchanges too).</p><p>You listen on ITCH to maintain an exact real time mirror of the order book and send on OUCH or FIX when the algorithm finds something worth acting on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/1*07ix-6txadqsV1P1ovrrEg.png" /><figcaption>Fig: ITCH for market updates and OUCH or FIX for order entry</figcaption></figure><p>One thing to note is: ITCH sends changes and not snapshots i.e. there’s no periodic “here’s the order book” message. You need to build the book yourself and if the packet drops you’re on your own.</p><h3>The wire format</h3><p>ITCH is a binary protocol and it has a very simple and fixed format. Every message type has a fixed length, fixed offsets and every multi-byte integer is big endian. Now, this is the perfect shape for hardware because “fixed offset” means a decoder is just wires.</p><p>The parser handles three message types:</p><pre>Type  Message                           Length<br>----  --------------------------------  --------<br> A    Add Order (No MPID Attribution)    36 bytes<br> E    Order Executed                     31 bytes<br> X    Order Cancel                       23 bytes</pre><p>These three are the useful minimum for maintaining an order book: Place, execution and cancellation.</p><p>The convenient part is that the first 19 bytes of these message types are identical across all three:</p><pre>Offset  Size  Field<br>------  ----  -----------------------------<br>   0      1   Message Type<br>   1      2   Stock Locate<br>   3      2   Tracking Number<br>   5      6   Timestamp (ns since midnight)<br>  11      8   Order Reference Number</pre><p>Two of those fields are doing real work.</p><p><strong>Stock Locate</strong> is a numeric ID assigned to the symbol per session. NASDAQ publishes the ID for the symbol once at the start of the day and after that every message identifies the stock by number.</p><p><strong>Order Reference Number </strong>is a unique number assigned to every order when it’s added. Every later message about that order refers to it by that unique number.</p><p>(Tracking Number is internal NASDAQ bookkeeping. You can ignore it)</p><p>The first 19 bytes are always the same size. The rest differs:</p><pre>&#39;A&#39; Add Order (36 bytes)<br><br>Offset  Size  Field<br>------  ----  -----------------------------<br>  19     1    Buy/Sell Indicator (&#39;B&#39; or &#39;S&#39;)<br>  20     4    Shares<br>  24     8    Stock (ASCII, right-padded with spaces)<br>  32     4    Price (4 implied decimal places)</pre><pre>&#39;E&#39; Order Executed (31 bytes)<br><br>Offset  Size  Field<br>------  ----  -----------------------------<br>  19    4     Executed Shares<br>  23    8     Match Number</pre><pre>&#39;X&#39; Order Cancel (23 bytes)<br><br>Offset  Size  Field<br>------  ----  -----------------------------<br>  19    4     Cancelled Shares</pre><p>Add Order has lot of fields required to fulfill the order, Order executions have lesser detail to provide and order cancellations just need to let know the amount of cancelled shares.</p><p>Order executions and cancellations do not tell you about the symbol, side, or price. They don’t need to because it was already broadcasted when the order was added. Exchange sends it and assumes you kept it safely.</p><p>That’s the risk. Miss one add order and information about order updates on that order is unusable.</p><p>And lastly, price is a fixed point integer with four implied decimals. 100000 means $10.0000, 12356432 means $1235.6432. The lack of floating point in the protocol is on purpose.</p><h3>The design</h3><p><em>Note: This is a simplified design built to understand the protocol, not to survive a live feed. It processes one byte, one message at a time. The limitations section at the end talks more about it.</em></p><p>The parser accepts one byte each time byte_valid is asserted, at most one per clock cycle, and produces the decoded fields in parallel. There are four states with these transitions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*-qFSHkB1TAO3ylSIYxW2YA.png" /><figcaption>Fig: state diagram for byte-serial ITCH parser</figcaption></figure><pre>STATE      CONDITION                    NEXT STATE<br>-------    -------------------------    ----------<br>IDLE       byte_valid, known type       CAPTURE<br>IDLE       byte_valid, unknown type     ERROR<br>CAPTURE    byte_valid, more to go       CAPTURE<br>CAPTURE    last byte captured           DECODE<br>DECODE     always, msg_valid pulses     IDLE<br>ERROR      byte_valid, byte dropped     IDLE</pre><p>IDLE reads the first byte and immediately knows how long the message will be since the first byte gives information about the message type and every message type has a distinct fixed length. For example, A always means 36 bytes, E means 31 and X means 23.</p><p>In SystemVerilog that’s a case statement:</p><pre>function automatic logic [5:0] len_for_type(input logic [7:0] t);<br>   case (t)<br>   &quot;A&quot;: len_for_type = 6&#39;d36;<br>   &quot;E&quot;: len_for_type = 6&#39;d31;<br>   &quot;X&quot;: len_for_type = 6&#39;d23;<br>   default: len_for_type = 6&#39;d0; // unrecognized<br>   endcase<br>endfunction</pre><p>A length of zero means the byte was garbage, and the FSM (Finite State Machine) heads to ERROR instead of CAPTURE.</p><p><strong>What does that lookup cost?</strong></p><p>In a program, it costs time. The comparison needs to be done, and needs to be branched to different cases.</p><p>But in hardware, it costs nothing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/509/0*8Uzqhk7lL-wRPkQO" /></figure><p>Yep, free because it isn’t executed, it’s wired. Picture eight wires carrying the type byte in and there are six wires carrying the length out and a fixed pattern of gates between them. Every possible input already has its output physically connected.</p><p>That’s what combinational means: no memory, no clock and output is mapped to the input. There is no separate step to fetch the length, store it and decide what to do with it. It all happens on one clock edge.</p><h4><strong>CAPTURE: fill the buffer</strong></h4><p>CAPTURE does the boring and repetitive part. Every time byte_valid goes high, the byte is written into a buffer and the index moves forward. When the index reaches the length that IDLE looked up, the message is complete and the FSM moves to DECODE.</p><p>The buffer is 36 bytes because that’s the largest message parser can handle i.e. an A (add order). An E uses 31 and an X only 23.</p><p><strong>Why is the size adjusted for largest message type?</strong></p><p>It’s because in hardware we can’t allocate memory when we need it. The registers are physically on the chip whether a message fills them or not. We need to size it for the worst case (the longest message type) and the extra bytes goes unused.</p><h4>DECODE: slicing the buffer</h4><p>DECODE cuts the buffer into the output fields:</p><pre>// Six bytes become one 48-bit number. No shifting, no multiplying, no loop.<br>timestamp &lt;= {frame[5], frame[6], frame[7], frame[8], frame[9], frame[10]};</pre><p><strong>Why does that matter?</strong></p><p>A number that is too big for one byte gets split across several bytes. There are two ways to order the pieces either biggest first or smallest first. ITCH puts the biggest piece first. x86 stores the smallest piece first.</p><p>A price of $123 goes across the wire as <em>00 12 C4 B0.</em><strong><em> </em></strong>Read those four bytes on x86 without reversing them and you get $296,563.9680 instead. That’s why software calls a byteswap on every multi-byte field.</p><p>Hardware does not reverse anything because it never reads them in sequence. All four bytes sit in the buffer at once and you wire each to the place it belongs.</p><p>The rest of the fields work the same way. stock_locate is two bytes, order_ref is eight, price is four. Fixed offsets in and fixed wires out.</p><p>When the slicing is done, msg_valid pulses high for one cycle. That pulse is the parser saying that the message is ready. The outputs then hold their values until the next message replaces them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ddPgYXvItpLHVuZlFKnJRw.png" /><figcaption>The last five bytes of an A (Add Order) going in, price coming out as 100000 ($10.0000). byte_valid alternates because the testbench paces itself, not because the parser has to.</figcaption></figure><h4>ERROR: when the byte makes no sense</h4><p>Not every byte that arrives is a valid message byte. If you feed the parser something else and when IDLE looks it up and if it gets a length of zero then it goes to ERROR instead of CAPTURE.</p><p>ERROR raises frame_error, waits for the next byte then throws it away and goes back to IDLE. The byte after that gets treated as a fresh message type.</p><p>Is that a good way to recover?<br><br>Not really. It’s a guess. The parser has no idea where the next real message starts, so it tries the next byte, then the one after that, until something looks like a valid type. On a corrupted stream that can burn a lot of bytes before it lands on a real boundary. Worse, it can land on a false one, because a byte sitting in the middle of some other message can happen to equal A.</p><p>Real systems don’t guess. The transport layer underneath ITCH carries sequence numbers, so a receiver knows exactly which message it’s on and exactly what it missed. This parser doesn’t look at any of that.</p><h3>Where it falls over</h3><p>This is the part I actually wanted to write, because a parser that handles the happy path is the easy 80%.</p><p><strong>It needs a gap between messages.</strong> DECODE takes a full clock cycle and never checks byte_valid. So if the next message&#39;s first byte arrives during that cycle, it&#39;s gone. And since that&#39;s the type byte, the parser reads the <em>second</em> byte as a type instead, and almost certainly ends up in ERROR. Feed it a real back-to-back capture and it loses sync immediately.</p><p>The fix isn’t hard. Either check byte_valid during DECODE, or decode combinationally off the last CAPTURE byte so there&#39;s no dead cycle at all. The second is cleaner and saves a cycle of latency. I haven&#39;t done either yet.</p><p><strong>There’s no transport framing.</strong> Real ITCH doesn’t arrive as a bare byte stream. It comes over MoldUDP64 multicast, or SoupBinTCP for recovery, and every message has a 2-byte length sitting in front of it. This parser assumes someone already stripped that off.</p><p>That’s a fair boundary for a parser module. But it also means I’m deriving the length from the type byte when the transport already told me the length. A real version would compare the two, because a mismatch is exactly how you find out you’ve lost sync.</p><p><strong>Resync is a guess.</strong> The ERROR section covered this: drop a byte, try the next one, hope. Sequence numbers exist for precisely this problem and I&#39;m not using them.</p><p><strong>Throughput is the ceiling.</strong> One byte per cycle is the design’s limit, and DECODE costs another cycle per message on top of that. At 156.25 MHz, a natural clock if you&#39;re sitting behind a 10G MAC, that&#39;s roughly 1.25 Gbps. Saturating a 10G link needs eight bytes per cycle.</p><p>Byte-serial is the right way to get the offsets correct and understand the protocol. It’s not the right way to keep up with a live feed. The wide version is a genuinely different design, where a message can start at any byte lane and the decoder turns into a pile of barrel shifters.</p><p><strong>The testbench only covers A.</strong> The E and X paths are written but never exercised. Untested code isn&#39;t working code, and I&#39;m not going to pretend otherwise.</p><h3>What’s next</h3><p>Roughly in order of how much they matter:</p><ol><li>Kill the DECODE dead cycle so back-to-back messages work.</li><li>Testbench coverage for E and X, plus a deliberately corrupted stream to exercise the error path.</li><li>MoldUDP64 framing, with the length prefix checking the type-derived length.</li><li>A wider datapath, at which point this stops being a protocol exercise and becomes a real project.</li><li>The order book itself, which is where the interesting hardware problems are. You need a structure with bounded-latency insert, delete, and best-bid-offer lookup.</li></ol><p>The repo is at <a href="https://github.com/probablysamir/itch-fpga">github.com/probablysamir/itch-fpga</a>. It builds with Icarus Verilog and lints clean under Verilator. make run runs the testbench.</p><p>The ITCH 5.0 spec is <a href="https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NQTVITCHspecification.pdf">published by NASDAQ</a>. I really recommend you to go through this once.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=421dac8787ed" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I used Zero-Copy To Achieve Blazingly Fast File Transfers]]></title>
            <link>https://medium.com/@probablysamir/how-i-used-zero-copy-to-achieve-blazingly-fast-file-transfers-d93eb093a8fb?source=rss-2b9c9c442e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/d93eb093a8fb</guid>
            <dc:creator><![CDATA[Samir Kattel]]></dc:creator>
            <pubDate>Wed, 06 Aug 2025 13:49:59 GMT</pubDate>
            <atom:updated>2026-07-03T08:38:39.965Z</atom:updated>
            <content:encoded><![CDATA[<p>Zero copy. Sounds fancy right? But the idea is quite simple. Just as the name suggests, it moves data with (nearly) zero CPU overhead, and zero detours through your application’s memory and just directly from kernel to kernel to achieve those blazingly fast transfer speeds.</p><p>This is what file transfers usually look like in Go:</p><pre>io.Copy(w, file)</pre><p>Looks neat, looks clean, gets the job done. But wait, what is it actually doing?</p><p>So here’s what I assumed it was doing. Reading the file into a userspace buffer with read(), then pushing it back down into the socket with write(). That&#39;s two syscalls, a bunch of context switches, and the CPU babysitting bytes it never actually needs to touch. Might not matter for small files, but when you&#39;re pushing gigabytes, this slowly turns into a silent bottleneck.</p><p>Turns out I was wrong, and it’s worth knowing why.</p><p>For a plain file to TCP socket, io.Copy is smarter than that. Under the hood it checks if the destination implements io.ReaderFrom, and *net.TCPConn does. So instead of the read/write ping pong, it goes through (*net.TCPConn).ReadFrom, which calls the sendfile() syscall for you. Over net/http it&#39;s response.ReadFrom that does this, and it takes the fast path as long as a few conditions are met:</p><ul><li>the source is an *os.File</li><li>the response isn’t chunked (meaning you set Content-Length)</li><li>it’s not a HEAD request</li><li>you’re not doing TLS in userspace</li></ul><p>Meet those, and io.Copy(w, file) is already zero-copy. The read/write path I was worried about only kicks in when one of these fails. Forget Content-Length and Go switches to chunked encoding, and now it quietly falls back to bouncing bytes through userspace.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/459/0*1wfafUN49mHWc3Ty.png" /></figure><p>So the real question isn’t “how do I beat io.Copy with sendfile()?&quot;. It&#39;s &quot;what is sendfile() actually doing that makes this fast, and when would I ever call it myself?&quot;. Only one way to find out, so let&#39;s get our hands dirty with some code.</p><p>Before I talk about sendfile(), this is how I implemented the same thing using io.Copy():</p><pre>func handleWithIOCopy(w http.ResponseWriter, _ *http.Request) {<br>	fmt.Println(&quot;Handling with io.Copy&quot;)<br>	file, err := os.Open(videoPath)<br>	if err != nil {<br>		http.Error(w, &quot;File not found&quot;, http.StatusNotFound)<br>		fmt.Println(&quot;Error opening file:&quot;, err)<br>		return<br>	}<br>	defer file.Close()<br>	stat, err := file.Stat()<br>	if err != nil {<br>		http.Error(w, &quot;Failed to stat file&quot;, http.StatusInternalServerError)<br>		return<br>	}<br>	size := stat.Size()<br>	w.Header().Set(&quot;Content-Type&quot;, &quot;video/mp4&quot;)<br>	w.Header().Set(&quot;Content-Length&quot;, strconv.FormatInt(size, 10))<br>	io.Copy(w, file)<br>	fmt.Println(&quot;Video streamed with io.Copy&quot;)<br>}</pre><p>Now, let’s dump the comfy io.Copy() abstraction and dive deeper into the world of raw syscalls. But wait, what are syscalls actually?</p><p>Syscalls (also known as system calls) are the gateway or the interfaces through which the userspace application communicates with the kernel. They are how the applications ask stuffs like read from a file, write to a socket, allocate memory and more. The applications cannot directly access the kernel or hardware so the only way to communicate is via syscalls.</p><p>Now here’s the catch: <strong>each syscall has a cost.</strong></p><p>Why?</p><ul><li>Multiple mode switches for the CPU</li><li>Blocking behavior</li><li>Inefficiency at scale</li></ul><p>Here’s the thing though, and it’s easy to trip over: sendfile() is <em>also</em> a syscall. So the win isn&#39;t &quot;no syscalls&quot;, that&#39;d be impossible. The naive read-into-a-buffer-then-write path costs you <em>two</em> syscalls (read() then write()) plus a full copy of the data through a userspace buffer sitting between them. sendfile() collapses that into a single syscall and skips the userspace buffer entirely, the bytes never leave the kernel. That&#39;s the actual saving. Let&#39;s look at how it does it.</p><p><strong>sendfile()</strong></p><p>sendfile() is a syscall provided by linux that helps to cut off that extra layer of userspace entirely. No need to ping pong the data between kernel space and user space and just do all the transfers directly in the kernel space without ever involving the application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/0*t0T1vlAIMJoOGHvF.png" /></figure><p><strong>Implementing </strong><strong>sendfile() in Go</strong></p><p>Now that we have discussed what is sendfile() and why is it important. I wanted to implement it in a Go server. But the standard Go HTTP stack doesn&#39;t expose sendfile() directly.</p><p>Well, not directly. Like we just saw, it’ll use it for you when the conditions line up. But if you want to drive it yourself, say your framework won’t set Content-Length, or you&#39;re speaking a non-HTTP protocol, or you just want to hold the file descriptor yourself, then you drop down to the syscall package. That&#39;s what the rest of this is about. Not because io.Copy can&#39;t do zero-copy, but because it&#39;s worth seeing what it&#39;s actually doing on your behalf.</p><p>I’ll be referring to syscall.Sendfile() or sendfile() interchangeably in this article and both should mean the same thing.</p><p>But syscall.Sendfile() uses file descriptors and http.ResponseWriter does not expose them directly. So, to work around this we need to hijack the underlying TCP connection.</p><p>Let us see how we can do this:</p><pre>// Try to cast responsewriter to http.Hijacker and get access to<br>// underlying TCP connection<br>hijacker, ok := w.(http.Hijacker)<br>if !ok {<br>	http.Error(w, &quot;Hijacking not supported&quot;, http.StatusInternalServerError)<br>	return<br>}<br>conn, _, err := hijacker.Hijack()<br>if err != nil {<br>	http.Error(w, &quot;Failed to hijack connection&quot;, http.StatusInternalServerError)<br>	return<br>}<br>defer conn.Close()</pre><p>Here, we have hijacked the connection and we have bypassed Go’s response life cycle and are completely responsible for managing the connection which includes writing headers manually, streaming files ourselves and properly closing the socket after file transfer.</p><p>But wait, hijacker.Hijack() gives us the generic net.Conn interface but to use file descriptors, we need to assert this into *net.TCPConn:</p><pre>tcpConn, ok := conn.(*net.TCPConn)<br>if !ok {<br>	fmt.Println(&quot;Connection is not TCP&quot;)<br>	return<br>}<br>defer tcpConn.Close()</pre><p>Now, since sendfile() uses input file descriptor for the file, we open the file and extract the fd:</p><pre>outFile, err := tcpConn.File()<br>if err != nil {<br>	fmt.Println(&quot;Failed to get file from TCPConn:&quot;, err)<br>	return<br>}<br>defer outFile.Close()<br>outFD := int(outFile.Fd())<br>file, err := os.Open(videoPath)<br>if err != nil {<br>	fmt.Fprintf(conn, &quot;HTTP/1.1 404 Not Found\r\n\r\n&quot;)<br>	fmt.Println(&quot;File open error:&quot;, err)<br>	return<br>}<br>defer file.Close()</pre><p>(One heads-up on tcpConn.File(), since it bit me later: it hands back a dup of the socket fd and flips it into blocking mode, which takes it off Go&#39;s network poller. Fine for a demo like this, but keep it in mind, it matters when we get to the numbers.)</p><p>The next step is to call file.Stat() to get the information about the file and extract the size of the file. This is important because sendfile() doesn&#39;t know how much data to send unless we explicitly tell them to. Also, we set the headers manually by using fmt.Fprintf since Go no longer handles them for us.</p><pre>stat, err := file.Stat()<br>if err != nil {<br>	fmt.Fprintf(conn, &quot;HTTP/1.1 500 Internal Server Error\r\n\r\n&quot;)<br>	fmt.Println(&quot;Stat error:&quot;, err)<br>	return<br>}<br>size := stat.Size()<br>inFD := int(file.Fd())<br>// Write headers manually<br>fmt.Fprintf(conn, &quot;HTTP/1.1 200 OK\r\n&quot;)<br>fmt.Fprintf(conn, &quot;Content-Type: video/mp4\r\n&quot;)<br>fmt.Fprintf(conn, &quot;Content-Length: %d\r\n&quot;, size)<br>fmt.Fprintf(conn, &quot;Connection: close\r\n\r\n&quot;)</pre><p>Now we finally send the file using syscall.Sendfile():</p><pre>var offset int64 = 0<br>for offset &lt; size {<br>	sent, err := syscall.Sendfile(outFD, inFD, &amp;offset, int(size-offset))<br>	if err != nil {<br>		fmt.Println(&quot;Sendfile error:&quot;, err)<br>		return<br>	}<br>	if sent == 0 {<br>		break<br>	}<br>}</pre><p>We have successfully implemented zero copy in Go using syscall.Sendfile().</p><p>Now, one honest word of warning before you go copy-pasting this: don’t ship it. Treat it as an illustration of what’s happening underneath, not a drop-in upgrade. A couple of reasons. That tcpConn.File() call flips the socket into blocking mode and takes it off Go&#39;s network poller, so you&#39;re leaning on blocking behavior for the loop to even work (that&#39;s also why there&#39;s no EAGAIN handling here, remove the dup and this loop falls apart on a non-blocking fd). And by hijacking, you&#39;ve thrown away keep-alive and forced Connection: close, a fresh TCP connection per request. For the normal case of serving a file over HTTP, io.Copy with Content-Length set already does the exact same sendfile for you, with none of these sharp edges. This code is the &quot;here&#39;s the machinery&quot; version, not the &quot;use this in prod&quot; version.</p><p><strong>What do the numbers say?</strong></p><p>Okay, here’s where I have to be honest with you, because a sharp reader called this out in the comments and they were right.</p><p>When I first wrote this, I benchmarked my io.Copy handler against my hand-rolled sendfile() handler and saw sendfile() come out way ahead. Lower CPU, lower latency, higher throughput. I chalked it up to zero-copy and moved on.</p><p>But here’s the catch. Both handlers were already using sendfile(). My io.Copy version set Content-Length, which means it took the exact fast path I described above and called the same syscall my &quot;manual&quot; version did. So whatever gap I measured, it wasn&#39;t userspace copy vs zero-copy. It couldn&#39;t have been, because neither side was copying through userspace.</p><p>What I was actually measuring was two handlers that differ in a bunch of other ways. My hijacked sendfile() path sends Connection: close and tears down the TCP connection on every request, while io.Copy keeps it alive. The manual path also calls tcpConn.File(), which dups the socket fd into blocking mode and pulls it off Go&#39;s network poller. Add the per-request logging and plain old run-to-run noise on a laptop CPU that was probably thermal throttling by request 2000, and yeah, I was benchmarking my test setup, not zero-copy.</p><p>So I’m pulling the old numbers. They weren’t measuring what I said they measured, and it’d be dishonest to leave them up dressed as a zero-copy win.</p><p>The actual takeaway is better anyway: you’re probably already getting zero-copy and didn’t know it. If you’re serving a static file over net/http with Content-Length set, io.Copy is calling sendfile() for you. The lesson isn&#39;t &quot;reach for raw syscalls&quot;, it&#39;s &quot;make sure you&#39;re on the fast path&quot;, because it&#39;s easy to fall off:</p><ul><li>Forget Content-Length, chunked encoding kicks in, and you&#39;re back to userspace copies.</li><li>Wrap your response writer in something that doesn’t forward ReadFrom, and the fast path is gone.</li><li>Terminate TLS in Go, and sendfile() can&#39;t touch encrypted bytes (more on that below).</li></ul><p>That’s the real performance advice, and it’s basically a one-liner. Set your Content-Length and let the runtime do the zero-copy for you.</p><p><strong>When <em>not</em> to use sendfile()</strong></p><p>After going through all this you might be like “Screw userspace buffers, I’m using zero copy for everything”. I’m sorry my friend, it has it’s own limitations and you should not use sendfile() especially when:</p><ul><li><strong>You refuse to use linux:</strong> I wouldn’t say “You don’t have linux” because linux is openly available to everyone. But if you decide to use something else like macos, sendfile() doesn&#39;t work like it does on linux and has been reported to be slower than io.Copy() and for the windows users I&#39;m sorry I don&#39;t have any words for you.</li><li><strong>You need to modify data before transmission:</strong> Since sendfile() skips userspace, you cannot do stuffs like compression, encryption, etc.</li><li><strong>Your source isn’t a regular file:</strong> Hear me out, sendfile()&#39;s input fd has to be something mmap-able, basically a regular file, it flat out cannot be a socket. So socket-to-socket or pipe transfers are off the table. (Bit of trivia: since Linux 5.12, if the <em>output</em> fd is a pipe, sendfile() quietly turns into a splice() under the hood. And if you actually need arbitrary fd-to-fd transfers where one end is a pipe, splice() is the tool you want, not sendfile().)</li><li><strong>You are working with HTTPS:</strong> Yes, you cannot use sendfile() with HTTPS because you cannot sign or encrypt data in kernel space. TLS expects to handle encryption in userspace which sendfile() bypasses entirely. BUT.. you can offload this to the kernel using ktls however Go&#39;s official tls library natively doesn&#39;t support ktls (well proposal has been submitted and accepted so we might see it in the future but it has lot of issues and is relatively premature to be rolling out soon)</li></ul><p><strong>Conclusion</strong></p><p>So yeah, zero-copy with sendfile() isn&#39;t some theoretical optimization, it&#39;s a real one. The twist is that Go already hands it to you through io.Copy for the common file-over-HTTP case, as long as you don&#39;t accidentally knock yourself off the fast path. Knowing when it kicks in, and when it quietly doesn&#39;t, is the actual win here. Driving syscall.Sendfile() yourself is still worth understanding, but you&#39;ll reach for it less often than you&#39;d think.</p><p>Your CPU deserves a break. Turns out io.Copy was already giving it one.</p><p>Here’s the github repo with the implementation: <a href="https://github.com/probablysamir/zero-copy-go.git">https://github.com/probablysamir/zero-copy-go.git</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d93eb093a8fb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Vertical vs Horizontal Scaling]]></title>
            <link>https://medium.com/@probablysamir/vertical-vs-horizontal-scaling-d8cdd8db4caa?source=rss-2b9c9c442e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/d8cdd8db4caa</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[scaling]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[deployment]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Samir Kattel]]></dc:creator>
            <pubDate>Sun, 17 Dec 2023 13:41:22 GMT</pubDate>
            <atom:updated>2023-12-17T13:41:22.498Z</atom:updated>
            <content:encoded><![CDATA[<p>Surprise! The side project you’ve been working on just exploded in popularity, leading to a traffic surge beyond imagination. Suddenly, it dawned on you: the humble server you initially chose is no longer up to the task.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*ENi8_ba8aHddfUAMIWCdhg.png" /><figcaption>The sudden increase in traffic</figcaption></figure><p>You have two options now: Test the patience of your users or adjust your product to the increasing traffic. If you chose the later option then that’s where you encounter scalability.</p><h3><strong>Scalability</strong></h3><p>Scalability is just about how well your servers can handle the surge in traffic. When there’s an sudden increase in traffic, the system requires more resources like bandwidth, server capacity and processing power. If the server is not well scaled then the users may experience slow loading times, unresponsiveness and may even encounter server crashes which leads to inconvenient user experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*Ag7TgL_wtvwZ5rU1vK54bQ.jpeg" /><figcaption>Users after encountering frequent server crashes</figcaption></figure><p>There again are two options: Getting a powerful server (vertical scaling) or increasing the number of servers (horizontal scaling). Now let’s dive deeper to get better insights.</p><h3>Vertical Scaling</h3><p>Vertical scaling also known as scaling up is accomplished by upgrading the hardware and/or network throughput of a single server or system to handle a higher volume of workload or demand. In short, you need a powerful machine to handle all your needs. So how can you scale up vertically?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UvWthe5nf9XVl0tvrdYEgA.jpeg" /><figcaption>Vertical scaling</figcaption></figure><h4>Adding storage</h4><p>The data is continuously increasing in your server. You need to add sufficient storage to the server to store the increasing data.</p><h4>Upgrading storage</h4><p>Using a traditional hard disk leads to slower read write operation. So, replacing the hard disks with SSDs will significantly improve the time in storing and retrieving data from the disk.</p><h4>Increasing RAM</h4><p>Less amount of RAM means less working memory and cache memory for the system. So, increasing the RAM increases the amount of working memory and cache memory thus making the server more efficient.</p><h4>Upgrading NICs</h4><p>The network throughput in your server is equally important. Even more if you are continuously streaming media from your server. Thus upgrading or adding NIC to your server will increase it’s overall network throughput.</p><h4><strong>Upgrading CPU</strong></h4><p>Sometimes the problem lies with the CPU itself. It’s time to toss the decade old CPUs and replace them with new ones. You can also add more processors or virtual cores ( on cloud instances or VM ) to have a faster server.</p><h3>Advantages of Vertical Scaling</h3><h4>Easier to implement</h4><p>Vertical scaling is relatively easier as we do not need to redesign the code for implementation.</p><h4>Cheaper Initial Cost</h4><p>The initial cost of vertical scaling is relatively low because we can upgrade the server at an affordable cost.</p><h3>Disadvantages of Vertical Scaling</h3><h4>Limit in scaling</h4><p>While vertically scaling a system, there reaches a point where we cannot upgrade the system further because of the hardware and software constraints.</p><h4>Higher cost at later stages</h4><p>There exists a point after which the cost of hardware upgrade is extremely high as compared to the initial upgrades.</p><h4>Lack of fault tolerance</h4><p>If we vertically scale a single machine to handle the workload, the whole server will crash if there is a problem in the system.</p><h4>Peak Demand Charges</h4><p>We need to pay for the upgraded machine even if the workload is low which might lead to unnecessary working fees.</p><h3>Horizontal Scaling</h3><p>Horizontal scaling, also known as scaling out, is a method of increasing system capacity by adding more machines or nodes to your network. It’s like running each component on a separate server and being able to add more servers whenever necessary. In short, we are using many small machines instead of one big one. So, let’s see how we can achieve this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Eh-teHrriKeErFrE7tXXig.jpeg" /><figcaption>Horizontal scaling</figcaption></figure><h4>Load Balancing</h4><p>Load balancing is a technique used to distribute the incoming traffic to different servers making sure that no single server is overloaded and all the machines share the load. When there is an incoming request the load balancer will use the suitable algorithm to distribute the traffic. This helps the system to scale horizontally as we can add more servers and they will all share the load.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rWATDEYMIUTmYwwPqUn4iA.png" /><figcaption>Load balancing working diagram</figcaption></figure><h4>Clustering</h4><p>Clustering is a technique of grouping together multiple servers to work collaboratively as a single system. The servers in the cluster have awareness of one another in the group. It seems similar to normal load balancing but in load balancing the servers are working independently but the servers in the cluster work together to achieve the objective. Clusters are also more redundant and scalable than normal load balancing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*yDI1YeneBVJteIdBqvKDPQ.jpeg" /><figcaption>Simple diagram of a active-active cluster</figcaption></figure><h4>Replication</h4><p>Replication is a technique to create copies of database or database nodes to provide better fault tolerance and availability. The database copies are synchronized with each other to ensure consistency. The requests can come into different nodes thus decreasing the load in a specific node. Different types of databases uses different techniques for replication. In master-slave replication technique used by most relational databases, the master is used for write operations and the data is asynchronously propagated to the slaves and read operation can be performed on any node. In case the master node fails one of the slave is chosen as master node.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rhoga3bUzBiIyaceDqXlEw.png" /><figcaption>Design of a working replication mechanism</figcaption></figure><h4>Sharding</h4><p>Sharding is a technique of dividing or partitioning a large database vertically or horizontally into smaller pieces called shards. Unlike replication each shard doesn’t have a copy of all data but rather it is a subset of the large database. It helps to improve performance, scalability and parallelism by distributing the workload across multiple servers or nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*etEhKFZFFttU3-aho0Y5_g.jpeg" /><figcaption>Sharding</figcaption></figure><h4>Microservices Architecture</h4><p>Microservices architecture is a software development architecture where a complex application is divided into smaller independent services that can be developed, tested and scaled independently. Each service is called a microservice and has it’s own role and communicates with other services through well defined APIs. Since the microservices are loosely coupled we can deploy them separately thus helping in horizontal scalability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*cPhPkJDkbK9rk27Otk2rBQ.png" /></figure><h4>Containerization and Orchestration</h4><p>Containerization is a process which encapsulates application and their dependencies into lightweight, portable units called containers. This ensures consistency across different environments that allows for rapid deployment. Orchestration is the process of automating the management, deployment and scaling of these containers. So, when the workload increases, the containers can be added seamlessly to the system. So these two together help to seamlessly scale and handle the varying workloads.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/786/1*GdRp67YMmIs1HOperFVz7A.png" /></figure><h4>Content Delivery Network (CDN)</h4><p>CDN is a distributed network of servers placed at multiple locations that caches website content. It offloads a significant portion of traffic from the main server by providing the cached content directly. It also helps in availability of content as if one edge server experiences issues, requests are automatically rerouted to alternative servers that host copies of the same content.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s8Me7HLwlxkpzlKcmcACCw.jpeg" /><figcaption>Single server vs CDN</figcaption></figure><h3>Advantages of Horizontal Scaling</h3><h4>Higher Scalability</h4><p>You can horizontally scale a system many times more than you can achieve with a vertical scaling.</p><h4>Redundancy</h4><p>The system becomes more redundant and there’s a backup even if a server fails.</p><h4>Cheaper Final Cost</h4><p>Even if the initial cost is higher, the cost at the later stages is cheaper in comparison to vertical scaling.</p><h4>No Peak Demand Charges</h4><p>As you can scale according to demand there’s no need to pay for the peak demand capacity if not needed.</p><h3>Disadvantages of Horizontal Scaling</h3><h4>Complex implementation</h4><p>It is harder to implement because it needs significant effort to change the architecture of the application to add later thus it is wise to consider it before the application is even built.</p><h4>Higher Initial Cost</h4><p>The initial cost of horizontal scaling is higher than vertical scaling till a certain point.</p><h4>Complex management</h4><p>Managing a lot of servers is harder than managing a single server as there are more potential points of failure and require more effort to monitor and maintain.</p><h3>Horizontal or vertical scaling. What should I use?</h3><p>After delving into the depths of this extensive article, the bottom line is: What’s the ultimate choice for you? Let’s cut to the chase and discover the perfect fit!</p><p>There’s actually no hard answer but actually depends on your use case. Firstly we need to consider the running costs and manpower. If your service has low to moderate workload and has a steady predictable growth that can be accommodated within the limits of a single server then it is advised to go for vertical scaling.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/954/1*ngC2XRBZJTUAkO-FDqbK3w.png" /></figure><p>Similarly, if your workload is high or unpredictable and requires high availability and fault tolerance then it is advisable to go for horizontal scaling.</p><p>If you are doing a large-scale deployment then it might be wiser to go for a hybrid approach. Hybrid approach is like having the best of both worlds, you can vertically scale specific components to improve their performance and horizontally scale when you require additional capacity.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d8cdd8db4caa" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>