<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Damodaran on Medium]]></title>
        <description><![CDATA[Stories by Damodaran on Medium]]></description>
        <link>https://medium.com/@gautham.daamodhar?source=rss-170011d88c9c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*eUkIO3H-zGnQ1-MsrW-JEg.jpeg</url>
            <title>Stories by Damodaran on Medium</title>
            <link>https://medium.com/@gautham.daamodhar?source=rss-170011d88c9c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 12:21:40 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@gautham.daamodhar/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Marko JS and progressive HTML rendering bare bones]]></title>
            <link>https://blog.devgenius.io/marko-js-and-progressive-html-rendering-bare-bones-e731b08755ab?source=rss-170011d88c9c------2</link>
            <guid isPermaLink="false">https://medium.com/p/e731b08755ab</guid>
            <category><![CDATA[js]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[marko]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[Damodaran]]></dc:creator>
            <pubDate>Wed, 29 Mar 2023 15:36:54 GMT</pubDate>
            <atom:updated>2023-03-30T21:23:39.822Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*ZJrSRDK3SQrF29yyXrecdw.png" /></figure><h3>Introduction</h3><p>This is a set of experiments in an attempt to unravel at a 10K feet view of how <a href="https://www.markojs.com/">Marko</a> achieves progressive rendering.</p><p>I’ve usually been intrigued by how <a href="https://www.markojs.com/">MarkoJS</a> SSR can write chunks into the output stream and do all kinds of magic like <a href="https://tech.ebayinc.com/engineering/async-fragments-rediscovering-progressive-html-rendering-with-marko/">progressive rendering</a></p><ul><li>out-of-order async renders + in-order flush</li><li>out-of-order async render + out-of-order flush + in-order paint</li></ul><p>While the internal workings maybe overwhelming, the core idea that helps achieves this on the server is via the <a href="https://github.com/marko-js-archive/async-writer">async-writer</a> module &amp; its variations.</p><p><em>While Marko’s internal APIs may have evolved from V3 to V5, the core idea is what we are trying to get a hang of.</em></p><p>It’s important to strip away the details &amp; understand the barebones to be able to better appreciate the inner workings.</p><p>This write up pretty much explores that in steps &amp; expands upon every previous step.</p><h3>Contents</h3><p>This begins with a sample app that illustrates how to integrate a Marko template with a simple Node js HTTP server &amp; <em>render a template on the server</em> to an output response stream, with the following cases :</p><ul><li>Change-set-1: Basic template render via template.render(templateData, writeableStream);</li><li>Change-set-2: Why http doesn’t write anything to the wire after my template.render(templateData, writeableStream);</li><li>Change-set-3: What is this Async-writer and what happens when we wrap our writeableStream with it ?</li><li>Change-set-4: I have some async op, how do I make the http writeableStream wait &amp; render in the right order?</li><li>Change-set-5: Fixing #2, with the new knowledge of wrapping the response writeable stream with an Async-Writer.</li><li>Change-set-6: Fixing the time taken in #4, by flushing out the Async Writer as bytes are received.</li><li>Change-set-7: Fixing the chunks showing up out-of-order</li></ul><h3>Installation</h3><pre>&gt; npm i<br>&gt; node server.js</pre><h3>Change-set-1 (Render a basic Marko v3 template on the server)</h3><p>The <em>package.json</em> file we would use. Going forward, only the “<em>server.js</em>” file will be updated.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/742e02bfd82f025a89e3abf3d31fa1e1/href">https://medium.com/media/742e02bfd82f025a89e3abf3d31fa1e1/href</a></iframe><p>The sample <em>server.js</em> file would look like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c0aac3cdf7ec307872064484c01e6ea4/href">https://medium.com/media/c0aac3cdf7ec307872064484c01e6ea4/href</a></iframe><p><em>index.marko</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f791dfe7bbcb5af38401b363e05837f0/href">https://medium.com/media/f791dfe7bbcb5af38401b363e05837f0/href</a></iframe><p><em>footer.marko</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e0064aaf4ff2d4d8e0cd32cd3a13101c/href">https://medium.com/media/e0064aaf4ff2d4d8e0cd32cd3a13101c/href</a></iframe><p>What is the above code doing ?</p><ul><li>Here we are invoking the indexTemplate ~ index.marko.</li><li>We use the template.render(templateData, writeableStream).</li><li>our templateData is {name: &#39;Frank&#39;, count: 30, colors: [&#39;red&#39;, &#39;green&#39;, &#39;blue&#39;]}</li><li>The res or response object in http is not wrapped by an <a href="https://github.com/marko-js-archive/async-writer">Async Writer </a>i.e., its not an Async stream for that matter.</li><li>This is a simple writeable stream.</li><li>Note that templateData is an {}. This brings up our page</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*hNHoa5JLR4z8FsRLEgvg0Q.png" /><figcaption>Basic usecase of rendering a Marko template on the server via a Http Request</figcaption></figure><h3>Change-set-2 (Issue: whatever written after template.render() is not showing up</h3><p>Next lets write multiple stuff to our page:</p><p>Say we want to just output</p><ul><li>&quot;&lt;h1&gt;------Before body--------&lt;/h1&gt;&quot;</li></ul><p>before our page, &amp;</p><ul><li>&quot;&lt;h1&gt;------After body--------&lt;/h1&gt;&quot;</li></ul><p>after the page.</p><p>Let’s update the <em>server.js </em>file to reflect this.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6aa0e64c3975600c08b0f568111817ce/href">https://medium.com/media/6aa0e64c3975600c08b0f568111817ce/href</a></iframe><ul><li>Here we are invoking the indexTemplate ~ index.marko.</li><li>We use the template.render(templateData, writeableStream).</li><li>The res or response object in http is not wrapped by an Async writer i.e., its not an Async stream for that matter.</li><li>This is a simple writeable stream.</li><li>our templateData is {name: &#39;Frank&#39;, count: 30, colors: [&#39;red&#39;, &#39;green&#39;, &#39;blue&#39;]}</li><li>Here we are writing something before and after the templateData. The output is as below:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*HIKPw6tZv8rjiG98fVU0iQ.png" /><figcaption>Whatever we wrote into the response stream after template.render did not show up!</figcaption></figure><p>Note that, whatever we wrote after the templateRender() never showed up!. This is because, the res output writeable stream of http is not wrapped by an Async writer.</p><p>So, Marko creates an Async writer within it &amp; maps it to the http response stream and finally ends it if:</p><ul><li>The incoming response was not a Stream / not an Async Stream</li><li>if the current render had created the Async stream.</li><li>If this call was part of a child component that was being rendered, no new Async stream is created, as the incoming stream itself is normalized to be an Async stream by Marko &amp; this Async Stream will not be ended.</li><li>The Async stream will be ended by Marko only when it reaches the call stack that created it.</li></ul><p>So, we would get the following error upon executing this:</p><pre>Error: write after end<br>    at ServerResponse.OutgoingMessage.write (_http_outgoing.js:439:15)<br>    at Server.server.on (/Users/homedesktop/workspaces/browse/marko-http/server.js:18:7)<br>    at emitTwo (events.js:106:13)<br>    at Server.emit (events.js:191:7)<br>    at HTTPParser.parserOnIncoming [as onIncoming] (_http_server.js:546:12)<br>    at HTTPParser.parserOnHeadersComplete (_http_common.js:99:23)</pre><p>Note that res.send() is internal to express JS, which is not used here. res.end() is used to end the stream - which was done by Marko.</p><blockquote><em>So this error shows up because we are trying to write to an output stream which has already been ended.</em></blockquote><h3>Change-set-3 (Understanding the async-writer &amp; a basic use-case of wrapping our response stream with an Async-Writer)</h3><p>Let’s update the <em>server.js</em> file.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b8282f08ca22435eb3139db06b5b2562/href">https://medium.com/media/b8282f08ca22435eb3139db06b5b2562/href</a></iframe><p>The response will properly contain <strong>A B C</strong> in the right order.</p><p><strong>Now, what happens when a <em>.marko </em>template receives such a response stream wrapped by the async-writer?</strong></p><p>The index.marko is the initial template that gets rendered to the page. Its renderer&#39;s render method that takes an out parameter (which marks an output stream) will not be an async stream unless otherwise specified.</p><p>However, if there are further components within this index.marko like the case of</p><pre>&lt;include(&quot;./footer.marko&quot;) name=data.name/&gt;</pre><p>Then, footer.marko will receive an Async stream (and all subsequent components after it will be writing to an output Async stream)</p><p>The above case is the simplest usage of an Async-writer that provides us with an Async output stream. It wraps the response/out stream of the http.</p><p>This is done internal to Marko itself.</p><p>In the above example, we create an Async stream and continue to write to it as how we would, into any writeable stream. What is so special about this ? It appears to be just another output stream…</p><p>But take a look at what we can do next with it.</p><h3>Change-set-4 (Async writer with Out-of-order render, in-order flushing)</h3><p>In real world scenarios, most of the time, as we end up building our templates and populate it with data. There are always going to be sections in the page, that render from some service that takes a little more time than the rest.</p><p>So, if that is the case, say the Left Hand Nav of your page takes a while:</p><ol><li>would you continue to wait for it ? Or,</li><li>would you just go ahead and render other potions of the template that have data ?</li></ol><p>Our choice is obviously (2), to speed up the render process. But in doing so, if use the basic http output response stream, then the order of the final content will be incorrect. We would therefore be forced to render it in the same order. Async-Writer helps mitigate this.</p><p>Let’s update the <em>server.js.</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5f69ae72406ca4b1c867fd8a6b4f5820/href">https://medium.com/media/5f69ae72406ca4b1c867fd8a6b4f5820/href</a></iframe><p>The output of this example will still be ‘<strong>ABCD</strong>’. How?</p><p>This is a classic case of using the Async-Writer to write to an Async stream. As before, the incoming response writeable stream of http, is wrapped by the Async-Writer, to become an Async output writeable stream.</p><p>Next, we first write ‘A’ into it. Next, we want to write ‘B’, but we sense some delay in the response to ‘B’. So, we wrap into inside an Async operation, via,</p><blockquote><em>var asyncOut = out.beginAsync()</em></blockquote><p>The end of this Async operation, will be marked by a</p><blockquote><em>asyncOut.end();</em></blockquote><p>The end of the writeable stream, will be marked by a</p><blockquote><em>out.end()</em></blockquote><p>So, for now, to simulate an async op, we use a setTimeout and then write &#39;B&#39; into it. We then follow it up by writing &#39;C&#39;, then, another async op, where we write &#39;D&#39;.</p><p>After completing writing &#39;D&#39;, we just end the async operation via asyncOut.end().</p><p>But note that, we have still invoked out.end much ahead of asyncOut.end.</p><p>This implies,</p><ul><li>when out.end executes =&gt; There is nothing more to write, you can end the stream, if there are no async ops.</li><li>But, we have created an async op, so we wait.</li><li>Upon invoking, asyncOut.end we see if there are anything further left to write. There is nothing as the response stream has already been marked with out.end(). So its the end of the stream.</li></ul><p>This gives the output, ‘<strong>ACBD</strong>’.</p><p>You may have noticed, we wrote into the response stream in different order. Yet, our output is still rendered out in the right order as mentioned above : ‘<strong>ABCD</strong>’. How ?</p><ul><li>The Async-Writer makes it possible to flush bytes to the output stream out of order, while still flushing out bytes in the correct order finally.</li><li>So, you can write parts of the stream out of order. But the bytes will be flushed in the right order</li><li>Content that is written after an async op, will be kept in a buffer and after the async op is done, the output is buffered out properly in the right order.</li></ul><h3>Change-set-5 (Fixing our change-set-2 with our new knowledge of Async-Writer)</h3><p>Now that we know about what the <em>async-writer</em> can accomplish, lets see if we can fix our change-set-2</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d48a43c592bd518a0af6906ec7cfe54e/href">https://medium.com/media/d48a43c592bd518a0af6906ec7cfe54e/href</a></iframe><blockquote><em>Did you notice what we just did? We wrapped the incoming output response writeable stream with an Async Writer. Now, our initial template </em><em>index.marko&#39;s </em><em>render() function receives an Async Writer:</em></blockquote><ul><li>We write first the &#39;&lt;h1&gt;------Before body--------&lt;/h1&gt;&#39;</li><li>Then we render the template.</li><li>Then we write &#39;&lt;h1&gt;------After body--------&lt;/h1&gt;&#39;</li></ul><p>Since we passed an Async output response writeable stream into the marko-runtime&#39;s render(), it will not marko the stream as ended. Its our duty to end the response stream. So we end it with an</p><blockquote><em>out.end()</em></blockquote><p>But since this happens after the template.render() call, the response stream ends only after this function completes.</p><p>This brings us the output that we had desired so long!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*wYRZtat0Uq99tpIffbE4kQ.png" /><figcaption>Change-set-2 fixed!</figcaption></figure><h3>Change-set-6 (Improvising the render speed of change-set-4. Out of order render &amp; out of order flushing &amp; out-of-order paint)</h3><p>Let’s get back to change-set-4, what was it about :</p><ul><li>async rendering of our data.</li><li>out-of-order rendering (we wrapped async ops well, so that other sections can be rendered. In fact, ‘C’ gets rendered before ‘B’ did)</li><li>correct order of buffering out the response. (Despite rendering ‘C’ first, we buffered it out in the right order, thanks to the Async-Writer)</li></ul><p>So, What was wrong with it ?</p><ul><li>It took too much time.</li></ul><p>If you noticed how we do asyncOut.write(&#39;D&#39;). It takes a paltry 4s. This could be any service call on the page. What would happen then ? Our page render time would have also taken 4s. How do we avoid this ?</p><blockquote><em>The best way would be to flush the buffer as and when we get the data.</em></blockquote><p>Lets update the <em>server.js</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/22f2df7cc2daad0e7469d085bd08061f/href">https://medium.com/media/22f2df7cc2daad0e7469d085bd08061f/href</a></iframe><p>Here we are just using an EventEmitter to emit data in chunks (so that the response can be written in chunks to the output stream, as it arrives)</p><p>In the above example, we use the <em>Async-Writer</em> to fix the issues in change-set-4.</p><p>Async-Writer is used to write to an Async output writeable stream.</p><p>It first writes out &lt;h1&gt;Hello world!&lt;/h1&gt; and then &lt;h1&gt;Bye world&lt;/h1&gt; to the output buffer.</p><p>Followed by this, it receives the first data event from the event Emitter. <br>So it now has &lt;h1&gt;Hello world!&lt;/h1&gt;, &lt;h1&gt;Bye world&lt;/h1&gt; followed by &lt;h1&gt;1&lt;/h1&gt;. Now, comes the important line:</p><blockquote><em>asyncOut.flush();</em></blockquote><p>This line immediately flushes the buffer and writes out the HTML to the output stream. So, we immediately see these on the page. The page load no longer waits for 4s to finish loading all the data. The moment the first data arrives, its piped out.</p><p>Followed by this, as and when there is a data event, the buffer is flushed out &amp; written to the output writeable Async Stream.</p><p>But the caveat to this is that, <strong><em>the response comes out of order</em></strong>. We shall see how this can be fixed. The output we get now is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*tjLJZyc_FJ4UoGSLdvyr6g.gif" /><figcaption>Progressive rendering basics with out of order flushing.</figcaption></figure><h3>Change-set-7 (Fixing Change-set-6 which had out-of-order render, out-of-order flush, via in-order paint)</h3><p>We saw in Change-set-6 that, although we achieved out-of-order renders and out-of-order flush, the issue was that the response comes out of order. So you can see how the order had changed on the page.</p><p>How can this be fixed? Well. for starters, this is the same problem that Marko solves via client-reorder=true. Here, Marko would something of the following:</p><ul><li>setup placeholders for the async content</li><li>flush the async content via &lt;noscript/&gt; tags</li><li>also flush a JS snippet which takes care of moving the data mapped to a certain placeholder (which is maintained at a certain position in the DOM), into the placeholder.</li></ul><p>But the caveat for this is that, it absolutely needs JS and it can cause reflows (there are some ways around mitigating this).</p><p>Marko calls this technique as out-of-order flushing &amp; rendering but in-order paint. Even though the chunks are rendered and flushed out-of-order, they are “fixed” to appear in the right order in the DOM.</p><p>We will try to achieve the same with some very rudimentary code. Let’s update the <em>server.js </em>file once again.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/81b17c4ebfdacdcc6f1d99fe33128670/href">https://medium.com/media/81b17c4ebfdacdcc6f1d99fe33128670/href</a></iframe><p>Notice the line:</p><pre>asyncOut.write(&#39;&lt;div id=&quot;placeholder&quot;&gt;&lt;/div&gt;&#39;);</pre><p>This first writes a placeholder for the async content, in the right order. <br>This placeholder is empty, but will eventually serve as a placeholder to insert the content at the right spot.</p><p>Then, we do</p><pre>asyncOut.write(&#39;&lt;noscript&gt;&lt;h3&gt;&#39; + data + &#39;&lt;/h3&gt;&lt;/noscript&gt;&#39;);<br>asyncOut.flush();</pre><p>This is done as part of the .on(&#39;data&#39;) event. The data is written into &lt;noscript&gt; tags &amp; is flushed out of the buffer.</p><p>Finally, we insert a JS snippet at the end, to fix the order:</p><pre>asyncOut.write(`<br>      &lt;script&gt;<br>          console.log(&#39;executing script...&#39;);<br>          (function(){<br>              const noscripts = document.getElementsByTagName(&#39;noscript&#39;);<br>              const placeholder = document.querySelector(&#39;#placeholder&#39;);<br>              let data = &#39;&#39;;<br>              console.log(placeholder);<br>              Array.from(noscripts).forEach(function (noscript) {<br>                  data = data + noscript.innerHTML;<br>              });<br>              if (placeholder) {<br>                  placeholder.innerHTML = data;<br>              }<br>          })();<br>      &lt;/script&gt;<br>  `);<br>  asyncOut.flush();</pre><p>This, once flushed would query the &lt;noscript&gt; tags, gather the html and put them into the placeholder element, thereby, enabling to &quot;appear&quot; at the right spot &amp; the order is maintained.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xX5Scw_6ORV6ivBFMAs0NQ.gif" /><figcaption>Order fixed ! yay!</figcaption></figure><p>Note that in the last 2 examples, we have not used / rendered the Marko templates at all.</p><p>This is because, Marko helps you achieve this declaratively, via their core tags called <a href="https://markojs.com/docs/core-tags/#await"><strong><em>&lt;await&gt;</em></strong></a><strong><em> </em></strong>which has support for <em>out-of-order flushing</em> via the attribute <strong><em>client-reorder=true</em></strong></p><p>Hopefully this was a useful exercise in unraveling the various aspects towards Marko JS’s progressive rendering.</p><p>Incidentally, Marko JS has pioneered all these approaches long before today’s mainstream JS frameworks!</p><p><strong>For further reads:</strong></p><p><a href="https://tech.ebayinc.com/engineering/async-fragments-rediscovering-progressive-html-rendering-with-marko/">Async Fragments: Rediscovering Progressive HTML Rendering with Marko</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e731b08755ab" width="1" height="1" alt=""><hr><p><a href="https://blog.devgenius.io/marko-js-and-progressive-html-rendering-bare-bones-e731b08755ab">Marko JS and progressive HTML rendering bare bones</a> was originally published in <a href="https://blog.devgenius.io/">Dev Genius</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>