<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jack Oddy on Medium]]></title>
        <description><![CDATA[Stories by Jack Oddy on Medium]]></description>
        <link>https://medium.com/@jackoddy?source=rss-2362cc3d858------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*lQ9q4WuKQRpT-bk_gqo0cQ.jpeg</url>
            <title>Stories by Jack Oddy on Medium</title>
            <link>https://medium.com/@jackoddy?source=rss-2362cc3d858------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 21:25:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jackoddy/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Turbolinks and the Next.js <head>ache]]></title>
            <link>https://medium.com/@jackoddy/turbolinks-and-the-next-js-head-ache-6e71614eeb8?source=rss-2362cc3d858------2</link>
            <guid isPermaLink="false">https://medium.com/p/6e71614eeb8</guid>
            <category><![CDATA[react]]></category>
            <category><![CDATA[turbolinks]]></category>
            <category><![CDATA[nextjs]]></category>
            <category><![CDATA[middleware]]></category>
            <category><![CDATA[ruby-on-rails]]></category>
            <dc:creator><![CDATA[Jack Oddy]]></dc:creator>
            <pubDate>Thu, 06 Jul 2023 20:34:33 GMT</pubDate>
            <atom:updated>2023-07-06T20:34:33.731Z</atom:updated>
            <content:encoded><![CDATA[<p>This is a story about migrating the frontend away from small React apps distributed throughout a Ruby on Rails monolith to a standalone Next.js app. Spoilers: <strong>turn off turbolinks</strong> if you’re looking to do the same thing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nb5KQNfsSwjFGhHb3izQpw.png" /></figure><p>Why are we even talking about <a href="https://github.com/turbolinks/turbolinks">Turbolinks</a> in 2023, it’s been dead since late 2021? It has, but since it came into our codebase on the rails train it hasn’t really left and has been flying under the radar for a while. Minding it’s own business, quietly undermining our migration efforts and eroding trust in the benefits we hoped to see from the new stack.</p><p>If you’re familiar with Turbolinks and what it does, you may intuit what was <em>really </em>happening from the following description, but if you’re like me 5 days ago, prepare to be confounded.</p><h4>The setup</h4><p>Before we jump into what was happening it’s important to understand the architecture we were dealing with. In order to safely migrate pages away from rails, all traffic was being routed through Next.js. Then through a combination of <a href="https://nextjs.org/docs/pages/building-your-application/routing/middleware">Middleware functions</a> and <a href="https://nextjs.org/docs/pages/api-reference/next-config-js/rewrites">Rewrite config</a>, the decision is made to either serve new content from Next.js or proxy the request to rails.</p><figure><img alt="A diagram showing the relationship between Next.js, Ruby on Rails and an API" src="https://cdn-images-1.medium.com/max/1024/1*rQ7oVLOYhI39mHeyT4YfLw.png" /></figure><h4>So what was happening?</h4><p>Fairly early on in the migration process we started to occasionally see some strange behaviour. It appeared the page would get ‘stuck’ in a loading state. Nothing was clickable and certain parts of the page would disappear. Refreshing the page would load it correctly, and for a while this was the temporary accepted workaround. It wasn’t pretty but we hoped it wouldn’t be too disruptive while we completed the migration.</p><p>Well we were wrong. Temporary fixes and workaround <em>always</em> live longer than intended, we shouldn’t have been surprised this one came back to bite us.</p><p>As we started migrating more complex pages and features the buggy behaviour we saw seemed to mature and become far more problematic. Part of the problem was that we introduced animated skeletons into the initial state of many of our pages — so from the user’s perspective it wasn’t clear the page was broken, just loading. And, far more sinister, we started to see some infinite loops for requests for assets, which locked up the users browser. And the spookiest thing? The assets being requested were used on the Rails pages.. but the page was served from Next.js 👻</p><p>Time to really dig into this and solve it. We quickly discovered inspecting the DOM, namely the &lt;head&gt;of the document during the buggy behaviour revealed something utterly perplexing;</p><pre>&lt;head&gt;<br>  &lt;script src=&quot;/_next/...&quot;&gt;&lt;/script&gt;<br>  &lt;script src=&quot;/packs/v1/..&quot;&gt;&lt;/script&gt;<br>...</pre><p>It has a mix of both Next.js AND Rails script, asset and meta tags. Somehow the two worlds had collided and unsurprisingly they culminated in an unresponsive mess on the page.</p><h4>Where does Turbolinks come into this?</h4><p>In hindsight, it says it right there in the <a href="https://github.com/turbolinks/turbolinks">readme</a>;</p><blockquote>When you follow a link, Turbolinks automatically fetches the page, swaps in its &lt;body&gt;, and merges its &lt;head&gt;, all without incurring the cost of a full page load.</blockquote><p>Seems pretty obvious now — but Turbolinks had been quietly lurking in the background for so long it was easily overlooked. Next.js being the new technology and changing at such a fast pace, it was far easier to come to the conclusion that the source of the bug lay there.</p><p>Next.js does its own version of the same thing — loading javascript and pre-rendered pages asynchronously, swapping out the body and head. Laying it all out, it’s not surprising that these two technologies weren’t compatible.</p><p>The simple solution to a baffling bug presented itself almost immediately: just turn off Turbolinks. Bug fixed.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e71614eeb8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reading the body of a request in Vercel’s Edge middleware]]></title>
            <link>https://medium.com/@jackoddy/reading-the-body-of-a-request-in-vercels-edge-middleware-b4dbe446eabb?source=rss-2362cc3d858------2</link>
            <guid isPermaLink="false">https://medium.com/p/b4dbe446eabb</guid>
            <category><![CDATA[request-body]]></category>
            <category><![CDATA[http-request]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[edge-computing]]></category>
            <category><![CDATA[vercel]]></category>
            <dc:creator><![CDATA[Jack Oddy]]></dc:creator>
            <pubDate>Mon, 19 Jun 2023 19:19:28 GMT</pubDate>
            <atom:updated>2023-06-20T17:55:43.992Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DfVIDDj-tZ0rejr0Kkjc6g.png" /></figure><p><strong>edit:</strong><em> turns out the incoming request body was actually encoded form-data which is why I had so much trouble! 😂 I’ll leave this post up incase it’s useful to someone — but try </em><em>request.json() and </em><em>request.formData() first, or at least inspect the </em><em>Content-Type header!</em></p><p>It should be simple right? Years of frameworks abstracting away this step will likely lead many to scratching their heads when it comes to reading the body of a request in an edge function. There’s nothing surprising here really, this is probably more of a ‘how to read a ReadableStream in node’.</p><blockquote>Of course we’re talking about <a href="https://vercel.com/docs/concepts/functions/edge-functions/edge-runtime">Vercel’s edge runtime</a>, not node. but for this post they may as well be the same thing</blockquote><p>For the sake of this example let’s say we want to verify the signature header — for that we’ll need to extract the body. We won’t go into signature verification — but if you’re interested, I also wrote about verifying a request Slack in edge middleware:</p><p><a href="https://medium.com/@jackoddy/verifying-slack-signatures-using-web-crypto-subtlecrypto-in-vercels-edge-runtime-45c1a1d2b33b">Verifying Slack signatures using Web Crypto / SubtleCrypto in Vercel’s Edge runtime</a></p><p>Let’s start with a middleware edge function and go from there. Here is an edge function that returns an error response when the request signature cannot be verified.</p><pre>import { next } from &#39;@vercel/edge&#39;<br><br>async function middleware(request) {<br>  const signature = request.headers.get(&quot;X-Signature&quot;)<br>  const body = await extractBody(request)<br><br>  if (await isSigned(signature, body) return next() // allow request to proceed<br><br>  return new Response(&#39;unauthorised&#39;, { status: 403 })<br>}</pre><p>Now we have a context to work within, let’s jump into this function called extractBody</p><p>A few things need to happen in order to extract the data from request.body it is a ReadableStream — which you can read about <a href="https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream">here</a>. To read a stream you need to get a StreamReader <a href="https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream/getReader">which locks the stream until it is released</a>. While it is locked to the reader the data can be streamed/read and this is just what we need to do.</p><pre>async function extractBody(request) {<br>  const reader = request.body.getReader();<br>}</pre><p>Now a reader has been locked onto the stream, it needs to be read. Streams work by reading parts of data and doing something with that part, similar to iteration, until you’re done. In JS this is commonly achieved with a while(true){ ... loop. Let’s see what that looks like:</p><pre>  while (true) {<br>    const { done, value } = await reader.read();<br>    if (done) break;<br>  }</pre><p>Calling read() on a reader returns an object with two values { done, value } — done being true once the end of the stream has been processed, value being the part of the stream that has been read. So now the value is available, what do we do with it? In this case, it needs to be decoded before being appended to a string to build the value of the body over time:</p><pre>async function extractBody(request) {  <br>  const dec = new TextDecoder();<br>  const reader = request.body.getReader();<br>  let body = &quot;&quot;<br><br>  while (true) {<br>    const { done, value } = await reader.read();<br>    if (done) return body;<br><br>    body = body + dec.decode(value)<br>  }<br>}</pre><blockquote>Note: this will return the raw value of the body — you may be expecting JSON, but it will need to be processed further for that. For the sake of this example — signature verification, the raw version was all that’s needed</blockquote><p>This may not be earth shattering, but I do suspect that this could trip up quite a few engineers coming from express or next.js, who will be used to a preprocessed body property to a request. I hope you found this useful!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4dbe446eabb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Verifying Slack signatures using Web Crypto / SubtleCrypto in Vercel’s Edge runtime]]></title>
            <link>https://medium.com/@jackoddy/verifying-slack-signatures-using-web-crypto-subtlecrypto-in-vercels-edge-runtime-45c1a1d2b33b?source=rss-2362cc3d858------2</link>
            <guid isPermaLink="false">https://medium.com/p/45c1a1d2b33b</guid>
            <category><![CDATA[slackbot]]></category>
            <category><![CDATA[edge-computing]]></category>
            <category><![CDATA[security-token]]></category>
            <category><![CDATA[vercel]]></category>
            <category><![CDATA[nodejs]]></category>
            <dc:creator><![CDATA[Jack Oddy]]></dc:creator>
            <pubDate>Mon, 19 Jun 2023 16:39:49 GMT</pubDate>
            <atom:updated>2023-06-19T16:42:11.569Z</atom:updated>
            <content:encoded><![CDATA[<h3>Verifying Slack signatures using Web Crypto / SubtleCrypto in Vercel Edge Middleware</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V9rKKM7peY-2j9OqcGss8A.png" /></figure><p>The JavaScript ecosystem is a nightmare to keep up with — it moves fast and waits for no one. I’ve be writing a slack bot using <a href="https://vercel.com/docs/concepts/functions/serverless-functions">Vercel’s serverless functions</a> to keep things simple, but when it came to verification of the slack signature I hit a road block. Below is the explanation of what the problem was and how I solved it within the limitations <a href="https://vercel.com/docs/concepts/functions/edge-functions/edge-runtime">Vercel’s edge runtime.</a></p><p>When creating integrations with Slack, your apps used to be able to use a verification token to verify if a request was really coming from Slack. However, you might notice a little notice by this token. Verification tokens have been deprecated — better to use the signatures.</p><blockquote>This deprecated Verification Token can still be used to verify that requests come from Slack, but we strongly recommend using the above, more secure, signing secret instead.</blockquote><p>Let’s take a look at <a href="https://api.slack.com/authentication/verifying-requests-from-slack">Slack’s documentation </a>on how to verify a request. You can go and read it for yourself, but the short of it is as follows:</p><ul><li>Grab the signature from the header</li><li>Create the signature yourself using the secret token</li><li>Compare the two — if they match then success!</li></ul><p>This all sounds <em>relatively</em> simple, so why isn’t it? Well Vercel’s edge runtime includes a limited implementation of various global node APIs, including Crypto / crypto — which is attached to globalThis — the main limitation in this case being that it does not expose crypto.timingSafeEqual which is what you would use to compare two signatures safely.</p><blockquote>Comparison could be done with === but <strong>this is not considered secure</strong>. The time it takes a ===statement to return changes depending on how many sequential characters/values match, and as such it is open to brute force attacks. crypto.timingSafeEqual resolves in constant time meaning it is not possible to detect these changes.</blockquote><p>We havecrypto.subtle instead which is globally available in node 15 and above (on globalThis.crypto.subtle in vercel’s edge runtime). This represents some unique challenges due to just how slack forms it’s signature. Let’s talk about how Slack does that.</p><p>If it were written in JavaScript and the crypto.subtle module, this is how Slack would create it’s signature:</p><p>First it would create a <a href="https://developer.mozilla.org/en-US/docs/Web/API/CryptoKey">CryptoKey</a> with which to perform the hashing to create the signature. This is simply done using the importKey function and a couple of options.</p><pre>const key = await crypto.subtle.importKey(<br>  &quot;raw&quot;,<br>  enc.encode(SLACK_SIGNING_SECRET),<br>  { name: &quot;HMAC&quot;, hash: &quot;SHA-256&quot; },<br>  false,<br>  [&quot;sign&quot;, &quot;verify&quot;] // specify what this signature will be used for<br>)</pre><p>Great, simple enough — what comes next? We need to create the data that will be used to generate the signature. Slack uses a combination of variables to achieve this, see below what they are an how they’re combined. The string must then be encoded before being passed to the sign function, a promise resolving the signature.</p><pre>// create the data which will generate the signature<br>const data = &quot;v0:&quot; + timestamp + &quot;:&quot; + body<br><br>// create a signature from the key and data<br>const signature = await crypto.subtle.sign(<br>  &quot;HMAC&quot;,<br>  key,<br>  new TextEncoder.encode(data) // encode the data<br>)</pre><p>Now we have our signature, it’s currently not a utf-8 string. It needs to be digested into a string for the last step.</p><pre>const result = new Uint8Array(localSignature)<br>  .reduce((digest, b ) =&gt; digest + b.toString(16).padStart(2, &quot;0&quot;), &quot;&quot;)<br>//=&gt; a2114d57b48eac39b9ad189dd8316235a7b4a8d21a10bd27519666489c69b503 </pre><p>We now have our signature, and it would be much easier if Slack left it at that. But they have one finishing touch before sending it along with the request.</p><pre>const generatedSignature = &quot;v0=&quot; + result</pre><p>Yep. To denote the version of the signature, it is prefixed onto the signature like this. This makes it (only slightly) tricker when it comes to verifying.</p><p>Slack’s own documentation would lead you to following the above steps and then use a time-safe comparison function to verify the values match.</p><pre>// if only...<br>crypto.timingSafeEqual(signature, generatedSignature)<br>//=&gt; true</pre><p>Unfortunately timingSafeEqual doesn’t exist on <a href="https://developer.mozilla.org/en-US/docs/Web/API/Web_Crypto_API">Web Crypto</a> and is not available in Vercel’s edge runtime. This means we must rely on crypto.subtle.verify — this is slightly problematic as it is not a comparison of two values in a time-safe manner. This is an abstracted way of doing all of the above, data does in, boolean comes out. So whats the problem? Let’s look at what this would mean if you took the documentation at face-value:</p><pre>const enc = new TextEncoder()<br><br>const isSigned = crypto.subtle.verify(<br>  &quot;HMAC&quot;,<br>  key, // generated from importKey and the secret<br>  enc.encode(signature), // signature from header<br>  enc.encode(`v0:${timestamp}:${body}`) // creating the data that was signed<br>)<br>//=&gt; ERROR<br>  </pre><p>This doesn’t work. We’ve passed in the correct items, but the signature is a hex string, whereas it needs to be a buffer. This function will soft that out:</p><pre>function hexToBuffer (hex: string) {<br>  const matches = hex.match(/[\da-f]{2}/gi) ?? []; // grab hex pairs<br>  const { buffer } = new Uint8Array(matches.map((h) =&gt; parseInt(h, 16))<br>  return buffer;<br>}<br>    </pre><p>Let’s try that again, no need to encode the signature after it has been transformed with this function, as it is already a buffer.</p><pre>const enc = new TextEncoder()<br><br>const isSigned = crypto.subtle.verify(<br>  &quot;HMAC&quot;,<br>  key, // generated from importKey and the secret<br>  hexToBuffer(signature), // signature from header<br>  enc.encode(`v0:${timestamp}:${body}`) // creating the data that was signed<br>)<br>//=&gt; false</pre><p>It’s no longer returning an error — but it’s not working as expected, whats the difference?</p><p>Let’s imagine whats going on inside the verify code:</p><pre>// this is pseudo/an example. do not copy<br><br>const slackSignature = <br>  &quot;v0=a2114d57b48eac39b9ad189dd8316235a7b4a8d21a10bd27519666489c69b503&quot;<br><br>const localSignature = await crypto.subtle.sign(<br>  &quot;HMAC&quot;,<br>  key,<br>  new TextEncoder.encode(data) // encode the data<br>)<br><br>const result = new Uint8Array(localSignature)<br>  .reduce((digest, b ) =&gt; digest + b.toString(16).padStart(2, &quot;0&quot;), &quot;&quot;)<br>//=&gt; a2114d57b48eac39b9ad189dd8316235a7b4a8d21a10bd27519666489c69b503<br><br>slackSignature === localSignature<br>// =&gt; false</pre><p>Can you see it now? Because Slack prefixed that pesky v0= we’re now comparing the wrong things. If we remove that prefix and just compare signature to signature it should work just fine!</p><pre>const enc = new TextEncoder()<br><br>const isSigned = crypto.subtle.verify(<br>  &quot;HMAC&quot;,<br>  key, // generated from importKey and the secret<br>  hexToBuffer(signature.substring(3)), // signature from header without v0=<br>  enc.encode(`v0:${timestamp}:${body}`) // creating the data that was signed<br>)<br>//=&gt; true</pre><p>And there it is! Verifying Slack signatures using the SubtleCrypto module in the edge runtime. Full snippet below for your copy-paste pleasure.</p><pre>async function isSignedBySlack(<br>  slackSignature,<br>  timestamp,<br>  body,<br> ) {<br>  const enc = new TextEncoder();<br><br>  const key = await crypto.subtle.importKey(<br>    &quot;raw&quot;,<br>    enc.encode(SLACK_SIGNING_SECRET),<br>    { name: &quot;HMAC&quot;, hash: &quot;SHA-256&quot; },<br>    false,<br>    [&quot;sign&quot;, &quot;verify&quot;]<br>  );<br><br>  return crypto.subtle.verify(<br>    &quot;HMAC&quot;,<br>    key,<br>    hexToBuffer(slackSignature.substring(3)),<br>    enc.encode(`v0:${timestamp}:${body}`)<br>  )<br>}</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=45c1a1d2b33b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>