<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jon Mulligan on Medium]]></title>
        <description><![CDATA[Stories by Jon Mulligan on Medium]]></description>
        <link>https://medium.com/@jon.mulligan?source=rss-84f5da7d378e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*V8eD9Z1wnWy4zvLXJ3hWIQ.jpeg</url>
            <title>Stories by Jon Mulligan on Medium</title>
            <link>https://medium.com/@jon.mulligan?source=rss-84f5da7d378e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 17:29:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jon.mulligan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Modern networking in large React applications — How we decided on, and extended SWR]]></title>
            <link>https://medium.com/eureka-engineering/modern-networking-in-large-react-applications-how-we-decided-on-and-extended-swr-6fe8268c10d4?source=rss-84f5da7d378e------2</link>
            <guid isPermaLink="false">https://medium.com/p/6fe8268c10d4</guid>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[software]]></category>
            <dc:creator><![CDATA[Jon Mulligan]]></dc:creator>
            <pubDate>Thu, 12 Dec 2024 02:41:28 GMT</pubDate>
            <atom:updated>2024-12-12T02:49:39.821Z</atom:updated>
            <content:encoded><![CDATA[<h3>Modern networking in large React applications — How we decided on, and extended SWR</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ELvsi8SvFyY5yRGZwj3TXQ.png" /></figure><p>This is the 12th article of the Pairs Engineering Advent Calendar 2024.<br>Check out <a href="https://medium.com/eureka-engineering/nfc-how-it-works-700061c750d7">yesterday’s article about NFC</a>.</p><p>Networking and data caching are important parts of any modern website or web application. But for a large single-page application (SPA), it can create a number of interesting challenges.</p><p>In this article, you’ll read how we settled on SWR as the networking library at Pairs, the steps we took to get there, and how we extended it to fit our application’s specific set of challenges. You’ll read about how we added type safety, improved reusability, and built an advanced cache mechanism.</p><h3>Challenges</h3><p><a href="https://www.pairs.lv/">Pairs</a> is an online dating service based in Japan, which has a diverse range of useful features and screens from which users can find and match with their future partner. Applications like this come with a unique challenge for networking and data caching: <strong>data consistency</strong>.</p><p>With many different and exciting ways to discover and match with users, our client-facing API interface has evolved such that it is highly optimized for each view, but not strictly RESTful. Many API endpoints return lists of User objects (<em>user profiles</em>) which can appear in numerous sections/features of the app throughout a single session.<br>With our API also returning dynamic data such as <em>“online status”</em>,<em>“match” / “favorite”</em> states, and other user-generated content, client-side caching requires careful thought and attention.</p><h3>Existing Architecture</h3><p>With the current iteration of the Pairs web application being launched over six years ago, our networking stack was stuck in a slightly-dated Redux world.</p><p>Our store was essentially our network cache, and data was meticulously organized within the object tree by feature and view. But importantly, we used <a href="https://github.com/paularmstrong/normalizr">Normalizr</a> to deduplicate and “normalize” our user objects separately of the parent view’s data. <br>Essentially, our store would only keep the latest version of each user object, regardless of which API endpoint it came from, replacing the previously cached value on each update.</p><p>Whilst data consistency was not an issue, we ran into a number of maintenance issues of our Redux store:</p><h4>Highly Repetitive Code</h4><p>Because data was stored in our store, each network endpoint we added required defining a new <em>Action Creator</em>, <em>Reducer</em>, <em>Selector</em>, API access code, and a data <em>converter</em>. <br>Despite numerous helpers it was still repetitive, requiring 5 or 6 files for the full request.</p><h4>Painful Upgrades and Refactoring</h4><p>Due to the structure of our data, and large number of files involved, changes and upgrades to our API code was painful and time consuming. <br>For requests that implemented Normalizr, we had to manually change the logic each time.</p><h4>No Standard for Optimistic UI and Error Feedback</h4><p>A now common practice of modern web applications; <em>“Optimistic UI”</em> and proper Error Feedback were tricky to implement and non-standard. As the application matured and we built out our design system, we had standards that were not easily achievable due to technical limitations.</p><p>In addition to the above issues, the implementation was entirely custom. Whilst simple rules/standards had been established to ensure commonality, it required extensive documentation, and maintenance became a chore.</p><h3>Picking a Library: SWR</h3><p>The main catalyst for our migration to a more modern networking stack was the desire for a more responsive UI. <br>Even though network requests take time to complete, we can increase the <em>perceived performance</em> of our application by immediately providing feedback to an action, by either “optimistically” updating the state while we wait for the server response, or by showing some sort of skeleton UI. <br>We also wanted to avoid cases where we show a blank UI whilst loading new data.</p><p>You’re likely aware that loading states and immediate-feedback are all possible with most modern networking libraries. With the advent of modern React hooks, request state is kept very close to the UI, and is basically cost-free to implement. <br>Additionally, the past few years has seen libraries converge on a few common caching strategies.</p><p><a href="https://github.com/vercel/swr">Vercel’s “SWR”</a>, named after the popular caching strategy <em>“stale-while-revalidate”</em>, does exactly what its name suggests. Data is first returned from the cache (stale) while the latest value is fetched again from the server (revalidate).<br>Behavior-wise this is exactly what we want for our application, and is very similar to our existing manual implementation.</p><p>The SWR library was first discussed during a team study session. It was well documented, and the hook interface was fairly logical and easy to understand:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/289779fcd12ce348007c033b0092352c/href">https://medium.com/media/289779fcd12ce348007c033b0092352c/href</a></iframe><p>Direct access to the data, isLoading state and error state makes UI implementation super easy. Furthermore, easy access to a bound-mutate function allows for the implementation of optimistic UI updates in our React code. <br><em>(No more UI logic in Redux actions!)</em></p><p>SWR is also highly configurable. <br>The default cache provided by SWR and almost all other modern hook-based libraries works perfectly for RESTful API interfaces. But as we covered earlier, not all of our APIs follow this pattern — data can be duplicated across multiple endpoints. SWR’s option to specify a custom caching mechanism gives us the ability to write a custom caching algorithm.</p><p><em>N.B. Strictly speaking, the ability to rewrite the cache mechanism is officially designed to override the in-memory </em><em>Map with say, Local Storage. However as you keep reading this article, you’ll find that we can use this to create a more advanced cache mechanism.</em></p><h3>Upgrading our Stack</h3><p>Whilst we’d settled on the fact that SWR was a likely candidate for our new architecture, it still took some time to think through what we wanted to achieve with the change. <br>Unnecessary change just creates noise in such a busy project, so we took time to discuss and evaluate the plan ahead, with a short time spent prototyping the idea and testing the theory.</p><h4>Standardized Code</h4><p>An orthodox use of SWR was trailed in an auxiliary feature of the Pairs application with a good degree of success. However, providing an endpoint URL and custom fetcher for each hook instance didn’t satisfy our requirement for easy maintenance.</p><p>In a small application with only a few dynamic requests, writing your API code inline using React hooks is absolutely fine. However, this quickly becomes an issue as the application scales. It also limits testability.</p><p>So we asked the question: <em>“What is a network request?”.</em></p><p>A network request in Pairs — much like any other application — performs an authenticated HTTP GET/PUT/POST request to a target URL, with a known response type, and a known client-side type. <br>With this knowledge — and a personal penchant for <em>“config over code”</em> — we decided on defining all of our network endpoints <em>(of which there are now over 160 configured)</em> using a standard object that we call an <em>“API Fetcher Definition”</em>.</p><p>What was born was a strongly-typed representation of each network request:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d69a7d523e3b61fc25e212f537360767/href">https://medium.com/media/d69a7d523e3b61fc25e212f537360767/href</a></iframe><p>The above example shows a getUser definition, with all of the required HTTP request parameters, information about the server response/client-side types, and a type-safe converter.<br>This converter can be as simple or complex as required by the implementation. In the above example, it converts JSON-serializable types to JavaScript types such as Date or a TypeScript Enum.</p><p>With the ability to define our network requests in an object, we wrapped the SWR hooks with a custom implementation that we’re calling useAPI(). The core interface of the hook is the same as SWR, however instead of passing a URL and fetcher into the hook, we pass our getUser fetcher definition. This pattern is infinitely more maintainable, as the definition can be re-used as many times as needed.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/be801f5773e2c982ca3e18e93bb1d365/href">https://medium.com/media/be801f5773e2c982ca3e18e93bb1d365/href</a></iframe><p>Furthermore, the setting of auth headers and other system-level properties is made easy by the now standardized internals of the useAPI() hooks. All requests with the requireAuth flag have the correct headers automatically added, and <em>401</em> authorization errors are automatically handled according to our application spec.</p><h3>Trying it Out</h3><p>By mid 2023, we had prototyped and proven the ability to make network requests from React hooks, using type-safe and reusable definition objects. A design refresh and rebuild of the “My Tag” community feature was a good opportunity to try it out.</p><p>Overall, it went very well and we got some good hands-on experience with the new format. We also refined the object typing a little, and added the ability to define paginated requests. <br><em>(Courtesy of the </em><em>useAPIInfinite() hook, which is in-turn a wrapper for </em><em>useSWRInfinite().)</em></p><p>Despite its importance, entity deduplication and normalization actually came slightly later in the roadmap. This is because we wanted to avoid the added complexity during the initial transition. And actually, the final solution to the normalization problem evolved differently to how we originally intended.</p><p>SWR’s cache is essentially an HTTP response cache, meaning that the cache operates as a key-value Map, in which an entry’s <em>key</em> is the request URL, and the <em>value</em> is the response data.<br>This is fine for most cases, but would be the cause a temporary UI inconsistencies in our non-RESTful app, where the /community/users endpoint response contains one version of a user object, and /user/:id contains a newer version.</p><p>We had originally planned to use cache mutators to update related cache items when a User entity changes. However, the1:N design of our API (users can belong to many “My Tag” communities) meant that the update scope was indeterministic.<br>On receiving of fresh user data, we’d have to scan the entire cache and update N amount of community response caches to propagate the change.</p><p>As the name suggests, SWR will automatically revalidate the community user list upon revisiting the page, so any changes to the user object would eventually be consistent with the latest value, even without manually updating the cache. However, the network delay would get in the way of “app-like” experience that our users expect. <br>This became particularly noticeable when “joining” a My Tag community or adding a user to your favorites. The UI immediately reacts to the user action. But navigating back to a previous page would temporarily show the old state where that page’s API cache was still storing the older data.</p><h4>Entity Normalization: Prototype</h4><p>Given the above, and the growing complexity of the project’s APIs, we took inspiration from Normalizr and decided to write custom cache. <br>Just like the initial implementation of SWR, we started with a prototype and enabled it for a subset of requests.</p><p>Essentially, we wanted to keep SWR’s HTTP response cache mechanism but deduplicate standard entities such as User and Community. This would give us all of the benefits of the view-optimized API design (data returned from our API hooks is already in a convenient structure for the view), but the individual entity objects would always be the latest known version even if the cached HTTP response is very old.</p><p>The process for data normalization is fairly simple: Our input data is passed through a function that finds entities such as User and Community, and replaces them with a token. Then we store the tokenized version of the object and the entities in separate stores.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zUzjlF7v78WqJteR2epF2Q.png" /><figcaption>Data “normalization” flow</figcaption></figure><p>When reading back the value, we must read the normalized data and replace the tokens with the actual entity values from the separate entity store. This will return us data that is equivalent to the input value. <br>However, here’s the magic of the normalization process; since the entity cache can only store one version of each entity, any updates automatically propagate to each response.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j_NrecKtdiEmxrkz_mKVtA.png" /><figcaption>Data “denormalization” flow</figcaption></figure><p>For anyone that’s done database design in the past, this should be fairly familiar. It’s akin to storing your data in a separate table, unique by ID as our primary key.</p><h4>Entity Normalization: Implementation</h4><p>The idea behind normalization and denormalization is fairly simple, yet the implementation has a number of complexities. But if you’ve read this far, I imagine you’re interested in the details. I will be skipping over detailed specifics of the implementation, but will give you an overview of how it works.</p><p>At the core of the normalization/denormalization process is a recursive algorithm that reads and clones an object. This behaves very similarly to any deepClone implementation.</p><p>The following diagram shows the DeepNormalize algorithm, which runs every time an HTTP response is written to the cache. It clones the response just like a deepClone, but with a special extra step that uses a lightweight heuristic (a specially named key) to determine if an included object is a User or Community type.</p><p>Upon discovery of one/more of these entity types, they’re replaced with a token representation of the entity (the “Store Entity” and “Return token” steps).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bUeR0b8i91ZO3V_yDMQZwg.png" /><figcaption>The “DeepNormalize” algorithm</figcaption></figure><p>Next, the DeepDenormalize algorithm runs every time SWR fetches data from the cache. The timing of this is controlled by the SWR library implementation, however the value is memoized within the hook.</p><p>Just like the normalization process, the object is read back and cloned. But when it encounters one of our entity tokens, it’s replaced with the value from our entity store (the “Fetch Entity” step).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JvdYMFwGD0MdwKbZcxflbw.png" /><figcaption>The “DeepDenormalize” algorithm</figcaption></figure><p>This feels like a lot of work to happen every time a cache item is read or updated, but React’s <em>vDOM</em> and Redux’s <em>reducers</em> behave in more or less the same way. <br>In practice, this runs very well on both desktop and mobile devices, and introduces no noticeable lag.</p><p>Furthermore, this implementation is highly testable and predictable.<br>Day-to-day feature development requires no thought about normalization, yet this custom cache implementation wrangles the data automatically.</p><h3>A Layered Approach</h3><p>Our approach to the implementation of SWR takes the form of separate layers, where each is built and tested in isolation. This allowed us to trial the SWR library in our application without fully fleshing out the implementation from day one.</p><p>Additionally, maintenance is made easy by the separation of the hook implementation, custom cache implementation, and the base “fetcher” request code.<br>Yet when used together, they become the backbone of our entire application’s network and data architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TPS29soFOOFmeV-X25BGEQ.png" /><figcaption>A diagram showing the full lifecycle of request data from the server, showing each layer of the architecture in isolation.</figcaption></figure><h3>Summary</h3><p>This article was less a specific how-to, than it was a story about our implementation process. But we covered some key points about choosing a library that’s desirable to use, yet extensible for our own use-case.</p><ol><li>We chose SWR for its hook interface, and its standardized cache mechanism. However, the default lack of type-safety and low code reusability led us to write our own wrapper.</li><li>The useAPI() hooks and “Standardized request code” (fetcher) shown in the diagram above were developed together to provide a strongly-typed network request configuration object. These wrap the core SWR library in an interface that is much more useful to our large-scale application.</li><li>The “Response Normalization” and “Cache Denormalization” algorithms were additionally developed as a facade over the base SWR network cache. By utilizing the base network cache, but deduplicating individual entities, we were able to provide a much more stable UI experience to our users with no burden on day-to-day development.</li></ol><h3>Postscript</h3><h4>In review of SWR</h4><p>SWR is great, and generally well documented, but there were a few things that I wish were different:</p><ul><li>The term <em>“mutate”</em> is way over-used, and makes discussions confusing. Be careful of loaded terms.</li><li>useSWRMutation() should probably be called something else. It’s sometimes necessary to lazy-trigger requests, not necessarily trigger a mutating POST or PUT as the name somewhat implies.</li><li>Despite <a href="https://swr.vercel.app/docs/revalidation#revalidate-on-mount">documentation specifically on the topic</a>, the mix of revalidateOnMount and revalidateIfStale options can sometimes cause confusion.</li></ul><h4>Why not TanStack Query?</h4><p>TanStack Query is an absolutely fantastic library and we strongly considered using it for Pairs. However, the inability to specify a custom cache implementation proved to be a limitation too great. <br>SWR is simple yet allowed us to configure it just how we needed.</p><p><em>*Cover image photo by </em><a href="https://unsplash.com/photos/a-black-and-white-photo-of-a-tunnel-NfLZeAN7I6s"><em>Martin Sanchez</em></a><em> on Unsplash.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6fe8268c10d4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/eureka-engineering/modern-networking-in-large-react-applications-how-we-decided-on-and-extended-swr-6fe8268c10d4">Modern networking in large React applications — How we decided on, and extended SWR</a> was originally published in <a href="https://medium.com/eureka-engineering">Pairs Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[React Without the State Machine]]></title>
            <link>https://medium.com/eureka-engineering/react-without-the-state-machine-ce4fd25a6180?source=rss-84f5da7d378e------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce4fd25a6180</guid>
            <category><![CDATA[react]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[software-architecture]]></category>
            <dc:creator><![CDATA[Jon Mulligan]]></dc:creator>
            <pubDate>Wed, 20 Dec 2023 02:20:44 GMT</pubDate>
            <atom:updated>2023-12-20T02:20:44.519Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7sj8ozH4tCNQFluXWULweQ.png" /></figure><p>This article is posted as day 20 of the <a href="https://qiita.com/advent-calendar/2023/eureka">Eureka Advent Calendar 2023</a>. If you’re interested in hybrid conference production, check out <a href="https://medium.com/eureka-engineering/%E8%A6%96%E8%81%B4%E7%94%BB%E9%9D%A2%E3%81%AE%E5%90%91%E3%81%93%E3%81%86%E5%81%B4%E3%81%AE%E3%83%8F%E3%82%A4%E3%83%96%E3%83%AA%E3%83%83%E3%83%89%E9%85%8D%E4%BF%A1%E6%9C%AA%E7%B5%8C%E9%A8%93%E8%80%85%E3%81%8C%E7%B5%8C%E9%A8%93%E3%81%97%E3%81%9F%E4%B8%80%E5%B9%B4-708155406c5f">Buzz’s article on day 19</a>! (In Japanese language)</p><p>Here at Eureka, my rants about Redux have become a bit of a meme.</p><p>It feels like most of my recent team study presentations have focused on rewriting code without Redux — a topic that I usually preface with the phrase <em>“I don’t actually hate Redux.”</em></p><p>This is genuinely the case. However, I believe that thoughtful architecture, and the use of React Context can help us build more maintainable applications.</p><h4>“I don’t actually hate Redux. I promise!”</h4><p>Before everyone gathers around with pitchforks, I want to say that Redux is a fantastic state management library, and I use it in a bunch of personal projects when autonomy and cross-environment state have been required.</p><p>There are many great justifications for its use on the <a href="https://redux.js.org/">Redux homepage</a>:</p><ul><li><strong><em>Predictable:</em></strong><em> Redux helps you write applications that behave consistently, run in different environments (client, server, and native), and are easy to test.</em></li><li><strong><em>Centralized:</em></strong><em> Centralizing your application’s state and logic enables powerful capabilities like undo/redo, state persistence and much more.</em></li><li><strong><em>Debuggable:</em></strong><em> The Redux DevTools make it easy to trace when, where, why, and how your application’s state changed.</em></li><li><strong><em>Flexible:</em></strong><em> Redux works with any UI layer, and has a large ecosystem of addons.</em></li></ul><p>What’s more, it’s tried and tested. As of writing this article, it has over 9,000,000 weekly downloads on NPM, and is used on a huge number of popular websites and services.</p><p>However, <a href="https://redux.js.org/faq/general#when-should-i-use-redux">as documented in the Redux docs</a>, it’s not necessarily for every application.</p><h4>React has Changed. However, a Legacy Lives on…</h4><p>There are many historical reasons why we use some sort of store to do most of the heavy lifting in our React applications, but one beats them all;<br><em>“Prop drilling”</em> was/is highly discouraged. And rightly so. Passing countless props through a chain of components creates a composition nightmare. Implicit links are created between what would otherwise be transparent/reusable components.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h-Sc9qSCdFKbPZ6j9Zu5aA.png" /><figcaption>Storing state and logic in a high-level container is good practice. But then showing the data somewhere deep in our component tree requires “prop drilling” through many layers.</figcaption></figure><p>For a very long time — <a href="https://github.com/facebook/react/blob/0.9-stable/src/core/ReactContext.js">since at least version 0.9</a> — , React included a <em>“Context”</em> API which allowed us to make select props available to <strong>all child nodes</strong>. However, this API was experimental, not friendly to use, and never considered stable or safe for production use.</p><p>Even so, the early <a href="https://github.com/facebookarchive/flux/tree/main">Flux</a> library utilized this to make its data store globally accessible. These early store libraries — and nowadays, Redux—helped make our components much tidier and offered many quality of life improvements. Unidirectional data flow was enforced by their design, which enabled good practice.</p><p>Since React 16.3 (released in March 2018), a much more friendly and production-ready <em>Context</em> API was released. However, the store pattern had become a common tool, and the future of React application development was decided: <br><em>We can avoid prop drilling by using an external store…</em></p><h4>Most Web Applications Aren’t Monolithic State Machines</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6mg1IdyvmwBZKa6iSzIxKw.png" /><figcaption>Modern applications are rarely a giant control panel with every possible action and data value visible at once. (Source: Magnox — Pictured: Sizewell nuclear power station)</figcaption></figure><p>Most web services — and mobile applications — aren’t monolithic state machines. They are often layered and contextual. Content is viewed, and actions performed within the <em>context</em> in which the user found them.</p><p>Often, the state that drives these views has no need to exist until the user sees it, nor does it need to exist after said view has been closed. I personally believe that almost all state put in a Redux store would be better off in a more selective, bespoke context.</p><p>Data storage in a global store can even be the cause of repetitious code smells. How many times have you appeased the Typescript compiler by adding a <em>nullish</em> (null or undefined) check to the result of a <em>useSelector()</em> hook, knowing full-well that it’ll be defined during render?</p><p>A simple, yet common example of this is storing data about the currently authenticated user in a store:</p><p>We have a “<em>currentUser</em>” object in our store, which prior to logging in is <em>undefined</em>. <br>When displaying the user’s name, or other user property in our React UI, each and every use of the <em>getCurrentUser</em> selector must include a check for <em>undefined</em>, due to the default state.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b40000843916ba7f955a80f7317ff1e1/href">https://medium.com/media/b40000843916ba7f955a80f7317ff1e1/href</a></iframe><p>But we know — and more importantly React can know — that none of these views will ever be rendered before you log in.</p><p>A Redux <em>store</em>, its <em>actions</em>, <em>reducers</em> and <em>selectors</em> exist in a world without context of their usage. It’s an independent state machine trying to drive a UI that it has no knowledge about.</p><p>For this reason, an action creator, or UI component can have complex logic and logic forks to handle any of the infinite permutations of the store’s state.</p><p>On action dispatch a complex logic tree is executed:<br>“Am I logged in?”<br>“Is <em>A</em> <em>undefined</em>”<br>“Is arbitrary flag <em>B</em> true?”<br>“How about flag <em>C</em>?” <br>“If so, have we confirmed <em>X</em>?”<br>All to show some feedback in the UI before starting the cycle again.</p><p>Worse still; if we forget to reset all temporary state when finished, repeat runs can lead to horrible, undiscoverable bugs.</p><p>This isn’t a problem with Redux and the store pattern, but a problem with how it is often used.<br>Code like this rarely exists from day 0. It’s written with good intention but evolves over time and changes with requirements. It gradually becomes unwieldy and incomprehensible.</p><p>I eel like the scenarios described are the exact opposite of <em>“Predictable”</em> and <em>“Debuggable”</em>.</p><h3>React doesn’t have to be Reactionary</h3><p>In the prior examples, our code had to be defensive and reactionary, it had to handle all possible states.</p><p>But what if our applications were structured in a way in which we can create more predictable states, based on logical rules?<br>Things like: “We will only access the <em>currentUser</em> object from a context in which we are logged in.” and “We will only look at <em>messages</em> from the messaging screen.”</p><p>Modern applications with even the most simplistic or naive of IAs can offer developers guidance on how we can add context to our code and keep it segmented. Our applications are rarely a giant control panel with all buttons and data visible at once, so why should we store our data like that?</p><p>The easiest place to start would be to look at the URLs of your app. Provided these have sensible and logical structure, they can be a great hint at how to segment and contextualize your code.</p><p>By adding context at appropriate points of our component tree, we can use conditional rendering to protect children from rendering in an “invalid” state.</p><p>We can think of things at a macro level. Contexts don’t have to encompass the entire page.<br>In a split-view messaging screen where conversations are listed in one column, and the selected conversation’s messages are shown in another, we may choose to create another context here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JR0kKoqm5M0eAanAncoQxg.png" /><figcaption>Here we have a very basic messaging UI where out state is layered and contextualized. Our topmost context, the “User Context” (green) holds information about the current user. Next, we have the “Messages Context” (blue) which contains information about our conversations. Finally, our active conversation has its own “Conversation Context” (orange). These contexts are where we will keep the majority of each layer’s business logic and state.</figcaption></figure><p>To solve our earlier problem of the <em>undefined</em> <em>currentUser</em> problem, we can store the <em>currentUser</em> object in a bespoke React Context whose provider conditionally renders the router for logged-in pages.</p><p>Because the routes will never be rendered if the current user is <em>undefined</em>, there’s no need to do a check for that when using the contextual value.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7c72fb229556de96bb10128290c8a0e1/href">https://medium.com/media/7c72fb229556de96bb10128290c8a0e1/href</a></iframe><p>In the example above, <em>useCurrentUser()</em> is a type-safe non-nullable reference to the current user which can be safely used in any of our <em>authorizedRoutes</em>.</p><p>The error thrown in the <em>useCurrentUser()</em> hook may look scary, but it’s a safety-net that ensures that we as developers don’t use the hook incorrectly.<br>Sometimes it’s ok to make noise about errors instead of failing silently.<br>In fact, masking errors and failing silently can be the cause of many production bugs that we would’ve otherwise caught during development.</p><h4>Disadvantages of this approach</h4><p>It’s worth noting that React state, and therefore its <em>contexts</em> cannot be used outside of the React renderer. Whilst this is most often a non-issue, some applications have a genuine reason for accessing and manipulating the state outside of the UI. In this case, other tools within React, or even entirely different patterns, allow us to achieve a similar result.</p><p>If the value of a context must be available outside of the React renderer for any reason, your custom context’s <em>Provider</em> can store its state in an external service, and trigger re-renders using event bindings or <a href="https://react.dev/reference/react/useSyncExternalStore">useSyncExternalStore</a>. Much like the React-Redux library does itself. The key difference being that we are selectively hoisting this state into an external store, thus keeping the majority of UI state in our UI layer.</p><h4>Unlocking possibilities</h4><p>Conditionally rendering context children gives us the opportunity to unmount and lazy-load entire trees of UI components, and business logic. One of the main problems of a monolithic store is that it can’t easily be split into separate bundles. Whilst the popular “<a href="https://github.com/erikras/ducks-modular-redux">Ducks</a>” and “<a href="https://redux-toolkit.js.org/api/createSlice">Slice</a>” patterns help us segment our store, actually splitting and lazy-loading these segments can be a large technical task. In general, the whole store, and all associated logic has to be available in our first bundle.</p><p>Even outside of application-wide issues, bespoke contexts can be a powerful pattern for small, self-contained components too! In our application, “Pairs”, we make use of React Context in a number of our re-usable components which would otherwise require heavy use of prop drilling.</p><figure><img alt="Screenshots of a “Wizard” component and “Tabs” component used in the Pairs application." src="https://cdn-images-1.medium.com/max/910/1*uu8dXlDKy0RLa9ssm-fQ9A.png" /><figcaption>The Pairs application uses React Context in a number of re-usable and flexible components which make use of component composition for their configuration.</figcaption></figure><p>With smart use of contexts, we can even abstract configuration to the component composition. In our custom “Wizard” component, steps are ordered, and their <em>isComplete</em> condition configured using the Step component alone. <br>With this design, re-ordering the steps is a configuration change. Each step need not care about the previous/next step as the arrow buttons operate contextually. There’s no need to carefully re-configure actions when changing step order.</p><p>(Note: this pattern is not achieved exclusively by the <em>Context</em> pattern. However the re-use and nestable nature of React <em>Contexts</em> makes this an incredibly powerful tool)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b31007f1d1dbbeb52d47d29fd30caa9c/href">https://medium.com/media/b31007f1d1dbbeb52d47d29fd30caa9c/href</a></iframe><h4>Conclusion</h4><p>The words written in this article shouldn’t be taken at face-value. Every application is different, and every use case unique. It’s by no means saying “Redux bad. React Context good.”<br>Many teams <strong>are</strong> successfully using Redux in their applications.</p><p>However, I have seen under the hood of many large projects using a global store, and heard engineers comically saying <em>“start the washing machine spin-cycle”</em> or <em>“throw that action into the wind and hope for the best”</em>.</p><p>Given Redux isn’t the only pattern available, I hope that this opens your eyes to some new solutions. As engineers, our goal is to write maintainable software. We can better do this by taking time to understand logical rules, and make thoughtful decisions about our architecture.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce4fd25a6180" width="1" height="1" alt=""><hr><p><a href="https://medium.com/eureka-engineering/react-without-the-state-machine-ce4fd25a6180">React Without the State Machine</a> was originally published in <a href="https://medium.com/eureka-engineering">Pairs Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image Processing with WebGL]]></title>
            <link>https://medium.com/eureka-engineering/image-processing-with-webgl-c2af552e8df0?source=rss-84f5da7d378e------2</link>
            <guid isPermaLink="false">https://medium.com/p/c2af552e8df0</guid>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[web-performance]]></category>
            <category><![CDATA[webgl]]></category>
            <dc:creator><![CDATA[Jon Mulligan]]></dc:creator>
            <pubDate>Fri, 10 Dec 2021 01:23:25 GMT</pubDate>
            <atom:updated>2022-01-13T09:17:06.261Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*If8tfHS0cGq1LrvurFz7Cw.png" /><figcaption>Photo by <a href="https://unsplash.com/@pawel_czerwinski?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Pawel Czerwinski</a> on <a href="https://unsplash.com/photos/ruJm3dBXCqw?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p><em>This is the December 9 article for </em><a href="https://qiita.com/advent-calendar/2021/eureka"><em>Eureka Advent Calendar 2021</em></a>.</p><p>Native applications have set a high standard for image editing and processing features that users have come to know and love. With features ranging from simple cropping, to photo filters and decoration.</p><p>With the exception of a few standout services, the standard for image editing on the web is unfortunately still pretty poor. In even some of the most popular applications and social networks, tools are either very limited or non-existent. Users are expected to revert to native counterparts to process and crop images before uploading to a web service.</p><p>As a keen photographer and web engineer, I have made it my challenge to create a portable image cropping and filtering library which is able to achieve native-like performance entirely on the client.</p><p>This has been a long and interesting journey over the course of a year that taught me a lot about 2D canvas rendering all the way down to low-level use of WebGL and shaders.</p><p>If any of this sounds interesting to you, read on and I’ll show you everything I wish I knew a year ago!</p><h3>ImageData interface</h3><p>Throughout this article, we will be dealing with images via the ImageData interface. It’s perhaps the simplest and most understandable way of representing an image, whilst also being very easy to manipulate.</p><p>I’ll skip the exact details of how to get an image as ImageData, but the simplest way involves creating an &lt;img&gt; tag to fetch image data, drawing that to a canvas and calling the getImageData() method on the canvas context.</p><p>The ImageData interface stores the image’s width, height and a UInt8ClampedArray of pixel data in a repeating RGBA sequence where each channel is represented as a number between 0 and 255. If that sounds familiar, that’s because it is — you’re probably used to writing colours as hexadecimal, where #ffffff means an RGB value of 255, 255, 255.<br>For all of the examples given in this article, I will be ignoring the alpha channel as I’m assuming our images are fully opaque. Also note that the pixel data in the ImageData interface is ordered top-left to bottom-right, matching the coordinate system of the browser . This will become a key piece of information later on.</p><h3>Image processing with HTML Canvas</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IA8jeSyIquwXBGzA1fZFhg.png" /></figure><p>Before delving into WebGL rendering, I think it’s worth briefly covering 2D canvas. The 2D canvas APIs are well established and widely supported. I’d also used it a number of times before so this was a good starting point for my research.</p><p>To process an image based on a set of input parameters (e.g. brightness, contrast, saturation etc.), we need to take the red, green and blue channels of each pixel and perform a transformation which applies the given input parameters. The simplest example of a transform increases the brightness of our input image:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7f345d40b24203194fc976e2bd463742/href">https://medium.com/media/7f345d40b24203194fc976e2bd463742/href</a></iframe><p>Using this pattern, a number of different filters can be created. An exposure filter multiplies each of the channels by an exponent. The simplest of contrast filters is a combination of the above two addition and multiplication examples.</p><p>But after chaining multiple transformations together, the code becomes heavy and definitely won’t run at a smooth frame rate in the browser. One way in which we could optimise this would be to represent each transformation as an <a href="https://en.wikipedia.org/wiki/Identity_matrix"><em>identity matrix</em></a><em>.</em> If you’ve ever written filters in SVG, you may be familiar with the <a href="https://developer.mozilla.org/en-US/docs/Web/SVG/Element/feColorMatrix">&lt;feColorMatrix&gt;</a> element which changes colours based on a transformation matrix. The neat thing about identity matrices is that they can be mathematically combined into a single transformation, therefore our loop would only have to run once.</p><p>The problem arises when we want to apply more complex filters that can’t be represented as simple transformations. My goal for this project is to replicate all of the filters available in the Google Photos app, where adjusting the <em>color temperature</em>, <em>tint</em>, <em>highlights</em> and <em>shadows</em> requires calculating <a href="https://en.wikipedia.org/wiki/Dot_product"><em>dot products</em></a> and converting between <a href="https://en.wikipedia.org/wiki/Color_space"><em>colour spaces</em></a>. <br>Performing all of these transformations across even a small image quickly blows our <a href="https://developers.google.com/web/fundamentals/performance/rendering">10ms frame time budget</a>.</p><p>This leads us on to…</p><h3>Image processing with WebGL</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rGkJNCdrqVN4OJJHKtmYjA.png" /></figure><p>In order to perform complex transformations whilst maintaining good performance, we’re going to hand off the work to the GPU.</p><p>Unlike a device’s CPU which is very good at performing a multitude of tasks, its GPU is much more specialised to perform highly parallelised floating point calculations. With WebGL we can break out of the single-threaded Javascript world and hand off our processing to the hundreds of shader cores inside the GPU.</p><p>WebGL is an almost 1:1 wrapper for the OpenGL ES 2.0 API which contains all of the power, and all of the quirks and misdirection that comes from an API that was originally written in the early 1990s. WebGL is quite unlike any of the standard object oriented browser APIs you’re familiar with.</p><p><strong><em>Note</em></strong><em>: </em><a href="https://webgl2fundamentals.org/webgl/lessons/webgl2-whats-new.html"><em>WebGL 2 </em></a><em>— being based on OpenGL ES 3.0 — includes a number of additional texture formats, and includes a few browser-specific additions. Everything that follows is compatible with both WebGL and WebGL 2.</em></p><p>In the following section, I will be using terminology that is often used in 3D rendering and the use of WebGL. Below are some of these terms which — in some cases — are overly-simplified, but accurate enough to the context of image processing.</p><h4>Vertex</h4><p>GPUs draw shapes from a list of points or <strong>vertices</strong> which form a triangle. Our list of <strong>vertices</strong> can be represented in either 2D or 3D space. Any surface that is drawn in WebGL must be made of triangles, regardless of the shape. This will be important later when we come to draw a square on the canvas; we will actually be drawing two triangles.</p><h4>Texture</h4><p>A <strong>texture</strong> is essentially an image file that has been uploaded to the GPU. An uploaded <strong>texture</strong> can then be read or sampled by a <strong>shader</strong> when drawing pixels on the screen. There are many more complexities and settings that <strong>textures</strong> can utilise such as <a href="https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/generateMipmap"><em>mipmapping</em></a>, but we will skip over these as they are not important to our work today.</p><h4>Shader</h4><p>Shaders are small pieces of code that tell the GPU how to draw each pixel. The two types of <strong>shader</strong> available in WebGL are <strong>vertex shaders</strong> and <strong>fragment shaders</strong>. These are written using <a href="https://webglfundamentals.org/webgl/lessons/webgl-shaders-and-glsl.html#glsl"><em>GLSL</em></a> (<em>Graphics Library Shader Language</em>), which you may find very similar to the C language.</p><p>A <strong>vertex shader</strong> takes each of our <strong>vertices</strong> which are represented in either 2D (x, y) or 3D (x, y, z) space and maps them to our 2D canvas. <br>These are important in 3D applications where your <strong>vertex shader</strong> is used to ‘project’ the position of each <strong>vertex</strong> onto your 2D monitor through use of a <a href="https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/WebGL_model_view_projection"><em>projection matrix</em></a>, creating depth and perspective.<em> </em>This is why games still look 3D when viewed on a 2D monitor.<br>It’s easiest to imagine that a <strong>vertex shader</strong> takes a shape and decides which pixels on the screen should be filled. As our image processing library is only dealing with 2D data, ours will be incredibly simple.</p><p>A <strong>fragment shader</strong> (sometimes known as a pixel shader) takes each pixel of our shape and decides what colour it should be. A simple <strong>fragment shader</strong> would return the same colour for each pixel, colouring the entire surface a solid colour. With this knowledge, you may be able to deduce that our <strong>fragment shader</strong> will have to sample pixels from a <strong>texture</strong> to draw our image.</p><h4>Program</h4><p>A <strong>program</strong> in WebGL is a combination of both a <strong>vertex shader</strong> and a <strong>fragment shader</strong> that when enabled, tells the GPU how to draw <strong>vertices</strong>.<br>This <strong>program</strong> exposes a series of <em>locations</em> (think memory locations) to which we can pass an ArrayBuffer of <strong>vertices</strong>, or any other parameter that our compiled <strong>shaders</strong> will use.</p><p><strong><em>Note</em></strong><em>: Scene rendering is not strictly limited to a single program; in some cases you may wish to run multiple programs sequentially to build up a scene.</em></p><p>With our simple knowledge of <strong>vertices</strong>, <strong>textures, shaders</strong>, and <strong>programs</strong> we are ready to start using WebGL!</p><h3>Drawing a Triangle</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BSaTc-LmdwsujfHvfNObZw.png" /></figure><p>For graphics programming, drawing a triangle is the equivalent of <em>“Hello World!”</em>.</p><p>It requires some <strong>shader</strong> programming and a fair bit of control code. There’s quite a few parts to it, but we’ll tackle it step by step.</p><h4>Vertex Shader</h4><p>In order to draw a triangle on the screen, we must first create a <strong>vertex shader</strong> to draw our vertices in screen space. Screen space in WebGL is represented as a floating point number from -1.0 to 1.0 .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a1069c619b2e55e9aea067cebe321add/href">https://medium.com/media/a1069c619b2e55e9aea067cebe321add/href</a></iframe><p>attribute vec2 position; Defines an attribute on our <strong>shader</strong>/<strong>program</strong>. Think of it like a parameter that will receive the 2D coordinate of each of our triangle’s <strong>vertices</strong>. <br>The main() routine runs once per <strong>vertex</strong> and sets gl_Position to our x and y coordinate (feel free to ignore the 0, 1.0 at the end as they are not used for simple 2D drawing).</p><h4>Fragment Shader</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5bcb01a216f17f8bfd13685101b4dbdc/href">https://medium.com/media/5bcb01a216f17f8bfd13685101b4dbdc/href</a></iframe><p>The main() routine of our <strong>fragment shader</strong> sets gl_FragColor to this lovely shade of magenta for each pixel of the triangle. The colour is represented as a vec4 as it contains all four RGBA channels.</p><p>You’ll also notice the precision highp float; definition at the top of the file. This tells our <strong>shader</strong> to run floating point calculations with high precision.</p><h4>WebGL Control Code</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/253ef9fed0eacd753ecd51650a796ca8/href">https://medium.com/media/253ef9fed0eacd753ecd51650a796ca8/href</a></iframe><p>There’s quite a lot going on here, so let’s break it down:</p><p>After creating a WebGL context, we compile our two <strong>shaders</strong> and attach them to a <strong>program</strong>. We then tell our WebGL context to use this as our active program.</p><p>We create a new Float32Array containing the 3 <strong>vertices</strong> of our triangle. This follows the format [x1, y1, x2, y2, x3, y3] . This array is then passed to the GPU via bindBuffer() and setting bufferData() . There are a number of parameters here, but I will skip over the details for brevity.</p><p>Next we call getAttribLocation() on our active program to find the <em>location</em> of our position attribute we defined in our <strong>vertex shader</strong>. When our shaders are compiled and linked to a <strong>program</strong>, our attributes are assigned a <em>location</em> from which we can access them.<br>With our newly fetched <em>location</em>, we can use our buffer as <strong>vertex</strong> data using enableVertexAttribArray() and tell WebGL to interpret it as 2D vectors using vertexAttribPointer(); .</p><p>Finally, our WebGL context has received all of the data it needs to draw the scene. So we clear the canvas, clear our COLOR_BUFFER_BIT and call drawArrays() , specifying that 3 <strong>vertices</strong> should be drawn as triangles, from a starting point of 0.</p><h4>The Result</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcodepen.io%2FJonymul%2Fembed%2Fpreview%2FLYLxEJp%3Fdefault-tabs%3Djs%252Cresult%26height%3D600%26host%3Dhttps%253A%252F%252Fcodepen.io%26slug-hash%3DLYLxEJp&amp;display_name=CodePen&amp;url=https%3A%2F%2Fcodepen.io%2FJonymul%2Fpen%2FLYLxEJp&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=codepen" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/7101750b02e09871e9066703dae37548/href">https://medium.com/media/7101750b02e09871e9066703dae37548/href</a></iframe><h3>Drawing a Square</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j5kC3XcSlV7r5EZk131Rdw.png" /></figure><p>Drawing a triangle in WebGL is great, but you may remember that we’re trying to build a photo editing application. <br>I don’t know about you, but I’ve not personally seen any triangular photos before!</p><p>To draw a square, we will need to draw two triangles. To do so, we will add another triangle to our VERTICES array and tell WebGL to draw 6 vertices instead of 3!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1c8c0e5bf804252cd1dc0662a7cce5af/href">https://medium.com/media/1c8c0e5bf804252cd1dc0662a7cce5af/href</a></iframe><h3>Mapping Colour</h3><p>Before we try to map a <strong>texture</strong> to our square, let’s first make some modifications to our <strong>shaders</strong>.</p><p>To each of our <strong>shaders</strong> we will add a varying vec2 texCoords; . Think of this like a program variable that our <strong>vertex shader</strong> will write to and then our <strong>fragment shader</strong> will read from.</p><h4>Vertex Shader</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/958d3a7a0036ac78423019a69655760a/href">https://medium.com/media/958d3a7a0036ac78423019a69655760a/href</a></iframe><p>If you remember from earlier, WebGL’s screen coordinates are represented as a floating point number from -1.0 to 1.0. To make our lives easier, we first convert this range to 0.0 to 1.0 before setting it as the value of our texCoords variable.</p><h4>Fragment Shader</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c9518f38ef63eeff575a6db24323797f/href">https://medium.com/media/c9518f38ef63eeff575a6db24323797f/href</a></iframe><p>Instead of returning a solid magenta colour, our <strong>fragment shader</strong> now uses the texCoords x component as our red channel, y component as our blue channel, and finally our green channel as 1.0.</p><h4>The Result</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcodepen.io%2FJonymul%2Fembed%2Fpreview%2FpowRJKo%3Fdefault-tabs%3Djs%252Cresult%26height%3D600%26host%3Dhttps%253A%252F%252Fcodepen.io%26slug-hash%3DpowRJKo&amp;display_name=CodePen&amp;url=https%3A%2F%2Fcodepen.io%2FJonymul%2Fpen%2FpowRJKo&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=codepen" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/1e75b4d518e7ee6b32cff7251509ec56/href">https://medium.com/media/1e75b4d518e7ee6b32cff7251509ec56/href</a></iframe><p>Try reading the <strong>fragment shader</strong> in detail and imagine how the strength of the red component increases as the x value increases. Likewise, the green component increases as the y value increases.</p><h3>Using Textures</h3><p>Now we’ve successfully created a square in our scene, we’re ready to show an image.</p><h4>Loading ImageData as a Texture</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f18f03c825dec84abe8cf37a3d072096/href">https://medium.com/media/f18f03c825dec84abe8cf37a3d072096/href</a></iframe><p>With some more confusing WebGL magic, we’re able to pass an image from Javascript as ImageData to our WebGL context as a <strong>texture</strong>.</p><p>In short, we define a 2D <strong>texture</strong> in slot TEXTURE0 , set the texture data to our image, using the colour mode RGBA .</p><p>Following this, we can set a number of parameters such as <em>wrapping</em> mode, and <em>interpolation</em> mode (MIN_FILTER and MAG_FILTER). Wrapping mode isn’t important for our use case, but you may wish to experiment with the <a href="https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/texParameter">different <em>interpolation</em> modes</a>.</p><h4>Fragment Shader</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fc4f05ccb1ca1cbdec2871769112235e/href">https://medium.com/media/fc4f05ccb1ca1cbdec2871769112235e/href</a></iframe><p>We create a sampler2D which will be used to sample colour data from the <strong>texture</strong> we previously set as the <em>active texture</em> in slot TEXTURE0.</p><p>As you remember, our <strong>fragment shader</strong>’s main() routine runs once for each pixel of the output canvas. So for each pixel, we use the texture2D function to sample the <strong>texture</strong> at our current coordinate and set this as our gl_FragColor.</p><h4>The Result</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcodepen.io%2FJonymul%2Fembed%2Fpreview%2FXWgpmXg%3Fdefault-tabs%3Djs%252Cresult%26height%3D600%26host%3Dhttps%253A%252F%252Fcodepen.io%26slug-hash%3DXWgpmXg&amp;display_name=CodePen&amp;url=https%3A%2F%2Fcodepen.io%2FJonymul%2Fpen%2FXWgpmXg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=codepen" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/86e612e9e688dcf04c881e134cd84686/href">https://medium.com/media/86e612e9e688dcf04c881e134cd84686/href</a></iframe><p>The result is a square with our beautiful texture rendered upside down! You may recall that earlier in the article where I mentioned that browser coordinates run from top-left to bottom-right. For everyone’s sanity, Browser APIs in Javascript represent image data in the same order.</p><p>Because WebGL coordinates are bottom-left to top-right, our image data is sampled with the y-axis inverted!</p><p>This is fortunately a simple fix, by inverting texCoords.y in our <strong>vertex shader</strong>. Open the above Codepen source and try it for yourself!</p><h3>Finally, Applying a Filter</h3><p>Now we’ve rendered our image in our WebGL canvas <em>(correct side up!)</em>, we are ready to transform the pixel data.</p><p>All of our transformations will happen within the <strong>fragment shader</strong>.</p><h4>Fragment Shader</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/05164d40651c8abc133966bef6d9b7a2/href">https://medium.com/media/05164d40651c8abc133966bef6d9b7a2/href</a></iframe><p>We’ve added a number of transformation functions adjustBrightness() , adjustContrast() and adjustSaturation() to our <strong>shader</strong> and call them within our main() routine.</p><p>After sampling the <strong>texture</strong>, we can sequentially adjust the RGB components of our colour using the transformation functions before setting gl_FragColor .</p><p>You may have noticed how the adjustSaturation() function uses the mathematical dot() (<a href="https://en.wikipedia.org/wiki/Dot_product"><em>dot product</em></a>) function and WebGL mix() function that aren’t available in Javascript. These can absolutely be replicated in Javascript, but the GPU can handle these with ease.</p><p>If you wanted to take this example a step further, you could store each of the filter parameters as an attribute, and then set them from our Javascript code. This is exactly what you’d do if you wanted to add UI controls for each of the values.</p><h4>The Result</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcodepen.io%2FJonymul%2Fembed%2Fpreview%2FRwgKWQz%3Fdefault-tabs%3Djs%252Cresult%26height%3D600%26host%3Dhttps%253A%252F%252Fcodepen.io%26slug-hash%3DRwgKWQz&amp;display_name=CodePen&amp;url=https%3A%2F%2Fcodepen.io%2FJonymul%2Fpen%2FRwgKWQz&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=codepen" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/d80a051038bb444e9ef0e439600f7675/href">https://medium.com/media/d80a051038bb444e9ef0e439600f7675/href</a></iframe><p>Feel free to open the Codepen sample and play with the values. Maybe even try writing your own transform filter!</p><p><strong><em>Note</em></strong><em>: Whilst there are often simplified alternatives, the mathematics behind many photographic filters are often based on years of technical research and even tuned for human visual perception. To fully understand how these filters work, you may end up reading technical white papers or graduate theses. Fortunately for me who’s pretty bad at maths, I discovered </em><a href="https://github.com/BradLarson/GPUImage/tree/master/framework/Source"><em>Brad Lawson’s GPUImage library</em></a><em> which includes many standard filters in shader format. This vastly helped with my understanding of how they work.</em></p><p>It’s not immediately clear in this example, but the runtime performance of WebGL rendering is far better than looping in Javascript. We can chain many heavy transformations together and still render within our target frame time.</p><h3>Caveats</h3><p>As with most things, there are a number of caveats to processing images using WebGL.</p><h4>WebGL Rendering Blocks the Main Thread</h4><p>Even though the processing and rendering of our image is handled by the GPU, our Javascript main thread will wait for all drawing to finish before continuing. This means that if we write very inefficient or complicated shaders, we can still slow down the UI and miss our 60fps target.</p><h4>GPU Hardware / Driver Bugs Are Real</h4><p>Web browsers do a good job to hide bugs and performance issues in most GPU drivers. Sometimes specific draw modes are unreasonably slow on specific GPUs and our browsers are filled with hacks and workarounds to hide this. WebGL allows you to run your code directly on the GPU where it’s susceptible to all of the quirks you would’ve been otherwise protected against. However, our image processing application shouldn’t run into many of these issues.</p><h3>Conclusion and Links</h3><p>Thanks so much for reaching the end of this very long and complicated article! I hope you found it interesting and maybe even inspiring enough that you want to investigate the world of WebGL and image processing even more.</p><ul><li>How to process image data using regular 2D canvas and how being bound to a single CPU thread severely limits performance.</li><li>The basics of <strong>vertices</strong>, <strong>shaders</strong>, <strong>textures</strong> and <strong>programs</strong>.</li><li>The different coordinate systems used by the browser and WebGL.</li><li>How to set up a WebGL 3D canvas for 2D image rendering by drawing two triangles and applying a texture.</li><li>How to transform pixel data within a <strong>fragment shader</strong> in order to create image filters.</li></ul><h4>What I’m Working On</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dyJRLLlT_pbLwDAhK3ei1g.png" /></figure><p>All of this effort is being put into the creation of a library I’m calling <strong>Iris</strong>.</p><p>My aim is to create a dependency-free, lightweight image editing solution that can be used across any framework.</p><p>As I mentioned previously, I am aiming for feature parity with the editing tools of Google Photos. I still have a lot to learn, but it’s an enjoyable process.</p><p><a href="https://github.com/Jonymul/iris-gl">The code for Iris is on GitHub</a>, and I will be releasing the packages on NPM very soon!</p><p>If you’re interested in getting involved with the project, please reach out on GitHub</p><h4>A Note About three.js</h4><p>You will notice that I didn’t mention any third-party libraries through the duration of this article. Part of my goal was to explore the inner workings of WebGL, and remove the need for third-party dependencies.</p><p>If you are working on a full 3D experience however, I would strongly recommend the use of a graphics library such as three.js as low-level graphics programming really doesn’t scale well to large applications.</p><h4>Links and Resources</h4><ul><li>The awesome shader sources in <a href="https://github.com/BradLarson/GPUImage/tree/master/framework/Source">Brad Lawson’s GPUImage library</a></li><li><a href="https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API">WebGL on MDN</a></li><li><a href="https://github.com/Jonymul/iris-gl">Iris on GitHub</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c2af552e8df0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/eureka-engineering/image-processing-with-webgl-c2af552e8df0">Image Processing with WebGL</a> was originally published in <a href="https://medium.com/eureka-engineering">Pairs Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Navigable Modals With the History API: Adventures in Web Modals]]></title>
            <link>https://medium.com/eureka-engineering/navigable-modals-with-the-history-api-adventures-in-web-modals-27d94ae2014?source=rss-84f5da7d378e------2</link>
            <guid isPermaLink="false">https://medium.com/p/27d94ae2014</guid>
            <category><![CDATA[ux]]></category>
            <category><![CDATA[modal]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[react-router]]></category>
            <dc:creator><![CDATA[Jon Mulligan]]></dc:creator>
            <pubDate>Tue, 08 Dec 2020 03:40:10 GMT</pubDate>
            <atom:updated>2020-12-08T03:40:10.603Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M2Ybyhfs5iS1yqQoTvRnOw.png" /></figure><h3>Adventures in Web Modals</h3><p>Hi there! I’m Jon Mulligan, a Front End Web Engineer working on the Japanese version of Pairs.</p><p>This article is day 8 of the <a href="https://qiita.com/advent-calendar/2020/eureka">Eureka Advent Calendar 2020</a>. For day 7, Spice wrote about <a href="https://medium.com/eureka-engineering/svelte%E3%82%B3%E3%83%B3%E3%83%9D%E3%83%BC%E3%83%8D%E3%83%B3%E3%83%88-%E3%81%AA%E3%81%9Creact-vue-%E4%B8%8A%E3%81%A7%E5%8B%95%E3%81%84%E3%81%A6%E3%82%8B%E3%81%AE-9adab6246e77">using Svelte components in a UI Framework such as React or Vue</a> [Japanese language].</p><p>In application design, splitting large and complex interfaces into pages without the jarring context change can be a challenge. Modal views help to solve these problems by presenting additional content in situ as the user interacts with the page. But from an implementation perspective, they introduce a number of challenges.</p><p>In this article I’ll be diving into some of the common challenges and pitfalls of modal content, and how we solved them at Eureka.</p><p>There’s a code portion a little later on which covers a very rudimentary implementation using React Router but in theory, these concepts can be applied to any library/framework.</p><h3>What is a modal?</h3><p>I’m sure by now, you’re all familiar with modals. Even if you don’t know of them by name, you interact with them every day. They can be as simple as an alert/dialog box asking you to save your work before closing, or as complex as a setup wizard when starting an app for the first time.</p><p>Put simply, <br><em>A modal is a child view which covers the page beneath it, requiring the user’s attention and interaction before returning control to the parent page.</em></p><p>Simple alerts, lightboxes and dialog boxes are all contextual to an action performed on the parent page. We’ll be skipping these examples and focussing on an increasingly common pattern — Navigable modals.</p><p>Many modern web and mobile apps call for unique items, or even entire pages to be shown in a window which covers the parent page or a sheet that slides in from the side of the screen.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TrMlLLGw_jsakbIz3zrxiA.png" /><figcaption>Left: A Google Calendar event is shown in a modal. Right: A shopping cart is displayed as an overlay panel.</figcaption></figure><p>In the desktop version of Pairs, we show user profile pages in a modal to allow users to quickly jump between profiles and listing pages without losing context.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7lBLc0MWhHwAQc1aKXJ0qQ.png" /><figcaption>A design sample of the Pairs web app.</figcaption></figure><h3>Modal navigation: An idea</h3><p>A web page is usually a unique piece of content, therefore it has a URL. A way to access the page directly. Visiting the page adds it to your browser’s history which we can navigate using the back and forward buttons. To put an entire page in a modal, we first have to answer a few questions.</p><p>Most of the modals you see on the web don’t have URLs — and in many cases, this is fine. But how many times have you used an e-commerce site where a product was shown in a modal, or the help screen was shown in a popup? Without URLs, these pages wouldn’t be shareable. Refreshing the page, or clicking the browser’s back/forward buttons would take us to a completely different location to what we were expecting.<br>This behaviour creates an inherently different user experience to the rest of the web, and users will lose confidence in your application.</p><p>Single-Page Apps (SPA) solve the navigation problem using the History API. Without actually navigating to a new page, we can insert a new history item and preserve natural browsing behaviours like refresh and back/forward navigation. By utilising the History API it’s entirely possible to make modals navigable just like any other page.</p><h3>The difficult part</h3><p>By choosing to render something in a modal, we’re technically rendering two unique pieces of content. The modal content itself, and the parent page beneath.</p><p>Let’s say we show a piece of content in a modal and the user refreshes their browser. Upon reloading, how would we know what the parent page was? Our only clue is the current URL which directs us to the modal content! This is where we utilise another feature of the History API: state.</p><p>When navigating between pages your browser’s history doesn’t just store a list of previously visited URLs, it also has the ability to store an arbitrary JavaScript object with each history item. When we (or our routing library) use history.pushState() or it’s cousin history.replaceState(), not only do we pass a new URL, but we can also pass a state object. After navigating around the app or even refreshing the browser, returning to that item in history gives us access to that same JavaScript object we saved earlier.</p><p>If we were to store information about the parent page in state, we can read it back after navigation/refresh, to determine what content to render on the parent page. With this, we have a navigable modal!</p><p>Twitter is an example which gets this right. <br>If you click on a photo and refresh your browser window, you’ll notice that the parent page is still the same prior to refreshing. Closing the photo takes you back exactly where you were.<br>Well done Twitter. 10 points to you. 👏</p><h3>Shut up and show me some code</h3><p>The following (vastly simplified) example is inspired by our implementation in the Pairs web project. Modal routes are mounted just like any other. The example uses React and React router, but the core concepts can be applied to any framework.</p><p>Firstly, we define our routes. Both regularRoutes and modalRoutes share the same basic config.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/09f3f7532fb8c5814ee819d18d5590a2/href">https://medium.com/media/09f3f7532fb8c5814ee819d18d5590a2/href</a></iframe><p>Next, we render all of our routes in two separate &lt;Switch&gt; components. These switches are passed a location prop. This tells the switch which path to match. Without this, it would default to the current path. The parentLocation and modalLocation values are determined by the getLocations() hook, described next.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c4c124b4de60034cdace3dbef16864af/href">https://medium.com/media/c4c124b4de60034cdace3dbef16864af/href</a></iframe><p>Our getLocations() hook looks a little like this. If the history state contains a backdropLocation, we can assume it is the parent (background) location and return it as parentLocation . The modalLocation should be the current URL.<br>If backdropLocation is undefined, we can assume that the current URL is not a modal, and return is as parentLocation.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1a873b0b363f000a5e7df75498ac469b/href">https://medium.com/media/1a873b0b363f000a5e7df75498ac469b/href</a></iframe><p>Finally, we need a way of setting the backdropLocation in our state. This is achieved by using a custom &lt;ModalLink&gt; component which extends the React Router &lt;Link&gt; component. This can be used on any of our app’s pages, and will open a modal with the correct parent page beneath.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/52b4fa4a1346f01bdbddd170340eacb1/href">https://medium.com/media/52b4fa4a1346f01bdbddd170340eacb1/href</a></iframe><h4>The missing piece</h4><p>You’ll notice that our &lt;ModalLink&gt; component has a safety mechanism to stop a modal being rendered as a parent page to another modal. backdropLocation: backdropLocation || currentLocation <br>This is just one piece of the puzzle.</p><p>If the user opens a modal URL in a new tab or sends it to a friend, a backdropLocation will not be included in the history state and our code will assume it’s a regular page. Therefore we must add some sort of defaultBackdrop configuration to our modalRoutes and redirect if the current URL is a modal. <br>A specialised getModalRouteRedirector() hook returns a &lt;Redirect&gt; component in this situation which — unless falsey — should override the default render of App.tsx. This should re-write the history state once, after which the App can render as normal.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a69318370ffc9c9e6e552a1393c60a88/href">https://medium.com/media/a69318370ffc9c9e6e552a1393c60a88/href</a></iframe><p>And there we have it! A massive chunk of code to monkey-patch support for navigable modals.</p><h3>The real fun starts here</h3><p>If you’re only showing one simple page in a modal, the above solution is great. It’s simple and isn’t too invasive. But it does have limitations.</p><p>If we wanted behaviour even more like a native app — a design that calls for navigation within modals or navigation to other modals — we have to think of a more drastic solution. Our main limitation of the web is the linearity of its history model. It’s essentially a flat array of items. <br>We’re inventing concepts that the web’s never seen; such as creating a “branched” context, and “reverting” to a previous state on close.</p><p>By recording a tree of context branches, and appending it to the state object of each history item, we could theoretically implement such an experience. But my wacky ideas will have to wait for another day.</p><h3>Conclusion</h3><p>If your designer asks you to put a page in a modal; pay them to change their mind. Unless you&#39;re some sort of lunatic or tech masochist…</p><p>I’m obviously kidding.<br>Modals can be a great way to increase screen real-estate and avoid jarring context switches. By making them navigable, not only do we make them shareable, but we also preserve natural browsing behavior. Whilst we face some challenges along the way, it’s still entirely possible, and pretty fun if you ask me!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=27d94ae2014" width="1" height="1" alt=""><hr><p><a href="https://medium.com/eureka-engineering/navigable-modals-with-the-history-api-adventures-in-web-modals-27d94ae2014">Navigable Modals With the History API: Adventures in Web Modals</a> was originally published in <a href="https://medium.com/eureka-engineering">Pairs Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>