<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Adam Solove on Medium]]></title>
        <description><![CDATA[Stories by Adam Solove on Medium]]></description>
        <link>https://medium.com/@asolove?source=rss-4c309f3e3ba6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*l4EA1YBe7qwOtWoyU9nyTA.jpeg</url>
            <title>Stories by Adam Solove on Medium</title>
            <link>https://medium.com/@asolove?source=rss-4c309f3e3ba6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 21:04:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@asolove/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Designing a better podcast editor]]></title>
            <link>https://asolove.medium.com/designing-a-better-podcast-editor-9d846db4a025?source=rss-4c309f3e3ba6------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d846db4a025</guid>
            <category><![CDATA[ui-design]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[podcast]]></category>
            <dc:creator><![CDATA[Adam Solove]]></dc:creator>
            <pubDate>Thu, 04 Jun 2026 03:58:42 GMT</pubDate>
            <atom:updated>2026-06-04T03:58:42.112Z</atom:updated>
            <content:encoded><![CDATA[<p><em>I’m migrating my writing to my </em><a href="https://www.adamsolove.com/ui/ducking/2026/06/03/better-podcast-ui.html"><em>new personal site</em></a><em>. I’ll just share tidbits here as reminders for those who are subscribed. But I encourage everuone to switch over to the RSS feed on my personal site to make sure you keep getting my writing.</em></p><p>For the past few years, my partner has recorded and edited a niche podcast while I’ve helped a bit with selecting music and EQ setting. Our workflow was painful: one shared iCloud file, emails of notes coded to timestamps, carefully checking that our changes didn’t conflict.</p><p><strong>Editing audio was like traveling back in time twenty years: no track changes, no comments, and no multiplayer editing.</strong></p><p>The friction wasn’t just in the file-oriented workflow. The normal interaction model of Digital Audio Workstations (or DAWs) is not particularly well-suited for the task of editing spoken word audio.</p><p>So I decided to build the podcast editor we wanted: Ducking. It has a UI purpose-built for laying out spoken word audio, plus multiplayer editing, collaboration tools, and history management. In this post, I’ll talk about the improvements it makes to editing tools. Future posts will discuss the engineering challenges of multiplayer audio editing and the pleasure of building software for just a few users with design sketching techniques and LLM assistance.</p><p><a href="https://www.adamsolove.com/ui/ducking/2026/06/03/better-podcast-ui.html">See more over on my personal site.</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1007/0*63kiiKeN-uncQtP-.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d846db4a025" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[<iframe>able finance]]></title>
            <link>https://asolove.medium.com/iframe-able-finance-7c4a72c2917b?source=rss-4c309f3e3ba6------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c4a72c2917b</guid>
            <category><![CDATA[iframe]]></category>
            <category><![CDATA[fintech]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[ui]]></category>
            <category><![CDATA[payments]]></category>
            <dc:creator><![CDATA[Adam Solove]]></dc:creator>
            <pubDate>Tue, 07 Nov 2023 13:26:10 GMT</pubDate>
            <atom:updated>2023-11-07T13:26:10.524Z</atom:updated>
            <content:encoded><![CDATA[<h4>Designing and building embeddable UIs</h4><p><em>An aside: most of my writing is now on </em><a href="https://www.adamsolove.com/"><em>my personal site</em></a><em>, as Medium has become a slightly less pleasant place to write. I recommend browsing or subscribing to RSS over there. Nevertheless, I wanted to check that folks who have followed me for a while saw this interesting talk.</em></p><p>At <a href="https://fintechdevcon.io/">Fintech Devcon 2023</a>, I gave a talk on designing and building embedded UIs, based on my experiences contributing to Stripe Elements and then to Rainforest’s embedded components.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FKDu7A_LcvgY%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DKDu7A_LcvgY&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FKDu7A_LcvgY%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/c8e24104d6ba550401573083bcef785b/href">https://medium.com/media/c8e24104d6ba550401573083bcef785b/href</a></iframe><p>I got some great comments and questions afterwords and wanted to start a broader conversation to bring together engineers and product people working with &lt;iframe&gt;s.</p><h3>&lt;iframe&gt; friends community</h3><p>To get involved, sign up for the <a href="https://buttondown.email/iframe-friends">iframe-friends mailing list</a>.</p><p><a href="/service/mailto:asolove@gmail.com">Email me</a> if you have questions, topics you’d like to see discussed, or a request for review on your proposed new embedded UI.</p><p>I’ll shortly be sending out further resources based on those questions as well as sharing an invite for future group discussions.</p><h3>Slides</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*ZR2rMdWvj_6Ezr28.jpg" /></figure><p><a href="https://www.adamsolove.com/img/iframeable-finance.pdf">You can download the slides as a pdf.</a> although note that some of the diagrams have animations that won’t work well statically and really need video to do them justice.</p><h3>Thanks</h3><p>I want to repeat what I said in the talk: while I have tried to gather together this information and make it easily available to more people, I am not primarily responsible for having learned or designed most of the things in this talk.</p><p>A huge thanks to the people who taught me or learned together with me, especially: Alex, Jenan, Michelle, David, and Tyler. You know who you are. I’m just trying to pay it forward.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c4a72c2917b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learning about learning]]></title>
            <link>https://asolove.medium.com/learning-about-learning-98d1a22b8be0?source=rss-4c309f3e3ba6------2</link>
            <guid isPermaLink="false">https://medium.com/p/98d1a22b8be0</guid>
            <category><![CDATA[education]]></category>
            <category><![CDATA[psychology]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Adam Solove]]></dc:creator>
            <pubDate>Tue, 23 Jan 2018 04:43:21 GMT</pubDate>
            <atom:updated>2018-01-23T04:43:21.872Z</atom:updated>
            <content:encoded><![CDATA[<p>Thanks to an extended holiday vacation and some family illnesses, I was lucky enough to spend two weeks reading about how people and teams learn. It was really interesting.</p><h4><strong>Why read about learning?</strong></h4><p>A few months ago, I started working at Stripe. It has felt like Neo getting plugged into the training modules. Except instead of kung fu, I know about complex online business models, brand-new payment methods, and detailed financial regulations. (It might not sound like it, but this has been really fun!)</p><p>Stripe has lots of teams working on lots of things. The organization has to understand and adapt to everything going on in the worlds of finance and tech. So teams generate enormous quantities of knowledge. And the projects I work on have enough scope that I sometimes need to understand large parts of that outside knowledge to do my job well. (If this sounds fun, I’m obligated to mention <a href="https://stripe.com/jobs">we’re hiring</a> and you should contact me if you have any questions.)</p><p>So I need to read, systematize and retain more information every day. And my code and prose need to convey that context to others. This has been a big but fun challenge.</p><p>That’s why I started reading about how individuals and whole organizations can learn. There are a lot of great books about this, and I’ve only read a tiny number of them. But <strong>I hope these summaries encourage you to learn more about learning or to be more thoughtful about your own work.</strong></p><h4><strong>Study skills and meta-cognition</strong></h4><p>My first goal was to understand how I could learn and retain more by myself. I used the very scientific method of picking the first highly-rated book on Amazon, which was <a href="https://www.amazon.com/Teach-Students-How-Learn-Metacognition/dp/162036316X/"><em>Teach Students How to Learn</em></a>. This book is aimed at professors who want to help their students adjust to the self-directed learning of a college environment. But if you ignore the faculty-specific bits, it’s also a great guide to being a better learner yourself.</p><p>It covers a lot of material about study skills, meta-cognition, different failure modes for teaching and learning, and how to create a learning-friendly environment. A lot of the ideas were already in my brain somewhere, but I was mostly ignoring them in my day-to-day work. So a systematic reintroduction to them was really helpful. Here are some specific things I took away:</p><ul><li>Partway through this book is a section on <strong>how to retain what you read</strong>. I paused there to ask myself how much of the previous material I remembered, and it wasn’t a lot. That section described a strategy of pre-reading to build an outline for the material, then reading to fill in that structure, and taking notes for a specific purpose. So I started the book over again following that advice. I now have a great set of notes, but I also remember a lot of the material and chose specific ways to apply it. Now when I need to read documents, I apply that same strategy and am able to recall much more of it.</li><li>A section about meta-cognition described various ways <strong>you can be aware of yourself as learning</strong> and consciously choosing strategies to do it. It also describes patterns of mental self-talk that help or hinder this awareness. I identified pretty strongly with several of these. I like to tell myself that I read quickly so I can just breeze through things. Or there are specific topics that I’ve never really understood and have developed complex excuses about, so I don’t even expect to understand. (Like graph algorithms, for some reason.) Neither of these thoughts is helpful. The book describes a way to start <strong>noticing and gently changing your self-talk</strong>. I’m still working on this, but I already see some improvements. I’m much more likely to pause before reading something and consciously decide my goal and my desired level of recall so I can pick a reading strategy before I start.</li><li>Another section on the biology and psychology of learning reminded me to <strong>take care of myself</strong> before getting stressed out about work. If I take breaks, get in some stretches, and drink enough water, I can be calmer and more likely to be self-aware about how I’m thinking. I’m learning to recognize the sensation of shutting down and becoming too narrowly focused to really pay attention to the broad context of my work.</li></ul><p>After finishing the book, I was pretty excited to apply these strategies in my learning and teaching. But I was also a bit perplexed: all this advice seems to be about extra things students should do to turn lectures and textbooks into real learning. Couldn’t we cut out the middle step and find a way to expose information in a way that naturally invites interactive learning?</p><h4>Learning with computers</h4><p>So I picked up <a href="https://www.amazon.com/Mindstorms-Children-Computers-Powerful-Ideas/dp/0465046746"><em>Mindstorms: Children, Computers, And Powerful Ideas</em></a><em> </em>and was fascinated with its main themes. In part, the book describes the Logo children’s programming environment and how it can be used to teach math. But more importantly, it argues for a different method of teaching, where children figure things out for themselves by interacting with computers.</p><p>How do computers lead to a different way of learning?</p><ul><li>Children are often unaware of the strategies or patterns of thought they are using. They get angry when they arrive at the wrong answer, and have a hard time examining the thought process they used to see where they went wrong. (Which, I mean, me too.) Programming is specifically about putting these strategies into an external representation. And when the program is wrong, you don’t have to self-identify with its wrongness. You can look at the program “from the outside” and debug it until it’s right. Which is great! You learn to expect a first try to be a little wrong. You learn to examine <em>why</em> it is wrong. And you learn that this meta-cognition, rather than answering right or wrong on the first try, is what learning really feels like. Applying this same process to your thinking, not just your programs, is a good way to improve at learning.</li><li>Computers provide an interactive medium to model any kind of system. For most topics, you could take all the material in a textbook and rearrange it into an interactive version that allows exploration in different orders and encourages students to consciously build a mental structure to hold the information. This change, from a set curriculum to a self-directed exploration, forces real learning habits onto the student. (And turns excuses like “I just don’t get X” into actions like “I’ll try playing with related idea Y first”.)</li></ul><p>I really enjoyed <em>Mindstorms</em> as a critique of learning methods in general. It’s hard to convey in summary: there are a lot of interesting concrete ideas in there.</p><p><strong>The main idea I took from <em>Mindstorms</em> is a sharp discomfort when using static words to convey the interactive behavior of complex systems</strong>. But I don’t yet know what to do about this. How would I go about building an interactive model in anywhere near as little time as I would need to write prose to describe it? What kind of tooling would make this easier? I’m not sure.</p><h4>What actually happens when we learn?</h4><p>Several footnotes in <em>Mindstorms</em> referenced Bruner’s <a href="https://www.amazon.com/Toward-Theory-Instruction-Belknap-Press/dp/0674897013"><em>Toward a Theory of Instruction</em></a><em>,</em> so I continued my reading there. This book is a collection of essays that span from details of cognitive psychology all the way to philosophy and the ultimate goals of human life. The most interesting parts for me were about the question: what actually happens when we learn? What is the actual function of a teacher, a text, an image, a study session? What is the actual effect in our mind’s ability to think about the subject?</p><ul><li>One essays presents a theory that <strong>knowledge can be represented through action, through visual symbol, and then through abstractions like language</strong>. And that our knowledge of the same topic progresses through these over time. When looking at a spec, we might first role-play the interactions to understand the “purpose” or “motive” of different parts, then later summarize our understanding as a diagram, and only later learn to think in terms of the actual data and operations in code.</li><li>Another section drew this idea out further to argue that learning is a sequence of cognitive changes where each step may require a new representation of the topic. This has been a good reminder that <strong>there can never be one canonical explanation of any knowledge that different communities of people need to understand</strong>. The right representation of the information is relative to the knowledge and goal of the viewers.</li></ul><p>Further on, I was really struck by this idea:</p><blockquote>“It took the efforts of many highly talented mathematicians to discern the underlying structure of the mathematics that was to be taught”</blockquote><p>This is a comforting thought when a subject is hard to learn! It’s not clear to me, but it also wasn’t clear to many of the same people who made progress on particular bits of this material.</p><p>I arrived at an image of a math course as a large pile of unassembled legos. Over time, people shape them into local, overlapping structures which only coalesce at the last moment into a surprising finished product. It shouldn’t be surprising if students presented with just the raw materials and then just the finished structure have trouble seeing how they could possibly be related. They need in part to try assembling the bits themselves, or at least see some of the early results, for the picture to become clear.</p><p>It also helped me accept that for subjects I’ve had trouble with in the past (like graph algorithms), I should search out different partial explanations for the material. (Which I’m doing! About which, more in a future post.)</p><h4>How to teach it</h4><p>I can’t write a post about conscious problem-solving and different ways of learning without mentioning George Pólya. After reading so much about self-teaching and meta-cognition, I wanted some concrete suggestions that were specific to the domains of programming and math. So I picked up a copy of his <a href="https://www.amazon.com/How-Solve-Mathematical-Princeton-Science/dp/069111966X">How to Solve It</a> and am slowly making my way through the strategies. <strong>Stuck on a problem? Just flip to a method at random and try what it says</strong>. (This has worked for me more than once.)</p><p>I also found a great video of Pólya demonstrating his student-directed teaching technique. He poses a problem and then gently steers a room full of students to use problem-solving strategies. The video lasts an hour, but I found it worth the time to see his approach in action.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fh0gbw-Ur_do%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dh0gbw-Ur_do&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fh0gbw-Ur_do%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/c603357fd119c4b30af7548d3f491ee6/href">https://medium.com/media/c603357fd119c4b30af7548d3f491ee6/href</a></iframe><p>For extra fun, try pausing part-way through and proving the solution yourself. Guess if the students will follow the same path. I wrote a proof with a completely different path from the one in the video. Interestingly, I noticed that Polya side-steps or ignores some student questions and answers that would lead towards the way I solved the problem. Maybe my proof has a flaw in it. But this made me wonder if he really wants to solve the problem in one specific way, and whether his method might not work for some students.</p><p>It was painful for me to watch him ignore the line of inquiry that clicked for me. (Even if it’s faulty, I’d like to know why, and his method hasn’t showed me.) The memory of that pain is a regular reminder to me: don’t waste time on the one perfect explanation. Explain things several ways. And always let your audience figure it out themselves or ask questions that might lead along an unexpected but interesting path.</p><p>I hope these book suggestions and summaries lead to fruitful changes to the way you teach and learn! They certainly have for me.</p><p>In a future post, I’ll summarize a few other books I read which focus not on individuals but on how whole organizations learn and think together. This material was even more surprising to me, and has important consequences for how companies and whole disciplines work together. Look forward to that soon.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=98d1a22b8be0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pure UI Control]]></title>
            <link>https://asolove.medium.com/pure-ui-control-ac8d1be97a8d?source=rss-4c309f3e3ba6------2</link>
            <guid isPermaLink="false">https://medium.com/p/ac8d1be97a8d</guid>
            <category><![CDATA[user-interface]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[functional-programming]]></category>
            <dc:creator><![CDATA[Adam Solove]]></dc:creator>
            <pubDate>Fri, 14 Jul 2017 04:58:56 GMT</pubDate>
            <atom:updated>2017-07-14T04:58:56.639Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<p>This month marks 10 years that I’ve been paid to build web user interfaces. And today is the second anniversary of Guillermo Rauch’s essay <a href="https://rauchg.com/2015/pure-ui">Pure UI</a>. Those two anniversaries have led me to reflect on the past and future of UI development.</p><p>The past 10 years has seen major improvements in how cross-discipline teams work together to create interfaces. We are much better at thinking about all the different ways an interface should display, depending on the user, their data, their device, and other context.</p><p>In contrast, we still struggle to understand how complex interfaces behave given all the possible ways that users interact with them. Most interfaces still have bugs when you interact with them in unexpected ways. Even the developers don’t fully understand their behavior. We can do better.</p><p><strong>Understandable behavior is possible.</strong> We can improve how teams understand a UI’s interactive behavior by following the same strategy that helped us understand complex UI display states.</p><p>In this post, I’d like to show that:</p><ul><li>Teams improved at building complex UIs by changing the way they represented and communicated about them.</li><li>Unexpected behavior arises because teams struggle to represent an interface’s desired behavior.</li><li>By adopting the idea of <em>control state</em>, teams can gain a powerful way to represent a UI’s interactions and prevent unexpected behavior.</li></ul><h3>Understanding display states with Pure UI</h3><p>In his wonderful essay <a href="https://rauchg.com/2015/pure-ui">Pure UI</a>, Guillermo Rauch described how the React programming model changed his collaboration with designers. The major change is a focus on explicitly describing <em>display states</em>. The designer creates an artboard with side-by-side designs of each of the widget’s states. The React code similarly branches on those same states. There is a close correspondence between between each design on the artboard and the code in the render method. More importantly, the shared language of states means that when the developer discovers a new possible case, it has a natural place on the design artboard and requires only one new bit of thought from both design and development.</p><p>This way of working is now so natural that it’s easy to forget that it wasn’t always obvious. To see the difference, let’s consider how designers and developers worked together when I started building web UIs ten years ago.</p><p>The design was a Photoshop file with just a single state of the screen visible. A thoughtful designer might have considered a few other states and put them in layer groups to be switched on and off. But many other states were ignored or described only in informal conversation, as changes to be made to the original state. The secondary states never took on visual form and were therefore a frequent topic of miscommunication.</p><p>The code for the widget consisted of a template, and then a series of functions that described how to modify the displayed DOM in response to various user inputs or data changes.</p><p>These design and code representations share two features:</p><ul><li>Different states of the application can be viewed only one at a time, and require careful manipulation to reach.</li><li>There is no explicit list of possible states.</li></ul><p>These features make it hard for the team to think and communicate about the interface:</p><ul><li>There’s no easy way to see all the states at once, so it’s difficult to hold the interface as a whole in your head.</li><li>Without an explicit list of states, it’s easy to incorrectly judge progress if the default state looks finished.</li><li>When developers discover a new state, it doesn’t have a natural place in the design document, so often isn’t designed at all.</li><li>Adding a new design element to the interface becomes progressively harder because every transition method needs to consider whether it should show, hide, or make changes to every element.</li></ul><p>Notice that these aren’t primarily technical or process problems. They’re thinking problems. The design and the code describe a component as a default visual display and a set of changes that can be applied to it. But these representations don’t help us communicate across disciplines, identify questions that we still need to answer, or simplify the work itself.</p><p>The Pure UI approach improved our thinking and communication by first changing the way we represented display states. Now we describe a component as a set of states that each have their own visual display. This description provides a natural mapping between the designer’s artboard and the developer’s style guide. Both of these artifacts allow us see all the states side-by-side, so we can think about them as a whole and notice inconsistencies and open questions. They also make changes atomic and localized: when we make a change to one state, it only affects that state.</p><p>Generalizing the lessons of Pure UI, we can see a strategy to apply whenever a problem becomes too complex for teams to think and communicate about together:</p><ul><li>Look for a missing idea not explicitly reflected in the problem’s current representation.</li><li>Find a new representation that makes that idea explicit and helps us represent what we know and what we still need to think through.</li><li>Then, only at the end, worry about the technology and process to use to bring this representation to life.</li></ul><p>Keep that strategy in mind as we consider the problem that many UI teams currently struggle with: understanding all the possible interactive behaviors of their application.</p><h3>The problem of unexpected behavior</h3><p>For as long as I’ve built interfaces, regardless of the technology or testing plan, we’ve found unexpected behavior in production. Nothing obvious, certainly nothing that anyone would have written a user story about.</p><p>Instead, the bugs came up in unexpected cases that made you wonder “why would anyone do <em>that</em>?” But if it’s possible to do something, someone eventually will. What almost all of these bugs had in common was a dependence on <em>ordering </em>and <em>cases</em>, especially <em>unexpected </em>orders of cases.</p><p>You’ve probably experienced bugs like these in your own projects:</p><ul><li>The credit card validation widget stops working if you enter half of a card number, switch to another screen in the application, do something there, and then switch back to the payment screen.</li><li>If you’re in the middle of editing one object, click a button to load data about another, and if the timing of the response is just right, your edits may be saved to the wrong one. Or lost forever.</li></ul><p>Sometimes I see teams blame these problems on their own bad code, redux-thunk, or some other bit of technology. But it isn’t asynchronous tools or lazy developers that create these problems. It’s true that once you understand how the bug can happen, it’s obvious that the code is wrong. But when writing the code, would a reasonable developer have thought to handle this case? Did a designer consider this possibility and what should happen?</p><p>Usually not.</p><p>So we don’t have a technology or laziness problem, we have a thinking problem. We need representations that can help us think about an interface’s behavior.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/194/1*gq6Pv_BHE2zALG6rgiCgJw.png" /></figure><p>Let’s consider a simple calculator application. It has only one display state, but a surprising depth of behavior. Everyone knows in general what it should do, but unless you’ve sat down to think about it, you likely don’t know <em>how</em> it should do it. It’s trickier than it looks.</p><p>You probably don’t believe me. You think you could turn this out in five minutes. I encourage you to go try. (Please leave a comment here with a link to the code and how long it took.)</p><p>Let’s examine the artifacts that a UI team would use to think about this interface:</p><p>A designer, after producing one simple mockup, might stop there. If very thoughtful, they might write a few sentences about the expected behavior for a few sequences of input.</p><p>The developer will quickly get the display and events hooked up. I’ll assume they use a pattern like Redux or the Elm architecture where they need to define a function that takes the current state and an action and returns a new state.</p><ul><li>First they implement an action for the digits. A simple implementation just multiplies the existing value by 10 and adds the new digit.</li><li>Next they implement the decimal action. This doesn’t immediately change the numerical value but does need to show up on the display. So perhaps they add a new decimalPressed flag to the state. It’s set to true when decimal is pressed, and it causes the decimal point to show on the display even if the current value is integral. After this, the sequence of actions 2 0 . correctly shows 20. on the display.</li><li>Ah, but now a problem. When the decimalPressed flag is set to true, the existing handler for typing a digit is wrong. So any action that changes the number needs to check the flag. Oh, and any action that means we’re starting to type a new number needs to remember to set it back to false. Well, unless we pressed decimal right before the digit, then it should stay true. …</li></ul><p>There are better and worse ways to implement the complete logic and I don’t want to walk through the whole problem. The naive strategy is to slowly add flags and other non-visible bits of state to represent what each action needs to know about the history of past actions. This is workable, but creates problems:</p><ul><li>Adding new actions becomes progressively harder because they may need to react to arbitrary previous actions or influence arbitrary future ones. It’s not clear which combinations you need to consider, or when you’re done.</li><li>The nested conditions in the code don’t clearly match to any other representation, so code itself becomes the definition of correctness. It becomes difficult for designers, or anyone else, to have opinions about what should happen in any particular case.</li><li>There’s no way to know if you’re done. Unit tests can cover cases you think about. But there are infinite possible orderings of events, so can they cover everything?</li></ul><p>Clearly this is not a technical problem, but a thinking one. In particular, the representation of behavior as prose or reducer code doesn’t help us to think about the problem or check our understanding. Just as in the case of Pure UI’s display states, we need a new organizing idea and a new representation to aid our understanding.</p><h3>Data state &amp; control state</h3><p>I think the idea we’re missing is an application’s <em>control states</em>. Previously we enumerated all of an interface’s <em>display states</em> as all the meaningfully different ways it should display. A <em>control state </em>is all the possible states of the interface that have different sets of allowed interactions. Sometimes the two layers of states align, as in a loading state that displays a spinner and ignores all user input. In other cases, an interface’s display state can remain the same even while it’s control state changes.</p><p>Consider the calculator. When its display reads 20, it may actually be in one of two control states. In the “Entering operand” control state, typing another digit appends it to the end of the current number. In the “Result” control state (where a previously-calculated answer is shown), typing a digit causes it to be the start of a new operand, and transitions the control state to “Entering operand,” changing how the next action will be interpreted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*0smAw31yiwO7aEriW3NXrA.jpeg" /><figcaption>The high-level statechart for the calculator application. © Addison Wessley. From Ian Horrocks’ “Constructing the User Interface with Statecharts”, p215.</figcaption></figure><p>With the new idea of control state, we can proceed just as we did before, by imagining sequences of inputs and asking what the interface should be able to respond to next. But now we can take what we learn and make it into an enumerated set of all the allowed control states and how they respond to user actions.</p><p>Thinking about our interface’s behavior in terms of control states also makes it possible to produce more understandable artifacts. Non-programmers can easily follow a state transition diagram like the one at left for the calculator. Even for programmers, the visual structure of the chart may provide a high-level “aha!” understanding of what the interface really allows.</p><p>Control states also can be naturally represented in our code. No longer should we define a single handler for each button, with lots of cases inside it depending on the action history. Instead, transitions are defined for a specific pair of the current control state and the action.</p><p>If your language supports some kind of pattern matching, this can turn into quite lovely code. Below is part of an implementation of the calculator’s control states in <a href="http://facebook.github.io/reason/">Reason</a>. Each line communicates how a specific control state reacts to a specific action by returning a new control state.</p><p>(<em>An aside to Reason aficionados: </em>Notice an extra benefit of this structure, that each control state requires a specific, typed set of context data. This ensures that each state can only rely on data it is guaranteed to have, and that on leaving one state, any no-longer-active data is discarded and re-initialized when the state is re-entered. You may enjoy reading <a href="https://github.com/asolove/restate/blob/c93fb84ea6bf39fb9722cd7593ac7c636d35c9e1/motivation/naive/calculator_state.re#L43">the full code</a>.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CFqSy8oH0ntTmiBLDGFzaA.jpeg" /></figure><p>You don’t have to understand the details of Reason as a language to see that this code, even if translated to your language of choice, maps quite naturally to the diagram above. If a designer adds a new line to the diagram, it corresponds clearly to one spot in the code. And if a developer adds a new transition in the code, it has a natural place on the diagram.</p><p>There is still a lot of thought required in the design of the control states. But they provide an organizing structure that allows for more natural representations of our thought. As a result, questions that used to require detailed thought can be reduced to simple visual perception. Cases that developers previously struggled to explain to designers now just require pointing.</p><p>The team can <em>think together</em> about their interface’s behavior.</p><h3>Summary</h3><p>Over the past few years, web teams have adopted new state-centric representations for their design work. These representations made it easier to think about what an interface should look like depending on data and user context.</p><p>Despite that new way of working, most interfaces still have unexpected behavior when subjected to all the possible permutations of user interaction. Teams have no way to represent all these possibilities or to communicate about all the different ways their application needs to respond to sequences of input.</p><p>To solve this problem, teams can adopt the notion of <em>control state</em> as a description of all the states the interface can be in from the perspective of how it will respond to user interactions. By explicitly designing the graph of control states, they can arrive at an understandable, finite set of behaviors that the application needs to exhibit. With a consciously-designed control state space, teams can more easily reason about what they create together.</p><h4>Any questions?</h4><p><em>The calculator is a simple example: does this approach scale to larger interfaces? </em>With simple state machines, things quickly grow too complicated. But a more powerful formalism, statecharts, can model more complex systems by allowing them to be decomposed into hierarchies (just like components, hmm…) and to represent several charts that can proceed concurrently with one another. Statecharts are a common method for modeling complex embedded software that has many more inputs, and less room for error, than our interfaces.</p><p><em>What are the semantics of the diagram? How are control states best encoded in my language/framework? </em>I don’t know! I think there are lots of possible ways to use the idea of control states, just as there are many concrete ways that teams use display states to organize their work.</p><p><em>That’s it? You wrote 2000 words to tell us about state machines? </em>Well, sort of. First, because many web developers don’t have a CS background, so just saying “state machines” doesn’t help them. Second, because I wanted to talk about the idea on a non-technical level. The benefit of statecharts comes from using them as a representation to aid cross-discipline thinking and communication. And hey, maybe hearing about it from a different angle will help developers, too.</p><h4>Notes</h4><ul><li>I am very thankful to <a href="https://medium.com/u/9a6e84e34ca9">Kevin Lynagh</a>, who started bugging me about the idea of using statecharts in UI development almost three years ago. Despite my slow uptake, he kept raising the topic until I saw its value.</li><li>The calculator example and the statechart for its behavior come from Ian Horrocks’ book <a href="https://www.amazon.com/Constructing-User-Interface-Statecharts-Horrocks/dp/0201342782"><em>Constructing the User Interface with Statecharts</em></a>. The book was written in 1999, but his complaints about the poor technical design and confusing behavior of most UIs read like a complaint about web development in 2017. I hope we can improve before another 18 years pass.</li><li>The idea of focusing on how teams communicate and represent their work came from a happy coincidence. I reread “<a href="https://rauchg.com/2015/pure-ui">Pure UI</a>”, for which I am very grateful to Guillermo, at the same time that I was reading Edwin Hutchins’ <a href="https://mitpress.mit.edu/books/cognition-wild"><em>Cognition in the Wild</em></a>. Hutchins’ book is a critique of traditional cognitive science, arguing that humans mostly don’t use their critical thinking skills to solve problems. Instead, we adopt tools and representations that allow groups of people to solve what would be very complex cognitive problems with fairly minimal use of abstract reasoning. That critique could easily be pointed to normal discussion of programming. We praise the individual who by sheer force of thought arrives at a full understanding of a topic. But often all that effort would be avoidable with a good architecture diagram. I saw that the effect of Pure UI was to replace what had been a large amount of abstract thought with a more convenient representation. This led naturally to the question: what representation would help us understand behavior?</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ac8d1be97a8d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What are the important problems in UI engineering?]]></title>
            <link>https://asolove.medium.com/what-are-the-important-problems-in-ui-engineering-8b7f8b305611?source=rss-4c309f3e3ba6------2</link>
            <guid isPermaLink="false">https://medium.com/p/8b7f8b305611</guid>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[user-interface]]></category>
            <dc:creator><![CDATA[Adam Solove]]></dc:creator>
            <pubDate>Fri, 21 Apr 2017 04:16:48 GMT</pubDate>
            <atom:updated>2017-08-11T05:12:45.244Z</atom:updated>
            <content:encoded><![CDATA[<h4>Let’s escape the day-to-day to survey what’s coming next</h4><p>Very early in my programming career, I read a transcript of the talk <a href="http://www.cs.virginia.edu/~robins/YouAndYourResearch.html">“You and Your Research”</a>, by Richard Hamming, a mathematician whose name appears in the title of at least six Wikipedia pages for important things he invented.</p><p>I was struck by a question which he persistently asked of his colleagues:</p><blockquote>What are the important problems of your field?</blockquote><p>and the observation he made of the successful scientists around him:</p><blockquote>Great scientists have thought through, in a careful way, a number of important problems in their field, and they keep an eye on wondering how to attack them.</blockquote><p>and also by his scheme to devote at least a little time each week to that question and long-term thinking:</p><blockquote>I finally adopted what I called “Great Thoughts Time.’’ When I went to lunch Friday noon, I would only discuss great thoughts after that. By great thoughts I mean ones like: “What will be the role of computers in all of AT&amp;T?’’, “How will computers change science?’’</blockquote><p>This evening I spent my regularly-allotted time thinking about the important long-term problems in UI engineering and catching up with the research on attacking them.</p><p>I thought it might be worthwhile to share my own thinking and ask for your suggestions of important problems or ways of attacking them. <strong>Please comment</strong> if you think I’ve left something out or if you know of relevant attack vectors that I don’t mention.</p><p>The three problems I think the most about are: structured concurrency, understandable behavior, and data synchronization. Let’s look at each of them:</p><h3>Structured concurrency</h3><p>Especially on the web, but also in native mobile development, it is still very easy to get concurrent behavior wrong. It is rare to see the code of a user interface that doesn’t have data races or use-after-dead problems related to components that fire off asynchronous chains that eventually trigger mutations. Promises are a great API for asynchronous behaviors, but not a great structure for composing behaviors together or ensuring they play nicely with other state changes in the UI.</p><p>On the web, the problem is even more severe because most JavaScript runs on the main thread and there is no convenient way to yield control back to the event loop while retaining your place in a long-running computation. For this reason, React had to rewrite their core algorithm to remove its recursive call structure and replace it with a data structure so that it can pause to let the event loop run and to re-evaluate whether it is working on the most important thing.</p><p>There are some available, partial attacks on these problems:</p><ul><li><a href="https://github.com/machty/ember-concurrency">Ember-concurrency</a> provides a shallow coroutine system for writing pausable, cancellable asynchronous tasks. This works well if your computation has a linear structure, but doesn’t solve the problem React has, of needing to pause in the middle of a recursive walk through a tree.</li><li><a href="https://www.youtube.com/watch?v=ZCuYPiUIONs">React Fiber</a> solves the problem of yielding and scheduling in the middle of walking through your component hierarchy doing updates. But if your component callbacks or data-layer code are what’s taking up time, fiber doesn’t solve the problem.</li></ul><p>And also more general, but not yet mainstream, approaches:</p><ul><li>For UI cases in particular, a natural fit is <a href="https://medium.com/@asolove/synchronizable-abstractions-for-understandable-concurrency-64ae57cd61d1">the Concurrent ML primitives</a>, which make it easy to express common UI needs like cancellable asynchronous transactions &amp; speculative calculation. They don’t play nicely with the current JavaScript execution model, but…</li><li>Other languages support coroutine- and continuation-based concurrency. This makes it possible to get pausable, cancellable tasks, even in arbitrarily-nested code. Some of those languages compile to JS, though generally with slower performance because they have to build data structures to replace the JS stack. There is research on how to do this <a href="http://users-cs.au.dk/danvy/sfp12/papers/thivierge-feeley-paper-sfp12.pdf">less slowly</a>. <em>Possibly</em>, the perceived performance improvements of scheduling work in the right order and yielding to the browser event loop every frame would outweigh the overall throughput decrease of slower-executing code.</li><li><a href="http://webassembly.org/">WebAssembly</a> will eventually be widely available and compiling other languages to C and then WebAssembly might let you eke out more performance. Current tests compiling existing languages to WebAssembly show a big performance penalty for languages like Scheme that use tail calls, because the WebAssembly VM assumes a normal stack discipline and you have to use trampolining and other indirect techniques. But hey, those other languages have a lot of baggage and features we don’t need. <em>Possibly, y</em>ou could implement a tiny language with just the bits you needed for Concurrent ML events. If you made a really great demo or could generalize your technique so that other high-level languages could use it, you could build a coalition to push for the features that would make this faster in post-MVP versions of WebAssembly. (There are some Scheme and ML folks in the webasm community, so they might not even need much convincing!)</li></ul><p>I’m quite excited about Concurrent ML and the idea of making it work in the browser, even if slowly, in WebAssembly. So, though it’s well out of my comfort zone, I’ve been trying to attack that problem for an hour or two a day for the past few weeks. So far, I understand interpreter implementations and continuations a lot better. But my C is rusty enough I still have trouble getting things to a state that emcc will build. But it’s a fun little problem and one I expect that someone, though probably not me, will make progress on over the next few years.</p><p>I’m also thinking about how structured concurrency can be taught to web developers like me. Could we use a “Synchronized Schemer” book to teach how to work with channels and CML events? What’s the best way to make this salient to developers who just want to get something built and don’t currently know their code has data races?</p><h3>Understandable behavior</h3><p>When building interactive systems, it is enormously difficult to understand their behavior because of the enormous state-space. You have to consider all the system’s data and operations multiplied by all the possible sequences of the user’s interactions. Unlike concurrent systems that don’t involve humans, which can often be defined narrowly in terms of fixed protocols, once a human being is involved and allowed to make decisions, the possible paths through the system become uncountable.</p><p>This raises a number of problems:</p><ul><li>Modifying an existing interface is often frightening because of the unknown and untestable myriad paths through it.</li><li>Reproducing customer-reported bugs can be difficult even if you have their exact current data, because the problem may depend on a specific sequence of operations, or even the timing of asynchronous responses. We need not just snapshots, but records of change over time, to understand</li><li>Trying to specify the behavior of an interactive system is enormously difficult. A fairly simple feature request can imply disproportionately difficult changes if it introduces new state space or operation interleaving possibilities.</li></ul><p>This problem is under attack from a number of angles:</p><ul><li>Parts of this problem are made easier with disciplined use of types or contracts, immutable data, the Elm architecture, etc. But while these can certainly aid reasoning, they mostly do not solve the problem of large state-spaces and enumerating possible orderings of operations.</li><li><em>Designing with data</em> tools like <a href="https://subformapp.com/">Subform</a> make it easier to set up specific data states and manipulate them graphically, rather than in code. This can make it faster to explore the state space and notice potential problems.</li><li><em>Simulation testing</em> with tools like <a href="https://www.youtube.com/watch?v=N5HyVUPuU0E">Simulant</a> attacks the problem of testing in large state-spaces by randomly selecting operations and searching for states where expected properties don’t hold.</li></ul><p>In reflecting on this problem, I haven’t been able to find an angle of attack where I think I can help out. My current thinking is stalled on two ideas:</p><p>(Update on 8/10/17: after thinking about this problem more, I wrote the essay <a href="https://medium.com/@asolove/pure-ui-control-ac8d1be97a8d"><em>Pure UI Control</em></a>, which describes the idea of “control state” and an explicit visual representation of UI behavior. This is an attempt to make it easier for programmers and non-programmers to share an artefact that allows them to reason about <em>all</em> the possible valid behaviors of a UI. I think that the conceptual separation of “data state” and “control state” can help us focus on the characteristics specific to control, which is what really creates the unique complexity in building interfaces.)</p><ul><li>We need a first-class construct that maps to an operation on the level of the user’s intention. It seems odd that we first encode the abstract operations of our system’s specification into states and transitions in our UI, then go back and write black-box simulations to test them. Both our code and tests could use a shared vocabulary for discussing user-initiated transitions in the state space. Then it would be easy to reproduce any case for manual or automated testing by simply specifying it with a name and some configuration and letting the system set it up for us. And building a simulation test would be as easy as telling the system to loop through and select any of the “user-level operations” that the UI is currently making available. How do you specify that the visual output of a UI offers a set of operations? How do you map those operations to low-level details like event handlers and the internal state machines of activity that implement them?</li><li>One social difficulty is that UI tools tend to win popularity by how well they work for making glitzy demos where logic and visual display are intertwined. But making progress on understanding UIs depends on splitting state and logic from visual display. Can this conflict be reconciled by a UI system that funnels developers into a productive and scalable happy path by making many early decisions for them? (One inspiration for this comes from the world of back-end frameworks, where the Phoenix has <a href="https://pragprog.com/book/lhelph/functional-web-development-with-elixir-otp-and-phoenix">great educational materials</a> to provide all the get-started-fast properties of Rails while funneling new developers into writing scalable OTP applications decoupled from the web frontend.)</li></ul><p>My current work on this problem boils down to using various methods to try to make sense of the complex behaviors in the interfaces I work on. I’ve tried some formalisms, different sketching techniques, etc., all things above the level of actual code but that aspire to fully encapsulate the code’s behavior. I try to read up on formalisms about effects, like <a href="https://www.manning.com/books/type-driven-development-with-idris">Idris</a>’ effects and concurrent typing. I’ve had some success with specific diagram techniques, but not anything I could make systematic. So I think about this a lot, but haven’t made any progress.</p><h3>Data synchronization</h3><p>The network eventually absorbs every application that used to live only on a single computer. From making small web apps work while offline to correctly merging multiple users’ edits on large documents, we are going to need a wide array of options for synchronizing data. Perhaps more importantly, we need a social solution for packaging these options and helping developers use them in practice. The domain of distributed systems, currently only inhabited by experts, will be an expected feature in run-of-the-mill apps and websites.</p><p>If the community gets this wrong and doesn’t package up good options and help steer people towards the right tool for their use-case, we’re going to end up with some truly terrible problems. Our current problem with slow-loading sites will look like nothing compared with sites that have data cacheing problems or that silently merge away your changes when they sync with the server.</p><p>I know of a couple fronts in the battle against this problem:</p><ul><li>In-browser declarative query languages and network-aware data stores like <a href="http://graphql.org/">GraphQL</a>/<a href="https://facebook.github.io/relay/docs/relay-modern.html">Relay</a> provide the structure needed to do systematic cacheing and offline query resolution. (Doing this correctly yourself, for arbitrary REST APIs with joins and other dependencies, is not fun.) But they don’t help much with the problem of resolving local edits made while offline against changes made on the server.</li><li><a href="https://www.youtube.com/watch?v=9xFfOhasiOE">CRDTs</a> are data types that support eventual consistency when changed independently in multiple locations. <a href="https://www.youtube.com/watch?v=n338leKvqnA">Phoenix Presence</a> is an interesting use-case that uses CRDTs to sync data both between distributed backend servers and to JavaScript clients. (The core of it, implementing the same CRDT with the same semantics in both Elixir and JS, could be a great basis for a generic data synchronization system.) Another interesting project is <a href="http://lasp-lang.readme.io/docs/what-is-lasp">Lasp</a>, a distributed programming model that uses CRDTs and some programming limitations to guarantee eventual data consistency.</li><li>Model-theory-oriented approaches use database theory ideas like MVCC and Datalog and try to map them to useful properties of distributed data systems. Peter Alvaro gave a mind-bending talk <a href="https://www.youtube.com/watch?v=R2Aa4PivG0g">“I See What You Mean”</a> about how Datalog’s semantics might be helpful in a distributed system. Yifan Wu gave another talk about <a href="https://www.youtube.com/watch?v=IZe3yvVMwqM">what consistency means for user interface development</a> that makes the problem very concrete. The two also have a great <a href="https://www.youtube.com/watch?v=yRNbZSUsre0">Papers We Love talk twofer</a>: one short talk by Yifan Wu on how database consistency models differ from the mental models humans inately have about consistency of UIs, and then a longer talk by Peter Alvaro about how model theory deals with the idea of causation.</li></ul><p>An example of a real system built with these ideas is <a href="http://witheve.com/">Eve</a>, a new model for programming that is built on top of a fast distributed data store and uses a Datalog-like pattern matching language for expressing rules. I spent some time writing toy Eve programs a few months ago and really enjoyed it. The current programming IDE and docs have some rough edges, and the things it can currently do aren’t that hard to with JS. But because Eve is built on top of a solid model for distributed data, once the distribution and permissions parts are available, it’s going to be a very interesting way to write distributed apps.</p><p>This is a hard but important problem, probably the most important one for the field. But I don’t know enough to help with it yet. So I’m keeping my eye on CRDT stuff in Erlang and JavaScript, and waiting for distributed Eve to become available, but not actively working on the problem.</p><p>Those are the three important problems for UI engineering that I think about the most.</p><p>Did I miss some vectors of attack for those problems? What other important problems do you think about? Leave a comment or write me a long email!</p><h3>Responses</h3><ul><li><a href="http://ward.bay.wiki.org/view/problems-in-ui-engineering">Ward Cunningham summarizes</a> this article and offers the reflection that <a href="http://video.fed.wiki.org/view/welcome-visitors/view/federated-wiki-videos">Federated Wiki</a> has often chosen to “sidestep or postpone” features that interact with the three problems I discussed. That’s an overly humble response. Federated Wiki takes a very clever approach: solve the data syncing problem by deliberately not trying to solve it. Instead, it encourages users to make their own copy of any page and edit that separate copy. Combined with features like distributed search and social discovery, this is a great way to deal with distributing data. Letting the data just live at the network’s edges is a great solution when you can get away with it.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8b7f8b305611" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Preact in pictures]]></title>
            <link>https://asolove.medium.com/preact-in-pictures-2007ebbae54c?source=rss-4c309f3e3ba6------2</link>
            <guid isPermaLink="false">https://medium.com/p/2007ebbae54c</guid>
            <category><![CDATA[preact]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Adam Solove]]></dc:creator>
            <pubDate>Wed, 19 Apr 2017 03:58:59 GMT</pubDate>
            <atom:updated>2021-02-12T01:00:16.488Z</atom:updated>
            <content:encoded><![CDATA[<p>I gave a talk at tonight’s <a href="https://www.meetup.com/ReactJS-Denver/">ReactJS Denver</a> meetup about <a href="https://preactjs.com/">Preact</a>.</p><ul><li>Part 1 describes Preact, why you might want to use it, and the tradeoffs you have to make by not using full React.</li><li>Part 2 walks through what Preact is doing internally to render and diff a simple application. To many people, the virtual dom and component model seem like deep mysteries that mere mortals shouldn’t try to understand. As I hopefully demonstrated, this isn’t true. It’s a reasonably simple algorithm with a few fiddly little details. While React is implemented somewhat differently, knowing Preact’s internals is still a good first mental model for approaching it.</li></ul><p>The video doesn’t pick up details on the slides, so you may want to <a href="https://docs.google.com/presentation/d/1L9bTrSMHGbe7doYi-4SlJt7DCOFT4NrqRWfhhCoDH3k/edit?usp=sharing">view them directly</a>. If all you want is to see the Preact algorithm diagram, a high-res version of the final state is below the video.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FCAg3eJ_pXgc%3Fstart%3D2206%26feature%3Doembed%26start%3D2206&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DCAg3eJ_pXgc&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FCAg3eJ_pXgc%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/6cc761f4bcfee64004a4b70f647479ba/href">https://medium.com/media/6cc761f4bcfee64004a4b70f647479ba/href</a></iframe><h4>The finished diagram</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7EC-5a_AHvYKLj04jmlV5w.png" /><figcaption>Diagram of the Preact algorithm</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2007ebbae54c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The new fast: advanced web performance best practices for 2017]]></title>
            <link>https://asolove.medium.com/the-new-fast-advanced-web-performance-best-practices-for-2017-28d1684414e0?source=rss-4c309f3e3ba6------2</link>
            <guid isPermaLink="false">https://medium.com/p/28d1684414e0</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[web-performance]]></category>
            <dc:creator><![CDATA[Adam Solove]]></dc:creator>
            <pubDate>Tue, 18 Apr 2017 03:37:53 GMT</pubDate>
            <atom:updated>2017-04-24T18:26:21.523Z</atom:updated>
            <content:encoded><![CDATA[<h4>Or: what Souders’ books didn’t tell you because it only happened in the last two years.</h4><p><em>(This is a blog version of a presentation I gave at work a few weeks ago. Because I talked about a couple internal-only things, I can’t put up the video, but I hope it is worthwhile to share the slides and some notes.)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rAUL7ls-N8PzIaULiYFvzg.png" /><figcaption>The New Fast</figcaption></figure><p>This presentation is focused on bringing you up to date on the best practices for building very fast web sites. It assumes as a pre-requisite that you already know the best practices of a few years ago, which I take to be the two Steve Souders books: <a href="http://shop.oreilly.com/product/9780596529307.do">High Performance Websites</a> and <a href="http://shop.oreilly.com/product/9780596522315.do">Even Faster Web Sites</a>. It focuses on the first page load rather than local cacheing strategies for speeding up subsequent loads.</p><p>To be clear, what I’m going to touch on is advanced techniques for after you already have a CDN for your static assets, minimize your JavaScript, and (most importantly of all!) minimize the bytes you send over the network.</p><p>If you haven’t read those books and followed as much of their advice as makes sense, don’t start with the things in this post.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zOS6Z7iLp1cCf1Olr2sT9g.png" /></figure><p>One idea present in the books needs to be repeated loudly today: send less stuff over the wire! It is ridiculous the size of JavaScript apps, ad tracking scripts, and other random cruft that appears on the majority of websites.</p><p>But entropy is the natural progression of everything. If you’re going to start fast and stay fast, you need to have automated tools to identify changes that increase the size or time required to load your page. That way, when you add a new dependency or try out some new images, you don’t have to <em>remember</em> to think about performance. Externalize your care for performance into automated measurements so you can’t forget!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*udkm9oItzhwkTyYN-HazHg.png" /></figure><p>This is a review of the most important bits of advice in the Souders books, especially as implemented for a site like the one I work on, which is built in Rails and uses the Asset Pipeline for some of its assets. For many of us who learned performance in the era of backend frameworks like Rails, this is as much as we learned.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RVnjLzs7LhQyabLpMcbbCw.png" /></figure><p>When we follow that advice, we end up with a waterfall diagram that looks something like this for a first pageview on a slow mobile connection from somewhere far away from our application server. <em>(Throughout this post I use idealized waterfall diagrams like this without precise measurements, just to indicate the outline of what is happening.)</em></p><ul><li>The initial html request takes a while to resolve DNS &amp; create an HTTPS connection (the blue line), then a while to send the request and wait for the first byte of the response (light blue bar), and then a while to receive all of the response (dark blue bar.)</li><li>As the response is received, the browser discovers css, image, and javascript assets that are needed. Each of these is fetched from the CDN and requires its own new HTTPS handshake, connection, etc.</li><li>When the css in the document head is received and processed, the browser can do a first paint (purple line).</li><li>When the js is all received, parsed, and executed, the page’s dynamic content becomes interactive (orange line).</li></ul><p>If we’ve been careful to minimize what we load on the page, then even on a very crummy network connection we might be able to paint at around 3 seconds and be interactive around 4 seconds.</p><p>The rest of this post is about doing better than that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C1qBFHrU41p8CZ-A1usWiA.png" /></figure><p>The first thing we notice about the waterfall is the wasted effort creating multiple HTTPS connections to our CDN server. The connection handshake just requires negotiations some crypto choices, so once we’ve done that once it would be nice to reuse them and save the round trips.</p><p>If our CDN supports HTTP2 multiplexing, we can save those round trips and pull in the response times of static assets fetched lower down in the page, saving up to a few hundred milliseconds in the connection creation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uMX-RUHvj-IUXrVZp46cmQ.png" /></figure><p>Reusing the same connection may also make the requests faster, because each TCP connection starts slow and speeds up over time until it finds the available bandwidth on the connection. Using an already-warm connection thus saves time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0mvUoAw7DRxXS2nVZgg4MA.png" /></figure><p>Next, we might notice that we still have two HTTPS handshakes: one to our main application server (for the html content), and another to the CDN server. Adding a CDN in front of our whole site allows even our first css request to reuse an existing connection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3n0LQqhHqio5-po2Ww6T7A.png" /></figure><p>Besides saving us the roundtrips for creating a new connection, adding a CDN between users and our site can drastically improve performance. Surprisingly, it can even improve performance if the CDN doesn’t do any cacheing!</p><p>Why? Well, because of the way TCP congestion control works, two things slow down our throughput: one is the latency between the server and client, and the other is the variability of the available bandwidth. Earlier we saw that TCP slowly increases the amount of data sent until a packet is lost, then it slows down and tries again. This behavior is reasonable over stable internet connections, but produces very bad results with variable cell connections. Because of network variability, the actually-used bandwidth may be substantially less than the theoretically-available bandwidth:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OjeHFYPZaSyj3lkAj-yeGA.png" /></figure><p>The very slowest moments in the network’s available throughput determine the actually used throughput. Notice that the connection can’t take advantage of any of the times when the network connection is good, because it’s still slowly ramping up from the last time the connection was terrible.</p><p>So: high network variability makes our connection slower.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FzpPvL1Jl7Kk1e8hg8KWWQ.png" /></figure><p>But, if the latency between the client and server is less, the connection can more quickly respond to changes in network conditions because packets are exchanged faster.</p><p>This isn’t how it works, but it’s a useful metaphor: Imagine that the cell phone is sending a message to the server saying “the network connection is good, send me lots of data right away!” If the server is 40ms away, it can send the data and have it arrive before network conditions have changed much. If the server is across an ocean, 300ms away, then by the time it receives the message and sends the data, the network conditions have changed, much of the data is lost, and the connection slows down to resend it.</p><p>So higher latency also makes our connection slower, especially in the presence of network variability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DSndXcKm4xlh3nfHrhgXyg.png" /></figure><p>With these two factors combined, adding a CDN between our client and server can speed things up even if it doesn’t cache but only acts as a middleman.</p><p>We’ve partitioned our two problems (latency and network volatility) into the two sides of the CDN. Between the CDN and the server, we have high latency. But the network connection is high-bandwidth and consistent, so they can still communicate quickly. Between the CDN and the client, network volatility is high, but latency is very low, so the CDN server can flush out bytes quickly when the network is good.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LOpmPTs5fiThxgFjyEeYhg.png" /></figure><p>With the change to a CDN in front of our whole site, we’ve made some good improvements:</p><ul><li>The initial HTTPS connection creation is quicker, because the HTTPS handshake is done over a much smaller network hop to the CDN. (Of course, the connection between the CDN and our server should also be over HTTPS so nothing secretive is leaked.)</li><li>Our CSS request can reuse the same HTTPS connection that’s fetching the html, so it is done sooner and we can get to first paint earlier.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WefllwQETBcz9fPDxAR4LQ.png" /></figure><p>Of course, once we have a CDN in front of our site, we can’t just let it serve as a middle-man. We’ll want to find (mostly-)static pages and let it cache them.</p><p>If you grew up in the era of backend frameworks like rails like I did, your idea of page cacheing might be to stick things in a redis cache or let a proxy right in front of your app server handle them. But if your application servers are in Amazon East (Virginia), having the page content cached there doesn’t do a lot of good for users in Europe or Japan. By letting the CDN cache the content, it’s closer to your customers, and therefore faster.</p><p>Of course, many pages are mostly static content, but also have an account menu in the top right or other small user-specific content. One strategy is to remove that content from the initial page request, so that it is completely cacheable, and then adding it back in via JavaScript on the client side. This isn’t a good idea for pages that are <em>primarily</em> about user-specific content, like an account screen. But for a home or marketing page that is static except for one little dropdown, it can save us the cost of all the round-trips needed to fetch the page content. (In this example, I’ve pegged it at about half a second!)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KFPRV1VYdIMrFUPIms3erA.png" /></figure><p>Now our initial document request is faster, and even our assets are getting here a lot sooner. But notice that even though the document and the assets all live on our CDN server, and even though it’s easy to tell that someone requesting our home page will need our main css file, we still don’t start loading it until the browser receives the reply and parses through it. We can do better than that!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-rIUUWIB3CFNhHoh3Yh12A.png" /></figure><p>With HTTP/2 server push, we can configure our CDN server to know what static assets a page requires, and start pushing them to the client right away. This makes such a big difference, I have to reformat the title on our waterfall diagram:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EHCA8_CjN2MdlbUBrzp-bw.png" /></figure><p>Using server push, we save all the time wasted between the server starting to send the response and the browser realizing it needs to make more requests and starting to receive them. Individual assets may take a little longer to load than before because they all fight for network bandwidth, but they start so much earlier it doesn’t matter.</p><p>Notice in this example that our css now loads before the browser has even received all of the document html, so it is able to paint the top of the document right away and then progressively paint as the rest of the content is received.</p><p>Being able to paint our header, page title, and the first bit of content is a huge win for perceived performance.</p><p>Now, server push requires some discipline. In particular, while it’s very effective for a first page load, it becomes wasteful on subsequent page loads if the server is pushing assets that the browser already has cached. So once we adopt server push, we also need a way to indicate to the server that we don’t want global assets pushed to us anymore. There are various ways to do this depending on how your site is structured. You can look them up when you need to.</p><h4>What we’ve accomplished</h4><p>With the best practices of just a few years ago, we had a site that was pretty darn fast:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jNmiG0Se4FrTiZOf_jI5rA.png" /></figure><p>By sprinkling in the new powers of HTTP2, whole-site CDN, and server push, we’ve now got a site that is really damn fast:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pVB4DuwAKWIkwewFzdYOvQ.png" /></figure><p>That’s a big difference. For an ecommerce site, it could be a difference of several percentage points in conversion rate. You do know your site’s funnel metrics, right? Multiply it out to see what it might be worth to you.</p><h3>Learn more</h3><ul><li>Read <a href="https://hpbn.co/#toc">High Performance Browser Networking</a>, a free online book that describes the web network stack in detail, all the way down to cell hardware.</li><li>Watch Alex Russell’s <a href="https://www.youtube.com/watch?v=4bZvq3nodf4">Progressive Performance</a> talk, which covers ways to measure real-world performance and some of the problems you wouldn’t expect.</li><li>Learn how to <a href="https://webpack.js.org/guides/code-splitting/">split bundles</a> and <a href="https://webpack.js.org/guides/tree-shaking/">remove unneeded code</a> in Webpack, or your build tool of choice.</li><li>Work through the <a href="https://www.udacity.com/course/offline-web-applications--ud899">Offline web applications</a> course on Udacity to learn about using service workers and client-side cacheing techniques to make your site even faster on subsequent loads.</li></ul><h3>A final word</h3><p>It is very tempting — I know, it happens to me! — to think about the steps in this blog post and all the advanced configuration and ops work that could make your site faster. Shiny new things are always fun to think about.</p><p>But you have a site that’s up now and could be faster. And there’s about a 98% chance you have a lot of work to do before you should start thinking about the stuff in this post.</p><p>But before you do any of this, please<em>, please,</em> first make sure that you:</p><ul><li>have real devices, slow Android phones, for doing measurements</li><li>know how to use network link conditioner to simulate realistic network conditions</li><li>set up automated performance measurements of your real site as it is today and of new builds to catch performance regressions</li><li>split your JS and CSS so that you only load the bare minimum you need for initial render, and async load the rest</li><li>know the weight of your dependencies and have consciously chosen to use them (evaluate <a href="https://medium.com/@asolove/preact-internals-1-the-easy-parts-3a081fa36205">using Preact</a> instead of React, for example)</li></ul><p>If you’re that far, then by all means, go have some fun with the shiny new things that can make your already-fast site even faster.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=28d1684414e0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Concurrent ML has a branding problem]]></title>
            <link>https://asolove.medium.com/concurrent-ml-has-a-branding-problem-ce0286eab598?source=rss-4c309f3e3ba6------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce0286eab598</guid>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[functional-programming]]></category>
            <dc:creator><![CDATA[Adam Solove]]></dc:creator>
            <pubDate>Sun, 02 Apr 2017 22:29:03 GMT</pubDate>
            <atom:updated>2017-04-02T22:29:03.852Z</atom:updated>
            <content:encoded><![CDATA[<h4>And some suggestions from a complete outsider</h4><p>Yesterday I published a post arguing that <a href="https://medium.com/@asolove/synchronizable-abstractions-for-understandable-concurrency-64ae57cd61d1">more programmers should learn about Concurrent ML</a>. Especially for UI development, CML offers better abstractions and some ready-made solutions for problems that are currently hard or ugly to solve.</p><p>But experts in this material may have noticed something odd about my post: I tried very hard to avoid saying “Concurrent ML” except when actually discussing the history of the ideas. Instead, I used names like “synchronizable abstraction” and “protocol” in an effort to make the ideas clearer to those who had never heard of CML.</p><p><em>So, disclaimer</em>: I’m not an expert in CML. Heck, I finished the book last week and just started trying to compose my own programs with events.</p><p><em>However,</em> over the last few weeks I spent a lot of time reading through the online discussion about CML (especially as compared with Go’s synchronous messages and Erlang’s asynchronous ones) and I came to a clear conclusion:</p><blockquote>There is no convenient, generally-useful way to refer to the core ideas of CML, and this naming problem makes it harder to introduce new people to the ideas.</blockquote><p>The two obvious names to use both have significant problems:</p><ul><li>The name “Concurrent ML” sounds like it’s just the API for programming concurrency in one particular (and fairly obscure) language. Anyone who doesn’t already know what CML is will immediately dismiss it as not of interest to them. The name also doesn’t cover or bring to mind the many other implementations of CML’s core ideas.</li><li>The “Event” abstraction has a similar naming problem. In standard programming parlance, an event is a description of an already-completed action, which we may choose to respond to. An event also signifies something fundamentally simple: “event combinator” sounds almost contradictory. These two connotations bring to mind the opposite of what CML’s events represent. In addition, any phrase with “event” in it, like “event-oriented”, is already taken by not just a different abstraction, but a completely different problem domain.</li><li>As a third option, The Racket Guide and other Racket docs consistently use the name “<a href="https://docs.racket-lang.org/reference/sync.html">synchronizable events</a>”. This is better, but still has a few connotation problems and has not been widely adopted.</li></ul><p>Without a convenient name that is at least somewhat meaningful to newcomers, it is very hard to discuss these ideas. As one example, try reading Andy Wingo on <a href="https://wingolog.org/archives/2016/09/21/is-go-an-acceptable-cml">adopting the CML primitives in Guile</a>. He’s a brilliant guy, and writing for an audience of knowledgeable Scheme programmers, but notice how awkward it is to just get a handle on the ideas. Compare it to the way he talks about Go, with very concrete names. For the ideas he actually wants to talk about, he is stuck using phrases like “the CML primitives” for lack of a better name.</p><p>I would humbly suggest that for these ideas to gain broader adoption, they <strong>need a clearer name</strong>. The name should describes the CML ideas, including as implemented in Racket or any other language, and should at least connote the right problem domain to programmers reading it for the first time.</p><p>When I sat down to think about what I learned from the CML book, it had to do with a way of thinking that provided communication, synchronization, composition, and abstraction. Combining those ideas with arbitrary positive technical qualities gives us some candidate names, like:</p><ul><li>Synchronizable abstractions</li><li>First-class communication</li><li>Composable protocols</li></ul><p>Which seem to correctly connote the problem domain of concurrent communication and a solution worth learning about.</p><p>I might also venture an uninformed opinion that, if you’re introducing it to a new environment, “event” is the wrong name to use. <em>Event</em> connotes something simple and past. We want a name that describes something compound and hypothetical. It also needs to be a noun that means <em>describing,</em> rather than carrying out, an action.</p><p>What about <em>protocol</em>? After all, a protocol is a description of all possible correct communications. That gets us compound, hypothetical, and a connection to safety. In this terminology, CML’s “primitive events” become the “trivial protocols”, like sending and receiving single messages. And CML’s main features become protocol combinators: ways to build more complex protocols from simpler ones. I also think this language makes the idea sounds attractive and mind-expanding to the right audience of programmers.</p><p>I apologize for the presumption of these suggestions after only a brief period working with the CML ideas. I hope you’ll forgive me and reply with your own thoughts.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce0286eab598" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Synchronizable abstractions for understandable concurrency]]></title>
            <link>https://asolove.medium.com/synchronizable-abstractions-for-understandable-concurrency-64ae57cd61d1?source=rss-4c309f3e3ba6------2</link>
            <guid isPermaLink="false">https://medium.com/p/64ae57cd61d1</guid>
            <category><![CDATA[functional-programming]]></category>
            <category><![CDATA[concurrency]]></category>
            <dc:creator><![CDATA[Adam Solove]]></dc:creator>
            <pubDate>Sun, 02 Apr 2017 02:32:19 GMT</pubDate>
            <atom:updated>2022-12-01T18:20:30.615Z</atom:updated>
            <content:encoded><![CDATA[<h4>Or: what is Concurrent ML all about?</h4><p>In this post I’d like to introduce you to a powerful idea for thinking about concurrent programs. Synchronizable abstractions were developed more than twenty years ago by John Reppy in his work creating the language <a href="http://cml.cs.uchicago.edu/">Concurrent ML</a>. Despite that age, and their use in several large-scale systems, synchronizable abstractions remain little known.</p><h4>Why learn about synchronizable abstractions?</h4><ul><li>They’ll expand your mind by unifying Go’s synchronous channels, Haskell’s MVars, and JavaScript’s promises under one new primitive.</li><li>They’re already used by programmers with trustworthy taste, like the implementers of <a href="https://docs.racket-lang.org/reference/sync.html">Racket</a> and <a href="https://github.com/wingo/fibers">Guile Scheme</a>.</li><li>Concurrent ML partly inspired React Fiber and can help us understand some of the trickiest concurrency issues in JavaScript UIs.</li></ul><p>One blog post isn’t enough to cover all of the detail of synchronizable abstractions. So my goals are less ambitions: to <strong>motivate</strong> the ideas by showing the problems they solve, to <strong>outline</strong> the primitives, combinators, and programming style they allow, and then to <strong>suggest resources</strong> if you want to learn more.</p><h3><strong>Understandable concurrency in Go</strong></h3><p>Let’s start by solving a small problem using Go-style concurrency, without the synchronizable abstractions primitives. In a typical Go program, many lightweight threads communicate with each other via synchronous messages sent over channels. When one thread wants to send or receive a message over a channel, it blocks until a suitable partner thread is ready to receive or send. This synchronous message passing makes it possible for messages to both communicate and synchronize between threads.</p><p>Let’s see an example of this style of program <em>(and also get used to the ES6-like pseudo-code used in this article)</em>. We’ll implement a simple communication protocol where two threads atomically swap values. Our swap function accepts the value to send and returns the value received. Here’s how we want it to work:</p><pre>let swap = new SwapChannel()</pre><pre>let swapper = oldValue =&gt; spawn(() =&gt;<br>  let newValue = swap(oldValue)<br>  console.log(“Swapped”, oldValue, “for”, newValue))</pre><pre>swapper(1)<br>swapper(2)<br>// “Swapped 1 for 2” &amp; “Swapped 2 for 1” are printed, in some order.</pre><p>Because channels only send messages in one direction, our implementation will need to send two message and make sure that they happen atomically.</p><p>A naive implementation of a swap channel might look like this:</p><pre>let SwapChannel = () =&gt; {<br>  let ch = new Channel() <br>  return (value) =&gt; {<br>    select {<br>      case send(ch, value):<br>        return receive(ch)<br>      case newValue = receive(ch):<br>        send(ch, value)<br>        return newValue<br>    }<br>  }<br>}</pre><p>It first constructs a plain channel and then returns a function that clients can call when they want to perform a swap. When a thread calls that function, it tries to both send and receive over the channel, blocking until another thread is also ready to swap. (Note the new select/case syntax, which allows threads to propose multiple communications, block until one of them succeeds, and then carry out the statements appropriate to that case.) The thread that succeeds in sending its value then waits to receive a value in return. The thread that first succeeds in receiving likewise then sends its value back.</p><p>Unfortunately, this code has a bug. If three threads all call swap at roughly the same time, we may not get an atomic swap between two of them, but instead a three-way cyclic exchange between all three threads. This is not the atomic swap we wanted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lFWJYuVbbvR0qbPUg_GoOw.png" /></figure><p><em>Each line represents a thread, with time moving from left to right. Threads offer to receive (empty circle) or send (full circle) on a channel. They then block (dotted lines) until one of the communications actually occurs (arrow).</em></p><p>To fix this problem, we need to ensure that once two threads do the first step in the swap, they can only communicate with each other when doing the second step. The solution is to create a new channel, send it as part of the first communication, then use it for the second communication.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1XfUilCl9hpvWpwxGCKmcQ.png" /><figcaption>A private channel ensures a correct swap</figcaption></figure><p>Even if a third thread comes along in the middle of the swap, it can’t interfere with the in-process transaction because it doesn’t have access to the private channel. It instead blocks and waits for another thread that wants to perform a swap.</p><p>Implementing this idea requires only small changes:</p><pre>let SwapChannel = () =&gt; {<br>  let ch = new Channel();<br>  return value =&gt; {<br>    <strong>let replyCh = new Channel();</strong><br>    select {<br>      case send(ch, <strong>[</strong>value, <strong>replyCh]</strong>):<br>        return receive(<strong>replyCh</strong>);<br>      case <strong>[</strong>newValue, <strong>replyCh]</strong> = receive(ch):<br>        send(<strong>replyCh</strong>, value)<br>        return newValue<br>    }<br>  }<br>}</pre><p>Notice that in this brief code, synchronous message passing is used for communication, synchronization, and security. And the correctness of our swap operation is encapsulated inside the returned function, preventing any thread from violating the protocol.</p><h4><strong>The tradeoff: abstraction, composition, or safety?</strong></h4><p>Now that we have a safe swap channel abstraction, we want to use it in other server processes. (Remember that while swap is trivial, it is an example of lots of interesting multi-phase communications, like request-response exchanges.)</p><p>Let’s write a simple server process that wraps some state. We’ll say that it needs to receive commands on one channel and make part of its data available over another channel:</p><pre>let OurServer = (state) =&gt; {<br>  let commandCh = new Channel<br>  let dataCh = new Channel</pre><pre>  let loop = (state) =&gt; {<br>    select {<br>      case command = receive(commandCh):<br>        loop(executeCommand(command, state))<br>      case send(dataCh, state.dataToShare):<br>        loop(state)<br>    }<br>  }<br>  spawn () =&gt; loop(startState)<br>  <br>  return {<br>    getData: () =&gt; receive(dataCh),<br>    sendCommand: (command) =&gt; send(commandCh, command)<br>  }<br>}</pre><p>The server thread loops forever, continually ready to either receive a command or send some of its data. Again, all the details of the internal communication are abstracted away from client threads, who only have access to the interface functions.</p><p>Now let’s add what should be a simple requirement: the server should also be able to participate in a swap operation. Just like receiving commands and sending data, the swap should be available every step through the loop. Like any other operation used in a select, it should complete if it’s the first operation to find a partner, but otherwise let the existing send and receive operations carry out their responsibilities.</p><p>What we would like is to write a select statement like this:</p><pre>select {<br>  case command = receive(commandCh):<br>    loop(executeCommand(command, state))<br>  case send(dataCh, state):<br>    loop(state)<br>  <strong>case value = swap(state.dataToSwap)</strong>:<br>    // do something with the value<br>    loop(state)<br>}</pre><p>Unfortunately, in languages that have channel operations but not synchronizable abstractions, this isn’t possible. To see why, let’s examine the semantics of our select/case syntax. Our case statements looks like they contain function calls to receive and send, but they clearly can’t be executed that way. Whichever was called first would block until it had been completed. Instead, select must be implemented by gathering descriptions of all the possible communications in the case statements, then offering all of them at the same time, and finally blocking until the first one completes.</p><p>Because select needs to introspect on the communications in its case statements, it doesn’t know what to make of our call to swap, which is just a variable holding a function we wrote. The select syntax can’t “see into” our function and get the description of its desired communication to offer it in parallel with the other case statements.</p><p>This is a limitation of most languages with channel-based concurrency. We can build new communication protocols on top of channels, like swap. And we can use functions to abstract their details from clients. But we can only abstract over the <em>execution</em> of our protocol. We can’t abstract the <em>description</em> of the protocol, such that it can be composed with other operations via select.</p><p>If we’re stuck in such a language and really need to both swap and perform other operations, we have a few choices:</p><ul><li>We can spawn a separate thread to only perform the swaps. This child thread loops forever: first receiving a message with the server’s current state, and then offering that as a swap. But when it swaps, the child thread may hold an out-of-date version of the parent’s state. So this solution enables us to compose our swap abstraction into a larger process, but only by trading away safety and understanding.</li><li>A more common solution is to break the encapsulation of our swap abstraction and inline its operations into our server thread:</li></ul><pre>select {<br>  <strong>let replyCh = new Channel</strong><br>  case command = receive(commandCh):<br>    loop(executeCommand(command, state))<br>  case send(dataCh, state):<br>    loop(state)<br>  <strong>// inline the definition of swap:<br>  case send(swapCh, [replyCh, state.dataToSwap]):<br>    let newValue = receive(replyCh)<br>    loop(state)<br>  case [replyCh, newValue] = receive(swapCh):<br>    // do something with newValue<br>    loop(state)<br></strong>}</pre><p>Now we have a server that is the composition of sending, receiving, and swapping values. But in order to compose it with other operations, we had to eliminate the abstraction around swap. The raw channel used for the protocol is now exposed to client threads that have to re-implement its steps, and might get it wrong in unsafe ways. At minimum we have lost encapsulation, but in all likelihood we’ve also lost safety.</p><h4>Imagining our way out of the problem</h4><p>Let’s stop and look at our problem. We would like to be able to build our own concurrent protocols that are encapsulated, easy to reason about, and also composable. The current language lets us abstract over the <em>execution </em>of our protocol, but not over a <em>description</em> of it that could be composed just like select can compose simple channel sends and receives.</p><p>What we need is a way to use <em>possible </em>communications as values, separate from executing them. Then our swap operation could just return a value that could be composed into a select statement.</p><p>Our possible communication values need a shorter name. We’ll call them <em>events</em>. Let’s imagine what we need from events:</p><ul><li>First, we need versions of send and receive that don’t perform the operation but just return a description that can be performed later. We’ll call those sendEv and receiveEv.</li><li>Next, we need ways of combining events together. For example, our swap function needs to return an event that represents its current select statement, the alternative between two events. We’ll call that selectEv.</li><li>Because selectEv receives event values, it can be a normal function rather than custom syntax. But to fully replace select/case, we also need a way to express the idea that certain statements should be executed if an event actually occurs. We’ll call that combinator wrap.</li><li>Our swap function returns an event value, which might be used and reused multiple times. So instead of creating a private channel when the event is built, we need to create a new one every time the event is used in a communication. We can do exactly this with guard, which wraps an event-generating function and returns an event that calls that function every time it is used in a communication.</li></ul><p>Here is our swap protocol with those changes so that it returns an event:</p><pre>let SwapChannel = () =&gt; {<br>  let ch = new Channel();<br>  return (value) =&gt; <strong>guard</strong>(() =&gt; {<br>    let replyCh = new Channel();<br>    <strong>return selectEv</strong>([<br>      <strong>wrap</strong>(receiveEv(ch), ([newValue, replyCh]) =&gt; {<br>        send(replyCh, value)<br>        return newValue<br>      }),<br>      <strong>wrap</strong>(sendEv(ch, [value, replyCh]), () =&gt; {<br>        return receive(replyCh)<br>      })<br>    ])<br>  })<br>}</pre><p>We changed a few function names, swapped our custom syntax for function callbacks, and otherwise changed very little. Client threads can now call the returned function to get an event that describes how to conduct the operation.</p><p>Next, we need to actually <em>execute</em> our events. We can call sync on an event to execute it, blocking until it succeeds. A very common pattern is to call selectEv on an array of events and then sync on the resulting single event. So we’ll give that combination a shorter name: select (which is no longer needed as a keyword).</p><p>Using our new event-returning swap operation and a select call to actually execute it, we can now write our original server exactly as we wanted to:</p><pre>let OurServer = (state, swapEv) =&gt; {<br>  let commandCh = new Channel<br>  let dataCh = new Channel<br>  let loop = (state) =&gt; {<br>    <strong>select</strong>([<br>      <strong>wrap</strong>(<strong>receiveEv</strong>(commandCh), <strong>(command) =&gt;</strong><br>        loop(executeCommand(command, state)),<br>      <strong>wrap</strong>(<strong>sendEv</strong>(dataCh, state), <strong>() =&gt;</strong><br>        loop(state),<br>      <strong>wrap</strong>(<strong>swapEv</strong>(state.dataToSwap), <strong>(received) =&gt;</strong><br>        loop(doSomething(state, received))<br>    ])<br>  }</pre><pre>  spawn () =&gt; loop(startState)<br>  <br>  return {<br>    <strong>getDataEv</strong>: () =&gt; <strong>receiveEv</strong>(dataCh),<br>    <strong>sendCommandEv</strong>: (command) =&gt; <strong>sendEv</strong>(commandCh, command)<br>  }<br>}</pre><p>When we started this section, we had an abstract and correct swap protocol. But limited to the language of channels, we couldn’t compose it with any other operation. We imagined a new type of value, the event, that could <em>describe </em>our protocol without executing it. Rewriting swap in terms of events, we were able to retain its encapsulation and safety while also being able to compose it with other events.</p><p>By now you’ve guessed it: the new <em>event</em> values that we imagined are the core primitive of synchronizable abstractions.</p><h3><strong>Synchronizable abstractions</strong></h3><p>We’ve already seen that an event is a description of a possible communication. We’ve also seen that some events aren’t just single operations, but instead encapsulate complex logic about multiple steps and alternatives.</p><p>The lifecycle of an event has three distinct moments:</p><ol><li>An event value is <em>constructed</em> and may encapsulate references to various channels, alternatives, and callbacks.</li><li>Next is is <em>sync</em>ed, that is: offered to the system as an available operation in search of a partner. When we sync a compound event, all of its alternatives are synced at the same time.</li><li>Finally, if the event <em>succeeds</em>, then its communication was carried out and it returns a value representing what was communicated. Alternately, it <em>fails</em> and is not carried out. Each time a compound event is synced, exactly one of its children succeeds, while the rest fail.</li></ol><p>Now that we know <em>what</em> an event is and <em>when</em> its different parts occur, we may ask <em>how </em>do we build programs from events. Ancient wisdom suggests we should ask three questions: What primitive events are there? How can we combine them together into more complex events? What means of abstraction do we have?</p><h4>Primitives</h4><ul><li>We’ve already used sendEv(ch, value)and receiveEv(ch) to build events representing channel sends and receives. When executed, a send has a result of nothing, while receive results in the value received.</li><li>Waiting on a promise or MVar, with the result of the value that it wraps.</li><li>Waiting for a semaphore to not be blocking.</li><li>System-level I/O, including reading and writing to disk, or sending and receiving messages from the network.</li><li>The CML implementation also includes useful primitive events like never (which never succeeds) and always(value) (which is always available and may succeed with the given value.</li></ul><h4>Combinators</h4><ul><li>selectEv takes a list of child events and return a new parent event. When the parent is synced, it syncs all of its children, but only one of them can possibly succeed. The parent event then succeeds with the value of whichever child suceeded.</li><li>We can use guard(callback) to build an event that will be initialized later. When the resulting event is synced, the callback is called and should return a new event, which will actually be used. This is useful if we want to do work when our event is first offered to the system, like sending a setup message and then waiting for a response, or initializing variables relative to the sync time. A small example is timeoutEv(secs), which represents an event that will be available a certain number of seconds after it is synced.</li><li>We’ve already used wrap(ev, callback), which returns an event that is synced exactly like ev but, if it succeeds, passes its result to the callback and then uses the return value of the callback as the event’s result. This is useful to chain together steps in a protocol, as we did in swap.</li><li>The opposite of wrap is wrapAbort(ev, callback), which wraps an event with a callback to execute only if the event is synced but fails. If you use guard to set up state or send a speculative message, wrapAbort is the place to tear it down or cancel it if the event does not succeed.</li></ul><h4>Means of abstraction</h4><p>Unsurprisingly, our means of abstraction is <em>lambda, </em>the encapsulation of detail provided by function calls and lexical scope. Indeed, the entire point of synchronizable abstractions is to produce values that can be passed to and returned from lambdas. This allow us to compose operations together while also abstracting their details from each other.</p><h4>Infrastructure</h4><p>One final set of details: our primitives and combinators will only work as promised with appropriate runtime infrastructure to avoid thread starvation or memory leaks. Two key properties of the CML runtime are:</p><ul><li><strong>Preemptive scheduling</strong>. Rather than allow individual threads to keep working until they block or are finished, the Concurrent ML runtime will interrupt a running thread to make sure others have a chance to execute. The scheduler is also deliberately designed to give enough time to both computation-intensive and communication-intensive threads. <strong>React Fiber </strong>is an attempt to get some of the benefits of preemptive scheduling in JavaScript, which only offers cooperative scheduling. To ensure React doesn’t prevent other script from running, it has been re-written so that it can do partial work, stop to let other scripts run, and then resume execution where it left off.</li><li><strong>Smart garbage collection</strong>. One common Concurrent ML idiom is to spawn a thread that waits on a communication, which may or may not ever occur. It would be quite wasteful if all those threads just sat around in memory, blocking forever. Fortunately, the CML runtime is quite intelligent. If a thread is blocked waiting to read or write to a channel, but no other threads have references to that channel, then both the channel and the blocked thread can be garbage collected. This avoids memory leaks and encourages a style that uses threads and channels for resource allocation and collection.<br> <br>This is especially relevant to concurrency in JavaScript. One of the most common source of bugs in JS apps is long promise chains which then perform some mutation of shared state as their last step. Consider a search widget that fires off an ajax request and then chains that with a setState to show the results. If a user searches for two things in quick succession, they may end up with the wrong results if the first search returns more slowly than the second. This is no different from any other concurrent program written without synchronization. A generic, boilerplate-free solution would be to treat the component and the promise chain as separate “threads” that can only share state via communication. The component would then listen for information from the channel of the most-recently conducted search request. All previous requests could be automatically cancelled and garbage collected, preventing them from interfering with the component’s state.</li></ul><h3>Resources to learn more</h3><p>I hope that I have motivated you to learn more about synchronizable abstractions and given you a high-level overview of what awaits you.</p><p>To deepen your understanding, I have three suggestions:</p><ul><li>If you have the time, I highly recommend Professor Reppy’s book, <a href="https://www.amazon.com/dp/0521480892/ref=cm_cr_ryp_prd_ttl_sol_0"><em>Concurrent Programming in ML</em></a><em>. </em>It explains the what, why and how of CML from scratch, with both many small examples and several chapter-length case studies of complex applications. (The only prerequisite for this book is learning to read basic ML type and function declarations. Skimming through chapters 1 and 2 of <a href="https://realworldocaml.org/v1/en/html/index.html"><em>Real World OCaml</em></a><em> </em>is probably sufficient.)</li><li>If you’re more of a Schemer, the chapter “<a href="https://docs.racket-lang.org/guide/concurrency.html">Concurrency and Synchronization</a>” in The Racket Guide provides an introduction to Racket’s embedding of events. This will not give you as thorough an introduction to patterns of event programming, but it does have many short code samples that are useful in understanding specific functions.</li><li>For an example of how synchronizable abstractions can implement complex system behavior, read the paper <a href="https://www.cs.utah.edu/plt/publications/pldi04-ff.pdf">“Kill-Safe Synchronization Abstractions”</a>, which describes an especially clever use of events in the implementation of DrRacket and uses several complex patterns of programming with event composition.</li></ul><p>Unfortunately, all of the resources I have seen present programs as finished artifacs. They lack information on <em>how</em> to arrive at a correct program, or even exercises to test and expand your knowledge. To remedy this and test my own knowledge, I have started <a href="https://github.com/asolove/explore-concurrent-ml">re-implementing the book’s examples</a> in Reason and writing tests for them. I have learned a lot from this process and assume that doing it in Racket or Guile would be even more educational.</p><p>I am also trying to write exercises appropriate for several sections in the CML book and hope to post them soon.</p><p>Of course, what the world could really use is <em>The Synchronized Schemer</em>, a book presenting a minimal form of events along with commandments and exercises for writing correct concurrent programs. Until that exists, we’ll just have to do the best we can with the resources we have.</p><h4>Acknowledgements</h4><p>I would like to thank many people for helping with my education and the content of this post:</p><ul><li><a href="https://medium.com/u/e17ec97253c5">Kris Kowal</a> read an earlier version of this post, provided helpful feedback, and suggested several related ideas worth exploring.</li><li>Several Facebookers whispered about their React-optimization experiments using CML, which was my first introduction to the idea.</li><li>Andy Wingo and Sam Tobin-Hochstadt had <a href="https://wingolog.org/archives/2016/09/21/is-go-an-acceptable-cml">a discussion on Andy’s blog about CML’s primitives in Guile</a>. This was my first sense that the ideas had broad applicability and were worth adopting outside ML.</li><li><a href="https://medium.com/u/44d51c4c2322">Chris Meiklejohn</a> was up late one night and responded enthusiastically to my tweet asking if reading Reppy’s book was worthwile.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=64ae57cd61d1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Preact internals #3: some fiddly little bits]]></title>
            <link>https://asolove.medium.com/preact-internals-3-some-fiddly-little-bits-f353b1ad7abc?source=rss-4c309f3e3ba6------2</link>
            <guid isPermaLink="false">https://medium.com/p/f353b1ad7abc</guid>
            <category><![CDATA[preact]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[react]]></category>
            <dc:creator><![CDATA[Adam Solove]]></dc:creator>
            <pubDate>Mon, 27 Mar 2017 01:05:36 GMT</pubDate>
            <atom:updated>2017-04-24T15:34:50.331Z</atom:updated>
            <content:encoded><![CDATA[<h4>How the virtual dom actually gets rendered to the DOM</h4><p>So far in this series, we’ve <a href="https://medium.com/@asolove/preact-internals-1-the-easy-parts-3a081fa36205#.c2tf1w39x">gotten a feel for the Preact repo</a> and looked at <a href="https://medium.com/@asolove/preact-internals-2-the-component-model-36a05e32957b#.v3ekjqydi">the component model</a> from a high level. When we left off last time, we had just finished our picture of the muturally-recursive process of diffing virtual DOM trees and rendering components:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H4ysOfvyT5BMKABhbyslwA.png" /></figure><p>And we had also made a long list of details that this picture left out, like:</p><ul><li>Crazy DOM special cases, like event binding and XML namespaces</li><li>Pairing DOM children to vnode children and how key works</li><li>Recycling DOM nodes for performance</li><li>Functional and higher-order components</li><li>Hydration, mount and unmount callbacks, and refs</li></ul><p>These details split nicely between things about the DOM (the first three) and things about components (the last two). We’ll tackle the DOM-related ones in this post, and the component ones next time.</p><p>This will be a <strong>much more detailed </strong>and<strong> code-heavy</strong> post than the ones so far. It’s also <strong>much longer</strong>.</p><p>Fortunately for those of us who use Preact, but unfortunately for those of us who want to read it, Jason is a masterful code golfer. A lot of this code is hard to understand because he has made it as short and fast as possible, often by reusing variables or mutating them in unexpected places. As we read through, I’ll do my best to point out and explain these cases.</p><p>Are you ready? Let’s do this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zA0lhxaRQOiK7Tp7sdV9UQ.png" /></figure><h4>Mucking about with the DOM</h4><p>We know that idiff is responsible for mutating the one DOM node to match a vnode. But most of the actual manipulation is done through helper functions defined in <a href="https://github.com/developit/preact/blob/8567c8bc13bc7fc60eeff4cad3bc9217822b4577/src/dom/index.js">src/dom/index.js</a>. There are a couple easy ones, for removing a node from the DOM:</p><pre>/** Removes a given DOM Node from its parent. */<br>export function removeNode(node) {<br>  let p = node.parentNode;<br>  if (p) p.removeChild(node);<br>}</pre><p>There’s another for safely setting DOM attributes without throwing exceptions: (although I wonder if it should at least log these to let the developer know)</p><pre>/** Attempt to set a DOM property to the given value. <br>  * IE &amp; FF throw for certain property-value combinations. */<br>function setProperty(node, name, value) {<br>  try {<br>    node[name] = value;<br>  } catch (e) { }<br>}</pre><p>And a wrapper for DOM event handlers that exposes a global hook for normalizing events, if you should desire. (We’ll see why the indirection of eventProxy is useful in just a little while.)</p><pre>/** Proxy an event to hooked event handlers <br>  * @private */<br>function eventProxy(e) {<br>  return this._listeners[e.type](<br>    options.event &amp;&amp; options.event(e) || e);<br>}</pre><p>But the real business of this file is in the exported function setAccessor, which is used to set an arbitrary key/value pair from a vnode’s props onto a real DOM node. The function’s signature includes some extra information that is sometimes needed, like the vdom value previously assigned for this attribute, and whether we’re inside an SVG context:</p><pre>/** Set a named attribute on the given Node, with special behavior for some names and event handlers.<br> * If `value` is `null`, the attribute/handler will be removed.<br> * @param {Element} node An element to mutate<br> * @param {string} name The name/key to set, such as an event or attribute name<br> * @param {any} old The last value that was set for this name/node pair<br> * @param {any} value An attribute value, such as a function to be used as an event handler<br> * @param {Boolean} isSvg Are we currently diffing inside an svg?<br> * @private<br> */<br>export function setAccessor(node, name, old, value, isSvg) {</pre><p>The function then proceeds to consider a bunch of different cases for the name and value that we want to set on the node.</p><p>Preact supports setting classes with the prop key class or className, so it normalizes those to the same name. It also supports setting classes via an object, like hashToClassName({button: true, active: false }) === &quot;button&quot; so it converts objects to a string if we’re setting the class:</p><pre>if (name===&#39;className&#39;) name = &#39;class&#39;;</pre><pre>if (name===&#39;class&#39; &amp;&amp; value &amp;&amp; typeof value===&#39;object&#39;) { <br>  value = hashToClassName(value);<br>}</pre><p>The key attribute is used by the diffing algorithm but not intended to be rendered to the DOM, so it’s ignored:</p><pre>if (name===&#39;key&#39;) {<br>  // ignore</pre><p>If we’re setting the class, it’s already normalized to a string so just set it:</p><pre>} else if (name===&#39;class&#39; &amp;&amp; !isSvg) {<br>  node.className = value || &#39;&#39;;<br>}</pre><p>If we’re setting the node’s styles, we first need to clear off the previous styles and then set the new ones, either via a string or an object of property/value pairs. Note the bit about NON_DIMENSION_PROPS. That’s so you can use numerical values like { width: 10 } and Preact will automatically add the &#39;px&#39; for you, unless the property name is in that list of non-dimensioned property names.</p><pre>else if (name===&#39;style&#39;) {<br>  if (!value || isString(value) || isString(old)) { <br>      node.style.cssText = value || &#39;&#39;;<br>  }<br>  if (value &amp;&amp; typeof value===&#39;object&#39;) {<br>    if (!isString(old)) {<br>      for (let i in old) if (!(i in value)) node.style[i] = &#39;&#39;;<br>    }<br>    for (let i in value) {<br>      node.style[i] = <br>        typeof value[i]===&#39;number&#39; &amp;&amp; !NON_DIMENSION_PROPS[i] ?<br>          (value[i]+&#39;px&#39;) : value[i];<br>    }<br>  }<br>}</pre><p>Next up is dangerously setting a string as inner html:</p><pre>else if (name===&#39;dangerouslySetInnerHTML&#39;) {<br>  if (value) node.innerHTML = value.__html || &#39;&#39;;<br>}</pre><p>Setting event handlers is an interesting optimization. Rather than re-set event handlers every time we render, which could be expensive, we save a map from event names to user-defined handlers on the DOM node as node._listeners. Then we only add/remove DOM event listeners if the set of events that is being listened to changes. The actual event handler attached to the DOM is just eventProxy, which we saw above, which looks into our map of event handlers to call the right one.</p><pre>else if (name[0]==&#39;o&#39; &amp;&amp; name[1]==&#39;n&#39;) {<br>  let l = node._listeners || (node._listeners = {});<br>      name = toLowerCase(name.substring(2));</pre><pre>  // <strong>adding a callback to a new event</strong> <br>  if (value) {<br>    if (!l[name]) node.addEventListener(name, eventProxy, <br>                                        !!NON_BUBBLING_EVENTS[name]);<br>  } </pre><pre>  // <strong>removing a callback from an event no longer listened to</strong><br>  else if (l[name]) {<br>    node.removeEventListener(name, eventProxy,        <br>                             !!NON_BUBBLING_EVENTS[name]);<br>  }<br>  l[name] = value;<br>}</pre><p>If we haven’t hit on the property name yet, then it might be a plain old DOM property, which we set directly, using the setProperty helper that we saw earlier. If the value is undefined, null, or false, then we just remove the attribute.</p><pre>else if (name!==&#39;list&#39; &amp;&amp; name!==&#39;type&#39; &amp;&amp; !isSvg &amp;&amp; name in node) {<br>  setProperty(node, name, value==null ? &#39;&#39; : value);<br>  if (value==null || value===false) node.removeAttribute(name);<br>}</pre><p>There’s one final case: SVG attributes. For these, we need to use the special namespace-aware attribute methods:</p><pre>else {<br>  let ns = isSvg &amp;&amp; name.match(/^xlink<strong>\:</strong>?(.+)/);<br>  if (value==null || value===false) {<br>    if (ns)<br>      <strong>node.removeAttributeNS</strong>(&#39;http://www.w3.org/1999/xlink&#39;, toLowerCase(ns[1]));<br>    else<br>      node.removeAttribute(name);<br>  }<br>  else if (typeof value!==&#39;object&#39; &amp;&amp; !isFunction(value)) {<br>    if (ns)<br>      <strong>node.setAttributeNS</strong>(&#39;http://www.w3.org/1999/xlink&#39;, toLowerCase(ns[1]), value);<br>    else<br>      node.setAttribute(name, value);<br>  }<br>}</pre><p>And that’s it. Those are the special cases that of attributes that can be set to the DOM from keys in the vnode’s props. To review, the cases Preact supports are:</p><ul><li>class and className, with either a string or an object, to set the class</li><li>style, either a string or an object, to set style properties</li><li>dangerouslySetInnerHTML for setting a string as the innerHTML of a node</li><li>event handlers via onEventName</li><li>DOM properties</li><li>SVG attributes</li></ul><p>Now that we understand what setAccessor does to set individual DOM attributes, let’s see how it’s used. It’s called by diffAttributes, which lives in the vdom diffing code and is called by idiff as it changes a node to match a vdom specification. That’s this part of our diagram:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zA0lhxaRQOiK7Tp7sdV9UQ.png" /></figure><p><a href="https://github.com/developit/preact/blob/8567c8bc13bc7fc60eeff4cad3bc9217822b4577/src/vdom/diff.js#L305">The source of </a><a href="https://github.com/developit/preact/blob/8567c8bc13bc7fc60eeff4cad3bc9217822b4577/src/vdom/diff.js#L305">diffAttributes</a> is pretty simple. It runs through any attributes in the previous state but not the current one and removes them, then goes through all the attributes in the current state and sets them. But there are a couple slightly crazy bits we need to remember.</p><p>First, <a href="https://github.com/developit/preact/blob/8567c8bc13bc7fc60eeff4cad3bc9217822b4577/src/vdom/diff.js#L317">certain attributes don’t even get sent to </a><a href="https://github.com/developit/preact/blob/8567c8bc13bc7fc60eeff4cad3bc9217822b4577/src/vdom/diff.js#L317">setAccessor</a> because we don’t want to even try to set them on the DOM. children and innerHTML get filtered out, since we never want to try to render them. Then there’s one additional test the attribute has to pass:</p><pre>// either it&#39;s a new attribute, or ...<br>!(name in old) || <br>  // its value is not equal to ...<br>  attrs[name]!==<br>    // if it&#39;s a DOM form value<br>    (name===&#39;value&#39; || name===&#39;checked&#39; ? <br>      // then the value on the DOM node<br>      dom[name] :<br>      // otherwise, the previous version of the prop<br>      old[name]))</pre><p>That boolean expression had me scratching my head for a while. But what it’s trying to say is:</p><ul><li>If the current value is the same as the previous one, there’s no need to set it again.</li><li>Unless it’s value or checked, because those might have been changed by user interaction since the last render. So to see if we need to re-set this attribute, we need to check against what’s on the real node, not what’s in the old props.</li><li>And unless this prop wasn’t in the old props, because then we definitely need to re-render it and can avoid running the logic above.</li></ul><p>That’s it for the first bit of complexity about diffAttributes, how it decides which attributes to send on to setAccessor.</p><p>The second bit of complexity is an extra responsibility that diffAttributes has. When removing attributes that were in the old state but not the new one, diffAttributes calls setAccessor this way:</p><pre>setAccessor(dom, name, old[name], <strong>old[name] = undefined</strong>, svgMode);</pre><p>And when updating an attribute, it calls it this way:</p><pre>setAccessor(dom, name, old[name], <strong>old[name] = attrs[name]</strong>, svgMode);</pre><p>Note that in both places, it doesn’t just pass in a value to set for this attribute on the DOM node, but also mutates the old object to have the new value. To understand why, we need to understand another little detail that we haven’t gotten to yet. In order to do faster diffing, and especially in order to respect the key property when diffing children, Preact needs access to the vnode properties used the last time it rendered a specific DOM node. So it caches these properties as an attribute on the node. And in this head-scratching code above, it turns out that old isn’t just an object of the old attributes. It’s actually the cache of attribute properties that is saved on the DOM node. Which means that diffAttributes is, in this weirdly indirect way, also responsible for updating the prop cache as it updates the DOM attributes. (Remember, this is some seriously-golfed code. Don’t write your application logic like this.)</p><p>We already knew that idiff is responsible for updating a DOM node to match a vdom tree. And now we know all the details of how it makes the attributes on the DOM node match the properties of the vnode. That’s a big chunk of the codebase, so savor the feeling of understanding it! Go get some chocolate or stand up and stretch. Seriously. The rest will still be here when you get back.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JLShS26Yv6c-qU20xvGWBg.png" /></figure><h4>Pairing children, or: how key works</h4><p>We now understand how idiff makes one DOM node’s attributes match one vdom node. The next thing to do is to make that DOM node’s children match the vdom node’s children. Most of the work — actually updating the childrens’ properties — will happen by calling idiff recursively. But before that can happen, innerDiffNode needs to decide which DOM child node to pair with each of the vdom children.</p><p>Here’s the overall plan of attack:</p><ol><li>Index the existing DOM children to make them easier to look up.</li><li>For each vnode, use those indexes to try to find a matching DOM node.</li><li>Diff the vnode and (possible) matching DOM node so they match.</li><li>Insert the DOM node into the right place in the DOM, if necessary.</li><li>Clean up any old DOM nodes that didn’t get used.</li></ol><p>Let’s walk through the code and look at how that happens. First we set up the necessary variables, especially ones that track the state of our indexes of the existing child DOM nodes:</p><pre>function innerDiffNode(dom, vchildren, context, mountAll, absorb) {<br>   let originalChildren = dom.childNodes,<br>       children = [],   // <strong>array of old child DOM nodes without keys<br>       </strong>min = 0,         // minimum index in `children` that has a node<br>       childrenLen = 0, // length of `children`<br>       keyed = {},      // <strong>object of old child DOM nodes by key</strong><br>       keyedLen = 0,    // how many nodes are in `keyed`<br>       len = originalChildren.length,<br>       vlen = vchildren &amp;&amp; vchildren.length,<br>       j, c, vchild, child;</pre><p>Then we iterate through all the old DOM children and sort them into the right index:</p><pre>if (len) {<br>  for (let i=0; i&lt;len; i++) {<br>    let child = originalChildren[i],<br>        props = child[ATTR_KEY], // <strong>props from last render</strong><br>        key = vlen ? ((c = child._component) ? c.__key : props ? props.key : null) : null; // <strong>key from last render</strong></pre><pre><strong>    // if it had a key, added it to index by key<br></strong>    if (key!=null) {<br>      keyedLen++;<br>      keyed[key] = child;<br>    }<br>    <br>    <strong>// otherwise, add it to the list of non-keyed old children<br>    </strong>else if (hydrating || absorb || props || child instanceof Text) {      <br>      children[childrenLen++] = child;<br>    }<br>  }<br>}</pre><p>Now that we’ve indexed all the DOM nodes, we loop through all the child vnodes and try to find each of them a match. If the vnode has a key, we look for a match in the keyed index; otherwise, we look for a matching tag in children.</p><pre>for (let i=0; i&lt;vlen; i++) {<br>  vchild = vchildren[i]; <strong>// current vnode seeking a match</strong><br>  child = null;          <strong>// the matched DOM node<br></strong>  <br>  // <strong>attempt to find a node based on key matching<br></strong>  let key = vchild.key;<br>  if (key!=null) {<br>    if (keyedLen &amp;&amp; key in keyed) {<br>      child = keyed[key];<br>      keyed[key] = undefined; // remove used DOM node from index<br>      keyedLen--;             // and decrement the size of the index<br>    }<br>  }</pre><pre>  // <strong>attempt to find a node of the same type from children</strong><br>  else if (!child &amp;&amp; min&lt;childrenLen) {<br>    for (j=min; j&lt;childrenLen; j++) {<br>      c = children[j];<br>      if (c &amp;&amp; isSameNodeType(c, vchild)) {<br>        child = c;<br>        children[j] = undefined;<br>        if (j===childrenLen-1) childrenLen--;<br>        if (j===min) min++;<br>        break;<br>      }<br>    }<br>  }</pre><p>(When this code block ends, we’re still inside the for loop over vnodes. The next block is still inside that loop.)</p><p>Notice that the state changes of children in the else case is a bit tricky. We start out with children being an array with entries 0 through childrenLen-1. As those children are paired with vnodes, holes are created in the middle fo the array. Or, if we remove the first or last item, we need to narrow the array range by incrementing min or decrementing childrenLen.</p><p>So we’re considering the child vnode vchild, and we may have found it a matching DOM node in child. The first thing to do is make their attributes match by recursively calling idiff on them:</p><pre>child = idiff(child, vchild, context, mountAll);</pre><p>If we hadn’t found a matching child DOM node, idiff created a new one, and either way the DOM node that it updated is returned back to us. The status of that node in the DOM is currently uncertain. It may be that it already is a child of the parent node and in the right place among the siblings. It may be that we reused a node, but it should now be at a new spot among the siblings. Or idiff may have made a brand-new node, and it needs to be placed inside the parent in the right place. Sorting out these cases and putting the node where it needs to go are handled next:</p><pre>if (child &amp;&amp; child!==dom) {<br>  // <strong>if the current child is past old DOM length, we can just append</strong><br>  if (i&gt;=len) {<br>    dom.appendChild(child);<br>  }<br>  // <strong>if the current child differs from what used to be at this index</strong><br>  else if (child!==originalChildren[i]) {<br>    if (child===originalChildren[i+1]) {<br>      // I don&#39;t think this line is necessary?<br>      removeNode(originalChildren[i]); <br>    }<br>    dom.insertBefore(child, originalChildren[i] || null);<br>  }<br>}</pre><p>And that’s the end of the work we need to do for each of the child vnodes. We exit out of that loop and just need to clean up any lingering DOM nodes that aren’t needed anymore. Remember that when we picked DOM node to use in the new content, we removed them from the keyed or children collections. So anything still left in them needs to be removed from the DOM. For now, we can just assume that recollectNodeTree makes sure the element is detached from the DOM and destroyed.</p><pre>if (keyedLen) {<br>  for (let i in keyed)<br>    if (keyed[i]) recollectNodeTree(keyed[i]);<br>}</pre><pre>while (min&lt;=childrenLen) {<br>  child = children[childrenLen--];<br>  if (child) recollectNodeTree(child);<br>}</pre><p>And that’s the end of innerDiffNode. It has paired up the new vnode children with existing DOM nodes, made them match, and now cleaned up any unneeded child nodes.</p><p>But what exactly does recollectNodeTree do? Preact’s support for recycling DOM nodes and component instances is what we’ll look at next.</p><h4>Recycling DOM nodes</h4><p>While performing diffing, Preact often needs to create or remove DOM nodes. A lot of those nodes are of the same type, like &lt;div&gt;s. Rather than constantly destroy and instantiate new nodes, Preact maintains a cache of unused nodes that are available for reuse. To make this work, the diffing code never directly creates or removes DOM nodes, but instead uses helper functions exported from src/dom/recycler.js.</p><p>The cache of reusable nodes is stored in an object keyed by element name:</p><pre>const nodes = {};</pre><p>The function collectNode removes a node from the DOM and adds it to the cache for elements of its node name:</p><pre>export function collectNode(node) {<br>  removeNode(node);<br>  if (node instanceof Element) {<br>    node._component = node._componentConstructor = null;<br>    let name = node.normalizedNodeName || toLowerCase(node.nodeName);<br>    (nodes[name] || (nodes[name] = [])).push(node);<br>  }<br>}</pre><p>When the diff algorithm needs a new node, it calls createNode, which looks in the cache for a suitable node to recycle, or else instantiates a new one:</p><pre>export function createNode(nodeName, isSvg) {<br>  let name = toLowerCase(nodeName),<br>      node = nodes[name] &amp;&amp; nodes[name].pop() ||<br>       (isSvg ?<br>         document.createElementNS(svgNS, nodeName) : <br>         document.createElement(nodeName));<br>  node.normalizedNodeName = name;<br>  return node;<br>}</pre><p>The diffing algorithm frequently needs to compare node types. But the DOM <a href="http://ejohn.org/blog/nodename-case-sensitivity/">nodeName method is a little confused</a> about whether to return capitalized or lowercase names, so it’s convenient to have access to a version of the name that we know will be normalized to lowercase. That’s stored as node.normalizedNodeName for use elsewhere.</p><p>Now we know how to recycle individual DOM nodes. But when the diffing algorithm finds a node it no longer needs, it doesn’t just need to get rid of that node, but all of its children too. And that’s what <a href="https://github.com/developit/preact/blob/8567c8bc13bc7fc60eeff4cad3bc9217822b4577/src/vdom/diff.js#L275">the </a><a href="https://github.com/developit/preact/blob/8567c8bc13bc7fc60eeff4cad3bc9217822b4577/src/vdom/diff.js#L275">recollectNodeTree function</a> is for. When a whole DOM tree is no longer needed, it walks through the tree recycling each node and also unmounting any component that may have rendered it.</p><p>The code for this is pretty clear:</p><pre>export function recollectNodeTree(node, unmountOnly) {<br>  let component = node._component;<br>  if (component) {<br>    // <strong>if node is owned by a Component, just unmount it<br>    </strong>// <strong>to let callbacks run before node is removed</strong><br>    unmountComponent(component, !unmountOnly);<br>  } else {<br>    // <strong>null out ref pointing to this node</strong><br>    if (node[ATTR_KEY] &amp;&amp; node[ATTR_KEY].ref) <br>      node[ATTR_KEY].ref(null);</pre><pre>    // <strong>recycle node unless asked to only handle unmounting</strong><br>    if (!unmountOnly) {<br>      collectNode(node);<br>    }</pre><pre>    // <strong>recurse on child nodes: `lastChild` is faster than other ways</strong><br>    let c;<br>    while ((c=node.lastChild)) recollectNodeTree(c, unmountOnly);<br>  }<br>}</pre><p>And now we understand how innerDiffNode deals with no-longer-needed children: it calls recollectNodeTree on them and that walks through the tree unmounting the component instances and recycling the nodes.</p><h4>Next time</h4><p>In part four, we’ll continue looking at the helpers that surround the main render/diff loop, by examining code that helps with rendering component contents.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f353b1ad7abc" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>