<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ryan Shackleton on Medium]]></title>
        <description><![CDATA[Stories by Ryan Shackleton on Medium]]></description>
        <link>https://medium.com/@ryshackleton?source=rss-910e778425b1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*0-H9ojPWnHCjDu3wlqwSPA.jpeg</url>
            <title>Stories by Ryan Shackleton on Medium</title>
            <link>https://medium.com/@ryshackleton?source=rss-910e778425b1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 20:14:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ryshackleton/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Creating data stories with ScrollyTeller]]></title>
            <link>https://medium.com/ihme-tech/creating-data-stories-with-scrollyteller-601a34327545?source=rss-910e778425b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/601a34327545</guid>
            <category><![CDATA[data-stories]]></category>
            <category><![CDATA[data-storytelling]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[scrollytelling]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Ryan Shackleton]]></dc:creator>
            <pubDate>Tue, 08 Sep 2020 22:02:19 GMT</pubDate>
            <atom:updated>2020-09-08T22:02:19.667Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><a href="https://github.com/ihmeuw/ScrollyTeller">ScrollyTeller</a> is a JavaScript library that streamlines the process of building scrolling data stories from tabular data</blockquote><figure><img alt="ScrollyTeller helps take chart and tabular data and convert it to scrollytelling HTML" src="https://cdn-images-1.medium.com/max/1024/1*65zhDyFI3pg935GMrsvKZQ.png" /></figure><h3>Introduction</h3><p>In the last 5+ years, the data visualization community has seen a burgeoning of data storytelling in various forms. A popular format is the “scrollytelling” data story, where scrolling is the primary user interaction; scrolling controls both the pace of narrative text that moves across the screen, and animations within a dynamic data visualization or relevant content. Unlike video content, users can move forwards or backwards at their own pace and the data visualization can contain other interactions that reinforce the data story or allow further exploration of the data. From a web development perspective, the basic technical implementation of scrollytelling is relatively straightforward, and there are great examples of tutorials for how to create them that are described below. That said, there are many moving parts that must be managed: scrolling text, multiple interactive data visualizations, and the usual mobile responsiveness that is made more difficult by scrolling and “sticky” elements.</p><figure><img alt="Bloomberg measles scrolly" src="https://cdn-images-1.medium.com/max/1024/1*NyNF6N-thhHFn9Rn9g8jQw.gif" /><figcaption>Bloomberg’s 2015 scrollyteller on <a href="https://www.bloomberg.com/graphics/2015-measles-outbreaks/">measles outbreaks</a></figcaption></figure><p>I work at the Institute for Health Metrics and Evaluation (<a href="http://www.healthdata.org/">IHME</a>), where multiple stakeholders collaborate to create data stories about global health issues. In an organization context, managing the many components of a data story and the technical infrastructure around them is quite complex. Subject matter experts, data scientists, graphic designers, copywriters, and many other people play a role in the creation of a data story, but few are fluent in HTML or JavaScript. As data visualization developers it is our job to manage all of these elements while also creating complex interactive data visualizations. Because many data stories are time sensitive, streamlining all aspects of the creation process becomes very important, especially the technical aspects. Good storyboarding can make a huge difference in reducing unnecessary code changes and text edits, but modifications to all parts of a data story are common during the creation process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o7artev1M-oTYfMjkdgtbw.png" /><figcaption>The Guardian’s groundbreaking 2017 <a href="https://www.theguardian.com/us-news/ng-interactive/2017/dec/20/bussed-out-america-moves-homeless-people-country-study">Bussed Out data story</a></figcaption></figure><h3>Motivation</h3><p>Like many other software libraries, <a href="https://github.com/ihmeuw/ScrollyTeller">ScrollyTeller</a> was created out of frustration. When creating scrollytelling data stories, I encountered several frustrations that centered around the management of editorial text and code.</p><h4>Too much HTML boilerplate</h4><p>First, I found that scrollytelling data stories can require a lot of very brittle, boilerplate code and HTML. Here’s an example of some HTML boilerplate you might need to create three text elements that scroll past your chart. There’s a lot of repetition here: each &lt;div&gt; element representing some narrative text (a “step” in scrolly parlance) differs only by the paragraph content and the index of the step, making this ripe for some templating.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/390dd95e264d13df25a572c8b4354bd3/href">https://medium.com/media/390dd95e264d13df25a572c8b4354bd3/href</a></iframe><h4>Too much JavaScript boilerplate</h4><p>Another example of brittle, boilerplate code in scrolling stories is the event handling code that links scroll events to actions in the data visualizations. Take <a href="https://medium.com/u/4d76dff3297c">Jim Vallandingham</a>’s excellent <a href="https://vallandingham.me/scroller.html">how-to article</a> describing how to build a scrolling data story. While the article is exceptional and very easy to understand, it’s easy to see how slight changes to the story can result in a lot of code change. I’ve modified Jim’s code slightly below, but the approach is still the same: defining an array of functions to respond to each scroll step.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f5690743077929769164a8cf0629c932/href">https://medium.com/media/f5690743077929769164a8cf0629c932/href</a></iframe><p>What if we wanted to switch the order of steps 2 and 3? In that case, we would need to change the HTML above to re-order the steps, then ensure that the new indices of steps 2 and 3 correspond to the functions we want to trigger. In longer data stories with much more complex edits, code changes like this become unwieldy and incredibly error prone. Furthermore, different developers might use different methods, such as hash mapping instead of array indices, to link “steps” with chart events, making collaboration between developers even more difficult.</p><h4>Too many text changes</h4><p>Another frustration is that, as a data visualization developer in a large organization, I find myself responsible for making many text changes to the narrative part of the story. Those text changes may come from copywriters, researchers, designers, or my own minor changes to update text styles. While the collaborative aspects of the editorial process are incredibly rewarding, modifying editorial text in code is tedious and places me (the web developer) between the editor and the copy they are working on. That “editorial friction” can slow the process down and stifle the collaborative creative process that makes visual stories possible.</p><figure><img alt="Office Space meme: could you come in on Sunday to make text changes?" src="https://cdn-images-1.medium.com/max/999/1*dTJQ98Xtsc23LRSFOMerHg.png" /><figcaption>Thaaaanks Peter</figcaption></figure><h3>ScrollyTeller features and use cases</h3><p>So how does ScrollyTeller help fix these issues? Let’s talk about a few key features that alleviate these problems, and a few more that help multiple developers work together on a story. The use cases below are just brief overviews that highlight key features of the library. For a much more in-depth look, see the <a href="https://ihmeuw.github.io/ScrollyTeller/">ScrollyTeller tutorial</a>, which serves as both a template for creating scrolling data stories a tutorial for web developers on how to use the library.</p><iframe src="https://share.getcloudapp.com/Blu5G9dv?embed=true" width="2048" height="866" frameborder="0" scrolling="no"><a href="https://medium.com/media/6adb2d0f4690374b3abc6b11226d351f/href">https://medium.com/media/6adb2d0f4690374b3abc6b11226d351f/href</a></iframe><h4>Reducing HTML boilerplate</h4><p>ScrollyTeller’s first functionality was to use JavaScript to template HTML text boxes that scroll in a data story using data from .csv files, Google/Excel spreadsheets, or database tables. Each row in the data table represents one scrolling text box, with various text fields to populate a title, paragraph, and some link text. This relatively simple functionality reduces the boilerplate HTML that data visualization developers need to manage and puts editorial power back in the hands of copywriters because almost anyone can edit a data table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4G8QINgoKEBQCuDWhGtMTg.gif" /><figcaption>Animation from the <a href="https://ihmeuw.github.io/ScrollyTeller/">ScrollyTeller tutorial</a> showing which columns in a data table correspond with each HTML element in a scrolling text box</figcaption></figure><h4>Reducing text edits and ceding editorial changes to copywriters</h4><p>When creating the <a href="https://vizhub.healthdata.org/tobacco-control/mexico">Tobacco Control</a> and <a href="https://vizhub.healthdata.org/child-mortality">Child Mortality</a> visualizations, we configured a development version of our data story to fetch data from a shared Google Spreadsheet using the <a href="https://developers.google.com/sheets/api/samples/reading">Google Spreadsheets API</a>. The data visualization team set up the basic structure of the spreadsheets from storyboards that outlined the basic story flow, then we shared the spreadsheets with our editorial teams. Copywriters and other stakeholders could then update the text in the Google Spreadsheet and reload the visualization to see the changes in real time, significantly reducing the burden of making minor text edits in code. Giving editorial teams control over the text also allowed them to feel much more in control of the story, and propose new changes to the story by adding new rows to the spreadsheet. The data visualization team could then add the relevant events and chart animations to complete the story.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4MW5qbzOYdyWEnImDpfppA.png" /><figcaption>A Google Spreadsheet containing the tabular data for a scrolling data story.</figcaption></figure><h4>Facilitating easier text translation</h4><p>At <a href="http://www.healthdata.org/">IHME</a>, many of our web tools are used by policymakers around the world, so translating text and localizing our applications is an important, if not very time consuming task. While creating the the <a href="https://vizhub.healthdata.org/tobacco-control/mexico">Tobacco Control</a> visualization, we found that translation of the application into several languages was much easier with the text components of the visualization already in tabular form. In that case, we were able to share our spreadsheets containing the story “narration” as-is with translators, who returned the spreadsheets, translated, in our data format (we also hid the columns that didn’t need to be translated using the “hide columns” functionality in Google Spreadsheets). Upon receiving the translated text, we only needed to add a language column to our data tables, fetch data by language, and our visualization was translated without any major code changes (other than adding the dropdown to change language).</p><iframe src="https://share.getcloudapp.com/GGuRKbRd?embed=true" width="1436" height="772" frameborder="0" scrolling="no"><a href="https://medium.com/media/c58346e7edde494610dd79afbd650c4b/href">https://medium.com/media/c58346e7edde494610dd79afbd650c4b/href</a></iframe><h4>Standardized event handling</h4><p>Creating the interactive data visualizations (charts) that underpin many data stories is complicated enough, so it’s nice to have a framework in place to handle the scroll events that trigger animations in the visualizations. There are many libraries that handle scroll events, and ScrollyTeller relies heavily on the much lighter weight <a href="https://github.com/russellgoldenberg/scrollama">Scrollama library</a>, written by Russell Goldenberg, to detect scroll events and scroll progress.</p><iframe src="https://share.getcloudapp.com/2NuymOjn?embed=true" width="1998" height="876" frameborder="0" scrolling="no"><a href="https://medium.com/media/41107d2ef39fe1376c77acc7d1826517/href">https://medium.com/media/41107d2ef39fe1376c77acc7d1826517/href</a></iframe><p>Like Scrollama, ScrollyTeller has several event handling functions that are triggered when a block of text scrolls to a predefined percentage of the page height. ScrollyTeller also provides framework for triggering chart events: each narrative text block contains a <strong>trigger</strong> field that can contain a JSON string. The JSON string is parsed to a JavaScript object literal and passed to an event handler to trigger changes to the chart. Below is an example of two rows representing narrative text with their <strong>trigger</strong> field shown. In this case, there is just a <strong>year</strong> property that changes from 1950 to 2008 between the two rows.</p><iframe src="https://share.getcloudapp.com/jkuZXwWr?embed=true" width="1370" height="318" frameborder="0" scrolling="no"><a href="https://medium.com/media/0cdf36438c766273bf94774afc2b9fe2/href">https://medium.com/media/0cdf36438c766273bf94774afc2b9fe2/href</a></iframe><p>We can configure a chart component to update when new state is passed to it, in this case, in the form of the <strong>year</strong> property that transitions from the 1950 data to the 2008 data.</p><iframe src="https://share.getcloudapp.com/v1u2OpY0?embed=true" width="1300" height="539" frameborder="0" scrolling="no"><a href="https://medium.com/media/6ff2cf215d4c84bc340a0bea297797d4/href">https://medium.com/media/6ff2cf215d4c84bc340a0bea297797d4/href</a></iframe><p>A code snippet is shown below that summarizes how the event handling functionality works in ScrollyTeller. Here we are using es6 destructuring to pick properties from an object passed to <em>onActivateNarrationFunction()</em>. In this case, we pick the <strong>year </strong>property from the <strong>state</strong> variable, and pass that to our graph’s render function. The ScrollyTeller configuration also provides a function (<em>buildGraphFunction()</em>) to build a data visualization component (a graph) and stores a reference to the component as a <strong>graph</strong> property for use in the event handler. Notice that the state variable could contain any JSON parse-able entity: arrays, strings, numbers, or even nested objects.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6110247627dad54ff9eeb792e6d0c36d/href">https://medium.com/media/6110247627dad54ff9eeb792e6d0c36d/href</a></iframe><p>In practice, we find that passing JSON to chart functions makes the process of modifying and updating charts highly flexible, so responding to requests to change the story don’t usually require code changes at all. Story changes might just mean modifying or adding a property to JSON in a Google Spreadsheet. This means that during collaborative meetings, we can sometimes make changes in real time without having to redeploy code to see the results.</p><p>It’s worth noting here that like many scroll based libraries, ScrollyTeller provides a <strong>progress</strong> property to an <em>onScroll()</em> function for smoothly transitioning between chart states as text scrolls up and down the page. We find that animations that link directly to scroll progress are much more satisfying and allow much finer control over chart interactions.</p><iframe src="https://share.getcloudapp.com/bLuRBE8G?embed=true" width="1278" height="607" frameborder="0" scrolling="no"><a href="https://medium.com/media/7946e063354247481f515941b96cf188/href">https://medium.com/media/7946e063354247481f515941b96cf188/href</a></iframe><h4>Management of trigger “state”</h4><p>With increasingly complicated components come increasingly complicated JSON triggers. Sometimes we might need three or four properties to control chart styles, series visibility, highlighted data points, or pre-specified ranges of data. To maintain consistent state when scrolling forwards or backwards, we would have to store all of the state as JSON in the <strong>trigger</strong> field of every data row. This turns out to be very verbose and difficult to maintain, so we added functionality to accumulate trigger properties from the top down to ScrollyTeller.</p><iframe src="https://share.getcloudapp.com/nOukX02P?embed=true" width="1354" height="370" frameborder="0" scrolling="no"><a href="https://medium.com/media/94812bb7dc02b63c94374ddf4266b805/href">https://medium.com/media/94812bb7dc02b63c94374ddf4266b805/href</a></iframe><p>In the table above, the series of JSON triggers is shown in the center column, with the merged state property that ScrollyTeller passes to the event handlers. Notice that the state at each row consists of the accumulated triggers that preceded it, so advancing backwards removes or updates the previous state variable accordingly.</p><h4>Other useful features: sections</h4><p>ScrollyTeller configuration is grouped by sections, so each section can have its own data, chart components, and state. We found this feature to be very useful when multiple developers collaborate quickly on the same project. Adding a new chart usually just means importing the relevant configuration and adding it to a JavaScript object that ScrollyTeller receives upon instantiation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N5gahmUlg1PKxmGl9KybjQ.png" /></figure><h4>Other useful features: standardized control flow</h4><p>ScrollyTeller configuration also standardizes the control flow associated with data stories, providing asynchronous functions to fetch narrative text tables and chart data, modify the chart data, then build the chart when the application loads. The <strong>data</strong> and <strong>graph</strong> in each section are stored as properties and are accessible as arguments in all of the event handlers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SQqwxdGVwpckjsoEOPFUrw.png" /></figure><h4>Other useful features: scroll tracking with Google Analytics</h4><p>ScrollyTeller also provides configurable tracking of scroll events in Google Analytics at the section level, or even down to the level of each narrative text block. We found this feature useful for determining how far users progressed in a data story, where we “lost” them, and where users spent the most time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/870/1*j9eqKqoLvezboH54lRgBTw.png" /></figure><h3>Wrapping up</h3><p>I hope this provides a high level view of ScrollyTeller and how it can be used to make creating scrolling data stories much smoother, more collaborative, and more flexible. To get started, I highly recommend checking out the <a href="https://github.com/ihmeuw/ScrollyTeller">ScrollyTeller repo</a>, which can serve as a template project for your own data story, and going through the <a href="https://ihmeuw.github.io/ScrollyTeller/">ScrollyTeller tutorial</a> in detail to understand how to set up a project.</p><p>I would like to acknowledge IHME for supporting this project and the developers on IHME’s Data Visualization team who contributed to ScrollyTeller and the data stories described in this article, including <a href="https://medium.com/u/84f9a353f380">Evan Laurie</a>, <a href="https://medium.com/u/21fc74c54339">Katherine Beame</a>, <a href="https://medium.com/u/efa5739ce635">David Schneider</a>, <a href="https://medium.com/u/54ebfb92896b">Komal Ali</a>, <a href="https://medium.com/u/549db7daeffb">Michael Fernandes</a>, and <a href="https://medium.com/u/bebbfed40f26">Ben Hurst</a>. Please don’t hesitate to get in touch if you want to contribute to this project or are interested in working with our team. I would also like to acknowledge the many other developers upon whose open source software ScrollyTeller is based, especially <a href="https://github.com/russellgoldenberg/scrollama">Russell Goldenberg’s Scrollama</a> library that provides the scroll event handling functionality that underpins ScrollyTeller.</p><p>Best of luck telling great data stories!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=601a34327545" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ihme-tech/creating-data-stories-with-scrollyteller-601a34327545">Creating data stories with ScrollyTeller</a> was originally published in <a href="https://medium.com/ihme-tech">IHME Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How we build data visualizations for a global audience]]></title>
            <link>https://medium.com/ihme-tech/how-we-build-data-visualizations-for-a-global-audience-550b2cb7e4e6?source=rss-910e778425b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/550b2cb7e4e6</guid>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[d3js]]></category>
            <category><![CDATA[data-storytelling]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-visualization]]></category>
            <dc:creator><![CDATA[Ryan Shackleton]]></dc:creator>
            <pubDate>Wed, 05 Feb 2020 19:50:38 GMT</pubDate>
            <atom:updated>2020-03-02T21:18:29.278Z</atom:updated>
            <content:encoded><![CDATA[<p>The Institute for Health Metrics and Evaluation (<a href="http://www.healthdata.org/">IHME</a>) is an independent research organization whose mission is to improve the health of the world’s populations by providing the best information on population health. A simpler way to summarize what IHME does is:</p><blockquote>We collect, process, and distribute big data for global population health.</blockquote><p>Part of that mission is to distribute data in ways that inform a diverse user group including expert global health researchers, ministers of health, policymakers, and the general public. To that end, we provide a suite of more than twenty <a href="http://www.healthdata.org/results/data-visualizations">data visualization products</a> that run the gamut from static charts to highly interactive tools with dozens of controls and chart types. The variation in our data visualizations reflects the diverse needs of our audiences, from the layperson who prefers a curated story to the expert user who wants the flexibility to ask their own questions of our population health datasets.</p><figure><img alt="Data visualizations page at IHME" src="https://cdn-images-1.medium.com/max/1024/1*qlayiy8wFNj3MpzSqmI9_g.png" /><figcaption>The first few entries in a list of IHME’s more than twenty data visualization tools.</figcaption></figure><p>On the Data Visualization team, we commonly get this question from users of our products:</p><blockquote>How did you build that visualization?</blockquote><p>The goal of this article is to answer that question in as much depth as we can provide, and hopefully provide a starting point for anyone interested in building their own web-based visualization.</p><h3>TL;DR</h3><ul><li>Rather than using an off-the-shelf data visualization platform, we build bespoke data visualizations using JavaScript, D3, and React, including two of our own open source libraries: <a href="https://github.com/ihmeuw/ihme-ui">IHME-UI</a> and <a href="https://github.com/ihmeuw/ScrollyTeller">ScrollyTeller</a></li><li>Our back end technologies have historically used the LAMP stack, but we are moving toward microservices architectures with NGINX and Node/Express back ends</li></ul><h3>Who built that?</h3><p>Our visualizations are created by the Data Visualization team, which currently consists of six developers, a technical product manager, a development manager, and a team lead. We are also supported by multiple technology teams including database developers that maintain our MySQL databases, an infrastructure team that manages the hardware and virtual environments we use to deploy our web technology, and a central computation team that handles organization-wide computational assets.</p><p><em>What we build</em> is guided by requests from health data researchers at IHME, external collaborators at health ministries, academic institutions, and the general public, who also supply IHME’s data in the first place. In that sense, the entire IHME community has a hand in building the tools we create. Functionally speaking, the Data Visualization team at IHME operates as many software development teams do, using agile development practices, two-week sprints, and code reviews. We track work efforts using agile project management software and use Git for source control. Our developers are very much Full Stack in the sense that we write our own API’s, create and manage small databases, write the visualization code, and manage application deployments.</p><h3>How we build it</h3><p>A major difficulty in summarizing <em>how we build</em> visualizations is that it’s not entirely consistent across our twenty-plus applications. Like many organizations that have been building web applications for more than a decade, we have a mix of legacy code and newer technologies that we are moving toward. With that said, the following sections generalize the most common technologies we use on the <a href="https://en.wikipedia.org/wiki/Front_and_back_ends">front end and back end</a> of our applications.</p><h3>How do you make those impressive charts?</h3><p>Most people asking “How did you build that?” are probably most focused on how we created the chart itself. In other words, “How did you convert rows of data into that great bar chart/tree map/line chart/scatter plot?”.</p><figure><img alt="A pyramid chart, or dual stacked bar chart from IHME’s GBD Compare tool." src="https://cdn-images-1.medium.com/max/1024/1*2bkf4xa1yLMJnIpS7C0Hng.png" /><figcaption>A pyramid chart view of death rates by cause for selected countries in the GBD Compare visualization.</figcaption></figure><p>The short answer is: from scratch using JavaScript and CSS. The longer answer is: we use custom <a href="https://d3js.org/">D3.js</a> code in older applications, and commonly use D3.js in combination with <a href="https://reactjs.org/">React.js</a> in our newer applications.</p><h4>Vanilla JavaScript &amp; D3.js</h4><p>Some examples of D3.js code can be found on <a href="https://bl.ocks.org/">bl.ocks.org</a> or <a href="https://observablehq.com/">ObservableHQ.com</a>. Many of these code samples offer a good starting point for learning code patterns to build charts in JavaScript, but generally aren’t modular enough, don’t handle component state very well, and don’t follow our code style practices (a modified <a href="https://github.com/airbnb/javascript">AirBnB style</a>) to be very usable. Thus, many of our older D3.js components are coded from scratch and are not open source. <a href="https://observablehq.com/@ryshackleton/us-county-basemap-counties-nested-within-state-groups">This US map</a> and a <a href="https://observablehq.com/@ryshackleton/choropleth-from-imported-basemap">derived choropleth</a> on ObservableHQ are examples with code patterns similar to the way some of our internal D3.js code is structured.</p><h4>React with D3.js</h4><p>To standardize some of our visualization code, the Data Visualization team created an open source repository called <a href="https://github.com/ihmeuw/ihme-ui">IHME-UI</a>. Like many React-based data visualization frameworks, IHME-UI uses <a href="https://reactjs.org/">React</a> to manage component rendering to the DOM, while leveraging D3.js for low-level chart scaling, layout, and map transformation functionality. <a href="https://medium.com/u/b8fe3c1518c1">Elijah Meeks</a> has an excellent discussion of the tradeoffs of this approach in his <a href="https://medium.com/@Elijah_Meeks/interactive-applications-with-react-d3-f76f7b3ebc71">article</a> and <a href="http://elijahmeeks.com/">book</a> on the subject. <a href="https://medium.com/u/48764f382701">Amelia Wattenberger</a> also has an excellent instructional <a href="https://wattenberger.com/blog/react-and-d3">blog post</a> on the topic.</p><p>The example below is from the IHME-UI demo files, and shows how a chart is <em>composed</em> from individual React components that represent different parts of the chart such as scales and symbols (lines in this case). An &lt;AxisChart&gt; component encloses &lt;XAxis&gt;, &lt;YAxis&gt;, and &lt;MultiLine&gt; components to compose a complete line chart. The &lt;XAxis&gt; and &lt;YAxis&gt; components leverage D3.js to compute transformations from x/y space to pixel space, and format ticks and tick labels. The &lt;MultiLine&gt; component uses D3.’s svg path function to place the lines in the appropriate position in pixel space within the <a href="https://developer.mozilla.org/en-US/docs/Web/SVG/Element/svg">svg</a> element.</p><figure><img alt="An example from the demo files of IHME-UI showing React code to generate the chart." src="https://cdn-images-1.medium.com/max/703/1*3s6WdJNxuDo3QIP0A6YnKQ.png" /><figcaption>An example from the demo files of IHME-UI showing React code to generate the chart.</figcaption></figure><p>IHME-UI was created primarily to unify the look, feel, and behavior of some IHME charts, but is far from complete in terms of its available chart types. For those interested in building some of the more common chart types using React, several other React-based charting libraries like <a href="https://formidable.com/open-source/victory/docs/victory-chart/">Victory</a>, <a href="https://semiotic.nteract.io//">Semiotic</a>, <a href="https://uber.github.io/react-vis/">React-Vis</a>, and <a href="http://recharts.org/en-US/">Recharts</a> use similar approaches to customizing axes and chart symbols. Other React-based libraries like <a href="https://nivo.rocks/">Nivo</a> offer higher level implementations of individual charts where props determine axis and shape behaviors.</p><h4>Scrollytelling</h4><p>Scrolly-what? Several of our visualizations, such as the <a href="https://vizhub.healthdata.org/child-mortality">child mortality</a> and <a href="https://vizhub.healthdata.org/tobacco-control/mexico">tobacco control</a> visualizations, are designed to guide the user through a story on a given topic, with the primary interaction being that the user scrolls to continue through the story. Whereas many of IHME’s visualizations are <em>exploratory</em> to allow experienced users to ask their own questions of data, scrollytelling visualizations are <em>explanatory</em> to appeal to users with less experience in a given global health topic.</p><figure><img alt="A scrolling data story about mapping global child mortality rates." src="https://cdn-images-1.medium.com/max/1024/1*wfOYxmaip3wnS6Glro8iVA.gif" /><figcaption>A scrolling data story about mapping global child mortality rates.</figcaption></figure><p>To create these visualizations, our team wrote an open source library called <a href="https://github.com/ihmeuw/ScrollyTeller">ScrollyTeller</a>, which provides a framework for dynamically creating a scrolling data story from configuration files and tabular data containing the story “narration”. See <a href="https://ihmeuw.github.io/ScrollyTeller/">this link </a>for a scrollytelling data story that explains how ScrollyTeller works.</p><h3>What about the back end?</h3><p>As most web developers are well aware, the front end code that renders our visualizations couldn’t exist on the web without a significant amount of development infrastructure. Visualizations with interactive, dynamically changing charts and multiple views require robust web technology, with flexible backend APIs to organize and serve the data from databases, not to mention varying amounts of web traffic, data caching, etc. We won’t go into too much detail about how we deploy, but for anyone interested, we describe some generalities about the types of development methods we use. In other words, what does our <em>stack</em> look like?</p><h4>A typical stack: <a href="https://tedium.co/2019/10/01/lamp-stack-php-mysql-apache-history/">LAMP</a> &amp; JavaScript front end</h4><p>The <a href="https://en.wikipedia.org/wiki/LAMP_(software_bundle)">LAMP</a> acronym might not mean much to those unfamiliar with web technologies. In our case, it stands for <strong>L</strong>inux, <strong>A</strong>pache, <strong>M</strong>ySQL, <strong>P</strong>HP, which forms both the web server and the back end API for many of our older applications. In most cases, these projects are <a href="https://en.wikipedia.org/wiki/Monorepo">monorepos</a> containing all of the code necessary to build and deploy the applications. An easy way to explore our stack is to break down the file structure of one of these monorepos for a typical project. The main components are:</p><ul><li>An index.php file, which is the entry point into the web server, and routes the web and api server using Apache or sometimes using the <a href="http://www.slimframework.com/">Slim PHP framework</a>.</li><li>An api folder to host backend PHP files. The API connects to IHME’s MySQL databases and uses SQL queries to query data via http routes consumable by the front end.</li><li>A php-templates folder to serve the base web components (usually just header, footer, body in HTML format).</li><li>A Docker folder + <a href="https://jenkins.io/doc/book/pipeline/jenkinsfile/">Jenkinsfile</a> to support automated builds via <a href="https://jenkins.io/">Jenkins</a>, a well known open source automation server. The Jenkinsfile is written in <a href="http://groovy-lang.org/">Groovy</a> and orchestrates the <a href="https://hub.docker.com/_/php">containerization of the Linux/Apache</a> web server environment, which is deployed via <a href="https://rancher.com/">Rancher</a>.</li><li>A src directory containing the front end JavaScript code, CSS, and any static resources such as images or icons. Most of our D3.js (and/or React) code that renders the data visualizations is here, along with any CSS and sometimes static resources like images and icons.</li><li>A variety of .files and other configuration files that configure <a href="https://getcomposer.org/">Composer</a> (PHP dependency management), <a href="https://nodejs.org/en/">Node</a> (JavaScript dependency management), and <a href="https://webpack.js.org/">Webpack</a>/<a href="https://babeljs.io/docs/en/index.html">Babel</a> for <a href="https://scotch.io/tutorials/javascript-transpilers-what-they-are-why-we-need-them">transpiling</a> and <a href="https://webpack.js.org/concepts/">bundling</a> our front end code.</li><li>A README.md file to tell our developers how to set up the project.</li></ul><figure><img alt="File structure for a generic IHME-application to illustrate API, source code, developer tools, and deployment." src="https://cdn-images-1.medium.com/max/646/1*ICG9mJTRaLA3_qZHExmQlg.png" /><figcaption>File structure for a generic IHME-application to illustrate API, source code, developer tools, and deployment.</figcaption></figure><p>This type of architecture has served us well, and because many of our projects are structured in this way, we can get up and running relatively quickly with this stack. That said, many of our developers like to work exclusively in JavaScript and sometimes Python, which has prompted us to explore some different project setups.</p><h4>A more modern stack: LEMN(?) stack &amp; React front end</h4><p>Many of our more recent projects have replaced Apache web servers with NGINX, and replaced PHP/Slim back end servers with Node/Express. Thus, many of our stacks use variations of Linux/NGINX/MySQL/Node. In most cases, we still stick to using monorepos, but break each of the services up into their own containers that run separate Node or Apache servers. The main components of this stack are:</p><ul><li>An app directory containing the front end JavaScript code. The typical React/Redux file structure may look familiar here, and may vary depending on the project. Upon deployment, the app files are bundled and copied to a separate app container (using Docker/app.Dockerfile).</li><li>A variety of .files and other configuration files that configure <a href="https://nodejs.org/en/">Node</a> (JavaScript dependency management), and <a href="https://webpack.js.org/">Webpack</a>/<a href="https://babeljs.io/docs/en/index.html">Babel</a> for <a href="https://scotch.io/tutorials/javascript-transpilers-what-they-are-why-we-need-them">transpiling</a> and <a href="https://webpack.js.org/concepts/">bundling</a> our front end code.</li><li>A server folder that hosts an Express backend. In this case, the backend is a separate Node/Express application that is built into its own container upon deployment (using Docker/api.Dockerfile).</li><li>A Docker folder + <a href="https://jenkins.io/doc/book/pipeline/jenkinsfile/">Jenkinsfile</a> that contain code to <a href="https://hub.docker.com/_/php">containerize the Linux</a>/NGINX web server environment, and configure NGINX to route (proxy) traffic to each of the app and api containers. Again, all of this is deployed via <a href="https://rancher.com/">Rancher</a>.</li><li>It’s worth noting that this configuration still uses a containerized version of Apache (hence theindex.php file) for the web server, which exists primarily to conform with some existing Apache-based infrastructure at IHME.</li><li>A README.md file to tell our developers how to set up the project.</li></ul><figure><img alt="File structure for more modern applications using NGINX/Express/Node back ends" src="https://cdn-images-1.medium.com/max/728/1*qs3YLoW8NcQMbRKGrHfkrA.png" /><figcaption>File structure for more modern applications using NGINX/Express/Node back ends.</figcaption></figure><h4>Microservices architecture in Local Burden of Disease visualizations</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Qy7SJ-A1WpcetnApYau2pQ.png" /><figcaption>The map-based Local Burden of Disease visualization.</figcaption></figure><p>A notable exception to our development environments are the <a href="https://vizhub.healthdata.org/lbd/cgf">Local Burden of Disease visualizations</a>, which are map based and are deployed using a microservices architecture with completely separate components in different code repositories. Because the same codebase is used to display many different health indicators, the codebases must also be highly configurable. Configuration is accomplished via a <a href="https://github.com/choroscope/choroscope-spec">JSON based configuration language</a> that defines the dimensions and shape of the data, database specifications, and how different controls and chart components should be rendered in the view. This stack is partially open sourced as the Choroscope platform, which is <a href="https://github.com/choroscope/about-choroscope">summarized nicely here</a>. On the front end, the Local Burden of Disease visualizations also utilize <a href="https://leafletjs.com/">Leaflet.js</a>, or more specifically, <a href="https://react-leaflet.js.org/">React-leaflet</a> to handle functionalities such as basemaps, map layering, and zooming/panning.</p><h3>Got all that?</h3><p>Hopefully this gives a flavor for the complexities of building, deploying, and maintaining the twenty-plus visualizations that IHME distributes. Please visit the <a href="http://www.healthdata.org/">IHME website</a> for the latest on what the organization is up to, our <a href="http://www.healthdata.org/results/data-visualizations">data visualizations page</a> for a complete list of our tools, or <a href="http://www.healthdata.org/get-involved/careers">our careers page</a> if you are interested in working with us.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=550b2cb7e4e6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ihme-tech/how-we-build-data-visualizations-for-a-global-audience-550b2cb7e4e6">How we build data visualizations for a global audience</a> was originally published in <a href="https://medium.com/ihme-tech">IHME Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>