<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Tableau Engineering Blog - Medium]]></title>
        <description><![CDATA[Engineering, Research, and Culture from the Tableau Development Team - Medium]]></description>
        <link>https://medium.com/tableau-engineering-blog?source=rss----e066fecef8f2---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Tableau Engineering Blog - Medium</title>
            <link>https://medium.com/tableau-engineering-blog?source=rss----e066fecef8f2---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 12:52:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/tableau-engineering-blog" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Digging into Data Landscapes]]></title>
            <link>https://medium.com/tableau-engineering-blog/digging-into-data-landscapes-c3d8faf1ad6f?source=rss----e066fecef8f2---4</link>
            <guid isPermaLink="false">https://medium.com/p/c3d8faf1ad6f</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Ana Crisan]]></dc:creator>
            <pubDate>Wed, 26 Oct 2022 16:50:44 GMT</pubDate>
            <atom:updated>2023-03-24T15:50:21.741Z</atom:updated>
            <content:encoded><![CDATA[<p>Automatically creating visually coherent chart combinations</p><figure><img alt="A dotted, circular, and meandering path on which is overlain icons representing individual datasets, data types, and dataset combinations. Additionally there are icons of such as an eye, a pie chart, and people talking, to reflect the socio-technical processes that are involved in iteratively finding a the right, and often combined, dataset for analysis. We call this a process of data reconnaissance." src="https://cdn-images-1.medium.com/max/1024/1*EQKSJdz2K_o7HUyE8108aA.png" /><figcaption>Putting together the right data set is a complex task that is usually done manually. In the above image, we show a meandering path an analyst usually has to take, from discovering individual datasets, gathering and combining data sets of different types, and finally exploring these data, either themselves or with stakeholders. We propose a way to speed up with process through automated support.</figcaption></figure><p><em>This research was presented at the </em><a href="http://ieeevis.org/year/2022/welcome"><em>IEEE VIS’22 conference</em></a><em>. The </em><a href="https://research.tableau.com/sites/default/files/Crisan_GenEpi_Recommender.pdf"><em>publication</em></a><em> and an </em><a href="https://youtu.be/57yWVCnp15E"><em>accompanying video</em></a><em> are available online for viewing.</em></p><h4><strong>What is a data landscape?</strong></h4><p>It is increasingly rare that any single dataset provides a solution to a complex question. Instead, it is incumbent on data analysts to discover and examine multiple datasets and consolidate them to form a final dataset for analysis. The consolidation process is even more complex when integrating data of different types, for example, integrating tabular data (a typical spreadsheet) with related image, text, or audio data. These varied, but interrelated datasets, exist in a landscape of all possible datasets available for analysis, some that are useful to particular data questions and some that are not. The analyst often acts like a scout on a reconnaissance mission — painstakingly crafting a map of these dataset relationships to derive actionable insights.</p><p>In prior research, <a href="https://amcrisan.github.io/assets/files/papers/Data_Recon_and_Task_Wrangling.pdf#news=0">we proposed that data reconnaissance</a> is an overlooked process or lumped together with data exploration — the latter focuses on exploring an <em>existing</em> dataset and not <em>creating</em> a dataset of analysis. Importantly, we stipulated that techniques for data exploration do not work well for reconnaissance. With concern, we found that overlooking the challenges of data reconnaissance leaves many unaddressed needs that analysts are burdened to resolve for themselves.</p><h4>We can automate aspects of data landscape reconnaissance</h4><p>Data reconnaissance is a highly bespoke process, so what can be done to automate aspects of it and lower burdens to data analysts? One critical component of reconnaissance is to <em>look at the data</em> to get an initial impression of whether an individual dataset or a combination of datasets, is analytically useful. However, the iterative creation of data visualizations is not always easy, especially when attempting to combine multiple different datasets with different data types. To lower this burden, we proposed an approach for automatically creating visually coherent chart combinations.</p><p>Briefly, our approach has several steps. First, it constructs a data source graph by analyzing the data and identifying potential connections between datasets. This step makes it easier to leverage data relationships automatically without having to manually discover them or define them a priori (e.g. as a database schema). In the second step, paths in the data source graph are explored and ranked. Our default ranking mechanism prioritizes paths in the graph that connect multiple datasets. This ranking approach also incorporates prior visualization practices and the qualities of the present dataset. In the final step, our technique creates specifications for visualizations in order of the ranked paths. These specifications are just recipes telling the computer how to build the chart so that an analyst can then view it. Our approach to generating visualization specifications is our secret sauce. Our technique generates specifications for visually coherent chart combinations that allow the analysts to <em>see and link</em> commonalities between multiple datasets. Why is this special? Using current data visualization tools the analysts need to manually coordinate shared information between charts. Our approach automates this process and makes it easier to quickly generated visually coherent chart combinations that integrate shared data between charts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-hWHbbzcFK0RgoavgRvFCA.png" /><figcaption>Using current data visualization tools the analysts needs to manually coordinate shared information between charts. Our approach automates this process and makes it easier to quickly generated visually coherent chart combinations that integrate shared data between charts.</figcaption></figure><p>Let’s say our analyst has a dataset of sales made across different geographical regions. Exploring this data question might require combining two datasets of different types — a spreadsheet of sales and some geographic data. The analyst might make a bar chart visualization summarizing their sales per region and a separate visualization of a map that overlays the total profits per region. In this example, the ‘region’ information is shared between the bar chart and map are the same, and the analysts might want to use a common color to show the connections between these two vizzes — in our terminology, this makes the two vizzes (bar chart and map) <em>visually coherent</em>. With most available visualization tools today, the analyst must manually resolve the color consistencies between these vizzes. But using our approach, this is done automatically! Let’s say the analyst then wants to change up their data or include a new dataset. Right now, they would need to once again manually identify and specify shared data between vizzes. However, our approach can be quickly re-run to generate a new set of visually coherent chart combinations on the fly. Doing so helps to speed up the reconnaissance process by helping analysts quickly visualize and examine the data landscape.</p><p>The details of our approach are intentionally brief here. If you want to know more, please check out <a href="https://research.tableau.com/sites/default/files/Crisan_GenEpi_Recommender.pdf">the research publication</a>.</p><h4>Automatic visualization creation was helpful for the reconnaissance</h4><p>We tested our technique by collaborating with experts in the field of microbial genomic epidemiology (GenEpi). GenEpi is a field of research that, among other things, explores how disease-causing microbes (like the SARS-CoV-2 virus that causes COVID) are transmitted between people. GenEpi data is complex and integrates data from a variety of sources, such as electronic health records, digitized genetic data, geography, and contact networks (the social network of who infected whom). The data landscape that GenEpi experts have to deal with is broad and heterogeneous but also highly integrated. Beyond manual analysis, they have little support for finding the relationships between datasets in this landscape and even less for visualizing these data.</p><p>We developed a domain-specific implementation of our approach called GEViTRec in the R programming language, which is the analysis language that GenEpi experts regularly use. Leveraging R and its visualization capacities, we allow experts to perform data reconnaissance activities <em>in the flow of their analysis</em> without having to switch tools. Here is an example of the R code and the visualizations that are produced:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D58l0VD4aY_5OVZyQBPrww.png" /><figcaption>Example of a domain specific implementation of our techniques as GEViTRec. In the above figure we show : <strong>A)</strong> the code required to run GEVitRec <strong>B)</strong> the visualization of the data source graph and<strong> C)</strong> the top-ranked viz. It takes approximately 30 seconds to go from the code to the visualization in panel C.</figcaption></figure><p>In the above visualization we see <strong>A)</strong> the code required to run GEVitRec <strong>B)</strong> the visualization of the data source graph and<strong> C)</strong> the top-ranked viz (although only one viz is shown here, we produce visualizations for <em>each </em>path in the data source graph). Visual coherence across the visualizations is achieved through aligning y-axis positions in the top row, alignment of color (the tree diagram and the map), and the use of common variables in the axes. Again, this visual coherence is done automatically by our technique and does not need to be specified by a data analyst.</p><p>We collaborated with ten GenEpi experts to evaluate whether the visualizations our approach recommends are useful and understandable. Here are some examples of what experts told us:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rHifHok7nTYKJcJ0hVMaww.png" /><figcaption>Expert feedback on our algorithm for automatically creating visually coherent chart combinations</figcaption></figure><p>Overall, the response to our technique was positive and experts pointed to interesting further directions of linking visualizations to the hypothesis (or data questions) they have a priori. In our research paper, we also have examples using different datasets.</p><h4><strong>What’s next?</strong></h4><p>The volume and variety of data that organizations store produces more complex data landscapes that analysts must contend with. We think there is a lot of potential in continuing to explore opportunities that facilitate and support data reconnaissance through automation and visualization. Speeding up reconnaissance will help analysts surface actionable insights they can share with stakeholders to drive data-informed decision-making. We continue to explore this trajectory of research to examine its extensibility and utility to varied data landscapes and data questions.</p><figure><a href="https://www.salesforce.com/form/other/privacy-request/"><img alt="" src="https://cdn-images-1.medium.com/max/100/1*ImBy4Z_J1PCcv4T0kP9DJg.png" /></a><figcaption>Your Privacy Choices</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c3d8faf1ad6f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tableau-engineering-blog/digging-into-data-landscapes-c3d8faf1ad6f">Digging into Data Landscapes</a> was originally published in <a href="https://medium.com/tableau-engineering-blog">Tableau Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How the Tableau Product Internationalization Engineering Team translates millions of words into…]]></title>
            <link>https://medium.com/tableau-engineering-blog/how-the-tableau-product-internationalization-engineering-team-translates-millions-of-words-into-95d3f73630e3?source=rss----e066fecef8f2---4</link>
            <guid isPermaLink="false">https://medium.com/p/95d3f73630e3</guid>
            <dc:creator><![CDATA[Karla Wagner]]></dc:creator>
            <pubDate>Thu, 11 Aug 2022 18:01:18 GMT</pubDate>
            <atom:updated>2023-03-24T15:56:17.470Z</atom:updated>
            <content:encoded><![CDATA[<h3>How we translate millions of words to localize Tableau into over 10 languages</h3><h3>Our mission</h3><p>Our mission on the Tableau Product Internationalization Engineering Team is to help people around the world see and understand data in their own language and culture. We are tasked with localizing all of our <a href="https://www.tableau.com/products">products </a>and all of the <a href="https://www.tableau.com/support/help">Tableau Help</a> content into 11 different languages and growing.</p><h3>Writing localizable code</h3><p>It is very difficult and expensive to translate strings that are directly embedded in source code files. Therefore, developers must avoid hard-coding strings by isolating all text into localizable string resource files.</p><p>For example, if a developer is adding a new navigation option in Tableau Server called “<strong>Shared with Me</strong>”, they should not hardcode the English term directly in the source code like this:<br><strong>name=&quot;Shared with Me&quot;</strong></p><p>Instead, the developer adds the English (US) source string to a localizable string resource file with a unique string ID. In this example, the string lives in a file called <strong>en_US.json</strong>:</p><figure><img alt="Screenshot of “Shared with Me” string in English source file" src="https://cdn-images-1.medium.com/max/742/1*_DRv84zljPHG6uY4u4YdrA.png" /><figcaption>“Shared with Me” string in English source file</figcaption></figure><p>In the source code, the developer calls the string by the string ID, for example: <br><strong>name={Messages.navigation_sharedWithMe()}</strong></p><p>At runtime, the “Shared with Me” string appears in the navigation pane like this:</p><figure><img alt="Screenshot of English “Shared with Me” string in Tableau Server" src="https://cdn-images-1.medium.com/max/408/1*oMMiRgYGtG1E80EfAZKfSw.png" /><figcaption>English “Shared with Me” string in Tableau Server</figcaption></figure><p>The <strong>en_US.json</strong> string resource file is then translated into all supported languages. For example, the German strings in this example live in a file called <strong>de_DE.json</strong>:</p><figure><img alt="Screenshot of “Shared with Me” in German target file" src="https://cdn-images-1.medium.com/max/790/1*5kC_mB1KIsrh3Z6OJAMaLA.png" /><figcaption>“Shared with Me” in German target file</figcaption></figure><p>Now, when the Tableau Server User Account Language is set to German, the “Shared with Me” and other navigation options appear in German:</p><figure><img alt="Screenshot of German “Shared with Me” string in Tableau Server" src="https://cdn-images-1.medium.com/max/444/1*riiSFqNqk0ONHpeNGOp-Mw.png" /><figcaption>German “Shared with Me” string in Tableau Server</figcaption></figure><p>This process seems pretty simple so far. All of the English strings live in a string resource file called <strong>en_US.json</strong> and all we have to do is translate those strings in language specific files (<strong>de_DE.json</strong>,<strong> ja_JP.json</strong>, etc.) in all supported languages and submit our changes for the localized strings to appear when the customer changes their Tableau language preferences.</p><p>However, there are hundreds of string resource files called <strong>en_US.json</strong> scattered across hundreds of GitLab source code projects. Furthermore, Tableau products run on Windows, macOS, Linux, iOS, Android, and in the browser. For flexibility and scalability, the code is developed in various programming languages such as C++, Java, etc. Each programming language has its own file extension for the localizable string resource files, including: *.json, *.resx, *.html, *.xml, *.wxl, *.properties, *.data, *.strings, and many others. Every resource file type has a different file format that we need to create parsers for. For example:</p><figure><img alt="Screenshot of *.data string resource file example" src="https://cdn-images-1.medium.com/max/884/1*fUfU3Mknr-aCPeIaLDjUhA.png" /><figcaption>*.data string resource file example</figcaption></figure><figure><img alt="Screenshot of *.xml string resource file example" src="https://cdn-images-1.medium.com/max/1024/1*U3CWSUZRcisk6f135KNqPw.png" /><figcaption>*.xml string resource file example</figcaption></figure><figure><img alt="Screenshot of *.properties string resource file example" src="https://cdn-images-1.medium.com/max/1024/1*Mi7q3xIwSuVvc3NeSm9gKg.png" /><figcaption>*.properties string resource file example</figcaption></figure><p>To top that off, Tableau source code lives in multiple source code repositories including GitLab, GitHub, and Perforce. We not only need to create parsers to handle all of the various file formats, but we need a way to keep track when new strings are added, removed, or edited in existing string resource files and when new string resource files are created in the various source code repositories. We also need a way to collect all of the strings and hand them off to our translation vendor. And we need a way to merge all of the localized string files back into the various source code repositories.</p><h3>Centralized String Repository</h3><p>Our solution to this problem was to create a centralized string repository to process and store all of the strings from various source code repositories, programming languages, and file formats. The centralized string repository is accompanied by a number of string repository scripts that are deployed using various CI/CD pipelines to facilitate the flow of hundreds of individual modules to the central string repository database. As Tableau embraced modularization, this was needed to support the growing organization.</p><p>The centralized string repository database contains a separate folder for each individual source code module. Under each of those module folders, we include a JSON file (xx_XX.json) for every language that that we support, plus a couple of extra “pseudo” languages that we use for testing purposes only.</p><figure><img alt="Centralized String Repository file structure example" src="https://cdn-images-1.medium.com/max/1024/1*WzgWCB3rE5uJ1VgpaP9BNA.png" /><figcaption>Centralized String Repository file structure example</figcaption></figure><p>The structure of the centralized string repository JSON schema is the same across all string resource file types and all languages. We specify the schema version, the module name (source code location), and the language of the strings in the file. We parse out the string ID, the text, and the optional developer comments from the original source code string resource file. Each string is added as an individual record in the centralized string repository JSON file for that module. Each string record includes the string ID, the text to be translated, the status (“new”, “pending”, or “translated”), a timestamp to indicate when we last updated the string in the file, and the developer comment associated with the string (if available). <br>{<br> &quot;schemaVersion&quot;: &quot;2.0.0&quot;,<br> &quot;moduleName&quot;: &quot;@tableau/[moduleName]&quot;,<br> &quot;language&quot;: &quot;en_US&quot;,<br> &quot;records&quot;: [<br> {<br> &quot;id&quot;: &quot;[string_ID]&quot;,<br> &quot;message&quot;: &quot;[English (US) source string]&quot;,<br> &quot;status&quot;: &quot;[new/translated/pending]&quot;,<br> &quot;updated&quot;: &quot;[time_stamp]&quot;<br> &quot;comment&quot;: &quot;[Optional developer comment to the translators]&quot;<br> },</p><p>For example, the record for the “Shared with Me” string looks like this:</p><figure><img alt="Screenshot of “Shared with Me” string in central string repository" src="https://cdn-images-1.medium.com/max/590/1*-qnZBeqOgqeVDXmUsPZ5TA.png" /><figcaption>“Shared with Me” string in the centralized string repository</figcaption></figure><p>We developed another library that developers must import into their projects in order for our localization automation to detect changes to their string resource files. They must also include a configuration in their project that specifies the path to their localizable string resource files and they must grant permissions to our team service account to access their source code repositories. Another automated script runs nightly that connects to every source code repository and pulls down all of the new and updated strings and merges them into the centralized string repository. New strings are inserted as new records into the corresponding <strong>en_US.json</strong> files in the centralized string repository. The status is set to “new” or “pending” and the timestamp in the ”updated“ field in the record is updated to the time that the changes were merged into our centralized string repository.</p><h3>Translation Process and Tools</h3><p>Our strings are not translated in-house. We contract with a Localization Services Provider (LSP) that employs native speaker translators located in each of our target markets. We use a popular cloud-based Translation Management System (TMS) to create localization jobs to share with our LSP vendor. Every 2 weeks, a scheduled automated script runs which pulls all of the latest versions of the English source strings from centralized string repository into a new localization project in the TMS and alerts our LSP vendor to begin translation. The localization project contains separate jobs for each language. The vendor then assigns each language-specific job to their translators.</p><figure><img alt="Translation Management System (TMS) screenshot" src="https://cdn-images-1.medium.com/max/1024/1*E6BO9WQtGXURmTynNUXmeg.png" /><figcaption>Translation Management System (TMS) example</figcaption></figure><p>Translation Management Systems (TMS) are powerful tools to help with efficient and quality translations. We use Translation Memory (TM) to recycle existing translations for existing identical strings to ensure translation consistency across our products. We create rules and regular expression keywords to identify terms that should not be translated, such as product names like “Tableau Desktop”, placeholders, HTML tags, and other formatters. We utilize Term Bases (TB) to store Tableau jargon, feature names, and other terms that we want to make sure to translate consistently. The TMS informs the translator when terms should not be translated or when they should be translated using an approved term from the Term Base. The TMS also includes quality assurance tools to validate a variety of common translation mistakes including, typos, inconsistent translations, terms which differ from the suggestion in Term Base, inconsistent punctuation, missing placeholders, etc.</p><p>The TMS indicates which translations have been fully translated and approved. The translator can filter the content to display only new and updated strings. They typically do not retranslate any strings that have already been signed off except to fix translation defects. The TMS also includes a way to preview how the strings will look in the target file. In our case, the preview is how the localized file will be formatted in the .json file in our centralized string repository.</p><h3>Putting it all back together again</h3><p>Typically, it takes our Language Services Provider (LSP) vendor one week to fully translate all new and updated strings in all languages. Once the vendor hands back the updated files through the TMS tool, we run various automated scripts to process and validate the files before they are submitted.</p><p>Once all validation tests have passed, automation runs that merges all new and updated strings into our centralized strings repository. The status is updated to “pending” and the timestamp is updated to match the time of the merge request. For example:</p><figure><img alt="Screenshot of Japanese translation for the new “Clear Settings” string merged into centralized string repository" src="https://cdn-images-1.medium.com/max/656/1*S4-7nJgaK6yb3DvKPsEvQg.png" /><figcaption>Japanese translation for the new “Clear Settings” string merged into centralized string repository</figcaption></figure><p>Another automated script runs that updates the target localized files with the new and updated strings. Validation checks are run to make sure that we have formatted and encoded the files correctly and to make sure that we have not over-translated any placeholders. If the validation checks succeeds, then our automation sends a merge request to each source code repository.</p><figure><img alt="Screenshot of Merge Request for the Japanese “Clear Settings” string translation in the target source code repository:" src="https://cdn-images-1.medium.com/max/1024/1*CAry7yL30zhsZFcs_4FVYw.png" /><figcaption>Merge Request for the Japanese “Clear Settings” string translation in the target source code repository</figcaption></figure><p>Once the changes are merged, then the new translations appear in the next pre-release build of the product.</p><h3>Quality Assurance Measures</h3><p>Our translations and product builds undergo a variety of quality checks to verify linguistic quality and functionality, including the following:</p><h4>Localization Freeze</h4><ul><li>To ensure that we release fully localized software, we set a “Localization Freeze” milestone on every release calendar which is the last date that developers are permitted to make changes to their English (US) strings. Any changes to strings after that date need special approval before they can be made to avoid risk of not being translated in time for the release.</li></ul><h4>File Format Validation</h4><ul><li>Various file format validation test are run against the files during the hand back processing to ensure that files are formatted and encoded properly.</li></ul><h4>Merge Request Auditing</h4><ul><li>Any errors that are generated when merge requests are created are sent to a monitored Slack channel for quick resolution.</li><li>Automated auditing of un-merged requests notify affected teams when they have pending merge requests that are older than 2 weeks. Engineering teams have the ability to configure how they want to be notified (via email or <a href="https://slack.com/">Slack</a>) when they have un-merged requests.</li></ul><h4>LSP Quality Checks</h4><ul><li>Our Language Services Provider (LSP) runs a variety of validation checks before handing back the translations to us.</li><li>The translation tool includes various built-in validation checks, including: spell checkers, grammar checkers, punctuation checkers, verification that all strings have been translated, verification that the suggested terms from the Term Base were used, etc.</li><li>The Language Service Provider (LSP) runs regular reviews of the translations to check for linguistic accuracy. They also periodically review the terms in our terminology database for linguistic accuracy.</li></ul><h4>Geopolitical Reviews</h4><ul><li>Every quarter, we send a batch of string files to an external consultant who reviews the English and translated terms for geopolitical and cultural sensitivity issues and to make sure that we are using inclusive language.</li></ul><h4>Pseudo-Localization</h4><ul><li>Nightly pseudo-localization builds are generated that creates a fake language for testing purposes so engineers and test automation can validate localizability of our products without waiting for the fully localized strings to be handed back from the Language Service Provider (LSP).</li><li>We add a unique string ID to help us identify the location of the string in our numerous string resource files for faster defect fixes.</li><li>The string ID also lengthens the string to help us verify that the text will not be truncated after translation.</li><li>We append delimiter characters around the string to help us identify the end of the string to spot potential truncation issues.</li><li>We replace some of the English letters with non-English characters to help validate encoding and garbage character issues. But the text is still legible to English speaking engineers.</li><li>The “Shared with Me” example above looks like this in our pseudo-localized internal testing builds:</li></ul><figure><img alt="Screenshot of pseudo-localized “Shared with Me” string on Tableau Server" src="https://cdn-images-1.medium.com/max/464/1*lr6UCemgCY2HdNCucgEyIg.png" /><figcaption>“Shared with Me” string after pseudo-localization</figcaption></figure><h4>Automated and Manual Testing</h4><ul><li>In-house test automation is run regularly in all languages to verify functionality of our products with localized user interface.</li><li>Multiple times during each release, we send the Tableau product pre-release builds to our translation test vendor to run manual test passes of the localized product UI on localized operating systems. These test passes uncover untranslated strings, truncation, overlapping, functionality issues, etc.</li><li>During both manual and automated test passes, we capture screenshots of the localized user interface. These screenshots are then sent to our native speaker translators so they can see their translations in context. They compare the localized screenshots side-by-side to the corresponding English (US) screenshots to check for linguistic accuracy and consistency.</li></ul><h4>Tableau Language Quality Game</h4><ul><li>At the end of every Tableau product release, we upload all of the latest product UI screenshots that we captured into an internal tool that we call the “Tableau Language Quality Game”.</li><li>We invite native speaker Tableau employees around the globe to participate in the game for a chance to win prizes. They review the localized and English (US) screenshots side-by-side to help us identify any translation issues we may have missed during all of our other quality efforts.</li></ul><figure><img alt="Screenshot of the Tableau Language Quality Game comparing Portuguese UI to English" src="https://cdn-images-1.medium.com/max/1024/1*9aVvh3gOxSaAe3GwEuZBIw.png" /><figcaption>Tableau Language Quality Game comparing Portuguese screenshots to English with the magnifier</figcaption></figure><p>We try our best to translate our products fully and with the most accurate terminology. But sometimes bad translations slip through. For blocking issues, we ask customers to contact <a href="https://www.tableau.com/support">Support</a>. For less urgent questions and suggestions, we love to hear from our customers in our <a href="https://community.tableau.com/s/internationalization">Tableau Internationalization Community</a>.</p><h3>Summary</h3><p>Developers across Tableau are continuously making product improvements and developing new features. Many strings are updated, added, or removed daily. Our automation runs nightly to pick up their changes and merges them into our centralized string repository.</p><p>Every 2 weeks, our automation prepares the latest English source strings to be handed off to the localization vendor (LSP). The translators typically take 1 week to translate all new and updated strings. After the vendor hands back the completed translations, our automation gets the updates from the Translation Management System (TMS) and updates the records in centralized string repository. Then the changes are merged back into their respective source code repositories and modules.</p><p>This cycle of localization hand offs and hand backs repeats continuously, so we call it the “Localization Train” or “Loc Train” for short. The Loc Train never stops and stays on a mostly consistent schedule.</p><p>The “Loc Train“ process for our <a href="https://www.tableau.com/support/help">Tableau Help</a> content is very similar, except that we use a different Translation Management System (TMS) tool and translation schedule. All of the Help content lives in a single source code repository and there are fewer file formats to parse and translate. However, the amount of content to translate is extremely large.</p><h4>The whole “Loc Train” process looks like this:</h4><ol><li>The English (US) source string are pulled from the source code repository.</li><li>A snapshot of the latest English (US) files are packaged up and stored in our artifactory.</li><li>The localizable strings are parsed from each source file into a single en_US.json file.</li><li>A new localization job is created for each language using our Translation Management System (TMS)</li><li>The Language Services Provider picks up the jobs from the Translation Management System (TMS) and assigns the jobs to translators across the globe.</li><li>The translators use the TMS tool to translate the strings and the vendor alerts us when they are ready for hand back.</li><li>The localized strings are parsed from the .json files and the target localized files are created.</li><li>The localized files are zipped up and stored into our artifactory and validation is ran against the files.</li><li>The localized files are pushed to each source code repository and merge requests are created.</li><li>Once the changes are merged successfully, the localized strings will be displayed in the next build and the “status” field in centralized string repository is set to “translated”.</li><li>Quality assurance is an ongoing part of the process that happens at various stops along the “Loc Train” route.</li></ol><figure><img alt="Flow chart of the end-to-end Tableau localization process" src="https://cdn-images-1.medium.com/max/1024/1*LxkQPgOpcGZk0BuSPOpThA.png" /><figcaption>Flow chart of the end-to-end Tableau localization process</figcaption></figure><figure><a href="https://www.salesforce.com/form/other/privacy-request/"><img alt="" src="https://cdn-images-1.medium.com/max/100/1*ImBy4Z_J1PCcv4T0kP9DJg.png" /></a><figcaption>Your Privacy Choices</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=95d3f73630e3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tableau-engineering-blog/how-the-tableau-product-internationalization-engineering-team-translates-millions-of-words-into-95d3f73630e3">How the Tableau Product Internationalization Engineering Team translates millions of words into…</a> was originally published in <a href="https://medium.com/tableau-engineering-blog">Tableau Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Software Defense Spending]]></title>
            <link>https://medium.com/tableau-engineering-blog/software-defense-spending-408cb1aefae4?source=rss----e066fecef8f2---4</link>
            <guid isPermaLink="false">https://medium.com/p/408cb1aefae4</guid>
            <category><![CDATA[depreciation]]></category>
            <category><![CDATA[test-automation]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[e2e-testing]]></category>
            <category><![CDATA[test]]></category>
            <dc:creator><![CDATA[Andy Kim]]></dc:creator>
            <pubDate>Thu, 21 Jul 2022 18:05:29 GMT</pubDate>
            <atom:updated>2023-03-24T15:56:50.218Z</atom:updated>
            <content:encoded><![CDATA[<p>First, a brief history. Like many tech companies, Tableau has invested heavily in E2E automation as a daily quality indicator, which left us with thousands of desktop tests written in Python 2.7. These tests still run on the daily promotion candidate build under several configurations but are not part of our checkin quality gates. As tests that exercise the application through the UI or command system, this code is completely independent of the development code and generally has a different set of engineers with unique skills and interests. The value of each test has ranged from essential to redundant, and while I will certainly touch on a larger conversation to be had about where a team, organization, and company should focus its testing efforts, most of this article focuses on how my team eliminated most of these tests.</p><p>When Python 2.7 ceased to be supported, the company decided to port our tests to Java, forcing us to weigh the benefits of each test against the cost of reimplementing it in Java. The decision to port our tests to Java certainly deserves its own post so I’ll again defer. Essentially, we had to decide whether to port the test to Java, if it made better sense to cover the scenario upstream in a module or unit test, or whether we could eliminate it.</p><p>We asked ourselves many questions, but they can mostly be distilled to the following:</p><ol><li>Does this test exercise a unique path in our product? In other words, is this test coverage provided elsewhere?</li><li>From a cost / benefit analysis, where is the best place for this coverage? Unit/module/integration tests add unique value (run on each checkin, no developer bootstrap required) but often carry a much higher engineering cost. E2E tests have many benefits as well. For example, we’ve found these tests being critical in validating refactoring efforts even in modules with great unit test code coverage. Drawbacks have included test reliability and longer validation times due to test setup, UI and command synchronization, and teardown.</li><li>Can we live with a coverage gap? For example, if the test would only discover a small styling issue with a corner case data source, we likely would invest our time elsewhere.</li></ol><p>Here is a high-level summary of the analysis broken down by feature team (my team is IA1):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/1*3Y7q3o-uMldvMs1ZooPwDQ.png" /></figure><p>There is quite a bit of variance amongst the teams. This is due to inherent variability in judgement, certainly, but it’s also largely due to the varying needs of each team. For example, one team historically needed protection against changes from other teams so keeping a disproportional number of E2E tests made sense, particularly with those other teams not interested in investing a large engineering effort to move the tests upstream. Another team dedicated more resources to the project, mobbing as a team for several months to move tests upstream to module and unit tests.</p><p>Focusing on my team, we went into this exercise treating our E2E test coverage as the way I view the US defense budget: excessive, but much of it necessary. You don’t want to be the one slashing the budget just before a catastrophe happens. We could have easily said to port all of the tests to ensure that we wouldn’t cut anything of value, but that would be deferring the cost to others to do work that we didn’t want to do ourselves. This wasn’t consistent with our ‘We Work As a Team’ value. Also, if you’ll indulge my defense budget analogy further, we didn’t want to rebuild a bunch of ported and landlocked aircraft carriers and jets so cuts were desired.</p><p>Instead, we decided to treat this as an opportunity to familiarize ourselves with the legacy code, which we occasionally need to dive into for defect fixes. So, this investment had value outside of this E2E test deprecation exercise. It’s important to note that we had already shifted away from excessive E2E automation in more recent features so it only stood to reason that we would do the same for legacy features. To illustrate, here is a chart of our features over time, sized to the number of E2E desktop tests we had:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/1*wJoSp8ppg947tZCORQItsA.png" /></figure><p>Look at those big bubbles in the 2015–2016 time frame. What happened here versus starting in 2017? Certainly, each feature is unique, and there are exceptions. However, we generally weren’t implementing less complicated features over time. In fact, the Set Control was a multi-year project, and Set and Parameter Actions have a large footprint across multiple Tableau features. Essentially, the team decided to take a more precise approach to testing and make larger investments upfront in module and unit tests. For those legacy 2015–2016 features, we would identify variables and their domains like this:</p><p>· Variable 1 (e.g. data type): integer, float, Boolean, date, date time, string</p><p>· Variable 2 (e.g. legend type): size, color, shape</p><p>There would be test suites with all 18 combinations of the two variables. There is certainly merit to this on cursory evaluation. Sometimes you don’t know what you don’t know, particularly in this type of blackbox testing, so taking a shotgun approach can be pragmatic, particularly if you can pack these into a data driven test (DDT) that abates the upfront engineering cost. Part of writing test automation is defect discovery and putting those scenarios down for future guardrails can be important. So, this isn’t to disparage the past but rather recognize that while the intent was sound, a lot of these tests from 2015–2016 are likely redundant. We still needed proof, and it was just a matter of doing the work.</p><ol><li>Find the entry points into the features</li><li>Set some breakpoints</li><li>Step through</li><li>Find references to the underlying functions and see if they’re already being tested</li><li>Evaluate bugs attributed to these tests’ failures. Were we getting one-off defects or were all 18 passing and failing together?</li></ol><p>For example, regarding legend types: color, shape, and size. While certainly there is merit to testing each of those at a rendering level, it did not make a difference in how our features behaved; there were no unique code paths forked from the legend type. As another example, copy and paste utilized common parsing and formatting functions already tested in other E2E scenarios. While testing the copy and paste engine has merit, writing a bunch of tests specific to our features was redundant to the parsing and formatting tests we already had. Finally, while there were certainly a handful of bugs attributed to this suite of tests, there were no unique failures to any variant, and they were mostly test case maintenance issues.</p><p>So, we ultimately decided to keep 344 of the 1132 tests with a little over half of those to keep as E2E tests. We may have been able to make deeper cuts, but, frankly, analyzing test cases full time for several months got tedious, and we wanted to do some more interesting work. We also decided to err on the side of caution, and we may find deeper cuts as we port those tests. So, we were happy with getting rid of 70% of our tests. As an added benefit, this was the first opportunity for our test engineers to work in the development code, widening their breadth of technical skills. Some of them even liked it! This kind of potential career growth for test engineers is a related topic of interest that I may touch on in a future post.</p><p>As of 7/2022, we have a vendor team working on the 184 items to port to Java and with a few dozen already complete. When we work on the 160 tests to move upstream is still a matter of priority, but the items are in the backlog. Getting started is often the most difficult part, but once your team decides to take a look at its E2E test inventory, I hope this can serve as an example of how to be pragmatic. Of course, the biggest risk is losing some coverage that would have caught a bug so we cannot immediately tell how successful this endeavor was. However, being frank about the trade-offs and not letting perfection stand in the way of progress has already made us more efficient. In the worst case scenario of letting a bug slip by, we’ll be honest about it, fix it, learn, and continue to move forward.</p><figure><a href="https://www.salesforce.com/form/other/privacy-request/"><img alt="" src="https://cdn-images-1.medium.com/max/100/1*ImBy4Z_J1PCcv4T0kP9DJg.png" /></a><figcaption>Your Privacy Choices</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=408cb1aefae4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tableau-engineering-blog/software-defense-spending-408cb1aefae4">Software Defense Spending</a> was originally published in <a href="https://medium.com/tableau-engineering-blog">Tableau Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating Richer Tables Using Blocks]]></title>
            <link>https://medium.com/tableau-engineering-blog/creating-richer-tables-using-blocks-aa3a8779802c?source=rss----e066fecef8f2---4</link>
            <guid isPermaLink="false">https://medium.com/p/aa3a8779802c</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[vizql]]></category>
            <category><![CDATA[tableau]]></category>
            <dc:creator><![CDATA[Dirk Karis]]></dc:creator>
            <pubDate>Tue, 21 Jun 2022 19:06:51 GMT</pubDate>
            <atom:updated>2023-03-24T15:44:57.271Z</atom:updated>
            <content:encoded><![CDATA[<p>For over fifteen years, the VizQL visual query algebra and Tableau’s Rows/Columns/Marks Card user interface for it have been the fastest, most flexible way to see and understand your data in the way you want to see it without writing a line of code.</p><p>Still, there are always some tasks that take a bit more — or in some cases, much more — effort. The Tableau Community Forums and the wider community are filled with recipes to build visualizations that do not come easily to a casual user.</p><p>In Tableau Research, a team of us combed through these recipes and interviewed users, looking for commonalities that might form the basis for a more expressive algebra than VizQL. We settled on the distinction between “headers” and “marks” in Tableau visualizations. In the examples below, everything shaded in blue is a header, formed by the blue discrete pills on the Rows and Columns shelves; everything shared in yellow is a mark, formed by the encodings selected on the Marks Card.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/1*kDyU1eCqmjGTJZvDiOPQVg.png" /><figcaption><em>In a scatter plot, the distinction between headers and marks is obvious.</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*Gko6zjYYGeI9p6E90CItjA.png" /><figcaption><em>In this table, the user has made the measures discrete and added them to the headers; usually, they would hide the circle marks, which I have left visible here for illustrative purposes. By making the measures headers, the user could re-arrange the order of the columns if desired, but they lose the ability to apply visual encodings.</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/1*1Y_iHxH3VcR6te7EZOwffA.png" /><figcaption><em>This table, almost visually identical to the one above, uses Measure Names/Values rather than headers to display the measures. This would allow the measures to be visually encoded, but prevents reordering the columns of measures to the left of Category or Order Date.</em></figcaption></figure><p>Headers and marks have distinct properties in Tableau; headers define the structure of the viz, but only marks can have rich visual encodings such as color, shape, size, or even customized tooltips. Users often wish they could have both. They resort to workarounds such as putting discrete measures on Rows and Columns, the <a href="https://playfairdata.com/3-ways-to-create-charming-crosstabs-in-tableau/">“MIN(1)” trick</a>, assembling multiple visualizations onto a dashboard to appear as one, and other techniques. All of these have their own limitations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/890/0*NTKyUKkGohD9VPW3" /><figcaption><em>A visualization built as a dashboard, aligning two Tableau worksheets both with Sub-Category on Rows. This allows display of Sales at the Sub-Category LOD and percent of total at the Region, Sub-Category LOD, both with a color encoding, which would not be possible on a single sheet. Using two sheets makes maintenance more difficult; if the number of sub-categories increases, the two sheets will not scroll together.</em></figcaption></figure><p>We asked ourselves — what if there were a new, single type of thing that could both define the structure of your visualization and carry all of Tableau’s rich visual encodings? Many whiteboard conversations and false starts later, we came up with the idea of Blocks and created a prototype implementation. Our paper<em>, </em><a href="https://research.tableau.com/blocks"><em>Blocks: Creating Rich Tables With Drag and Drop Interaction</em></a><em>, </em>won an honorable mention at <a href="https://www.eurovis.org/">EuroVis 2022</a>, June 13th-17th.</p><h3>What is a Block?</h3><p>A Block, roughly, is a single Tableau Marks card — plus the ability to define structure and to have parent/child relationships with other Blocks. Each Block supports the familiar Tableau encodings, plus a structure type of Rows, Columns, or Inline, plus optional X and Y axes. Moving the X and Y axes to individual Blocks, rather than having them on global Rows and Columns shelves, supports sparklines and other complex visualizations.</p><p>An Inline Block is essentially a Tableau visualization with no discrete fields on Rows or Columns. Here we build the same scatter plot in Tableau on the left and the Blocks prototype on the right.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ezFQDPnMQI0ds3dW" /><figcaption><em>A simple scatter plot built in Tableau (left) and Blocks (right).</em></figcaption></figure><p>A Rows Block creates one row for each set of dimension values from its encoding; the simplest version of this looks just like a single field on Rows in Tableau, but unlike Tableau headers, Rows Blocks can also have visual encodings; if these encodings are also dimensions, they will result in more Rows; a Columns Block is similar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YOZzWerthvhWsJJg" /><figcaption><em>Single Rows blocks, with just one dimension, one dimension and a measure, and two dimensions.</em></figcaption></figure><p>The real power of Blocks comes from relationships between Blocks. Each Block inherits its Rows or Columns structure from its parents, if it has any. For example, below is the dashboard from earlier, rebuilt using a group of Blocks, and with line graphs added. We wanted to build actual sparklines, but the prototype doesn’t yet have the options to turn off the axis labels and reduce the height.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*APR7Jzs1HDup3nDIbh7jjQ.png" /><figcaption><em>The arrows indicate parent-child relationships; the rightmost two inline Blocks have both Rows and Columns parents, inheriting Sub-Category from one and Region from the other. The next block to the left is only a child of Sub-Category, so the measures are computed at that level instead, no Level of Detail calculations needed. The prototype doesn’t support table calculations currently, so we show Quantity here rather than the percent-of-total from the Tableau version.</em></figcaption></figure><p>The next example shows shape marks used to define rows and nested columns with the sub-categories ordered by highest sales for each category and ship status.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*E1pbn0BDa4_kCtTu" /></figure><p>We conducted a user study on the usability of the Blocks prototype. All of the participants were Tableau employees of varying roles and expertise. Reactions ranged from confusion at the different interface to “this is ridiculously awesome!”. You can see a video summary of the user study <a href="https://cloud.fraunhofer.at/s/5q2qXLJss8oWYkj?dir=undefined&amp;path=%2Feurovisshort2022%2Fmm%2F061-065&amp;openfile=2704047">here</a>.</p><p>In this article, we show how Blocks can be a promising UI paradigm for supporting the creation of rich tables. We plan to further build on this paradigm to explore additional analytical and interaction capabilities along with useful visual scaffolds to guide users during visual analysis. We hope that insights from our work inspire interesting research directions to balance expressivity, ease of use, and analytical richness in visual analysis tools. For more details on Blocks, please refer to our <a href="https://research.tableau.com/blocks">paper</a>.</p><figure><a href="https://www.salesforce.com/form/other/privacy-request/"><img alt="" src="https://cdn-images-1.medium.com/max/100/1*ImBy4Z_J1PCcv4T0kP9DJg.png" /></a><figcaption>Your Privacy Choices</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aa3a8779802c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tableau-engineering-blog/creating-richer-tables-using-blocks-aa3a8779802c">Creating Richer Tables Using Blocks</a> was originally published in <a href="https://medium.com/tableau-engineering-blog">Tableau Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beyond the Dashboard: Facilitating Data Conversations]]></title>
            <link>https://medium.com/tableau-engineering-blog/beyond-the-dashboard-facilitating-data-conversations-973c7a43afba?source=rss----e066fecef8f2---4</link>
            <guid isPermaLink="false">https://medium.com/p/973c7a43afba</guid>
            <category><![CDATA[decision-making]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[dashboard]]></category>
            <category><![CDATA[future-of-work]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Brittany Fiore-Gartland]]></dc:creator>
            <pubDate>Wed, 25 May 2022 18:32:49 GMT</pubDate>
            <atom:updated>2023-03-24T15:57:15.986Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vglm0Jlgj0P4rm5F" /></figure><p><em>This is a blog post was co-written by Britta Fiore-Gartland, Melanie Tory, Lyn Bartram , and </em><a href="https://medium.com/@amcrisan"><em>Anamaria Crisan</em></a><em>, and is associated with the paper “</em><a href="https://research.tableau.com/sites/default/files/tory_data_conversation.pdf"><em>Finding their data voice: practices and challenges of dashboard users.”</em></a><em> For more information check out the research paper!</em></p><p>Organizations want their employees to use data to drive smarter, more efficient decision-making. And while many organizations are awash in data, fewer are effectively leveraging data within the broader range of workflows and conversations that matter for decision-making. The current tooling ecosystem makes data access possible, but falls short of supporting the wide range of data conversations that need to happen across organizations. In general, dashboards are the primary interface to data for many individuals. They work to engage people in some cases, but in many more cases they stop short of supporting the expressivity, dynamism, and fluidity required for people to have conversations with their data, and for people to position data within decision-making conversations.</p><p>For organizations, the spectrum of potential data workers is broad, and yet our understanding of dashboard uses and practices is often limited to individuals that identify as professional analysts. Those that do not identify as analysts are often overlooked because their dashboard uses are deemed too simplistic and their interactions with data are too intermittent. These non-analysts, what we termed “data workers”, are people that use data to support decision-making in their role, but for whom data is not their primary job. We argue that these data workers are an under-recognized and under-served population, and they are essential for organizations that aim to support data-informed decision-making. This gap in knowledge shows up in the research and subsequently in the tools and ecosystems that then fail to support these real work practices and goals with data.</p><p>We conducted a research study with a spectrum of data workers that use dashboards (not Tableau-specific) in their work. Our study interviewed 20 professionals that hold diverse roles, from individual contributors to managers, and across different companies, nonprofit organizations, education, and government; for example, market campaign management, product management, operations, and employee success. We draw a distinction between professionals who produce dashboards and other data-driven artifacts for others (termed here as analysts) and people who use those artifacts (dashboard users). We capture and describe the practices, breakdowns, and workaround strategies of dashboard users and characterize their data work as conversations that occur both through and around data. In framing dashboard use as data conversations, we reveal the unmet needs of data workers and propose avenues to resolve them.</p><h3>Data Conversations</h3><p>Data and dashboard artifacts serve different roles within organizations. They can function as documentation, in which data highlights the provenance of decision making and can support collaboration by establishing a shared agreement or baseline. They can also function as circulation, distributing data artifacts throughout an organization, making information and ideas accessible across data workers and organizational contexts. Finally, and most pertinent to our research, is the role of data in supporting conversations.<em> </em>When data is conversational it supports sensemaking practices and importantly, it becomes the mode through which new ideas emerge and organizations may begin to change. But this<em> </em>depends on:<em> </em>Who can have a data voice? Who can ask and answer questions with data?</p><p>We can further break down these data conversations into categories: those with data, and those through and around data. <strong>Conversations with data</strong> involve interactions directly between the user and their data (individual analysis work). <strong>Conversations through and around data </strong>use the data as a medium for conveying information, a story, or a particular view to others. Key here is that people reshape the data and its associated artifacts as a strategy for communication, anticipating the translation necessary for data to jump contexts. Talk around data may generate further questions and insights, lead to actions, or simply support connections among people.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iddK9LW7UJgPMVWD" /><figcaption>Figure 1 Conversations with data and conversations through and around data</figcaption></figure><p>For these data workers, many of their conversations with data include common analysis goals, such as explain, predict, or find anomaly, that overlap with goals that an analyst would have. The difference, however, might be whether they have time or bandwidth to do the work, whether they have the data and analytic skills to do what they need to do, or perhaps they are lucky and have an analyst to support them answering the questions they have with their data. The goals of monitoring, lookup, and summarizing stood out as uniquely important for data workers, and are often overlooked when building tools for analysts.</p><p>Through mapping participant tasks to goals we are able to see how tasks contributed toward many goals while others were more specific. For example, data might be sliced and diced (i.e. grouped, filtered, and aggregated) as part of almost any conversation with, through, or around data, whereas data was only modeled to predict or discuss with others. Similarly, participants only bothered to format if the goal was to circulate or discuss. We are also able to see how there are tasks distributed across <strong>accessing, enriching, shaping, and construction</strong> in addition to <strong>sharing </strong>when thinking about what data workers do to have conversations through and around data (see Figure 2).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*u6E979EPjsxEniG_" /><figcaption>Figure 2 <strong>Goals and Tasks,</strong> Goals (rows) and tasks (columns) for Dashboard users. <strong>Breakdowns and Strategies,</strong> Dashboard users’ strategies upon breakdown, grouped by task in which the breakdown occurred. Size = number of participants.</figcaption></figure><h3>Breakdown and Mitigations</h3><p>Our study participants aimed to have conversations with, through, and around data, but encountered an array of breakdowns along the way. We used these moments of breakdown, when participants couldn’t complete their task within their current tool, as probes for innovation and cataloged the strategies they employed to mitigate the breakdowns (see Figure 2). We found that even deceptively simple tasks like constructing and sharing insights encountered multiple breakdowns as data workers tried to stitch together customized narratives and format it for presentation that led participants, most commonly, to switch tools, but also do manual workarounds, and unfortunately, give up when tooling didn’t support their task.</p><p>A key finding was that data work practices did not follow a linear model of data analysis. Instead these conversations with data often required moving back and forth between tasks, from constructing a narrative back to enriching the data back to sharing and reconstructing that narrative. Both analysts and data workers alike would return to the dashboard and, when applicable, different analysis phases in iterative cycles of refinement to realize their goals and tasks. However, this iterative cycling was especially problematic for data workers when they encountered breakdowns that prevented them from completing their tasks. Because data tools don’t support the full scope of how data workers need to engage with data, when they tried to achieve their goals with data, it often required manual workarounds, data dumps or requesting assistance from an analyst (if they were lucky enough to have one dedicated to their needs). For data workers, data is not their primary job so they may not have advanced levels of data literacy or bandwidth for these more data-intensive tasks and workarounds. The cost of having to stitch together tools and tasks to achieve their goals with data is so high that it means many will find sub-optimal workarounds or abandon their goal entirely.</p><h3>Addressing Unmet Needs</h3><p>By attending to the work practices and challenges of data workers, we identified the top breakdowns and unmet needs of data workers. These represent opportunities for better tooling that directly supports the conversations with, around, and through data. These opportunities lie in developing experiences with data that don’t over-rely on the dashboard, but instead invest in highly interactive and flexible tool ecosystems. Rather than one-way communication tools, these tools need to leverage augmentation to enable more proactive monitoring, fluid asking and answering of questions within their workflows and greater ability to construct insights and stories through and around data. There are opportunities to help people more directly see and manipulate data values and records, which provides an easier on-ramp for people to find their data voice.</p><p>This study highlighted a core practice for data workers is content creation as a way to share and construct insights for others. There is an opportunity to recognize the value of this work by finding ways to integrate qualitative and quantitative frames to better reflect the way people actually have conversations through and around data. It is not just the what of these conversations that matter for supporting these conversations, but also the when and where. Integrating insights and analytics into current data worker workflows is an unmet need across many industries and job functions.</p><p>More people will find their data voice when our tools and systems support the types of conversations not just with data, but through and around. So let’s get to work!</p><figure><a href="https://www.salesforce.com/form/other/privacy-request/"><img alt="" src="https://cdn-images-1.medium.com/max/100/1*ImBy4Z_J1PCcv4T0kP9DJg.png" /></a><figcaption>Your Privacy Choices</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=973c7a43afba" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tableau-engineering-blog/beyond-the-dashboard-facilitating-data-conversations-973c7a43afba">Beyond the Dashboard: Facilitating Data Conversations</a> was originally published in <a href="https://medium.com/tableau-engineering-blog">Tableau Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Being an industry research scientist]]></title>
            <link>https://medium.com/tableau-engineering-blog/being-an-industry-research-scientist-e9a2749245a1?source=rss----e066fecef8f2---4</link>
            <guid isPermaLink="false">https://medium.com/p/e9a2749245a1</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[tableau]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[industry-research]]></category>
            <category><![CDATA[semantics]]></category>
            <dc:creator><![CDATA[Vidya Setlur]]></dc:creator>
            <pubDate>Wed, 20 Apr 2022 21:59:00 GMT</pubDate>
            <atom:updated>2023-03-24T15:46:52.744Z</atom:updated>
            <content:encoded><![CDATA[<p>I recently became the head of research at Tableau and thought it would be a good time to reflect back on the decade since I joined the company. I’d like to spend some time talking about my personal experience being in industry research, covering topics that have been helpful in my own personal journey — crafting a research portfolio, following through on research ideas, setting up productive collaborations, listening, communicating, and writing as a way to organize my thoughts.</p><p>I come from a rather eclectic research background, a mixed skillset of seemingly disparate disciplines — computer graphics with natural language processing (NLP). When I joined Tableau Research back in 2012, I had limited experience in data visualization, with the only visualization-related projects being map displays and navigation at Nokia Research. Folks were perplexed by my unconventional research background and what I was doing here; the computer graphics experience somewhat justified my existence at that time. Fast forward ten years later, with the help of wonderful collaborators, I have carved out an exciting research area that combines these two disciplines, essentially looking at the semantics of the data with user intent to inform the meaningful visual depiction of data. It’s also heartening to see how this work is still evolving in terms of innovation — inspiring interesting research papers from the academic community as well as products and features from the Tableau development teams. The work also provided me an opportunity to be an engineering manager on the Tableau NLP product team for a couple of years, shipping production code to customers. As I reflect back on the 17 years of my research career, I thought I would document my personal experiences during my time here at Tableau, and what it takes to do impactful work as an industry research scientist. So here goes.</p><h4>Managing a research portfolio with a point of view</h4><p>While it is helpful to have a set of high-level research goals to guide one’s work, translating that to day-to-day research tasks can feel a bit abstract; a common problem for any research scientist, be it in academia or industry. Over the years, I have found that developing a collection of short- to long-term research ideas (i.e., a research portfolio) helps one take a crack at a bigger, abstract problem and identify smaller nuggets of ideas to help move the research forward. Let me walk you through a few project examples from my research portfolio to illustrate what I mean.</p><p>When I started at Tableau Research, one of my goals was to identify how NLP and semantics can help support meaningful visual encodings for data. I started off working with <a href="https://research.tableau.com/user/jock-mackinlay">Jock Mackinlay</a> on a project to <a href="https://research.tableau.com/sites/default/files/semanticshapes-final.pdf">automatically generate semantically relevant icon encodings</a> for categorical dimensions of data points. The algorithm employed NLP to find relevant imagery from the Internet, such as the example below where each animal is automatically associated with a semantically meaningful icon, making a chart becomes more useful to understand and follow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*CV1kbClIdw7R81zZ" /></figure><p>A natural segue from that project was exploring with <a href="https://research.tableau.com/user/maureen-stone">Maureen Stone</a> how linguistic information about the data could be used to generate <a href="https://research.tableau.com/sites/default/files/setlurstoneinfovis2015_0.pdf">semantically meaningful color palettes</a>. For example, here is a color palette automatically generated based on the ice cream flavor names in the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/651/0*8-JQLnXyNCx3qiGq" /></figure><p>While these projects did not lead to specific features in the Tableau product per se, they did accomplish two things: (1) the projects got researchers thinking about interdisciplinary research where NLP techniques could be applied to different aspects of data visualization, and (2) got leadership at Tableau excited about the opportunities where concepts from NLP can bring value to its customers.</p><p>These little initial successes led to bigger and more impactful projects, such as the deployed <a href="https://help.tableau.com/current/pro/desktop/en-us/maps_mapsearch.htm">map search feature</a> in Tableau 9 to explore conversational interfaces like <a href="https://dl.acm.org/doi/pdf/10.1145/2984511.2984588">Eviza</a> later. Eviza was a research prototype that supported vague natural language queries in the context of a visualization, such as, “Find large earthquakes near California.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*y4ZU0pXKmZ7BbDoJ" /></figure><p>While it was Eviza that ultimately excited executive leadership to seriously think about investing in NLP, every project in my research portfolio played a little part in helping communicate my point of view as to how and why semantics and user intent are important in supporting Tableau’s mission of helping people see and understand their data.</p><p>A well thought out research portfolio consists of a variety of short to longer-term creative projects that try to solve a set of problems that are important to the company’s mission. The ideas need to be adequately discussed with peers, managers, and product teams, to validate the approach along with understanding the applicability and value for the research in the near future. Having a coherent research portfolio also provides an effective communication medium for demonstrating a specific point of view to relevant decision-makers in the company like product owners, engineering managers, and senior tech leads, along with establishing a track record for doing interesting research.</p><h4>Prototyping to demonstrate an idea</h4><p>Ideas are cheap, and building something helps determine how viable that idea is. Most of my projects involve developing a prototype. I have found that prototyping is an effective way to clarify one’s thinking, demonstrate a research idea through an algorithm or novel experience, show the benefits of the approach, and even convey the limitations. These proofs-of-concept provide a tangible window into the bigger picture of your work. There is also the visceral thrill of building something that feels real, getting people to use and play with the system for feedback. I often demo to product teams — be it a new way of implementing auto-completion for search or a novel way of handling ambiguity in Slack and chatbot interactions. Prototyping is a tangible way to elicit their reactions, followed by additional iteration.</p><p>A prototype also gets product teams excited, particularly engineers, as they are able to identify pathways for taking the idea forward more concretely in their own code and feature ecosystem. It also helps foster creativity, aimed at solving important problems that inspire the engineering teams to innovate. Videos of prototypes are also an effective medium to keep colleagues informed of my progress and share my own excitement of chipping away at the bigger picture.</p><p>While Eviza was a prototype that inspired the formation of the <a href="https://help.tableau.com/current/pro/desktop/en-us/ask_data.htm">Ask Data</a> product team, I will say that prototyping is an effective tool for even smaller ideas. For example, <a href="https://research.tableau.com/sites/default/files/Autocomplete-UIST2020.pdf">SneakPique</a> was a prototype that explored visual autocompletion techniques to help users during data exploration. Aspects of that work inspired the Ask Data team to think about ways to improve the product’s autocompletion experience, particularly using a calendar widget for formulating temporal queries that filter to dates. In addition, a software engineer from the <a href="https://aerospace.org/">Aerospace Corporation</a> reached out to me and said that they found SneakPique to be useful for validating their approach for users to query their systems (SQL-like syntaxes being too limited and natural language being too free). However, to make prototyping part of one’s research practice, it is important to be fluent in programming languages and libraries that can be used to realize these ideas. Whether it is a programming language or a design tool, I think that it is important for research scientists to invest in themselves, being adept at picking up a tool and using it quickly to realize an idea.</p><h4>Value productive collaborations and keep them time-boxed</h4><p>Over the years, I have considered myself to be very lucky, having worked with a range of amazing internal and external collaborators. Many of my collaborators come from backgrounds different from mine, including software engineers, product managers, visualization practitioners, and UX researchers. I have always found that a good mix of different perspectives makes for a really enjoyable and fruitful end result. I also like to observe how people work on a given problem, their approaches, “secret sauce,” and bag of tricks. I often feel inspired to pick up some of these traits and skills, and improve my own way of working as a research scientist. I have found that the best collaborations are time-boxed with clear deliverables and role clarity.</p><p>I worked with the Tableau Maps Team for a period of six months to develop a search relevance algorithm that would rank places in the data domain higher than just place prominence. For example, when users search for “Paris” in a map of the US, “Paris, Texas” would be ranked higher in the drop-down list than “Paris, France.” In other words, I was tasked with researching a search relevance algorithm that took into account the domain of the data rather than just based on how popular the place is. Engineers carved out a separate module for me to work in, as I iterated and tweaked the ranking parameters. Beyond being part of shipping the feature, it was a valuable learning experience. I closely watched how a product team operated, prioritizing decisions, and writing production code. The expectation was that once we were happy with the results, the code-reviewed algorithm, along with relevant unit and integration tests would be integrated into production. The work expectations and timeline were clearly defined at the beginning of the collaboration and my value-add was known to the team, making for a very pleasant and productive collaboration all around.</p><p>For collaborations involving paper deadlines, setting such a timeline comes (almost) for free. However, it is equally important to have self-imposed deadlines when consulting with feature teams, developing strategy road maps, vetting startups for acquisition, and other community service obligations, including being on conference program committees. I often explicitly ask the other party what timeline they have in mind so that I can load-balance my work accordingly to suit their expectations. Deadlines also help me prioritize my time, work on what’s important, pacing myself without burning out.</p><p>I will also add that a time-boxed collaboration can help bring a less effective group project to a close if it isn’t going too well. For example, I started looking into mobile interfaces for search with the UX Team at Tableau. After a couple of months, midway into the project, we realized that the project was no longer a priority to the product roadmap at that time, and as a group, we were spinning and not moving forward. We made the decision to stop continuing, document what we learned till then, and decided as a group that we would revisit this if and when the timing was right. No bridges were burned, and everyone was in fact relieved to move on.</p><h4>The importance of having good mentors</h4><p>I believe that mentors are a critical aspect of one’s professional journey where you can have grounded and often candid conversations to shed light on new opportunities and overcome challenges in a way that feels actionable. <a href="https://www.radicalcandor.com/the-book/">Radical Candor</a> is a great book that I have found helpful in both identifying good mentors and deriving value from such a partnership.</p><p>Over the years, I have been fortunate to have had mentors who have helped shape my career in different ways, providing guidance and instilling confidence as I continue to grow professionally. One of my earliest mentors has been <a href="https://graphics.stanford.edu/~hanrahan/">Pat Hanrahan</a>, who really supported and nurtured my love for semantics and graphics from my early Ph.D. days. I can still count on him to talk through research and work over a good cup of coffee. Jock, who was my first manager here at Tableau has evolved to being a mentor and a friend who provides sound, technical advice, and feedback. Mentors are also helpful for offering their perspective on other soft skills that are required to grow professionally and personally. Being a woman of color, I regularly seek guidance from other women who have navigated similar issues in their own lives.</p><p>Having great role models as mentors has also helped me pay forward and mentor upcoming research scientists and young women in the field. I find it immensely satisfying now to spend time and work through topics that my mentees bring to the conversations. Reflecting on their questions and struggles helps me introspect on my own experiences, helping me be a more empathetic, sensitive, and effective leader to my team and colleagues.</p><h4>Listen, listen, listen</h4><p>While part of a research scientist’s job is to give talks and communicate their ideas to a broader audience, the most important skill that I have learned over the years is to listen and observe what people are saying. Whether seeing how others react to my ideas or how they animatedly talk about their own, I follow body cues, voice intonation, and language to calibrate my thoughts as I make sense of my world. Being part of a bigger company, I like to listen to people from different parts of the organization — product managers, architects, UX designers, engineers — understanding what matters to them and to their team and how that translates to my own work. I am also a voracious reader (yes, that’s part of listening too) and try to stay abreast of what is going on in the market and industry at large.</p><p>Tableau has an annual customer conference, which is fantastic as it provides a close-up view of how customers react to new product feature releases, presentations, as well as concept ideas. For example, in 2019, we ran user studies for various chatbot interfaces (voice- and text-based) to 30 study volunteers at the Tableau Conference. Feedback from these studies provided valuable insights regarding people’s expectations and interactions to a chatbot and which modalities are worth pursuing. While voice-based chatbots seemed rather cool and novel at that time, we realized that their utility for users conversing with data was very niche and limited. Text-based interactions turned out to be promising, and the ideas are even more relevant now given that Slack was recently acquired by Salesforce. So, keenly listening and observing actual people who would potentially be the ones using these ideas, brings ecological validity to your work, functioning as early assessments when formulating and identifying longer-term projects to tackle.</p><h4>Writing as an organizing principle for communication</h4><p>Publishing for the sake of publishing has never been a priority for me, and that is one of many reasons I shied away from academia. However, writing is an effective way to help organize my thoughts and clearly describe why a certain approach was taken and what research problem it solves. For deadline-driven people like myself, submitting academic papers is a great way to make progress on research ideas and have that work peer-reviewed. Having a paper that addresses customer needs and the company’s mission and is accepted at a top conference can be a rare “ double-word score.” Even during my stint as an engineering manager on the Ask Data team, I published several papers with folks on my team, articulating the thought process that went into the feature. These included work on <a href="https://research.tableau.com/sites/default/files/IUI_vsetlur.pdf">handling ambiguity</a> in the input utterances,<a href="https://research.tableau.com/sites/default/files/intent_modifiers.pdf"> visually representing vague concepts</a> like “high” and “low”, and supporting an <a href="https://research.tableau.com/sites/default/files/intentpaper_2019.pdf">analytical conversation</a> through pragmatics. Research-inspired engineering is not new to Tableau; after all, it is a company that was founded on a set of award-winning academic papers such as <a href="https://research.tableau.com/sites/default/files/Tableau-CACM-Nov-2008-Polaris-Article-by-Stolte-Tang-Hanrahan.pdf">Polaris</a> and <a href="https://research.tableau.com/sites/default/files/200711-infovis-showme.pdf">Show Me</a>.</p><p>While academic papers can demonstrate external thought leadership, it is just as important to work on relevant problems that don’t feel esoteric. In fact, white papers and blogs are also effective ways to communicate and share ideas with a less academic audience. Working at Tableau, I also enjoy writing technical specifications, state-of-the art survey writeups on a topic of interest like natural language generation, or how ElasticSearch APIs can be used for search. The exercise keeps me abreast of what is going on in the industry. The art of good writing is to clearly convey the intended message to the audience at the right level of detail to help with action-driven decision making at a company. I often study good exemplars of writing as inspiration — maybe it’s a strategy document that was well received by leadership or a best paper at a seminal conference. Often conferences publish author guides that I have found useful, such as “​​<a href="https://uist.acm.org/uist2022/author-guide.html">How to Write a Successful UIST paper</a>.”</p><h4>Closing thoughts</h4><p>Industry research is invigorating and constantly keeps me on my toes. I find myself continuously recalibrating and balancing longer-term blue-sky ideas while trying to stay relevant to the company. As an industrial research scientist, one needs to be flexible and adapt to whatever makes sense to push that idea forward in a way that fits the mission of the company. Being passionate, creative, and persistent with an entrepreneurial mindset also helps. These characteristics have served me well at Tableau, particularly with my unconventional background. I embraced various opportunities, learning and evolving along the way; from collaborating with research scientists who have strong visualization or NLP backgrounds to working as a partner to the Maps and Search product teams, or jumping headfirst into being a production-coding scrum-attending engineering manager embedded for several years in the Ask Data team, being part of all the sausage-making was incredibly educational and shaped who I became as a research scientist. Each journey has given me something to carry forward, whether it’s the art of writing quality software, crafting persuasive strategy documents, or compiling beautiful LaTeX for research papers. Most importantly, I have found a community of amazing people who continue to inspire me.</p><figure><a href="https://www.salesforce.com/form/other/privacy-request/"><img alt="" src="https://cdn-images-1.medium.com/max/100/1*ImBy4Z_J1PCcv4T0kP9DJg.png" /></a><figcaption>Your Privacy Choices</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9a2749245a1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tableau-engineering-blog/being-an-industry-research-scientist-e9a2749245a1">Being an industry research scientist</a> was originally published in <a href="https://medium.com/tableau-engineering-blog">Tableau Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Explain Data, Explained]]></title>
            <link>https://medium.com/tableau-engineering-blog/explain-data-explained-c780ed905a07?source=rss----e066fecef8f2---4</link>
            <guid isPermaLink="false">https://medium.com/p/c780ed905a07</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[tableau]]></category>
            <dc:creator><![CDATA[lara.r.thompson]]></dc:creator>
            <pubDate>Wed, 30 Mar 2022 21:14:59 GMT</pubDate>
            <atom:updated>2023-03-24T15:57:55.731Z</atom:updated>
            <content:encoded><![CDATA[<p>Explain Data is an autoML feature in Tableau that lets anyone ask “Why?” questions about their data. Why are sales down this month? Why did people ride bikes more in August? Run Explain Data on any mark or data point in a viz that you want to understand better. Tableau will build and test statistical models of the data, including <em>unvisualized</em> data from your workbook, to build possible explanations of why your selected mark is high or low.</p><p>Today, I’ll walk us through the various explanations that can be surfaced in the order that they would appear in the Explain Data panel. I’ll discuss what can and can’t be concluded for each type of explanation. Then I’ll go into more technical details of how Tableau decides whether or not an explanation is good (feel free to skip this section!).</p><p>At Tableau, I am a Principal Data Scientist, working on augmented analytics features such as Explain Data. But I’m also an avid rock climber: I boulder and trad climb (but, strangely, I don’t sport climb! Bouldering = climbing boulders with pads and people spotting to protect a fall; sport climbing involves ropes and clipping into bolts drilled into the rock; trad or traditional climbing involves ropes also but gear should not alter the rock and be cleaned after the climb).</p><p>For this post, I’ll analyse a climbing <a href="https://docs.google.com/spreadsheets/u/0/d/1J6d45EqIlIsIqNdi2X-Zl-EGFxf9d9T3R_W55xrpEAs/edit">dataset</a> that compiles survey responses from over 500 climbers. Questions ranged from height and weight to specific details of training for climbing. I’ll look into how these aspects relate to the bouldering grade that they climb at.</p><p>Boulders are graded in North America by the V-grade system and rate the level of difficulty; they go from V0 (pretty easy) to V17 (climbed by only 2–5 people, pending grading consensus on three very hard boulders). Note that not all ‘V10 climbers’ (the climbers that can boulder V10) are alike: there’s the climber who focussed all year and only climbed a single V10 vs the climber that can do several in a session. That’s a huge source of noise in this data!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Pey70HuLeQmAoP67" /><figcaption><strong>Halfway up Middle Monk in The Ruins above Kelowna, BC.</strong></figcaption></figure><h3>Extreme Values</h3><p>I expect bouldering grade to correlate highly with various strength metrics — pull-ups, finger strength, etc — but when it comes to core strength, visualized here as the maximum duration of a hanging L-sit, the climbers with the highest average value only climb V6. While this is the most commonly climbed max grade for the male respondents, it’s not even half the hardest grade climbed in this dataset. The world record is under 2 min and yet the average for V6 climbers is over 3 times higher!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UkQxLdVx0euYrW4lO_TaCA.png" /><figcaption><strong>The hanging L-sit is a difficult pose with a </strong><a href="https://www.youtube.com/watch?v=OFOwA7vaOws"><strong>world record</strong></a><strong> under 2 minutes. The viz shows the average L-sit time vs grade (as a measure) with the grade (as a dimension) setting the level of detail.</strong></figcaption></figure><p>To explain this suspiciously high L-sit time, select the V6 mark and then click on the lightbulb. Each of the 2 measures in the viz is shown but I’ll start by investigating the AVG(max L-sit time) by clicking into that tab. The first explanation we see is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/0*AXPlwksO1tYfnuij" /></figure><p>Out of all of the records for V6 climbers, one climber had an L-sit time of 10,000 seconds! Given the world record is under 2 min, this value must be wrong. Explain Data calls out this extreme value and shows record details for the suspicious record: many other values are suspicious so I want to drop that entry entirely. The next viz does that for me: now with a far more reasonable mean L-sit time of 30s, the V6 climbers follow the overall trend. I can pop out that viz as a new sheet and keep working without that entry.</p><h3>Explaining V10 Climbers</h3><p>As a climber myself, what I really want to know is how to climb harder grades. Can the data help me direct my training? Now I’ll explore the V10 climber data (V10 is a grade that’s high but can be climbed by quite a few climbers in this dataset — Explain Data requires at least 5 rows to explain a mark).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dNjBMDEAqp7xmmpb" /></figure><h3>Contributing Dimensions</h3><p>For Contributing Dimensions explanations, an unvisualized dimension (or the single value of one) is used to predict the bouldering grade. Here are the top results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/0*sHG0PD2vUcwC3RHS" /></figure><p>Just over half of the climbers who’ve climbed a V10 in the last 3 months don’t do any kind of roped climbing, answering “I don’t climb routes”; compare that with only 30% of climbers who boulder at any grade. Furthermore, the fraction of responses “I don’t climb routes” increases with bouldering grade.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*d6piqBA7OzmVPPwE" /><figcaption><strong>(left) Because there’s only one visualized dimension (the grade), the base model has no predictors: it predicts the average of other marks for all marks. (right) The fraction of climbers that don’t climb is used as a predictor in the explain model, vastly improving the model fit to the other marks and somewhat improving the prediction for the V10 climbers. The newbie V1 climbers are skewing the model so this explanation is even better when they’re excluded.</strong></figcaption></figure><p>I have explored this climbing data before and haven’t discovered this particular explanation: clearly, there should be a trend between the climbing grades, but I hadn’t considered the Nulls as predictors of the other. AutoML like Explain Data is especially good at looking at all combinations, even the “silly” ones, and sometimes it finds startling insights. To be more like the stronger climbers, I should hang up my ropes and focus on bouldering.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/0*OKuen99J_PPAw7Hc" /></figure><p>The explanation given by “Where do you climb?” considers all three values: climbers that climb inside or outside or both inside and outside. The higher grade climbers are more likely to climb both indoors and outdoors; only a few V10 climbers climb only indoors. Lucky for me, I started bouldering outdoors!</p><h3>Contributing Measures</h3><p>The explanations in this section use the average of an unvisualized measure to predict climbing grade. Here are the results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/0*Ixu4ULZLpixdyva7" /></figure><p>I see all the expected correlations: strong climbers have stronger fingers (purported to be the strongest predictor of climbing level), they climb more, train longer, are stronger generally, are taller, and have a longer arm span. Seems I just have to double to triple each of the relevant strength measures (and increase my training to get there), and grow another ~8cm taller and ‘spannier’. This is where I run into one of the limitations of any kind of analysis like this — the algorithms used by Explain Data can tell me what differentiates a V10 climber, but they don’t have any knowledge of which variables I can actually control!</p><h3>Other things to explore</h3><p>Back at the root level of the Explain Data pane, we can click through to see “What is unique about V10?”, completely independent of either measure in the viz (more details on scoring to come). This category of explanations might include totally random information, such as the percentage of climbers with blue eyes, or what kind of pet they own (if that were in the dataset). There may or may not be an actual relation with climbing grade but, in this section, the explanations highlight only what makes V10 climbers different.</p><p>V10 climbers weren’t unique and have no explanations in this section. To find some, consider instead a viz of the average ‘hardest V grade ever climbed’ across all months of the year. Why is November higher than other months? I shouldn’t expect many meaningful explanations for this peak (however, November is downtime for many climbers and possibly a good time to complete surveys).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wfKhEp0ryJDoxHzU" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yiktnv65lahUfYPf" /><figcaption><strong>Listed under What is unique about this November? we find the unvisualized dimension “Grip: pinch”: the rate of climbers that train a pinch grip.</strong></figcaption></figure><p>November has a standout high fraction of respondants that train pinches (either by hanging hands pinched to a block or pinching a heavy block and lifting it), but there is no discernible trend in bouldering grade as a result of training those pinches. This is why this explanation is called out as something that makes November unique, but isn’t offered as an explanation for a climber’s grade.</p><p>A dimension returned only as unique may still be a good explanation: the model may not be able to learn from the other marks to explain this one <em>because</em> it’s so different from the others. Use domain knowledge to pick out what is worth pursuing further.</p><h3>Scoring Model-Based Explanations</h3><p>Most of the explanations under the measure tab are <em>model-based explanations</em>. If a model including an unvisualized dimension or measure performs better than without, that dimension or measure is possibly <em>contributing</em> to the target measure. Under the “What is unique about V10?” tab, Tableau lists unvisualized dimensions along which the mark is rather different compared to the other marks.</p><p>The models being compared are two regression models of the target measure. Every model is fit to the marks, excluding the mark being explained. First a base model is fit using the visualized dimensions as predictors (if there are at least 2); when there are no predictors, the base model is just the average over other marks (left in the figure). Next, one by one, all unvisualized dimensions and measures in the dataset are used as additional features in the explain model (<em>y</em> in the right plot of the figure).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/0*VAZMjeq4ZjC7deSW" /><figcaption><strong>The example on the left depicts a viz with a very low x value for mark B. With only one visualized dimension, the base model will predict the average of marks A, C, and D (the dashed line). In the example on the right, an unvisualized measure, y, has been added as a predictor to the explain model (the dashed line). This model fits A, C, and D better and predicts a value of x for B a lot closer to the true value: y is a good Contributing Measure explanation.</strong></figcaption></figure><p>Each model is evaluated by the mean absolute fit error and the error in predicting the held out mark. The quality of the explanation is scored by comparing the errors of the two models. Roughly speaking, the score is a weighted average of the relative fit and predict errors, with adjustments to counter possible overfitting and a lucky prediction on the single held out mark (with an anomalously low prediction error most likely gotten entirely by chance).</p><p>Our models use <a href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html">ridge regression</a> with a regularization constant of 0.1; visualized dimensions are<a href="https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html"> one-hot encoded</a>; unvisualized dimensions are modelled using the rate across each of the top 10 values as predictors; unvisualized measures are averaged to the viz level of detail and modeled to second order; measures and rates alike are normalized.</p><p>Are all model-based explanations just finding strong correlations? Not only: the explained mark is not in the training set and the relation must generalize to predict it well. Explaining a different mark may have different explanations. Are all explanations also a strong correlation? If we mean <em>linear</em> correlation, then, again, no: we include quadratic order of measures and they might actually have zero linear correlation.</p><h3>Scoring of Other Explanations</h3><p>Non-model based explanations are evaluated differently. These include the Extreme Values (though some <em>are</em> model-based) and unique mark explanations shown with the climbing data in this article.</p><p>Extreme Values are row level values that are separated by at least 1.5x the interquartile range (the middle 50% of the data) from at least 75% of the data.</p><p>For unique mark explanations, the distribution of values across an unvisualized dimension in the mark is contrasted with that of the other marks; if the marks’s distribution is different enough from the distributions of other marks, this explanation will be displayed.</p><h3>How to use Explain Data</h3><p>Explain Data makes it easier to exhaustively explore a new dataset. It will consider all unvisualized dimensions and measures as predictors of the target measure; it looks for outliers at row-level within the mark and odd behaviour of the mark itself along unvisualized dimensions. But Tableau doesn’t know which correlations are spurious, has no understanding of the direction of causality (does being strong enable the climber to boulder V10 or does bouldering V10 make the climber strong?), and doesn’t know which variables I can control (training habits) and which ones I can’t (height and arm span). What I do with the explanations will then rely on my “domain expertise”.</p><p>Of course, any tool is only going to be as good as the data you put in, and any findings will mirror the bias inherent in the dataset. How would our findings have changed had the climbing survey recruited from a juniors team, or at a climbing gym, or at the physiotherapist’s office (where so many climbers end up after over-training)?</p><h3>Explain Data and Augmented Analytics</h3><p>Explain Data isn’t about closed loop analysis — it can’t be with autoML — but rather to keep the human in the loop. Could you find everything Explain Data shows you on your own? If you have the expertise, sure, but it would take a long time and you might miss something; otherwise, maybe not.</p><p>What might you find in your data?</p><figure><a href="https://www.salesforce.com/form/other/privacy-request/"><img alt="" src="https://cdn-images-1.medium.com/max/100/1*ImBy4Z_J1PCcv4T0kP9DJg.png" /></a><figcaption>Your Privacy Choices</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c780ed905a07" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tableau-engineering-blog/explain-data-explained-c780ed905a07">Explain Data, Explained</a> was originally published in <a href="https://medium.com/tableau-engineering-blog">Tableau Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[You should give Catch2 a try]]></title>
            <link>https://medium.com/tableau-engineering-blog/you-should-give-catch2-a-try-a06161f05b64?source=rss----e066fecef8f2---4</link>
            <guid isPermaLink="false">https://medium.com/p/a06161f05b64</guid>
            <category><![CDATA[unittest]]></category>
            <category><![CDATA[catch2]]></category>
            <dc:creator><![CDATA[Hung Dang]]></dc:creator>
            <pubDate>Wed, 16 Mar 2022 16:57:31 GMT</pubDate>
            <atom:updated>2023-03-24T15:58:22.729Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p><a href="https://github.com/catchorg/Catch2">Catch2</a> is a modern C++ test framework for unit tests, TDD, and BDD using C++11, C++14, and C++17 standards. This project is actively developed, with 200+ contributors. Our blog post gives a gentle introduction to writing unit tests using Catch2.</p><h3>Why Catch2?</h3><p><a href="https://github.com/catchorg/Catch2">Catch2</a> offers an idiomatic way to write our unit tests. Here are some <a href="https://github.com/catchorg/Catch2/blob/devel/docs/why-catch.md#top">key features</a>:</p><ul><li>Use standard C/C++ operators for the comparison.</li><li>Decompose the assertion expression, and log both <strong>LHS</strong> and <strong>RHS</strong> values when reporting errors.</li><li>Have only one core assertion macro for comparisons.</li><li>Test names are free-form strings — no more couching names as legal C++ identifiers.</li><li>No external dependencies.</li><li>Test cases are self-registering functions.</li><li>Can group test assertions into sections. And test sections are executed in isolation (eliminates the need for fixtures).</li><li>Support <a href="https://en.wikipedia.org/wiki/Behavior-driven_development">BDD-style sections</a> as well as traditional unit test cases.</li></ul><h3>Writing our first Catch2 unit test</h3><p>Assume we have our implementation for the factorial function using the recursive approach. We want to add some tests to verify that our code is correct, and we purposely introduce a mistake when computing the factorial of zero. Let’s ignore numerical overflow to keep our demonstration simple. The reader can access the source code from <a href="https://github.com/hungptit/catch2-benchmark-demo">this repo</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7e78a467427a47525be84fe960e888f6/href">https://medium.com/media/7e78a467427a47525be84fe960e888f6/href</a></iframe><p>How does the above code work?</p><ul><li><strong>CATCH_CONFIG_MAIN</strong> macro will generate the main function automatically.</li><li><strong>TEST_CASE</strong> will create a test case named <strong><em>Factorial</em></strong>.</li><li><strong>SECTION</strong> will create a new test section in the parent test case. We can also use nested <strong>SECTION</strong>s for more complex test scenarios.</li><li><strong>CHECK</strong> macro will check that <strong><em>Factorial(3)</em></strong> is equal to 6. <strong>Catch2</strong> will display an error using the to_string output of the test objects if the condition is false.</li><li><strong>REQUIRE</strong> is similar to <strong>CHECK.</strong> However, <strong>REQUIRE</strong> will terminate the test if the condition is false. This assertion is very handy if we want to stop our test early. For example, we will stop our test case if a database connection is invalid.</li></ul><p>That is all we need to do to write our first test case using <strong>Catch2</strong>.</p><h3>My data structure does not work with CHECK/REQUIRE assertions. What should I do?</h3><p>Catch2 only provides <strong>==</strong> and <strong>&lt;&lt;</strong> operators for standard data structures. We need to define <strong>==</strong> and <strong>&lt;&lt;</strong> operators for our classes before calling <strong>CHECK</strong> or <strong>REQUIRE</strong> assertions. Consider this simple struct:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/263581416a6b0e1a7ebf599891365181/href">https://medium.com/media/263581416a6b0e1a7ebf599891365181/href</a></iframe><p>If the <strong>==</strong> operator for the<strong> Person</strong> struct is not defined, then Catch2 assertions will fail with these confusing errors:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2abd199d122e360af3fa5b5c83ab110b/href">https://medium.com/media/2abd199d122e360af3fa5b5c83ab110b/href</a></iframe><p>We can compile our code successfully after adding the required operators, and all assertions give the expected output.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3756db57f42788e0c163ffcb9d493d58/href">https://medium.com/media/3756db57f42788e0c163ffcb9d493d58/href</a></iframe><p><em>Use </em><a href="https://en.cppreference.com/w/cpp/utility/tuple/tie"><em>std::tie</em></a><em> in your == operator to improve the </em><a href="https://accu.org/journals/overload/24/134/wakely_2271/"><em>readability and maintainability</em></a><em> of our code while offering similar performance to hand-rolled code.</em></p><h3>How do we write tests for exceptions?</h3><p>Catch2 provides several assertions for validating exceptions. The below examples cover all exception test scenarios.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e6e630dde1af09d089e4fd35762c5c70/href">https://medium.com/media/e6e630dde1af09d089e4fd35762c5c70/href</a></iframe><p>where:</p><ul><li><strong>CHECK_THROW</strong>: Check that a given expression will throw during the evaluation.</li><li><strong>CHECK_THROW_AS</strong>: Check that a given expression will throw with an expected expression type during the evaluation.</li><li><strong>CHECK_THROW_WITH</strong>: Check that a given expression will throw with an expected error message during the evaluation.</li><li><strong>CHECK_THROW_MATCH</strong>: Check that a given expression will throw with a specified exception type and the error message match our expectation. <strong>Use this assertion by default if possible</strong>.</li></ul><p>And this is the sample output</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b4b7b95624eba72199145877b97f2322/href">https://medium.com/media/b4b7b95624eba72199145877b97f2322/href</a></iframe><h3>Advanced Comparisons</h3><p>Catch2 provides <a href="https://github.com/catchorg/Catch2/blob/devel/docs/matchers.md">built-in matcher classes</a> for the standard data structures. Consider the default options before rolling your custom matcher class. Let’s consider complicated situations that require a customer matcher class; we need to verify that <strong>set A contains all values in set B</strong>.</p><h3>Step 1: Define our custom matcher</h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/21be838ee78f280cacd565dc4952fe5b/href">https://medium.com/media/21be838ee78f280cacd565dc4952fe5b/href</a></iframe><p>where:</p><ul><li><strong>match(): </strong>Compare two sets.</li><li><strong>describe():</strong> Create a readable output of the comparison. Will be used when the match method returns false.</li></ul><h3>Step 2: Update our test using the new matcher</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/697/1*vrQd6oquK7T4yyPBQT8xcw.png" /></figure><p>This is the error message when the <strong>CHECK_THAT</strong> assertion is failed</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c004ff0923bf7f71834ddd53cc3afd0/href">https://medium.com/media/3c004ff0923bf7f71834ddd53cc3afd0/href</a></iframe><p>The error message tells us the exact issue. In some test scenarios, the custom matcher classes will improve the readability and debuggability of our tests significantly.</p><h3>Behavior-driven development (BDD) with Catch2</h3><p><strong>Catch2</strong> supports the <a href="https://en.wikipedia.org/wiki/Behavior-driven_development">behavior-driven development</a> (BDD) style, which allows us to implement complicated test plans. Consider this example that tests several cases about the <a href="https://en.cppreference.com/w/cpp/container/vector">std::vector</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/07460dd62fb6538759982847d268c2cb/href">https://medium.com/media/07460dd62fb6538759982847d268c2cb/href</a></iframe><p>One<strong> Catch2</strong> advantage over other <strong>unit test frameworks</strong> is error reporting. We purposely fail one check, and the error messages give us a clear explanation about why and how our test failed.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/30304cfe85c5abfaccffc75a236f967b/href">https://medium.com/media/30304cfe85c5abfaccffc75a236f967b/href</a></iframe><p>We have illustrated that</p><ul><li>Tests written using Catch2’s BDD style are natural.</li><li>The error messages are descriptive, and it is easy to figure out why a test failed from reading the log.</li></ul><h3>How do we execute Catch2 tests?</h3><p>We can run all <strong>Catch2</strong> tests in a test file using this command:</p><pre>./unittests/algorithm_test</pre><p>Or run a specific test case using the test case filter:</p><pre>./unittests/algorithm_test &quot;Random*&quot;</pre><p>That’s all folks! Writing <strong>Catch2</strong> tests is simple and idiomatic. Let us know your thoughts in the comment.</p><figure><a href="https://www.salesforce.com/form/other/privacy-request/"><img alt="" src="https://cdn-images-1.medium.com/max/100/1*ImBy4Z_J1PCcv4T0kP9DJg.png" /></a><figcaption>Your Privacy Choices</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a06161f05b64" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tableau-engineering-blog/you-should-give-catch2-a-try-a06161f05b64">You should give Catch2 a try</a> was originally published in <a href="https://medium.com/tableau-engineering-blog">Tableau Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Massive Serverless Parallelization]]></title>
            <link>https://medium.com/tableau-engineering-blog/massive-serverless-parallelization-1e82b65fa4ed?source=rss----e066fecef8f2---4</link>
            <guid isPermaLink="false">https://medium.com/p/1e82b65fa4ed</guid>
            <category><![CDATA[dynamodb]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[parallelization]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[V Maharajh]]></dc:creator>
            <pubDate>Tue, 22 Feb 2022 16:31:58 GMT</pubDate>
            <atom:updated>2023-03-24T15:58:42.541Z</atom:updated>
            <content:encoded><![CDATA[<p>Who would have guessed that being able to quickly crunch data would be valuable to Tableau’s Prototyping team? Whether we need to comb through gigs of log files or test millions of permutations of columns in a data table, parallelization allows us to deliver prototypes that are responsive and compelling.</p><p>In <a href="https://medium.com/@annalee.barber/b51f8b0b8dae">Building Serverless Prototypes</a>, we introduced our team, and explained why we prefer serverless architectures:</p><blockquote>At the Tableau Prototyping team, our mission is to drive innovation through exploring and prototyping new ideas. Often these are most easily proven out through simple web apps that allow users to explore our ideas and allow us to verify our projects’ usefulness. We want to build fast and stay flexible — spending large amounts of time to set up a bespoke infrastructure for each of our new projects takes away from the time we have to work on core algorithms or project innovations.</blockquote><p>In this post, we’ll explore our recipe for implementing parallelized computations using AWS Lambda, S3, and DynamoDB. We’ll also discuss some approaches we tried along the way that did not work.</p><p>To develop and test our parallelization recipe, we came up with a “<a href="https://en.wikipedia.org/wiki/%22Hello,_World!%22_program">Hello, World!</a>” for parallel computing — a simple task that naturally lends itself to parallelization. Our task was to determine most frequent of words in a large text file for each length of word (1–20). We were able to accomplish this in 7 seconds.</p><p>We’ll dive into implementation details soon, but let’s start with the high level algorithm. We take a 1 gigabyte file <a href="http://mattmahoney.net/dc/textdata.html">from the Wikipedia dump</a> and segment it into many smaller blocks. Each block is processed in parallel using a standard <a href="https://en.wikipedia.org/wiki/MapReduce">MapReduce</a> pattern using AWS Lambda. Each Lambda reads the assigned block and populates 20 in-memory dictionaries (one for word length). These dictionaries are our word frequencies for this block, and are persisted in JSON files on AWS S3. We then start another MapReduce stage where each word length is processed in parallel, reading the word frequencies from S3 for all blocks for the specified word length and summing the frequency counts. The top 10 frequent words for the word length are saved on DynamoDB. Finally, the top 10 frequent words for all word lengths are read from DynamoDB and written to AWS CloudWatch.</p><p>Let’s take a closer look at the Lambdas that facilitate the above algorithm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u3DQ6AsOcptjgV8W_PPkag.png" /><figcaption>High-level overview of Lambda orchestration</figcaption></figure><p><strong>λ parseFile</strong></p><p>This is the entry point for our entire workload. It partitions the large text file into 5-megabyte blocks and invokes <em>λ parseBlock</em> in parallel for each of these blocks. We limited the maximum concurrency to 250. It waits for the completion of all of the parallel <em>λ parseBlock</em> invocations before finally invoking <em>λ getAllOccurrences</em> to continue our orchestration. Libraries like <a href="https://www.npmjs.com/package/p-map">p-map</a> make it easy to implement this MapReduce pattern:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c5985f6d65ed042d4e9661ddb94b9989/href">https://medium.com/media/c5985f6d65ed042d4e9661ddb94b9989/href</a></iframe><p><strong>λ parseBlock</strong></p><p>This lambda <a href="https://docs.aws.amazon.com/whitepapers/latest/s3-optimizing-performance-best-practices/use-byte-range-fetches.html">reads the text file from s3 for the specific block range</a> and keeps track of the frequencies of each word. The frequencies are bucketed by word length. This lambda has to deal with words that are cut off at the edges of the block: A split word at the start of the block is ignored, and a split word at the end of the block is read by over-fetching some bytes beyond our block’s specified range.</p><p>After reading the block, we need to save the word frequencies so that Lambdas further down in our orchestration can consume them. <a href="https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ServiceQuotas.html">DynamoDB has a 400KB limit on item sizes</a>, and these word frequencies were more blob-like in any case, so we store them in S3. In order to retrieve the object in the future, we use a prefix that is a hash of the length of the word and the block id.</p><p>Recall that we limited this phase to a concurrency of 250. This is because <a href="https://aws.amazon.com/premiumsupport/knowledge-center/s3-request-limit-avoid-throttling/">S3 can fail with 503 Slow Down errors if given a sudden burst of requests</a>. If we needed to improve our performance further, we could overcome this by gradually scaling up the concurrency or <a href="https://docs.aws.amazon.com/general/latest/gr/api-retries.html">by implementing retries with exponential backoff</a>, however we opted for the simpler concurrency limit since 7 seconds was sufficient for our needs.</p><p>We encapsulated this s3-backed blob storage with the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2956449799b7d1ad5aaadbb8ffb92b67/href">https://medium.com/media/2956449799b7d1ad5aaadbb8ffb92b67/href</a></iframe><p><strong>λ getAllOccurrences</strong></p><p>This simply kicks off the second MapReduce phase using λ getOccurrences — one for each word length (1–20). It reuses the mapReduce function described earlier.</p><p><strong>λ getOccurrences</strong></p><p>Recall that each λ parseBlock<strong> </strong>stored the word frequencies for each block via the BlobStore described earlier. In this lambda, we read all of those frequencies for all of the blobs of a specific word length and sum them. We also keep track of the 10 most frequent words. The final result is stored in DynamoDB.</p><p><strong>λ logResults</strong></p><p>The final lambda in our orchestration reads the results from DynamoDB and logs them using CloudWatch.</p><h3><strong>Things that didn’t quite work…</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rKMf6k056gcSkvrW.jpg" /><figcaption>Jacques Carelman, Catalogue d’objets introuvables, 1969</figcaption></figure><p>At this stage, we have a recipe that we are satisfied with; it is fast, easy to implement, and costs nothing when our prototypes aren’t in use. Along the way, we experimented with a couple other ideas. Here’s what didn’t quite work.</p><p><strong>Over-parallelization</strong></p><p>One of our first attempts stored the word frequencies directly in DynamoDB, using the words as keys. As we read words in the file, we’d update the count for that word in DynamoDB. This approach failed for two reasons. First, <a href="https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/WorkingWithItems.html#WorkingWithItems.AtomicCounters">atomic counters built on DynamoDB should only be used when minor overcounting or undercounting can be tolerated</a>. Secondly, we quickly exceeded DynamoDB’s allowed throughput. The table throughput is a soft limit and we were able to ask AWS to increase it for our account. However the partition limit is a hard limit, so we had to find another way.</p><p><strong>AWS Step Functions</strong></p><p><a href="https://aws.amazon.com/step-functions/">This service provides an orchestration framework for lambda</a>s. However it is quite slow — taking over 2 minutes to execute the lambdas above (compare this to the 7 seconds it takes when we handle the orchestration ourselves). The performance is likely due to the low maximum concurrency limit for the Map task and other <a href="https://docs.aws.amazon.com/step-functions/latest/dg/limits-overview.html">AWS Step Function limits</a>. If you’re determined to use Step Functions, <a href="https://stackoverflow.com/questions/65335230/working-around-aws-step-function-map-concurrency-limit">folks have found less-than-elegant workarounds for these issues</a>, however we could not justify proceeding down this path given how easy it is to do the orchestration ourselves.</p><h3>Conclusion</h3><p>We learned that we can indeed support highly parallelized workflows on our Serverless architecture. Identifying the right level at which to parallelize is key; too high and you’re not going to see perf improvements, but too low and the overhead of splitting and merging will exceed the gains you got from parallelization. We found that assembling our own lambdas worked well, and other architectures we tried did not work well.</p><figure><a href="https://www.salesforce.com/form/other/privacy-request/"><img alt="" src="https://cdn-images-1.medium.com/max/100/1*ImBy4Z_J1PCcv4T0kP9DJg.png" /></a><figcaption>Your Privacy Choices</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e82b65fa4ed" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tableau-engineering-blog/massive-serverless-parallelization-1e82b65fa4ed">Massive Serverless Parallelization</a> was originally published in <a href="https://medium.com/tableau-engineering-blog">Tableau Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Serverless Prototypes]]></title>
            <link>https://medium.com/tableau-engineering-blog/building-serverless-prototypes-b51f8b0b8dae?source=rss----e066fecef8f2---4</link>
            <guid isPermaLink="false">https://medium.com/p/b51f8b0b8dae</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[tableau]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[authentication]]></category>
            <dc:creator><![CDATA[Anna Lee Barber]]></dc:creator>
            <pubDate>Tue, 08 Feb 2022 22:56:13 GMT</pubDate>
            <atom:updated>2023-03-24T15:59:14.125Z</atom:updated>
            <content:encoded><![CDATA[<p>At the Tableau Prototyping team, our mission is to drive innovation through exploring and prototyping new ideas. Often these are most easily proven out through simple web apps that allow users to explore our ideas and allow us to verify our projects’ usefulness. We want to build fast and stay flexible — spending large amounts of time to set up a bespoke infrastructure for each of our new projects takes away from the time we have to work on core algorithms or project innovations.</p><p>When designing our prototyping systems, we needed to find an infrastructure solution that allowed us to adapt where necessary, but have a fairly consistent and maintainable setup. Crucially, we also did not want to sink much time into upkeep after a prototype is built. Ideally any prototyping project can stay “on the shelf” for anyone to find and work with after we’ve wrapped up the initial engineering. Our goal was to set up a repeatable infrastructure for prototypes that would meet these projects’ recurring needs: always on, low cost, and low maintenance.</p><p><strong>A Serverless Architecture</strong></p><p>The wide variety of potential projects and expected usage means running our own server-side code is inefficient. Amore traditional setup might make sense, if building a single app or new product, but is unwieldy for our purposes. Running the app in an EC2 instance means not running the server ourselves however it still requires the team to patch the instance OS. We would also need to carefully manage instance lifetimes or pay for always-on.</p><p>Instead, our requirements made using serverless computing an obvious choice. With these guidelines we settled on a simple <a href="https://aws.amazon.com/serverless/">AWS architecture</a> based on Lambdas using the <a href="https://www.serverless.com/">Serverless Framework</a>. The basic infrastructure with Lambdas, S3, and DynamoDB is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RAld6UJ1dCFZfLcWd62o6g.png" /><figcaption>The AWS architecture for our prototypes</figcaption></figure><p>In this setup, a user is verified through a lambda authorizer (connecting to a <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/what-is-amazon-cognito.html">Cognito</a> user pool in our example) and then makes calls through an API gateway. Any of our prototypes can have a custom set of functions that may read from and write data to a DynamoDB table and/or read from and write files to an S3 bucket. This infrastructure is simple and repeatable — the code the functions themselves run differs from prototype to prototype, but the fundamental structure can remain the same.</p><p>The Serverless Framework, in particular, is well-suited for our prototyping. It lets us set up our infrastructure as code, making it much easier to turn over the prototype to any interested consumer/team in the future, or for our team to pick it back up after significant time away from the project. Serverless also makes it simple to clean up resources automatically or remove old CloudFormation stacks, making our maintenance costs even lower. While these things can be accomplished with basic CloudFormation templates, as a team we found Serverless files less verbose and more readable. Serverless made writing and deploying infrastructure code much faster than it otherwise might have been. It is easy to have a repeatable <a href="https://www.serverless.com/framework/docs/providers/aws/guide/serverless.yml">serverless.yml</a> file for each project where just the names of resources are changed. These can all refer to one Cognito User Pool for authorization, while allowing customization in the actual function code or in adding multiple S3 buckets or DynamoDB tables if the project requires it. In addition, there are many useful plugins for development with Serverless, including <a href="https://github.com/dherault/serverless-offline">serverless-offline</a> to enable quick iterations on local development.</p><p>This setup hits our main goals:</p><ol><li><strong>Always On:</strong> AWS allows us to easily set up an always-on system, and Serverless provides many features that help simply start and maintain such a system.</li><li><strong>Low Maintenance:</strong> This infrastructure also gives us auto-scaling, allowing the team to be more hands-off on project upkeep, particularly with expected periods of low usage and then spikes as new users explore the prototype.</li><li><strong>Reasonable Cost: </strong>None of these features are storage-intensive, and AWS allows us to keep costs low for these systems that do not expect heavy usage in most cases.</li></ol><p><strong>Where this doesn’t work</strong></p><p>We’ve found that this simplified approach works very well for most of our prototypes. In the majority of cases, we just need a basic infrastructure set up and do not want to worry about fine-tuning the capabilities of that infrastructure. There are some drawbacks to the system that we’ve found.</p><ul><li><strong>Cold starts: </strong>The<strong> </strong>overhead in spinning up a lambda invocation on first request increases the response time, and is particularly noticeable with lambdas written in Java (to a lesser degree with JavaScript and Python). We’ve found that, while annoying, this is acceptable in almost all prototyping cases.</li><li><strong>Deploying ML models</strong>: In order to fully prove out ideas in this space we’ve found it necessary to maintain an always-on backend. AWS Lambda has <a href="https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html">limitations on deployment package size and memory</a> that are restrictive for larger models. The described set up does not work very well by default here, though the same general serverless architecture described above can be set up to communicate with a backend deployed in an EC2 container.</li><li><strong>Orchestrating parallelization: </strong>Making simple calls from an AWS Lambda out to S3 or Dynamo works without stress, but orchestrating a parallel response becomes quite challenging. Parallelized computations from a simple approach of invoking many functions and waiting for them to return varies from run to run in both overall timing and the aggregated results. Calling many functions at once to access the same AWS resources can quickly run up against AWS rate limits.</li></ul><p>Parallelization is particularly important to process massive amounts of data, a not-infrequent challenge at Tableau and for our prototypes. In our next post, we will discuss some alternative approaches to solve this issue.</p><figure><a href="https://www.salesforce.com/form/other/privacy-request/"><img alt="" src="https://cdn-images-1.medium.com/max/100/1*ImBy4Z_J1PCcv4T0kP9DJg.png" /></a><figcaption>Your Privacy Choices</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b51f8b0b8dae" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tableau-engineering-blog/building-serverless-prototypes-b51f8b0b8dae">Building Serverless Prototypes</a> was originally published in <a href="https://medium.com/tableau-engineering-blog">Tableau Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>