<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Spencer Taylor on Medium]]></title>
        <description><![CDATA[Stories by Spencer Taylor on Medium]]></description>
        <link>https://medium.com/@spencer_11278?source=rss-5b4db702bf63------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*jJxFoCp5s_0LeYX0.jpg</url>
            <title>Stories by Spencer Taylor on Medium</title>
            <link>https://medium.com/@spencer_11278?source=rss-5b4db702bf63------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 17:12:29 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@spencer_11278/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building a Data Product Organization, Part I]]></title>
            <link>https://medium.com/4mile-analytics/building-a-data-product-organization-part-i-ff4a8c593094?source=rss-5b4db702bf63------2</link>
            <guid isPermaLink="false">https://medium.com/p/ff4a8c593094</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-team]]></category>
            <category><![CDATA[product-design]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[modern-data-stack]]></category>
            <dc:creator><![CDATA[Spencer Taylor]]></dc:creator>
            <pubDate>Wed, 28 Sep 2022 17:24:12 GMT</pubDate>
            <atom:updated>2022-09-28T17:24:12.399Z</atom:updated>
            <content:encoded><![CDATA[<p>As a data consultant, I’ve had the incredible opportunity to understand the mechanics of a wide variety of data teams at different companies. I’ve seen very successful examples, and also organizational patterns that lead to mediocrity, failure, and in some cases, pure chaos.</p><p>In this series, I’ll discuss a theme that I have seen lead to success when adopted by organizations of various sizes and industries. My goal is to convince readers to consider moving away from a traditional data team setup, in which the data team operates as a services organization, and to adopt a new mindset that leads to a data team functioning as a product organization.</p><p><strong>Background</strong>:</p><p>Many of you reading this may be part of a data team that is operating in a model commonly referred to as a “data services organization”. This describes a team who has an intake process for data requests, most commonly for dashboards or specific data questions (e.g. “What was our average weekly ad spend by channel over the past 6 months”). The team sifts through the data requests, prioritizes them, and works through tickets in a sprint-based fashion. If you’re reading this and thinking, “Huh, that sounds like my company”, you’re definitely not alone. This is the way that many companies’ data teams operate. And it works, for a while…</p><p><strong>Why move away from this model:</strong></p><p>There are four cascading consequences that are a direct result of this having a data services organization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j4VDSnhnX4rkaw5_" /></figure><ol><li>Lack of scalability. Meme culture in the tech world has smashed that word into the ground, but it’s a real consequence of not adapting to a better model for your data team. When the business scales, so does the demand on the data team. This seems obvious — as the business grows and generates more data, there will be an increased demand in the volume and complexity of your data requests — but what’s not so obvious is how to enable your team to handle this increased demand. All too often I see organizations throw more money and people at this problem in hopes that the data team can at least scale linearly with the business, leading to the data team being a slow moving cost center.</li><li>Inherent lack of innovation. This type of data team is fulfilling requests from (in many cases) non-technical people, and making sure that the solutions they deliver are meeting the requirements to spec. This does not drive creative solutioning in the way that experienced data professionals — yes, I’m speaking for us all here — like to be challenged. This confinement quickly leads to a decrease in morale and stagnation in career development opportunities for your team. At risk of generalizing, most data analysts started their career because they realized at some point in their prior career that they loved playing with data and solving complex problems, and made the switch. Think about it — very few of us graduated from college with a degree in SQL and just continued on to their career in data. The point is, we all love diving into the data with our heads down and figuring out new patterns and sharing them with others. And if we’re focusing on building new dashboards and spending 2 days trying to figure out how to make our BI tool format a date in the right way, we lose out on that.</li><li>The detrimental effect on the rest of the business. Consider the folks who are submitting data requests… Perhaps a finance stakeholder needs to understand forecasted revenue for the next 3 months from the brick and mortar business in advance of a budgetary meeting. Without a quick answer to this question, they’re unable to make a valuable recommendation in the next meeting. Or perhaps a supply chain manager needs to know the typical demand of a certain product in November to begin making purchase orders. When the people that run the business rely on the data team to supply them with information to make key decisions, and with a backlog inhibiting the data team to meet that demand, the entire business suffers.</li><li>All of this leads to cost. Without changing your operating model, the increased demand on the data team is going to require more and more staff to fulfill. With a decrease in morale comes an increase in turnover — the ability to quickly onboard new team members and ensure continuity of knowledge transfer is going to be diminished. Of course, there are many ways to address issues with morale and turnover — an easy way is to just pay your team more money. But in lieu of changing your operating model, you’re putting a bandaid on a gaping wound and it’s going to cost the business more money than it would cost to address the root of the issue.</li></ol><p><strong>The solution: </strong>Productize your data org.</p><p>What does it mean to productize your data org? In essence, it means building new features and tools to allow your business to answer their own questions, instead of asking the data team to answer questions. It also means developing empathy for your stakeholders and respecting their expertise in their business domains.</p><p>Productizing your data team can also be hard. It requires dedication and buy-in from leadership, and it requires making some short term investments and concessions in favor of long term payoffs. That said, the earlier you make the shift in mindset the quicker you will realize the benefits. There are many strategies you can consider to help transform your team away from a service model.</p><p>To start discussing the “How” of productizing your data team, stay tuned for the next part of this conversation, lead by Hannah Galloway, our Director of Product Design, and Andrea Ramirez, our Director of Product Management.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ff4a8c593094" width="1" height="1" alt=""><hr><p><a href="https://medium.com/4mile-analytics/building-a-data-product-organization-part-i-ff4a8c593094">Building a Data Product Organization, Part I</a> was originally published in <a href="https://medium.com/4mile-analytics">4 Mile Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hub and Spoke Implementation in Looker]]></title>
            <link>https://medium.com/4mile-analytics/hub-and-spoke-implementation-in-looker-4f43cbb4f2c?source=rss-5b4db702bf63------2</link>
            <guid isPermaLink="false">https://medium.com/p/4f43cbb4f2c</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[looker]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[hub-and-spoke]]></category>
            <category><![CDATA[enterprise]]></category>
            <dc:creator><![CDATA[Spencer Taylor]]></dc:creator>
            <pubDate>Fri, 24 Apr 2020 20:36:43 GMT</pubDate>
            <atom:updated>2020-04-27T16:11:57.625Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/561/1*dN_h3M1NEt6gAioE9chnsQ.gif" /></figure><p>In a large organization with many segmented departments, data governance becomes a difficult task to handle. Looker is a great tool to help with data governance because it offers a rigid data model, but many teams need some flexibility with their data model and require the ability to redefine or add to certain parts of the data model. Enter Hub and Spoke.</p><p>Hub and Spoke architectures have been prevalent in many industries for quite some time, and the phrase hub and spoke can be interpreted differently depending on the type of technology used and the type of business using it. However, the basic principle remains the same. The hub controls a central piece of information, which is propagated out to the different spokes. The spokes can use that information and augment it any way they want for their own purpose, but cannot push changes to that information upstream back to the hub, nor laterally to other spokes.</p><p>In Looker, we can use a feature called project_import to implement a hub and spoke architecture between different Looker projects. This enables one team to define a central, immutable repository of code that gets propagated to the other development teams. Those teams in turn can extend the files created in the hub project, changing or adding to them for their own use.</p><p>Setting up the Hub and Spoke architecture is not straightforward. My goal is to help demystify the hub and spoke strategy and help you determine if this is the right approach for your instance.</p><h3>General Setup</h3><h4>Step 1: Create your hub project</h4><p>This project should impose the most stringent level of access control that Looker offers, as we only want a small group of developers to be changing this code. Therefore it may come as a bit of a surprise that all Looker developers will need to have access to this project. This is because they will need to pull changes down from the hub when they are in development mode. Since development mode is an instance-wide toggle in Looker, they won’t be able to be in dev mode in one project but not in another.</p><p>So, how do we lock down the hub project? We’ll need to enforce the restriction in github (or other git provider). While all Looker developers may have dev access to the hub project, only a select few who have access to the git repo will be able to open a pull request in github and actually merge their code with production.</p><p>In Looker, assuming you have already set up your git integration, all you need to do is navigate to your project settings in the hub and enable the “Require merge requests” option. With this setting enabled, all Looker developers will need to navigate to github in order to merge their changes with the production branch. Only users with credentials to the repo in which the hub project lives will be able to do so.</p><h4>Step 2: Build your hub project</h4><p>The hub project is organized differently than normal Looker projects. Since Looker does not allow model files to be extended into other projects, you won’t need a model file with a ton of explores defined. Instead, you’ll build your explores in another file (or files), which will be included in the model files within each spoke project.</p><p>But before we touch on explores, let’s get to the most important part of your hub project — view files. These are your files that the spoke projects will inherit, which contain governed and standardized logic. Some tips here:</p><ul><li>Use the IDE Folders feature in Looker, especially if you have multiple connections throughout your spoke projects. Name each folder according to the database dialect used within each view file.</li><li>Stick to a common format throughout each view file. Be aware that many developers will be referencing your views and likely using them as templates — use good coding practices as a baseline for everyone else.</li><li>Be as descriptive as possible in your code. Add descriptions to measures, and make sure that your naming conventions make sense to a broad audience.</li><li>Set up a formal review process to ensure that all dimensions and measures use logic that should be inherited everywhere. A measure that only one department uses should be defined in that team’s spoke project, not in the hub.</li></ul><p>If you want to surface a series of explores to your spoke projects, you’ll need to create one or more explore files. To create an explore file you can click the + button in your LookML IDE and select “Create generic LookML file”. Or, if you’re on an earlier version of Looker, you can simply create a blank view file and name it with the appropriate suffixes (file_name.explore.lkml). The suffixes will cause Looker to categorize this as a different type of file. Here is the official Looker <a href="https://docs.looker.com/data-modeling/getting-started/other-project-files#explore_files">documentation</a> on explore files</p><p>Some tips about the explore file(s):</p><ul><li>I would recommend calling it hub_explores.explore.lkml. Or, if there will be multiple connections defined in the across the spoke projects, you may need multiple explore files to account for different SQL dialects and database objects. I would recommend prefixing each file with the name of the connection or database dialect (e.g. bigquery_hub_explores.explore.lkml).</li><li>Be sparing in the number of explores you create. There is not much need for creating single-view explores. Only define explores if there is logic in them that should be inherited everywhere. Otherwise it is better to let the spoke projects create the explores they need, with their own descriptions, labels, and group labels.</li><li>Keep in mind that when spoke projects include an explore in their model file, it will show up in the explore dropdown once for each project that includes it for admins and developers with access to all projects.</li><li>If two spokes both include the same explore in their models, the same explore will appear twice in the dropdown. This is why it is best to allow the spokes to create their own explores, allowing for differentiation.</li></ul><h4>Step 3: Setup the spokes</h4><p>Each spoke project will need to inherit the governed code from the hub. To do this, we’ll take advantage of Looker’s <a href="https://docs.looker.com/data-modeling/learning-lookml/importing-projects">project import</a> feature. This feature is essential to the hub and spoke architecture. To do this, you’ll need to create a project manifest file in each spoke project. This file only requires a few lines of crucial code. You’ll need to specify the project name in wish the file lives (the name of the spoke project), as well as the name of the project serving as the hub.</p><p>Here’s an example of a manifest file:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kBaURuyy7oQTJUn1" /></figure><p>Each spoke will need a model file. The model file in each spoke should include the explore file(s) relevant to their business area, as well as view files from the hub that they need to build those and other explores. The syntax for including files from the hub is documented in the same <a href="https://docs.looker.com/data-modeling/learning-lookml/importing-projects">link</a> above.</p><p>Once those files are included, the developers in each spoke will be able to <a href="https://docs.looker.com/data-modeling/learning-lookml/extends">extend</a> code from the hub and augment/redefine it as they see fit.</p><h4>Step 4: Enable the spokes</h4><p>Training is a big component of ensuring the hub and spoke method will be able to scale as you roll out the platform to more teams. LookML developers will need to understand how to extend the code from the hub into their own projects and use it without cluttering up the Looker instance. Here are some tips for keeping the architecture functioning with few hiccups:</p><ul><li>Document everything, and record a short video with simple steps for how to <a href="https://docs.looker.com/data-modeling/learning-lookml/extends">extend</a> a file.</li><li>Remember to include all necessary files in whatever file you’re working on (a view from the hub needs to be included in the view file that is extending that view)</li><li>Emphasize that spoke developers will need to regularly pull code down from the hub in order to stay up to date. They might not realize that since their project depends on the hub, they need to pull down the most recent version of the hub onto their dev branch while in development mode.</li></ul><h4>Step 5: Untangle your access controls</h4><p>While it may seem complex, managing access to content and data is not terribly difficult in a hub and spoke architecture once you establish the building blocks. By using groups and roles, you can manage granular permissions and folder access with relatively low maintenance. Here’s how:</p><ul><li>Create the folder structure you need to facilitate your spokes. I would recommend one main subfolder for each spoke, with more subfolders within each spoke if necessary.</li><li>Create groups that have access to each folder. One main group per spoke should do the trick. Assign each group to its corresponding folder within the shared folders.</li><li>For each spoke, create roles that partition developers from dashboard builders from viewers. This will also require you to create a model_set for each spoke, and a permission_set for each type of user. Remember to always include the hub model in your model sets, as all developers will need access to the hub project. You can assign the same model set to your other roles within each spoke.</li></ul><h4>Here are some examples of what your permission and model sets might look like:</h4><p><strong><em>permission_set: developer</em></strong><em> -Contains developer permissions</em></p><p><strong><em>permission_set: business_user</em></strong><em> -Contains business user permissions (can explore, create dashboards, etc.)</em></p><p><strong><em>permission_set: viewer</em></strong><em> -Can only view dashboards, cannot explore</em></p><p><strong><em>model_set: finance</em></strong><em> -Contains the hub model, along with any models in the finance spoke project</em></p><p><strong><em>model_set: marketing </em></strong><em>-Contains the hub model, along with any models in the marketing spoke project</em></p><h4>Here’s an example of what the subsequent roles might look like:</h4><p><strong>Role = finance_developer</strong></p><ul><li>model_set = finance_model</li><li>permission_set = developer</li></ul><p><strong>Role = marketing_developer</strong></p><ul><li>model_set = marketing_model</li><li>permission_set = developer</li></ul><p><strong>Role = finance_viewer</strong></p><ul><li>model_set = finance_model</li><li>permission_set = viewer</li></ul><p><strong>Role = marketing_business_user</strong></p><ul><li>model_set = marketing_model</li><li>permission_set = business_user</li></ul><p>(Optional) Assign your roles to subgroups within each spoke. This allows a user that is added to a group to inherit a role associated with that group. Doing so will reduce the number of actions needed when a new user is created. If you create a group called “finance_developer” you can attach the finance developer role to that group.</p><p>You can add those subgroups to your main group for each spoke. For example, any user added to the finance_developer group will automatically gain access to the Finance folder, along with the model_set and permission_set defined in the role that is attached to that group. You’ve made it so that every new user only requires one group assignment to determine what they can access.</p><p>An additional note about access controls regarding multi-spoke developers -since development mode is instance-wide, a developer will not be able to be in dev mode in one project, and production in another. Therefore, developers will need dev access to spokes they have view access to, otherwise they will not be able to view data from those spokes while in dev mode (toggling out of dev mode fixes the issue for them, but is not a great experience). If you need spoke developers to be able to view content across many projects, you can implement git-level development access for each spoke, and give all developers the Looker role that allows them to develop in each spoke.</p><h3>Step 6: Leverage the API</h3><p>When managing a large scale Looker implementation, it becomes essential to familiarize yourself with the API and feel comfortable writing and executing simple scripts that make requests to the API.</p><p>Some examples of actions you’ll want to take regularly are:</p><ul><li><strong>Cleanup</strong>: Move old dashboards to either the trash or a designated folder for old content. You can create a report based off of the system_activity model that returns the dashboard_id for any dashboard that hasn’t been viewed in 6 months</li><li><strong>Add users in bulk</strong>: You may need to onboard a new spoke all at once, and if there are many members of that spoke, it will become much easier to leverage a script that adds those users for you than to manually add them via the UI.</li><li><strong>Disable users</strong>: As you continue to roll out the Looker platform, you’ll inevitably have users drop off (either by leaving the company or changing roles). It will pay off to have a script built that can disable users for you, either from a simple file of user_ids generated from the system_activity model, or by your own logic defined in your script.</li></ul><h3>Other notes/tips:</h3><ul><li>You may have groups of users that need access to multiple spokes. You can easily create a role that includes multiple model_sets and assign it to a new group. Or, you can create a model_set that include some or all models and assign it to a role/group.</li><li>Looker offers SAML integration. I would recommend exploring authentication options to make granting initial access to users easier.</li><li>Establish a process by which spoke developers can submit requests for getting a code change into the hub. It may be the case that a developer in a spoke project comes up with a better definition of a metric, or a more efficient way to write a derived table, and they should be able to voice their findings.</li><li>You can create a generic project that all users have access to that will allow you to use the code from the hub and surface it via a set of explores. This project will function in the same way as a spoke — changes made to it will not be reflected in the hub — but it will allow the hub explores to surface without any spoke modifications.</li><li>Feel free to enable the same git access control as the hub to prevent any other developers making changes to the code in that project.</li></ul><p>Hopefully this helps you make the decision about whether or not to go down the road of implementing a hub and spoke architecture. I have personally found it very scalable and flexible, but it requires a diligent setup and constant maintenance going forward.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4f43cbb4f2c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/4mile-analytics/hub-and-spoke-implementation-in-looker-4f43cbb4f2c">Hub and Spoke Implementation in Looker</a> was originally published in <a href="https://medium.com/4mile-analytics">4 Mile Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>