<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Stephen Day on Medium]]></title>
        <description><![CDATA[Stories by Stephen Day on Medium]]></description>
        <link>https://medium.com/@stevvooe?source=rss-d21a17509c63------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*DGA-Q7VAPGuaqW92.jpg</url>
            <title>Stories by Stephen Day on Medium</title>
            <link>https://medium.com/@stevvooe?source=rss-d21a17509c63------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 21:09:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@stevvooe/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Managing Kubernetes RBAC Groups]]></title>
            <link>https://medium.com/cruise/open-sourcing-rbacsync-48758df685b0?source=rss-d21a17509c63------2</link>
            <guid isPermaLink="false">https://medium.com/p/48758df685b0</guid>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[rbac]]></category>
            <dc:creator><![CDATA[Stephen Day]]></dc:creator>
            <pubDate>Thu, 21 Mar 2019 22:01:02 GMT</pubDate>
            <atom:updated>2019-03-21T22:19:34.038Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/390/1*C9kchyqGqVIcoiz7ji1f1Q.png" /><figcaption><a href="https://github.com/cruise-automation/rbacsync">github.com/cruise-automation/rbacsync</a></figcaption></figure><p>At Cruise, we use <a href="https://kubernetes.io/">Kubernetes</a> for many workloads across the organization. In the past, we had several clusters across different teams, deployed and managed in a variety of ways. While this worked early on, it quickly became an operational burden. To address this, we introduced a Kubernetes-based Platform as a Service (PaaS) . This effort has allowed us to consolidate workloads into just a few clusters, deployed in a common way, to allow teams to focus on building and shipping their software rather than operational concerns. Teams previously required a full time Site Reliability Engineer to setup and manage a cluster. Now they only need to get access to the PaaS cluster and can get started running their workload. This frees up our SREs to solve other problems and allows us to have more engineering resources focusing on our core problems.</p><p>When consolidating our users onto a single cluster, we recognized that controlling access to specific resources would become a problem. In the very least, we wanted to ensure that users on different teams couldn’t tamper with each other’s software and with services provided as part of the PaaS. Our plan was to use <a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/">Kubernetes RBAC</a> to manage this. It allows cluster operators to define <em>roles</em> for operations in Kubernetes. These <em>roles</em> can then be bound to specific resources, using <em>role binding, </em>to name one or more <em>subjects</em> that get those <em>roles</em>. These subjects can be <em>users, groups</em> or <em>service accounts</em>.</p><p>Because we’re using Google Kubernetes Engine (GKE), we have a strong login identity. This makes it easy to identify who is using the cluster, but not the groups to which a user belongs. This meant that for each user we added to the cluster, we needed to create <em>role bindings</em> to link them to a group. While you can add several users to a single role binding, keeping this updated became difficult. Not only did we want to assign specific roles to large numbers of engineers, we also wanted to define groups make it even easier.</p><h3>What did we do?</h3><p>With an understanding of the problem above, we put together a configuration example that would allow us to map <em>roles</em> to <em>groups</em>. Speaking in Kubernetes YAML, we were hoping for something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_1R7vYtQblvhQeeI" /></figure><p>The above simply says “bind cluster role ‘cluster-admin’ to the group ‘mygroup-admin@getcruise.com’”. If that group has the members “a@getcruise.com”, “b@getcruise.com” and “c@getcruise.com”, and it is honored by Kubernetes authorization system, it is equivalent to the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tLAezVPKPVVfx9ed" /></figure><p>Unfortunately, GKE does not make the group memberships of users available to the Kubernetes authorization system, and we couldn’t replace the component or change it to make it work as we needed. Because translating “groups” to a list of “subjects” is relatively straightforward, we wrote <a href="https://github.com/cruise-automation/rbacsync">RBACSync</a> to automate the process.</p><h3>How does it work?</h3><p><a href="https://github.com/cruise-automation/rbacsync">RBACSync</a> is what is called a <em>Kubernetes controller</em>. A <em>controller</em> watches the Kubernetes API for changes and takes action on those changes. In the case of RBACSync, it watches a configuration object, defined by a <a href="https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/">Custom Resource Definition</a>, that states a <em>group</em> and <em>role reference</em>. When the controller sees a new configuration or gets an update on an existing configuration, it looks up the group in an <em>upstream</em> and creates a <em>role binding</em> with all of the members of the group.</p><p>The above is a mouthful, so hopefully an example will help to illustrate it more clearly. The following is a minimal ClusterRBACSyncConfig that binds the <em>group</em> “mygroup-admin@getcruise.com” to the <em>role reference</em> “cluster-admin”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*h36xPw6FOe3qBbfC" /></figure><p>When the RBACSync controller see this configuration, it calls into the Google Groups API, looks up all the members of the group, then synthesizes the appropriate <em>role binding. </em>Let’s say that “mygroup-admin@getcruise.com” has the members “a@getcruise.com”, “b@getcruise.com” and “c@getcruise.com”. The following would be created, binding the members to the role “cluster-admin”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*g1t8pzj5J2ZfsAK1" /></figure><p>If you’re squinting at this YAML and asking yourself what happened, notice that the “subjects” field has all the members that we had called out above. Also notice that this structure that was automatically created matches the “equivalent” structure from the example in the section above. While this seems simple, this configuration makes Kubernetes RBAC significantly more powerful. Instead of deploying a large number of <em>role bindings</em>, we now just deploy a few RBACSyncConfig objects and let the controller do the rest. If the members of the group change, RBACSync will poll the upstream API and update the members on that group.</p><p>Hopefully, that provides enough of an overview to get the idea. If you want to understand it in more depth or deploy it in your cluster, please check out the <a href="https://github.com/cruise-automation/rbacsync">README</a>.</p><h3>In Production</h3><p>We’ve been running RBACSync in production for about four months and have discovered a few things in the process.</p><p>The first problem we ran into was that Google Cloud Platform (GCP) service accounts could not be added to Google Groups, causing us to split these out into separate <em>role bindings</em>. We fixed this by allowing one to augment groups in the configuration with extra members, explicitly called out. This feature also allows you to simply define the group directly in the RBACSyncConfig, such that the controller will automatically create role bindings for each reference.</p><p>Deploying RBACSync into our environment made it easier to map users to roles, but we quickly found that the existing roles were insufficient and had to create new roles to meet our requirements. After we deployed the new roles, we had to be responsive to user feedback and tune them to the right level of access. Whether or not you’re planning to use RBACSync, make sure to leave time when implementing RBAC to come up with a solid set of roles that implement your policy.</p><p>Another issue that we ran into was Google Group API availability. If the upstream for a group becomes unavailable, it currently removes all of the members from the <em>role binding</em>. To defend against a complete loss of access, critical group members can have cluster administration bindings through static configuration, ensuring that a group of users can always have access. We have found that this isn’t quite enough and would like to introduce group membership caching into RBACSync to prevent loss of access on loss of connectivity.</p><p>To avoid privilege escalation, we separated cluster configurations and namespace configurations, using ClusterRBACSyncConfig and RBACSyncConfig, allowing each to <em>only </em>reference ClusterRole and Role respectively. Don’t worry if you didn’t understand that on the first read through. If you’re curious, the details are <a href="https://github.com/cruise-automation/rbacsync#custom-resources">here</a>. The problem with this approach is that it requires one to deploy the same roles to every namespace. Since then, we’ve found that having common cluster roles that bind to a namespace is sufficient and does not risk privilege escalation as long as access to the configuration is also limited. We’ll relax this in the future.</p><h3>Conclusion</h3><p>In the future, we intend to continue hardening RBACSync based on our experience with it in production. While we don’t need it at this time, we’d also like to see support for other group up streams, such as Okta or Active Directory, as well as verifying it can work on Kubernetes clusters that aren’t on GKE.</p><p>The biggest thing we learned with RBACSync is that creating controllers is a reasonable path to make Kubernetes work in our organization. There are many steps required to make Kubernetes production ready, but the flexible API and the ability to extend it with new types and behavior means that we were able to deploy sooner and didn’t have to wait for upstream features.</p><p>If this project interests you and you want to work on our Kubernetes-based platform to support self-driving car development, <a href="https://getcruise.com/careers">we’re currently hiring in SF &amp; Seattle</a>!</p><p>GitHub: <a href="https://github.com/cruise-automation/rbacsync">https://github.com/cruise-automation/rbacsync</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=48758df685b0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cruise/open-sourcing-rbacsync-48758df685b0">Managing Kubernetes RBAC Groups</a> was originally published in <a href="https://medium.com/cruise">Cruise</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Docker Registry API to be standardized in OCI]]></title>
            <link>https://medium.com/moby-blog/docker-registry-api-to-be-standardized-in-oci-e36c0af08f3d?source=rss-d21a17509c63------2</link>
            <guid isPermaLink="false">https://medium.com/p/e36c0af08f3d</guid>
            <category><![CDATA[cncf]]></category>
            <category><![CDATA[oci]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Stephen Day]]></dc:creator>
            <pubDate>Wed, 11 Apr 2018 00:13:13 GMT</pubDate>
            <atom:updated>2018-04-11T00:13:13.227Z</atom:updated>
            <content:encoded><![CDATA[<p>We are excited to announce that the Docker Registry HTTP API V2 specification will be adopted in the Open Container Initiative (OCI), the organization under the Linux Foundation that provides the standards that fuel the containerization industry. The Docker team is proud to see another aspect of our technology stack become a de-facto standard. As we’ve done with our image format, we are happy to formally share and collaborate with the container ecosystem as part of the OCI community. Our distribution protocol is the underpinning of all container registries on the market and is so robust that it is leveraged over a billion times every two weeks as container content is distributed across the globe.</p><h3>What does this protocol do?</h3><p>Putting the protocol into perspective, part of the core functionality of Docker is the ability to push and pull images. From the first “Hello, World” moment, this concept is introduced to every user and is a large part of the Docker experience. While we normally sit back in our armchairs and marvel at this magical occurence, the amount of design and consideration that has gone into that simple capability can easily be overlooked.</p><p>When Docker was first released, the team quickly put together a protocol and implementation of the <em>Image Registry</em> and the magic truly began. An <em>Image Registry</em> provides a common service to store images across machines. It is what allows one to build an image on one machine, then pull down that same image and run it on others. One now had the power to pull down an entire software distribution and run it at the drop of a finger tip. This implementation powers the Docker Hub and eventually was open sourced as <a href="https://github.com/docker/docker-registry">https://github.com/docker/docker-registry</a>. This protocol and the implementation behind it eventually became known as the V1 protocol. Many an image was pushed and pulled and the developers rejoiced.</p><h3>Evolution</h3><p>Pushing and pulling images continued throughout the ages but, as users began to use Docker with other registries, issues with the V1 protocol arised. The central theme of the problems was around the concept of shared identity across registries and the tight coupling with the Docker implementation. The problem was that if a single <em>Docker Engine</em> pulls images from two separate registries, they may disagree on which image has which identifier. Something needed to change to ensure using multiple registries wouldn’t lead to problems for users.</p><p>Towards the end of 2014, Docker began addressing these problems with the introduction of <a href="https://github.com/moby/moby/issues/9015">a proposal</a> with the initial API structure. Key to this design was <a href="https://en.wikipedia.org/wiki/Content-addressable_storage">content addressable</a> images, which allowed registries to have common identifiers for images, and the decoupling of internal details of the image format from the Docker Engine, allowing it to evolve on its own. The community came together and produced 140 comments on that proposal that were incorporated into the specification and implementation. The result of this effort was the release of Docker Registry 2.0 with GA support in <a href="https://blog.docker.com/2015/04/docker-release-1-6/">Docker 1.6</a> in the spring of 2015. Since then, the Docker community has evolved to meet the growing needs of users.</p><h3>Going Forward</h3><p>As a result of the popularity of Docker, this protocol has become widely adopted across the industry. It is battle tested in a wide variety of environments. The protocol integrates well with complementary technologies such as signing and verification, as is available in <a href="https://docs.docker.com/datacenter/dtr/2.4/guides/">Docker Enterprise Edition</a>. By <a href="https://github.com/opencontainers/distribution-spec/pull/5">donating this specification</a> to the OCI, we can ensure that this important part of the container experience becomes an official OCI standard. The <a href="https://www.opencontainers.org/"><em>Open Container Initiative</em></a> previously introduced the <a href="https://github.com/opencontainers/runtime-spec/">runtime-spec</a> and <a href="https://github.com/opencontainers/image-spec">image-spec</a> used by container runtimes. With the acceptance of the <a href="https://github.com/opencontainers/distribution-spec/">distribution-spec</a> proposal, the protocol that has been a key part of using containers will flourish as part of OCI.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e36c0af08f3d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/moby-blog/docker-registry-api-to-be-standardized-in-oci-e36c0af08f3d">Docker Registry API to be standardized in OCI</a> was originally published in <a href="https://medium.com/moby-blog">Moby Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>