<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Philipp Belitz on Medium]]></title>
        <description><![CDATA[Stories by Philipp Belitz on Medium]]></description>
        <link>https://medium.com/@philipp.belitz?source=rss-131fe98392d6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*gELqppXwQuZT5GuLUrH1Qw.jpeg</url>
            <title>Stories by Philipp Belitz on Medium</title>
            <link>https://medium.com/@philipp.belitz?source=rss-131fe98392d6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 09:31:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@philipp.belitz/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Connaisseur 3.4.0: Minor Release, Major Change]]></title>
            <link>https://medium.com/sse-blog/connaisseur-3-4-0-minor-release-major-change-aae7b52ad8e9?source=rss-131fe98392d6------2</link>
            <guid isPermaLink="false">https://medium.com/p/aae7b52ad8e9</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[supply-chain-security]]></category>
            <category><![CDATA[kubernetes-security]]></category>
            <category><![CDATA[releases]]></category>
            <category><![CDATA[container-security]]></category>
            <dc:creator><![CDATA[Philipp Belitz]]></dc:creator>
            <pubDate>Wed, 27 Mar 2024 10:17:47 GMT</pubDate>
            <atom:updated>2024-03-27T11:06:38.257Z</atom:updated>
            <content:encoded><![CDATA[<h4>From Python to Golang: New Connaisseur Release</h4><h4>From Python to Golang: Learn how and why this change improved security, performance and compatibility in Connaisseur’s latest release.</h4><p><a href="https://github.com/sse-secure-systems/connaisseur">Connaisseur</a> released version <a href="https://github.com/sse-secure-systems/connaisseur/releases/tag/v3.4.0">3.4.0</a> and even though this is a minor release, there has been a major change: the programming language switched from Python to Go. This resulted in a multitude of improvements:</p><ul><li>Better security thanks to the use of FROM SCRATCH base image, getting rid of many dependencies</li><li>Native integration of validation frameworks like Cosign, Notary and future ones, yet to come</li><li>Smaller image size due to the removal of the Python interpreter and Cosign binary</li><li>Increased performance with new Redis-based caching mechanism</li><li>More stability through more robust goroutines</li></ul><p>To better understand why we made this change, let me embark on a little story.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*teZV-pzG8XnZqE3SqSmIPQ.png" /><figcaption>Created with DALL-E 3</figcaption></figure><h3><strong>Tough decisions to make</strong></h3><p>In software development it is not uncommon to start out simple — you have a specific problem and create a small proof-of-concept for solving it, trying to validate your initial ideas and then continue forward from there. Some ideas get fleshed out and new features are added over time, continuously building on the current state and organically growing the whole project into a full-fledged piece of software. Eventually, you might encounter problems that weren’t part of your initial assessment and whose solutions are difficult to incorporate into your existing code base, at which point you have a decision to make: find a quick but maybe hacky way of solving it, or rethink and rewrite most of your previous solutions so everything fits well together (or there may be something in between).</p><p>The right decision here is not obvious at all and is dependent on many factors, mostly on the time and resources that are available, compared to the benefits gained from investing them. So even if a “clean“ rewrite of your solution may feel like the right decision, it is not worth anything should you never have the time to finish it. In contrast, always choosing the quick way will bloat your code with technical debt that becomes increasingly hard to manage.</p><p>Those are the kinds of decisions we also had to make while developing the <a href="https://github.com/sse-secure-systems/connaisseur">Connaisseur</a> project, the Kubernetes admission controller for verifying container image signatures. While doing so, at times we did go for the quick and hacky way, while at others we re-evaluated our solutions and made major rewrites of our code base. This time, we took several steps back and re-evaluated one of the very first decisions we made back when the project started: which programming language should we use?</p><h3>From Python to Go in 5 seconds (or more)</h3><p>Back then, Python was the obvious choice as a programming language since everyone within the team knew it, but over time it became clear that we had to make compromises because of that choice.</p><p>Connaisseur was created with <a href="https://github.com/notaryproject/notary">Notary</a> as the only validation framework in mind, until <a href="https://github.com/sigstore/cosign">Cosign</a> entered the scene. Our integration of Cosign works by including its binary inside our container, since we didn’t see value in writing our own Cosign Python library. A choice that led to quick results, but made maintaining it cumbersome in the long run. Speaking of the Connaisseur container — inside you will find a full Python interpreter, whose only purpose is to run Connaisseur, although it could do much more. That means we leave a very powerful tool inside our container, just so we can conveniently run our application.</p><p>Now let’s look ahead to what the future of supply chain security brings us 🔮. I can see something! It’s <a href="https://github.com/notaryproject/notation">Notation</a> that also wants to be integrated into Connaisseur and doesn’t have a Python library as well. Looks like we are out of luck.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CiK67swLH9dYjaylYj407Q.png" /><figcaption>Created with DALL-E 3</figcaption></figure><p>As you can see, many reasons ranging from security to maintainability issues piled up over time that made Python a questionable choice. As a replacement, one language stood out from the rest: we wanted to go with Go!</p><p>Rewriting Connaisseur in Go was very tempting for several reasons:</p><ul><li><strong>Compilability</strong>: Golang is compilable and having a compiled Connaisseur binary in the end would allow us to get rid of a great number of dependencies in the Connaisseur image. We hoped to be able to drastically decrease the image size and, more importantly, reduce the attack surface to an absolute minimum by making use of the possibility of building our image from <a href="https://hub.docker.com/_/scratch">scratch</a> — and this dream came true: The Connaisseur image size went down from 193MB to 92,5MB and now only includes the new Connaisseur binary. No more Python interpreter or OS packages that can potentially be abused for attacks.</li><li><strong>Integrability</strong>: Along these lines, we were especially looking forward to eliminating the need to use the Cosign binary, embracing the native integration with the Go-written Cosign package offering access to many more functionalities bearing advantages for both end users but also us maintainers. However, during implementation, we noticed that not only Cosign but also the Notary integration benefits from having direct access to the package’s Go API. One example: Suddenly, it was a no-brainer to broaden the supported key types to cover all TUF-compliant key types. We haven’t even come to make use of all the new possibilities yet, so stay tuned: More features are on the way!<br>After all, our GOal to GO with the GOrgeous Go-GOverned Kubernetes eGOsystem (eh.. ecosystem of course) really paid out.</li><li><strong>Robustness and performance</strong>: Sporadically, Connaisseur was facing stability issues for two different reasons:<br>Once, there were issues with the event loop used for parallelization that we put quite some effort into, but although we were able to solve these issues in the cases we didn’t fully eradicate them. Parallelization based on Goroutines worked out of the box and like a charm (at least up to now).<br>Secondly, Connaisseur needed its time. Despite several performance improvements, in some rare cases, Conaisseur still exceeded the maximum waiting time of Kubernetes admission controllers. The performance of the new Connaisseur release improved significantly, not only tackling the aforementioned timeout issues but also smoothening the user experience in general. First quick measurements testing a sample deployment covering both, containers as well as initContainers based on 10 unique images show a performance gain by a factor of 5.</li></ul><p>In the end we committed to the Go switch, knowing it would take a lot of time and are very happy with the result!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fj6sTkryZbvPprbDOoQqYA.png" /><figcaption>Created with DALL-E 3</figcaption></figure><h3>But wait, there is more</h3><p>Besides the advantages we gained from the language switch, we didn’t pass up the chance to implement some extra features, two of which had been on our list for quite some time.</p><ul><li>We introduced a Redis-based caching mechanism that allows storing digests of validated images for 30 seconds. Before, deployments of workload objects with multiple container instances of the same image entailed multiple network calls to the registry or Notary retrieving the exact same signature data. Admitting an image within 30 seconds of the initial validation allowed a huge performance gain for these kinds of deployments without compromising security with regard to rollback attacks.</li><li>It was brought to our attention by several users (🙏) that Connaisseur was quite difficult to integrate with GitOps solutions. This is due to Connaisseur being a <em>mutating</em> admission controller, mutating the image references at all places by replacing the human-readable image tag with the SHA256 content hash, the image digest. (Why Connaisseur does this is explained <a href="https://sse-secure-systems.github.io/connaisseur/v2.0.0/#trusted-digests">here</a>.) As a consequence, the workload object description inside the cluster differs from the workload description from the git repository that has originally been applied even though the referenced workload object itself is exactly the same. However, the core job of GitOps solutions is to detect exactly those diffs and to sync the workload objects from the git repository to the cluster. The result was an infinite loop of GitOps reapplying content from the git repository and Connaisseur changing it.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*BnyE7SjJMgd4cHbwZ-fTbw.gif" /></figure><p>The old Connaisseur applied this behavior to all workload objects, be it Deployments, ReplicaSets, Jobs, Pods… With the new release, Connaisseur now offers a mode to only mutate Pod resources. This mirrors the behavior of other admission controllers e.g. the Kubernetes native PSA (Pod Security Admission). As for every workload object the actual core consists of one or several underlying Pods, this does not come with any decrease in security.</p><p>However, in terms of usability, one could argue for one or the other. With Pod-only mutation, Connaisseur will officially admit the parent workload object, say a deployment even though the Pod resources <em>inside</em> the deployment are being denied, so the deployment is being admitted while, at the same time being rendered dysfunctional, due to the denial of the actual Pod resource doing the work. For human users, this won’t be a problem, because there is a warning being issued, but it might not be the best solution for CI/CDs only watching for the status code. This is why we made the pod-only mutation optional, being configurable in the values.yaml configuration file.</p><h3>Clara Futura Est</h3><p>As you can see, a lot has changed with release <a href="https://github.com/sse-secure-systems/connaisseur/releases/tag/v3.4.0">3.4.0</a> of Connaisseur, but we are not done yet! There are many changes yet to come:</p><ul><li>First of all we want to make the Cosign integration feature complete, meaning we finally want to implement <a href="https://edu.chainguard.dev/open-source/sigstore/cosign/an-introduction-to-cosign/#keyless-signing">keyless signature</a> verification for Cosign. This will be another step towards a more seamless integration of supply chain security inside your organization, as you no longer need to worry about key management.</li><li>Similarly we want to look into OpenPubKey, as it will be the way to go for <a href="https://www.docker.com/blog/signing-docker-official-images-using-openpubkey/">verifying Docker Official Images</a>, giving you more baseline security without the need for any configuration.</li><li>As already mentioned, Notation is yet another signature verification framework that we aim to support, giving you more choices on signature solutions you want to use, or even mix and mesh.</li><li>We are also looking into streamlining some of our configuration options that might have been confusing to use (looking at you alerting), but that will be part of a future 4.0.0 release since this will incorporate breaking changes.</li></ul><p>In the meantime, we are happy for any feedback from the community on how the Golang switch turned out for them and what other features they would like to see implemented. If you are interested in helping us but haven’t dared so far because you don’t feel comfortable with Python — now is your chance to <a href="https://sse-secure-systems.github.io/connaisseur/latest/CONTRIBUTING/">Go</a>!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t9SPa13Ml_YHBc598QTU1Q.png" /><figcaption>Created with DALL-E 3</figcaption></figure><p>Hopefully you could take away some insights from our little journey from Python to Go, perhaps looking at your own projects and taking a few steps back, reconsidering the decisions you have made and why you made them. It doesn’t need to be as drastic as we did, by switching out the entire underlying programming language, but reworking smaller parts of existing solutions may improve your overall project bit by bit.</p><p>Take care and cheers.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aae7b52ad8e9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/sse-blog/connaisseur-3-4-0-minor-release-major-change-aae7b52ad8e9">Connaisseur 3.4.0: Minor Release, Major Change</a> was originally published in <a href="https://medium.com/sse-blog">SSE Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Container Image Signatures in Kubernetes]]></title>
            <link>https://medium.com/sse-blog/container-image-signatures-in-kubernetes-19264ac5d8ce?source=rss-131fe98392d6------2</link>
            <guid isPermaLink="false">https://medium.com/p/19264ac5d8ce</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[docker-content-trust]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[integrity]]></category>
            <dc:creator><![CDATA[Philipp Belitz]]></dc:creator>
            <pubDate>Fri, 07 Aug 2020 17:13:39 GMT</pubDate>
            <atom:updated>2020-08-11T21:14:01.316Z</atom:updated>
            <content:encoded><![CDATA[<p>Container image signatures are a rarely implemented security feature, even though images’ contents are ever changing and hard to get a grasp of, making it easy for attackers to hide malicious content in them. A main reason for that is that the most popular container orchestrator Kubernetes has no native support for image signatures or their verification. <a href="https://github.com/sse-secure-systems/connaisseur">Connaisseur</a> is a Kubernetes admission controller that tries to change that, by allowing only signed images into a cluster and ensure only trusted and unmodified content is deployed, thus amp up your security.</p><h3><strong>tl;dr</strong></h3><ul><li><em>Docker Content Trust </em>(DCT) is a way to sign your Docker images</li><li>It uses <em>Notary</em> to store signing data</li><li>Kubernetes doesn’t support DCT natively</li><li><a href="https://github.com/sse-secure-systems/connaisseur">Connaisseur (GitHub)</a> is a Kubernetes admission controller that intercepts requests sent to the cluster</li><li>It verifies the signatures of all image references found in the requests</li><li>It denies any requests trying to deploy unsigned images</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r6OKeU4h3YFtjb_5PSTIAg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@sctgrhm?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Scott Graham</a> on <a href="https://unsplash.com/s/photos/signature?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Digital signatures are a well-known approach for maintaining the integrity of any data transferred all around the web. Whether it’s for signing emails, using TLS certificates or app signatures for popular stores such as Google Play or Apple’s App Store. It’s an overall appropriate solution that provides a lot of trust and security, in a world where your credentials are at a constant risk of being stolen and your machine is at a constant risk for abuse, such as bitcoin mining.</p><p>With Docker and Kubernetes, a new landscape has been opened up in this world, full of containerized applications in the form of Docker images, all ready to be pulled from your favorite image registry. But when pulling these images, how sure can you be of their contents? Are there no malicious services hidden in there somewhere? Docker images change all the time and that freshly pulled nginx image could already look a bit different, updated with the latest security patch, or maybe updated with something else, more sketchy…? It’s hard to say, not only for images coming from a public registry, but also for the ones you built yourself. You would have to go into the image and scan for this “malicious content”, how ever that may look like.</p><p>So why not use digital signatures for solving this problem, like we always do? That way we make sure we know from whom the image is coming and that there are no bitcoin miners in there (except you pulled a bitcoin mining image of course). For docker-compose, my colleagues at <a href="https://securesystems.de/">SSE</a> have developed <a href="https://medium.com/sse-blog/bringing-content-trust-into-the-world-of-docker-compose-12836b3537d">trusted-compose</a>. But how does the situation look like in the context of Kubernetes? Well, let’s see.</p><h3>A Game of Tag</h3><p>Before talking about image signatures themselves, a quick reminder on how images can be addressed. There are two options, either address them by tag or by digest. Using the tag, the image content can vary over time as the tag might be overwritten with newer versions of the image, especially if you use the tag latest. The digest on the other hand, is unique for each image, as the digest is a SHA256 hash over the image content that cannot be changed. So for the same image you can either use image:tag or image@sha256:d19357…, while the second option always gives you the same content and the first could change with newer versions of tag.</p><p>Now you could say, why not use digests all the time, if tampering with the images content is your concern. The same argument could be made around the Domain Name System (DNS) system. Why not exclusively use IP addresses to get rid of DNS spoofing? Just use 172.217.20.110 instead of google.com and no one can redirect you to a fake version of Google. Obviously we are not doing that. Domain names are much more tangible and give you a better understanding on where you are or going. 172.271.20.110 seems like a random number that is hard to remember. Maybe you haven’t even noticed that the second IP address isn’t valid at all, which shows how easy it is to mess things up here.</p><p>The same is true for image tags and digests, which is why it’s a lot more common to use tags instead, even though this opens up this issue. And even if you were to use digests exclusively, one cannot tell whether it is a legitimate one just by looking at it, as the source of trust is not clear at all.</p><h3>Docker Content Trust and Notary</h3><p>Container image signatures are nothing new per se. Docker actually developed its own system to sign images, called <a href="https://docs.docker.com/engine/security/trust/content_trust/"><em>Docker Content Trust</em></a>, which is tightly connected to yet another system called <a href="https://docs.docker.com/notary/getting_started/"><em>Notary</em></a>. Notary functions as a server that stores manifest files of “trusted resources”, which are signed by a trusted entity (that’s you). In the context of Docker Content Trust that means that all signed or to be signed images are the “trusted resources” for which some manifest files exist. These files contain a mapping between tags and digests in a 1:1 relation, with the manifest files being signed with a private key. That is the image signature. It’s not created from the image itself, but only from a mapping that pins down a specific distinct version of an image, when referencing its tag.</p><p>You can activate Docker Content Trust, by setting the DOCKER_CONTENT_TRUST environment variable to 1 and DOCKER_CONTENT_TRUST_SERVER to a Notary instance URL. When you then try to pull let’s say image:tag, a lookup for image’s manifest files will be made in the Notary instance. The signatures of these manifests will be verified, so you can trust the tag-digest mappings. Should the signatures all be valid, the manifests will be searched for tag to determine the digest. This digest can be referred to as the “signed” version of the image, which is then used for pulling the image from the registry instead of the original tag(The registry never gets consulted for looking up a tag’s image digest; Docker Content Trust sidesteps this lookup …). If the signatures are not valid, or the tag can’t be found (or no manifest files exist to begin with), the image never gets pulled, as there is no signature.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RwJ0gZNi26Xg4om2814v6w.png" /><figcaption>An Overview on how DCT changes the Docker pull and push behavior</figcaption></figure><p>How do you create signatures with Docker Content Trust? You can just use Docker as usual. Under the hood, it will update the manifest files for the image you are trying to sign or create new ones should they not exist. The current tag is added or updated with the digest and afterwards the manifest file is newly signed.</p><p>This covers the basic functionalities of Docker Content Trust and Notary, leaving out some more complex additions such as freshness guarantee and delegation roles. If you want to learn more about Notary and the underlying system called <a href="https://theupdateframework.io/"><em>The Update Framework</em></a>, go to their <a href="https://github.com/theupdateframework/notary">Github page</a>.</p><p>All that is left is to activate Docker Content Trust in Kubernetes and we are all good, right? Well there lies the problem. Activating this feature in the Docker daemon of Kubernetes doesn’t prevent you from pulling/deploying unsigned images whatsoever. It essentially ignores the feature and thus defeats the whole purpose of doing image signatures in the first place. So, what can we do?</p><h3>Connaisseur</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0kTxKVVC_fbMdNfLzw-YwA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@magicetea?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Ice Tea</a> on <a href="https://unsplash.com/s/photos/french-wine?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Connaisseur solves this problem. It’s an open source solution developed by SSE, inspired by the fact that image integrity for Kubernetes emerged as a recurring topic during our work. Portieris (by IBM) is a solution simliar to ours, but its functionality is currently restricted to the IBM Cloud. That’s one of the reasons why we decided to take things into our own hands and work on the matter.</p><h4>Design</h4><p>Connaisseur is implemented as an admission controller, which means after a resource that is to be deployed to a cluster has gone through the authentication and authorization steps, it will be verified by Connaisseur before actually hitting the cluster. In order to do this verification, Connaisseur searches for all image references in the resource and collects them in a list. For each image in the list, Connaisseur will look up the manifest files in a Notary instance and validate their signatures. If valid, the corresponding digest for the tag is extracted and the original image reference is changed to use the digest instead of the tag. Otherwise, should the signature not be valid, or if the given tag is not present in the manifest files or the manifest files themselves are not there, Connaisseur will deny the resource, stopping its deployment completely.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HDs_SXJeY1e3YzUNrxJojQ.png" /><figcaption>Overview on Connaisseur’s basic workflow</figcaption></figure><p>That’s Connaisseur in a nutshell: You want to deploy an image? Connaisseur will only deploy the signed version, or none at all.</p><h4>Signing your Images</h4><p>In theory that sounds great. But what about reality? Well see for yourself. First you are going to create an unsigned and signed image, using Docker Hub as your image registry and their Notary instance, then install Connaisseur in your cluster and lastly check whether the signature verification works.</p><p>So take two arbitrary images, let’s say image:unsigned and image:signed.</p><pre>$ docker image ls<br>REPOSITORY      TAG         IMAGE         ID  CREATED      SIZE<br>phbelitz/image  signed      e6d91653fd50  19  minutes ago  83.7MB<br>phbelitz/image  unsigned    1601ff33dbe9  21  minutes ago  83.7MB<br>python          3.7-alpine  6ca3e0b1ab69   3  weeks ago    73.1MB</pre><p>You can go ahead and push the image:unsigned image into you Docker Hub repository, as you don’t want a signature for it. For the the one you want signed, you have to activate Docker Content Trust before pushing. As the Notary instance you can use the public one from Docker.</p><pre><em>export</em> DOCKER_CONTENT_TRUST=1<br><em>export</em> DOCKER_CONTENT_TRUST_SERVER=https://notary.docker.io</pre><p>Now push the the image, just as you would do with the unsigned one. When doing so, you’ll be asked to enter a passphrase for a new root and repository key. These keys are automatically generated and are used for creating the image signatures. They will reside on your machine at ~/.docker/trust/private.</p><p>The repository key will be used, should you create new signatures for image, whether that means updating the existing tags or adding new ones. The root key will be needed when creating signatures for completely different images (e.g. image2). In this case a new repository key will be generated as well. Meaning your organization will always have only one root key, but many repository keys, one for each image.</p><pre>$ docker push phbelitz/image:signed<br>The push refers to repository [docker.io/phbelitz/image]<br>6dc8ae05970a: Pushed<br>13d4f082f19a: Pushed<br>122b7df5671a: Pushed<br>5afde73d5bad: Pushed<br>b10be96d1b4e: Pushed<br>003d0b48eda4: Pushed<br>408e53c5e3b2: Pushed<br>50644c29ef5a: Pushed<br>signed: digest: sha256:6e7a18a418d4996681ea3bd1757fcf61f70d7cb32902fc4ce85d025c4a633465 size: 1994<br>Signing and pushing trust metadata<br>You are about to create a new root signing key passphrase. This passphrase<br>will be used to protect the most sensitive key in your signing system. Please<br>choose a long, complex passphrase and be careful to keep the password and the<br>key file itself secure and backed up. It is highly recommended that you use a<br>password manager to generate the passphrase and keep it safe. There will be no<br>way to recover this key. You can find the key in your config directory.<br>Enter passphrase for new root key with ID 4d8aa7e:<br>Repeat passphrase for new root key with ID 4d8aa7e:<br>Enter passphrase for new repository key with ID 7082f7c:<br>Repeat passphrase for new repository key with ID 7082f7c:<br>Finished initializing “docker.io/phbelitz/image”<br>Successfully signed docker.io/phbelitz/image:signed</pre><p>After that you’ll need to get the public part of your root key, as Connaisseur will need it to verify all the signatures. Go to your ~/.docker/trust/private directory and use OpenSSL to generate the public key:</p><pre>$ cd ~/.docker/trust/private<br>$ sed &#39;/^role:\sroot$/d&#39; $(grep -iRl &quot;role: root&quot; .) &gt; root-priv.key<br>$ openssl ec -in root-priv.key -pubout -out root-pub.pem<br>read EC key<br>Enter PEM pass phrase:<br>writing EC key<br>$ rm root-priv.key<br>$ cd -</pre><h4>Connaisseur in Action</h4><p>Ok! The final step is the verification. Clone the Connaisseur repository from Github (git clone git@github.com:sse-secure-systems/connaisseur.git) and adjust the configuration. Take a look at the helm/values.yaml for that. There are three values you have to change, the notary.auth.user, notary.auth.password and notary.rootPubKey. For notary.auth.user and notary.auth.password use your Docker Hub credentials, as the Notary instance will use the same here. For notary.rootPubKey put in the public part of the root key you just created. Save the changes and go into the root directory of the Connaisseur repository.</p><p>A simple make install should do the rest:</p><pre>$ make install<br>bash helm/certs/gen_certs.sh<br>Generating RSA private key, 4096 bit long modulus (2 primes)<br>.....................................................................................................................++++<br>..........................................................................................................++++<br>e is 65537 (0x010001)<br>Generating RSA private key, 4096 bit long modulus (2 primes)<br>.......................++++<br>...............................................++++<br>e is 65537 (0x010001)<br>Signature ok<br>subject=CN = connaisseur-svc.connaisseur.svc<br>Getting CA Private Key<br>kubectl create ns connaisseur || true<br>namespace/connaisseur created<br>kubectl config set-context --current --namespace connaisseur<br>Context &quot;minikube&quot; modified.<br>helm install connaisseur helm --wait<br>NAME: connaisseur<br>LAST DEPLOYED: Wed Jul 29 13:42:05 2020<br>NAMESPACE: connaisseur<br>STATUS: deployed<br>REVISION: 1<br>TEST SUITE: None</pre><p>The moment of truth has come. Create a sample namespace and switch to it:</p><pre>$ kubectl create ns sample<br>namespace/sample created<br>$ kubectl config set-context --current --namespace sample<br>Context &quot;minikube&quot; modified</pre><p>Now try creating a pod with the unsigned image. It should give an error, since there is no signature for it:</p><pre>$ kubectl run unsigned --image=phbelitz/image:unsigned --port=5000<br>Error from server: admission webhook &quot;connaisseur-svc.connaisseur.svc&quot; denied the request: could not find signed digest for image &quot;docker.io/phbelitz/image:unsigned&quot; in trust data.</pre><p>Great success! Works as intended. Do the same with the signed image. Here the pod should be created, as a signature is present and valid:</p><pre>$ kubectl run signed --image=phbelitz/image:signed --port=5000<br>pod/signed created</pre><p>Tada, that works as well 🎉. Now if you check all the pods that are currently running, all you’ll see is the signed pod, but not the unsigned one, since this one got denied:</p><pre>$ kubectl get pods<br>NAME    READY  STATUS   RESTARTS  AGE<br>signed  1/1    Running  0         2m6s</pre><p>And that’s it!</p><h3>Fade-out</h3><p>A quite simple example on how container image signatures in Kubernetes can work. In more sophisticated approaches, you can also let your images be signed during a CI/CD pipeline and then be verified at deployment time. This can especially be useful in the growing GitOps approaches that are getting more and more popular. Solutions like <a href="https://github.com/fluxcd/flux">Flux</a>, where an operator inside Kubernetes listens on image registries and applies any changes, whether they include malicious content or not, could be secured with Connaisseur. With Flux and Connaisseur together, you need no kubectl access to your cluster anymore, thus reducing the attack surface and at the same time ensuring that only signed images are admitted for deployment. Security all over the place!</p><p>If you want to learn more about Connaisseur, for example how the image policy works, go visit the <a href="https://github.com/sse-secure-systems/connaisseur">Connaisseur GitHub page</a>. Also feel free to give us feedback or add some contributions, it’s much appreciated. Now go ahead and secure your clusters. Cheers!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=19264ac5d8ce" width="1" height="1" alt=""><hr><p><a href="https://medium.com/sse-blog/container-image-signatures-in-kubernetes-19264ac5d8ce">Container Image Signatures in Kubernetes</a> was originally published in <a href="https://medium.com/sse-blog">SSE Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>