<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sebastian Röbke on Medium]]></title>
        <description><![CDATA[Stories by Sebastian Röbke on Medium]]></description>
        <link>https://medium.com/@boosty?source=rss-b6b351cf7f1c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*loo5mu_F_Bd3hzow.</url>
            <title>Stories by Sebastian Röbke on Medium</title>
            <link>https://medium.com/@boosty?source=rss-b6b351cf7f1c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 03:59:14 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@boosty/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[XING Kubernetes components published as Open Source]]></title>
            <link>https://tech.new-work.se/xing-kubernetes-components-published-as-open-source-368829289c96?source=rss-b6b351cf7f1c------2</link>
            <guid isPermaLink="false">https://medium.com/p/368829289c96</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[prometheus]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[operations]]></category>
            <dc:creator><![CDATA[Sebastian Röbke]]></dc:creator>
            <pubDate>Tue, 05 Feb 2019 16:58:30 GMT</pubDate>
            <atom:updated>2019-02-06T09:07:56.893Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p><a href="https://kubernetes.io/">Kubernetes</a> is an amazing container orchestration system with a lot of functionality provided out of the box. We use it as the basis for our internal Platform as a Service, which we call “Olympus”. To offer a complete and coherent deployment platform for the XING engineering teams, we sometimes need to write custom components on top of Kubernetes. In particular, these components fill a gap when it comes to integrating with other systems or automating common tasks.</p><p>While these components are usually developed out of the necessity to solve a specific problem within our environment, many of them turn out to provide generic functionality which can also be useful to others. This is why the Olympus team recently published some of them as part of the first ever <em>Olympus Open Source Week</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*WpKHLIDsJZgWKJe-SkOtcg.png" /><figcaption>Kubernetes Logo</figcaption></figure><p>Here we want to share what we have been working on 🚀</p><h3>kubernetes-event-forwarder-gelf</h3><p>Kubernetes events are an indispensable part of the Kubernetes API for monitoring and debugging Kubernetes clusters. Events get created when interesting things happen, e.g. when a node is detecting memory pressure or when a container keeps crashing. You can read more about using events for debugging your application in the <a href="https://kubernetes.io/docs/tasks/debug-application-cluster/debug-application-introspection/">Application Introspection and Debugging section</a> of the official Kubernetes docs.</p><p>As the name suggests, our kubernetes-event-forwarder-gelf forwards these events to a <a href="https://www.graylog.org/">Graylog</a> endpoint so that they can be monitored in a central place. It uses the structured Graylog Extended Log Format (<a href="http://docs.graylog.org/en/2.5/pages/gelf.html">GELF</a>), so that you will find the event details nicely separated as individual fields for proper searching and filtering, e.g. to only see critical events or those coming from a specific cluster or namespace.</p><p>GitHub: <a href="https://github.com/xing/kubernetes-event-forwarder-gelf">kubernetes-event-forwarder-gelf</a></p><h3>kubernetes-oom-event-generator</h3><p>As written above, Kubernetes events are an indispensable resource for monitoring and debugging. In addition to the built-in Kubernetes events, we can also use this mechanism to create custom events.</p><p>Our kubernetes-oom-event-generator is doing exactly that. When a container getsOOMKilled – which might happen when it uses more memory than requested or when a configured limit is reached – there is no built-in event we could use. But we found a workaround: We can ask the Kubernetes API about the previous state of a container after it got restarted. When we see that the previous instance was OOMKilled, we create a Warning event with the reason PreviousContainerWasOOMKilled.</p><p>And thanks to the generic kubernetes-event-forwarder-gelf introduced above, these events will be visible in Graylog just like all the others.</p><p>GitHub: <a href="https://github.com/xing/kubernetes-oom-event-generator">kubernetes-oom-event-generator</a></p><h3>kubernetes-deployment-restart-controller</h3><p>In Kubernetes we mainly use two built-in resource types to provide configuration to containers: ConfigMaps and Secrets. For example, we use ConfigMaps to create environment variables or config files and Secrets to maintain configuration items that should be kept, well, secret (database passwords, API tokens, etc.).</p><p>You might be surprised, but when changing a ConfigMap or Secret, the containers using them are not automatically restarted to pick up the new values. Yet that would be the behaviour we usually want, and the Kubernetes community has been <a href="https://github.com/kubernetes/kubernetes/issues/22368">discussing</a> this requirement for quite some time.</p><p>Our kubernetes-deployment-restart-controller solves this problem as it watches ConfigMaps and Secrets referenced by Deployments and StatefulSets and triggers restarts as soon as the configuration or secret values change.</p><p>To enable this functionality, adding a single annotation to your deployment manifest is all you need:</p><pre>apiVersion: apps/v1beta2<br>kind: Deployment<br>metadata:<br>  annotations:<br>    com.xing.deployment-restart: enabled<br>  # the rest of the deployment manifest</pre><p>GitHub: <a href="https://github.com/xing/kubernetes-deployment-restart-controller">kubernetes-deployment-restart-controller</a></p><h3>passenger-prometheus-exporter-app</h3><p>At XING we are heavily using the <a href="https://www.phusionpassenger.com/">Passenger application server</a> by Phusion. When moving to Kubernetes, we wanted to expose Passenger’s status metrics in the <a href="https://prometheus.io/">Prometheus</a> format. Some solutions already exist but they require a sidecar container. For us it was easier to have something we could integrate directly within our Docker images, so we created a small Ruby app.</p><p>It exposes a Prometheus metric endpoint that runs <a href="https://www.phusionpassenger.com/library/admin/standalone/overall_status_report.html">passenger-status</a> and returns information about the processes:</p><ul><li>passenger_capacity – Number of processes spawn</li><li>passenger_processes_active – Number of processes currently working on requests</li><li>passenger_wait_list_size – Requests in the queue</li></ul><p>In our setup, <a href="https://github.com/influxdata/telegraf">Telegraf</a> scrapes these metrics and pushes them to InfluxDB for Grafana, where they can be observed using a custom dashboard.</p><p>GitHub: <a href="https://github.com/xing/passenger-prometheus-exporter-app">passenger-prometheus-exporter-app</a></p><h3>ingress-nginx</h3><p>Not all contributions have to happen in our own repositories. We also participate in Open Source projects we depend on.</p><p>Recently a few patches of ours have landed in <a href="https://github.com/kubernetes/ingress-nginx">ingress-nginx</a>, one of the most used load-balancing solutions for Kubernetes.</p><ul><li><a href="https://github.com/kubernetes/ingress-nginx/pull/3628">Fix 503 error generation on empty endpoints</a></li><li><a href="https://github.com/kubernetes/ingress-nginx/pull/3548">e2e test to ensure graceful shutdown does not lose requests</a></li></ul><p>Some older contributions from our team can be found <a href="https://github.com/kubernetes/ingress-nginx/pulls?utf8=%E2%9C%93&amp;q=is%3Apr+author%3Amaxlaverse+">here</a>.</p><h3>Summary</h3><p>Giving back to the Kubernetes community feels great! We have been using Kubernetes and related Open Source software to successfully build a Platform as a Service for XING that serves a massive amount of production traffic today and has become the default deployment stack for the XING engineering teams.</p><p>By publishing our work as Open Source we truly believe to improve our software and the whole platform by getting valuable feedback from the OS community. Thanks to everyone involved!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=368829289c96" width="1" height="1" alt=""><hr><p><a href="https://tech.new-work.se/xing-kubernetes-components-published-as-open-source-368829289c96">XING Kubernetes components published as Open Source</a> was originally published in <a href="https://tech.new-work.se/">New Work Development</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>