<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Turbine Labs - Medium]]></title>
        <description><![CDATA[Ship more. Worry less. www.turbinelabs.io - Medium]]></description>
        <link>https://medium.com/turbine-labs?source=rss----352a8f053fef---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Turbine Labs - Medium</title>
            <link>https://medium.com/turbine-labs?source=rss----352a8f053fef---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 06:09:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/turbine-labs" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Turbine Labs is shutting down and our team is joining Slack]]></title>
            <link>https://medium.com/turbine-labs/turbine-labs-is-shutting-down-and-our-team-is-joining-slack-2ad41554920c?source=rss----352a8f053fef---4</link>
            <guid isPermaLink="false">https://medium.com/p/2ad41554920c</guid>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Mark McBride]]></dc:creator>
            <pubDate>Fri, 19 Oct 2018 14:04:45 GMT</pubDate>
            <atom:updated>2018-10-19T14:04:45.287Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-u2bbby5y5ZpawOhw8HupA.jpeg" /></figure><p>Today is a bittersweet day for Turbine Labs; we are shutting down our product, Houston, and most of our team is joining Slack.</p><p>Glen and I started Turbine Labs a little over three years ago, to fundamentally change how developers worked with each other at scale. We helped Twitter move from a monolith to microservices, and we saw how important it became for teams to be able to fully own and operate their services. We started Turbine Labs to help unlock this superpower for everyone.</p><p>What we didn’t realize nearly four years ago is just how much everybody else agreed with us. Service mesh has taken off in a way that neither of us expected, and we’ve worked with some of the highest-traffic websites in the world to help them roll our better SLO management, traffic shifting, and security to teams of hundreds of developers. There is clearly a need for better operational tooling for teams with all architectures, from monolith-first to monoliths with supporting services to fully distributed microservices.</p><p>There is a TON going on in the service mesh space right now. Envoy and Istio are fantastic projects, with Consul, Heptio, and more building integrations into orchestrators like Kubernetes to help weave them into a coherent product. Infrastructure engineers have their hands full just sorting out which stack to use, and everybody has their own definition of a service mesh.</p><p>At its core, this is where we got bogged down. We never wanted to create tools just for the infrastructure team. Service meshes should be useful to every engineer. The UIs and API to make changes should be both powerful and approachable. With constantly shifting definitions of what technologies constituted a service mesh, we found it impossible to build a product that worked for everybody, much less a sustainable business.</p><p>To be clear, Slack isn’t acquiring Turbine Labs; we’re simply going to work for them. We know and trust many people across their organization, and we’ve been consistently impressed with their work. In particular, their thoughtful approach to scaling and developer tooling has been crucial to their growth. We found that they have invested deeply in their infrastructure, and their culture — from empathy for coworkers, to customer focus, to technical vision — matched ours to a degree that we never imagined. We literally built our company on Slack, and we’re excited to join them.</p><p>We’d like to thank our customers, our investors, and everybody in the Envoy community for joining us on this journey. We know there’s a bright future for Envoy, Istio, and all the other service mesh projects in open source. To the communities around these projects, we’ll be contributing back much of what we created to help you, and we hope we can find a way to continue working with you in the future.</p><p>We’d also like to thank our friends and families for their unending support over the past three years.</p><p>Feel free to email us at <a href="/service/mailto:hello@turbinelabs.io">hello@turbinelabs.io</a> about anything.</p><p>Thank you!</p><p>Mark McBride<br>CEO, Turbine Labs</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ad41554920c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/turbine-labs/turbine-labs-is-shutting-down-and-our-team-is-joining-slack-2ad41554920c">Turbine Labs is shutting down and our team is joining Slack</a> was originally published in <a href="https://medium.com/turbine-labs">Turbine Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Guide to Envoy’s Backpressure]]></title>
            <link>https://medium.com/turbine-labs/a-guide-to-envoys-backpressure-22eec025ef04?source=rss----352a8f053fef---4</link>
            <guid isPermaLink="false">https://medium.com/p/22eec025ef04</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[grpc]]></category>
            <category><![CDATA[proxy-server]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[TR Jordan]]></dc:creator>
            <pubDate>Fri, 07 Sep 2018 12:31:01 GMT</pubDate>
            <atom:updated>2018-09-07T12:31:01.538Z</atom:updated>
            <content:encoded><![CDATA[<h4>How Envoy handles overloaded microservices in a service mesh architecture</h4><p>One of the biggest advantages of a service mesh is that it frees up clients from implementing features like rate limiting, retries, or timeouts. It’s easy to think that with a service proxy like Envoy handling these features, there’s no further work to be done. While this is <em>almost </em>true, if there’s a real failure, (e.g. multiple retries that never succeed), the client will ultimately have to handle it.</p><p>Whether you handle this with custom code in each service or with a set of thin client libraries is up to you, but here are the behaviors you can expect to see from Envoy when things go wrong with HTTP/1, HTTP/2, or gRPC requests.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZiWbT267697LgO1f" /><figcaption>Sometimes it’s better to fail gracefully</figcaption></figure><p>Two quick definitions, used by Envoy:</p><ul><li><strong>Upstream</strong> connections are the service Envoy is initiating the connection to.</li><li><strong>Downstream</strong> connections are the client that is initiating a request through Envoy.</li></ul><p>These terms take their origin from traditional computer networking; information flows like water through Envoy from <a href="https://en.wikipedia.org/wiki/Upstream_(networking)">upstream</a> (the service that eventually responds to the request) to <a href="https://en.wikipedia.org/wiki/Downstream_(networking)">downstream</a> (the caller making the request). In a service mesh, all services are often co-located in the same cloud, but there’s still utility in thinking in terms of traffic flow.</p><h3>Envoy Failure Modes</h3><p>The following is a (hopefully comprehensive) list of ways Envoy can send an unsuccessful response to the downstream client.</p><h4>Rate Limiting Failure</h4><p><strong>What it is: </strong>The <a href="https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/global_rate_limiting">Rate Limiting</a> service determined that this request exceeds the quotas it sets and should be tried again later.</p><p><strong>Result:</strong> The client receives an <a href="https://http.cat/429">HTTP 429</a> response.</p><h4>Timeout Failure</h4><p><strong>What it is: </strong>The upstream connection took longer to respond than the <a href="https://www.envoyproxy.io/docs/envoy/latest/api-v1/route_config/route#route">timeout set on the route</a>.</p><p><strong>Result:</strong> The client receives an <a href="https://http.cat/504">HTTP 504</a> (<a href="https://www.envoyproxy.io/docs/envoy/latest/configuration/http_filters/router_filter.html?highlight=circuit%20breaking%20returns#x-envoy-retry-on">Read More</a>)</p><h4>Empty Cluster</h4><p><strong>What it is: </strong>There are no hosts available in a cluster. Note that this is <strong>NOT</strong> what happens when all hosts fail either <a href="https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/health_checking">health checks</a> or <a href="https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/outlier#arch-overview-outlier-detection">outlier detection</a>. In either of those situations, Envoy will notice the number of hosts is below the <a href="https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/load_balancing#panic-threshold">panic threshold</a> and balance across all known hosts.</p><p><strong>Result: </strong>Returns a <a href="https://http.cat/503">503</a> with the body text “no healthy upstream”.</p><h4>Unresponsive Upstream</h4><p><strong>What it is: </strong>The upstream host is unresponsive.</p><p><strong>Result: </strong>Returns a <a href="https://http.cat/503">503</a> with the body text “upstream connect error or disconnect/reset before headers”.</p><h4>Circuit Breaker Failed Open</h4><p><strong>What it is: </strong>Due to too many unsuccessful or outstanding requests to this upstream, Envoy has tripped the <a href="https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/circuit_breaking">circuit breaker</a> and is failing the request.</p><p><strong>Result: </strong>The client receives an <a href="https://http.cat/503">HTTP 503</a> with the x-envoy-overloaded: true header set. (<a href="https://www.envoyproxy.io/docs/envoy/latest/configuration/http_filters/router_filter.html">Read More</a>)</p><h4>Maintenance mode</h4><p><strong>What it is: </strong>Routes to this cluster have <a href="https://www.envoyproxy.io/docs/envoy/latest/configuration/http_filters/router_filter#config-http-filters-router-runtime-maintenance-mode">been configured with</a> upstream.maintenance_mode.&lt;cluster_name&gt;, and this request is in the fraction of requests that is unsuccessful</p><p><strong>Result: </strong>Returns a <a href="https://http.cat/503">503</a> with the x-envoy-overloaded: true header set. (<a href="https://www.envoyproxy.io/docs/envoy/latest/configuration/http_filters/router_filter.html">Read More</a>)</p><h4>Domain or Route not found</h4><p><strong>What it is: </strong>Envoy is not configured to serve this URL.</p><p><strong>Result:</strong> The client receives an <a href="https://http.cat/404">HTTP 404</a>.</p><p><strong>Note: </strong>Envoy only supports HTTP/1.1 and newer, which requires the presence of a Host header. In particular, this means that requests to 127.0.0.1 or the IP that Envoy is listening on will almost always return a 404, even if Envoy is correctly configured to serve one or more domains.</p><h4>Upstream Connection Closed</h4><p><strong>What it is: </strong>When the upstream closes the connection before the response is finished sending, Envoy cannot send a complete response to the downstream.</p><p><strong>Result:</strong> This depends on whether the downstream has started receiving data.</p><ul><li>If it has not (i.e. the upstream disconnects quickly), the downstream connection is reset.</li><li>If it has, the downstream receives an <a href="https://http.cat/503">HTTP 503</a> and the body text “upstream connect error or disconnect/reset before headers”</li></ul><h4>Envoy is Overloaded</h4><p><strong>What it is: </strong>When Envoy itself is overloaded, it will stall. Envoy stops reading from the downstream client when it wants to exert back pressure (e.g., local kernel buffer fills, then remote kernel buffer, then client blocks).</p><p><strong>Result: </strong>Downstream connections are blocked with no error.</p><h3>What Clients Should Handle</h3><p>In addition to the errors above, clients should also handle scenarios that out-of-process proxies cannot deal with. These include:</p><ul><li>Thread-local circuit breakers</li><li>Timeouts if Envoy is blocking</li><li>A code path that returns a sensible result when a failure has occurred</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F35e04f%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F35e04f%2F&amp;image=https%3A%2F%2Fucarecdn.com%2F9c5e5f35-9737-44ec-8aaa-76b453b3f4db%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3af9e28be2aa1a8842cfa806fdd6225e/href">https://medium.com/media/3af9e28be2aa1a8842cfa806fdd6225e/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22eec025ef04" width="1" height="1" alt=""><hr><p><a href="https://medium.com/turbine-labs/a-guide-to-envoys-backpressure-22eec025ef04">A Guide to Envoy’s Backpressure</a> was originally published in <a href="https://medium.com/turbine-labs">Turbine Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Service Level Objectives for Every Service]]></title>
            <link>https://medium.com/turbine-labs/service-level-objectives-for-every-service-53088558182b?source=rss----352a8f053fef---4</link>
            <guid isPermaLink="false">https://medium.com/p/53088558182b</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[TR Jordan]]></dc:creator>
            <pubDate>Tue, 21 Aug 2018 12:31:01 GMT</pubDate>
            <atom:updated>2018-08-21T12:31:01.076Z</atom:updated>
            <content:encoded><![CDATA[<h4>Using a Service Mesh to Set, Track, and Manage SLOs</h4><p>One of the biggest advantages of microservices is that it makes giant applications human-scale again. By introducing well-defined barriers, you can hold a whole service worth of functionality in your head at one time.</p><p>It’s easy to forget that your users don’t have the luxury of interacting with just a single service. No matter what happens between services, users see the aggregated response from all services, or at least the services in the critical path.</p><p>This can make it hard to talk about “is this service working well?” for anything more than a simple availability check. If users expect a page to load in 400ms with 99.9% availability, every service involved must load in (probably much) less than 400ms with (at least) 99.9% availability in order to hit the combined goals. These individual goals are referred to as Service Level Objectives, or SLOs. But even armed with the knowledge of what users expect, how do you set SLOs?</p><p>This post outlines an approach to set realistic targets and trade effectively between operational goals and feature goals.</p><h3>Setting SLOs: The Critical Path</h3><p>To set appropriate SLOs, the first thing is to identify the core use experience, or critical path. For Google, this might be making a search. For an ecommerce company, it might be purchasing an item. The key here is to pick a simple set of activities that can be easily described by somebody using the app.</p><p>Once you’ve identified the critical path, the next step is to collect data on how services typically contribute to this path. Distributed tracing is a great tool for doing this with real user data, but it could be done with other tools. For our hypothetical ecommerce site, loading a product page may break down into:</p><ul><li>Fetching the product information (50ms)</li><li>Fetching recommendations (100ms)</li><li>Generating a list of top reviews (100ms)</li><li>Assembling the HTML (50ms)</li></ul><p>Total page load time: 300ms</p><p>If 300ms is an acceptable p99 time, we can now set SLOs based on this data. Note that the product information service gets a lower SLO than the recommendation service! This may be unfair, because we have to start somewhere, and it’s deeply unfair to hand out latency SLOs that teams aren’t already close to meeting. If we want 99.9% uptime on page load, we can also set availability SLOs for each service of 99.975%.</p><p>(Note: if you have a statistics background, you’ll note that error rate / number of services and page p99 == sum[service p99] is wrong. It’s generally close enough, especially since you’ll have to explain it to all your colleagues in the next step, but feel free do it the right way!)</p><h3>Tracking SLOs</h3><p>The worst thing you can do at this point is start holding teams accountable to these numbers. Time to get their feedback!</p><p>The best way to do this is to start visualizing these numbers, without judgement. Depending on your level of instrumentation, you may already have this data, or you may have to add an observability layer to some services. If you have a service mesh in place, measuring proxy-to-service latency is a great standard of measurement. This lets you include network time in your SLOs, which can otherwise slip between the cracks.</p><p>Graphs are a great place to start a conversation with service teams about whether their SLOs are reasonable. Perhaps the recommendation service actually provides a lot of data to the BI team for purchasing, and those batch calls are slow! It may be appropriate to modify the service goals based on client, time of day, or anything else that matters to your users.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cH6THp8AR1Fi5CIoyp_WRA.png" /><figcaption><em>A simple SLO dashboard. Resist the temptation to add more metrics!</em></figcaption></figure><p>Once teams have SLOs in place, don’t change them too frequently! While it’s true that your user experience might drift from the initial measurements, the point is to provide a consistent framework to teams to help them prioritize features, tech debt, and operational improvements. Revisit them quarterly, at most.</p><h3>Managing SLOs</h3><p>With an SLO in place, teams can manage their own time much more effectively. Has their service been unstable? Time for database optimisation. No SLO violations for 90 days? Full steam ahead on feature work!</p><p>While the world of possible changes is enormous, a service mesh offers several tools that are easy to enable and can make an immediate impact on SLOs. In particular, many of these options allow for <strong>tradeoffs</strong> between speed and reliability.</p><ul><li><strong>Request Retries</strong> allow services to handle flaky backends gracefully, improving availability.</li><li><strong>Timeouts</strong>, when combined with retries, trade backend speed for availability.</li><li><strong>Circuit Breakers</strong> allow overloaded services to fail quickly, which may allow clients to fail over to a different region or service (e.g. with stale data) instead of timing out themselves.</li><li><strong>Health Checking </strong>allows services to remove underperforming infrastructure automatically.</li></ul><p>In many cases, services out of SLO have flexibility in one of these dimensions, and trading their budget back and forth is an effective way to get back to an acceptable level quickly, without holding up feature work.</p><p>Want a simple way to track SLOs and tools to manage them? Check out <a href="http://turbinelabs.io/">Houston, the self-service mesh</a>!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F35e04f%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F35e04f&amp;image=https%3A%2F%2Fucarecdn.com%2F9c5e5f35-9737-44ec-8aaa-76b453b3f4db%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/9e4ea59b35963a38598941f2413ba777/href">https://medium.com/media/9e4ea59b35963a38598941f2413ba777/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53088558182b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/turbine-labs/service-level-objectives-for-every-service-53088558182b">Service Level Objectives for Every Service</a> was originally published in <a href="https://medium.com/turbine-labs">Turbine Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Control Planes for Envoy]]></title>
            <link>https://medium.com/turbine-labs/control-planes-for-envoy-5453f004a5f0?source=rss----352a8f053fef---4</link>
            <guid isPermaLink="false">https://medium.com/p/5453f004a5f0</guid>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Brook Shelley]]></dc:creator>
            <pubDate>Thu, 16 Aug 2018 12:36:01 GMT</pubDate>
            <atom:updated>2018-08-16T12:36:01.109Z</atom:updated>
            <content:encoded><![CDATA[<h4>Understanding a vital part of connecting Envoy Proxy to your applications</h4><p>(<em>Previously appeared on </em><a href="https://blog.envoyproxy.io/">https://blog.envoyproxy.io</a>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PR9VzwbxuaWlAO86Ldi-uw@2x.jpeg" /><figcaption>Image by <a href="https://unsplash.com/@trommelkopf">Steve Harvey</a></figcaption></figure><p>In modern environments, hosts don’t live for months or years. Cloud VMs may live for days, and Kubernetes pods may only live for minutes. Envoy’s dynamic configuration allows it to keep up-to-date on what infrastructure is available and healthy. Modern apps at any scale need a standardized approach for dealing with this ever-changing infrastructure. A well-built control plane easily integrates with service discovery and Envoy connects to it for data. Routes and rules change as services are added and removed, and utilizing a control plane means each service team can fully utilize the power of your front and service mesh proxies.</p><h3>What makes a great control plane?</h3><p>Think of a control plane as a Docker-style metaservice for your Envoy proxy. A control plane helps manage configuration, as Envoy defines a management service API to allow dynamic updating of routing configuration across an entire fleet of proxies. A control plane harnesses this API and can connect it to your service discovery regime. This central point of control allows you to build whatever workflows you need: blue/green releases, test in prod, shadow traffic, etc. Having these tools at your disposal can help you build confidence in your code faster, ultimately leading to shipping more. There are five features of a great control plane:</p><ul><li>Service discovery support</li><li>Easy, well-documented configuration</li><li>Optional dynamic or static configuration</li><li>APIs for advanced control</li><li>Observability</li></ul><h3>Service discovery support</h3><p>To some extent, modern service discovery systems do a lot of this work for you. Kubernetes will create DNS entries for each service, so if you have a service named “auth,” you can make requests to auth.default.svc.cluster.local and it’ll work. Similarly, Consul creates auth.service.dc1.consul.</p><p>Both of these run over DNS, which comes with all the features of… DNS. Primarily, that all languages already speak it. Kube-proxy helps a bit here, but it’s still a purpose-built proxy. Consul is great at adding quick registration/deregistration of instances, but ultimately Envoy makes decisions on traffic and customer requests and how they route based on these services and instances’ existence. By adding a control plane, you allow envoy to use SD information to make high-level and vital decisions for each of your services at the network level.</p><h3>Easy, well-documented configuration</h3><p>Any tool you use regularly should come with clear documentation of its functions, overrides, and error states. <a href="https://www.envoyproxy.io/docs/envoy/latest/">Envoy’s documentation</a> is a great example of clear, thorough, and informative development reference. Further, your control plane should use a well-defined, well-documented configuration interface — eg Envoy’s [<a href="https://github.com/envoyproxy/envoy/tree/master/api/envoy/api/v2">xDS</a>], Istio’s [<a href="https://istio.io/docs/reference/config/istio.networking.v1alpha3/">traffic routing API</a>]or the [<a href="https://docs.turbinelabs.io/api">Houston API</a>]. This ensures that anyone using your control plane will start with a great level of understanding of the metaphors and language that leverages their existing understanding and knowledge.</p><h3>The option of dynamic or static configuration</h3><p>In an ideal world, all services use the same template and format, and behave the same way. In a real app, sometimes there are exceptions that must be hardcoded. But what if there’s a service that still lives in a different cloud provider, or one that hasn’t been added to the service discovery yet?. When using or building a control plane, these heterogeneous architectures are important to consider. Whether you’re defining clusters, routes, or templates, connecting to service discovery and dynamically collecting information for Envoy is only half of the story — static configurations are often a key component of infrastructure. A great control plane should act as an amplifier of your effort to guide and release network and microservice architecture across your teams, whether they’re using static or dynamically configured services.</p><p>An example of a well-implemented open-source control plane is <a href="http://github.com/turbinelabs/rotor">Rotor</a>. Rotor was designed so that it’s as easy to run Envoy out-of-the-box to use DNS-based resolution from your service discovery. It supports Kubernetes, Consul, AWS, static configuration files, and v1 Envoy configurations, to help populate many of Envoy’s endpoints with collected services. It also provides static configuration to handle overrides, or edge cases through a few environmental variables.</p><h3>APIs for advanced control</h3><p>A control plane should expose or allow for manipulation of advanced features of the Envoy API, like circuit-breaking, load-balancing, and outlier detection. The heavy lifting of these features could be enabled by a user-facing management tool, with the control plane interfacing between Envoy and a GUI.</p><h3>Observability</h3><p>Envoy collects incredibly useful information about your clusters, routes, and requests, to construct a picture of your infrastructure and team’s SLOs. Ensure you’re passing these into a service like statsd, Prometheus, or Lightstep, or into your own dashboards to get real-time feedback on your infrastructure controls and network.</p><h3>Further Consideration</h3><p>Whether you’re building your own bespoke control plane, or using a production-ready one like Rotor, there are a few things to keep in mind as you work to make your routing definitions as dynamic and flexible as your endpoints:</p><ul><li>How do you handle configuration changes? Do you add your files to version control and periodically redeploy your control plane? Envoy can reload in this way, but it can be cumbersome and slow.</li><li>Do you build out or use dynamic configuration, storing routes in a database with an API?</li><li>How do you manage dynamic configuration, and expose controls and monitoring to bridge multiple types of infrastructure and eases migration?</li><li>What types of service discovery do you use and do heterogenous systems need to work together?</li><li>How easy is it for individual teams to manage their own SLOs?</li></ul><p>A basic control plane helps with some of these goals, but doesn’t implement advanced load balancing, health checking, circuit breakers, and other advanced Envoy features. Implementing this API is left to another metaservice. Control planes also lack automatic collection of Envoy’s metrics for routes, clusters, and more, with easy integration into statsd, Prometheus, and other common dashboards, which are key for SLO management.</p><h3>Conclusion</h3><p>This overview of control planes in Envoy, as well as an example of how one control plane functions should help start a conversation about your team’s implementation of these tools. With a thoughtful approach, Envoy can simply integrate with your services, and manage SLOs at your traffic layer. There are a lot of great ways to get started, including the <a href="https://github.com/envoyproxy/go-control-plane">go control plane</a> and <a href="https://github.com/turbinelabs/rotor">Rotor</a>. You can also check out the many awesome <a href="https://www.envoyproxy.io/community">projects built on Envoy</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5453f004a5f0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/turbine-labs/control-planes-for-envoy-5453f004a5f0">Control Planes for Envoy</a> was originally published in <a href="https://medium.com/turbine-labs">Turbine Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Nav Moved to Kubernetes with Houston, by Turbine Labs]]></title>
            <link>https://medium.com/turbine-labs/how-nav-moved-to-kubernetes-with-houston-by-turbine-labs-2e276168dc57?source=rss----352a8f053fef---4</link>
            <guid isPermaLink="false">https://medium.com/p/2e276168dc57</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[migration]]></category>
            <dc:creator><![CDATA[TR Jordan]]></dc:creator>
            <pubDate>Mon, 13 Aug 2018 07:31:01 GMT</pubDate>
            <atom:updated>2018-08-14T00:16:02.863Z</atom:updated>
            <content:encoded><![CDATA[<h4>Financial services firm leveraged Envoy to save hundreds of engineering hours and reduce technical debt in the process</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*E8PyXG8zfEEI90SC.png" /><figcaption><a href="http://nav.com/">Nav: Get the credit your business deserves</a></figcaption></figure><h3>The Challenge</h3><p><a href="http://nav.com/">Nav</a>, a financial services company that helps small businesses understand and streamline the factors that impact their credit, was working with infrastructure that wasn’t scalable. Each of their services was modeled 1:1 with a Virtual Machine (VM) in AWS. As a financial services company, their uptime requirements were even more rigorous than most companies. The systems they’d built for stability were sluggish and bloated: it took roughly 160 engineer-hours to get a service into production. Nav sought to migrate their infrastructure to Kubernetes to resolve the 1:1 modeling issue and reduce the time spent releasing new services.</p><p>But setting up the new infrastructure was only one piece of their challenge. Nav also needed a way to manage both environments simultaneous during the migration. They had investigated traffic proxies, but none could shift traffic for individual services between AWS VPCs and Kubernetes clusters. It would take at least 6 weeks for them to write something in-house to handle these conversions, and they were already 3 months into the process, without time or money to spare. To compound this difficulty, their in-house solution would only be able to move over traffic and services abruptly, all at once — leaving Nav open to catastrophic consequences if a rollback or retry were needed.</p><p>Concerned that in-house efforts would be incur too much technical debt from an overly fragile system, Nav started to look for an external solution.</p><h3>The Solution</h3><blockquote>Infrastructure migration with Houston, as easy as a blue-green deploy.</blockquote><p>While looking for a solution, Nav contacted Turbine Labs. After seeing our release demo, they were excited to see how it could work for their use case. They were excited to see how easy it was to use <a href="https://blog.turbinelabs.io/deploy-not-equal-release-part-one-4724bc1e726b">traffic shifting for releases</a> to avoid hard cutovers with a high risk of failure. Faced with the challenge of releasing an entirely new infrastructure stack, they wondered “could it really be so easy to handle a large-scale migration of services?”</p><p>The answer, surprisingly, was yes.</p><blockquote>“What we had described was exactly what Turbine provided. It was exactly what we had talked about, but we didn’t have to build it ourselves” — Travis Jeppson, Director of Engineering</blockquote><p>Using <a href="https://www.turbinelabs.io/product">Houston</a> by Turbine Labs, Nav was able to migrate from their existing system to a Kubernetes environment. Houston allowed Nav to collect information from both environments and use tagging to direct traffic between services, one service at a time. It also allowed them to migrate incrementally, both on the service-level and the traffic-level.</p><h3>The Results</h3><ul><li>6-month migration achieved in 2 months, with zero risk</li><li>Rebuilt Nav’s Kubernetes Cluster (twice!) with no ill effects to uptime or compliance metrics</li><li>Time to release a new service to production reduced from 160 engineer-hours to 1 engineer-hour.</li></ul><p>As soon as the migration from the old system to Kubernetes was complete, engineers were able to deploy. The first engineer got in touch minutes after the migration finished, and within an hour, he had deployed a live service to production in the new Kubernetes environment. Previously, this would have taken two weeks worth of work. Using Houston, the engineer was able to set up routing rules by specifying the domain and path that he wanted to use as soon as the service was live in Kubernetes.</p><blockquote>“[Turbine Labs] saved us. Hands down. It saved us money and time, and it made the transition successful all in all” — Travis Jeppson, Director of Engineering</blockquote><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F1ae1b8%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F1ae1b8%2F&amp;image=https%3A%2F%2Fucarecdn.com%2Fbed08e3d-99ec-443c-96a5-c907bf87794f%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/045b4056cce6d22f68c25d14df1e6fa9/href">https://medium.com/media/045b4056cce6d22f68c25d14df1e6fa9/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e276168dc57" width="1" height="1" alt=""><hr><p><a href="https://medium.com/turbine-labs/how-nav-moved-to-kubernetes-with-houston-by-turbine-labs-2e276168dc57">How Nav Moved to Kubernetes with Houston, by Turbine Labs</a> was originally published in <a href="https://medium.com/turbine-labs">Turbine Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Local Development with Lots of Microservices (Part 1)]]></title>
            <link>https://medium.com/turbine-labs/local-development-with-lots-of-microservices-part-1-c2059a1e9906?source=rss----352a8f053fef---4</link>
            <guid isPermaLink="false">https://medium.com/p/c2059a1e9906</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[development]]></category>
            <dc:creator><![CDATA[TR Jordan]]></dc:creator>
            <pubDate>Tue, 07 Aug 2018 12:31:01 GMT</pubDate>
            <atom:updated>2018-08-07T12:31:00.883Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to use Envoy and Houston to develop on Kubernetes without running every service on your laptop</h4><p>One of the biggest advantages of microservices is that each service has a codebase small enough for you to fit it all in your head. The biggest disadvantage is that there’s more than one service. Unless you’re running a service that’s particularly simple, you need service-to-service communication. It can be difficult to figure out how to run all the dependencies for a service while quickly iterating on the code. If 10 services slow down your laptop, 100 services will make it unusable. Even worse, loading the entire app with development data can make it feel like you’re trying to recreate production on a single computer!</p><p>Generally, people take two approaches to solving this problem:</p><ul><li>Offload development to big machines and run remotely. Workable test data can be a problem to debug, and not everybody likes developing in a terminal on an EC2 machine.</li><li>Provide pre-fab staging environments for individuals to reserve. It still takes time to deploy local changes into any environment, and each environment has to be maintained separately.</li></ul><p>In both cases, hardware costs money, and depending on your budget, these environments may become expensive or a productivity bottleneck.</p><p>One way to fix this is to offload the dependent services to a remote cluster and only run the services that are under active development on your laptop. Done right, this can provide the best of all worlds: quick local iteration, stable dependencies and test data, and only a single shared development environment to maintain.</p><p>Let’s look at how to build this!</p><h3>Calling Remote Services</h3><p>First, let’s set up our laptop to call out to the remote services. We’ll assume that both the local and remote services are running in Kubernetes, but this approach works for any case. Let’s assume a simple service with three dependencies. High-level, its traffic patterns would look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/0*4aEoQ78pZIB8UXyB" /></figure><p>Without the same service discovery mechanisms that your app probably has in prod, the question is, how do requests from a laptop find the right instances in the staging cluster? We can create this bridge with two <a href="https://www.envoyproxy.io/">Envoy </a>instances: one to catch all local traffic and pass it to the remote cluster, and one in the remote cluster to find the running instances and load balance over them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/0*mrxCvu0tww_prRBg" /></figure><p>To see this in action, check out the local dev example on Github, using just two services in a similar model.</p><p><a href="https://github.com/turbinelabs/examples/tree/master/local-dev-kubernetes">turbinelabs/examples</a></p><p>There are three key points this workflow addresses:</p><ul><li>Envoy configuration</li><li>Adding new services</li><li>Keep staging secure (or at least hidden)</li></ul><h3>Envoy Configuration</h3><p>While Envoy can do a ton of fancy tricks to make services more stable, we only need its routing capabilities. In the example code, the Envoy at the edge of the cluster is a straightforward front proxy. Because Kubernetes Pods may restart with new IPs at any time, it is configured using <a href="http://turbinelabs.io/product">Houston</a> and <a href="http://github.com/turbinelabs/rotor">Rotor</a>: Rotor collects the Pod IPs from Kubernetes, passes it to Houston, and Houston generates the routes that Envoy serves. By exposing Envoy as a <a href="https://kubernetes.io/docs/concepts/services-networking/service/#nodeport">NodePort</a>, Kubernetes will give it an external IP, so anybody can hit it. (More on keeping this secure below!)</p><p>To figure out the route from laptop to cluster, we also run a Rotor that collects the NodePort IP and port of the remote Envoy and uses that to configure the local Envoy. This value gets stored in Houston, and whenever the laptop Envoy comes online, it gets the latest value of the IP for the remote cluster. No DNS needed!</p><h3>Adding New Services</h3><p>One of the big problems this request flow solves is that local laptops never need to update their routing to handle new services. The only configuration needed to make this work is that the default outgoing host and port for services should be localhost:80. Megan can add a new Service 5 in the Kubernetes cluster that’s a dependency of Service 1, and when Joe pulls in the latest master to his local copy of Service 1, it’ll automatically send all traffic through the local Envoy and up to Kubernetes, where Service 5 is ready and waiting.</p><p>If Joe wants to test a change to Service 5, he can spin it up on his local laptop and update the local configuration, while Services 2 through 4 still run in in the cloud.</p><h3>Keeping Staging Secure</h3><p>Another problem with staging is keeping it hidden from the outside world, while still allowing developers to get to it easily. There are a couple approaches to securing the cluster without getting in the way of users:</p><ul><li>Put everything on a VPN. This is simple, and it even means you can assign a DNS entry to the front proxy, making it easy to do integration tests against the cluster.</li><li>Use Envoy’s ext_authz to validate requests from users. These could even be automatically added to developers laptops using the request_headers_to_add option on their laptop Envoys, though if your application already relies on ext_authz for authentication, this could get messy.</li><li>Distribute client certificates to laptops and validate them in Envoy.</li></ul><p>In all cases, these can be dynamically handed out to laptops, cleanly separating access to staging from any development concerns.</p><h3>Trying It Out</h3><p>If you want to get started, check out <a href="https://github.com/turbinelabs/examples/tree/master/local-dev-kubernetes">the github repo</a> and <a href="http://turbinelabs.io/contact">sign up for a Houston account</a> today!</p><p><strong>Coming soon</strong>: sending requests from the development cluster to your laptop. If you want to get notified when it’s out, sign up for our email list!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F35e04f%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F35e04f%2F&amp;image=https%3A%2F%2Fucarecdn.com%2F9c5e5f35-9737-44ec-8aaa-76b453b3f4db%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3af9e28be2aa1a8842cfa806fdd6225e/href">https://medium.com/media/3af9e28be2aa1a8842cfa806fdd6225e/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c2059a1e9906" width="1" height="1" alt=""><hr><p><a href="https://medium.com/turbine-labs/local-development-with-lots-of-microservices-part-1-c2059a1e9906">Local Development with Lots of Microservices (Part 1)</a> was originally published in <a href="https://medium.com/turbine-labs">Turbine Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rotor: A Complete Control Plane for Envoy]]></title>
            <link>https://medium.com/turbine-labs/rotor-a-complete-control-plane-for-envoy-294acb3fc92?source=rss----352a8f053fef---4</link>
            <guid isPermaLink="false">https://medium.com/p/294acb3fc92</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[consul]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[grpc]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[TR Jordan]]></dc:creator>
            <pubDate>Tue, 17 Jul 2018 12:31:01 GMT</pubDate>
            <atom:updated>2018-07-17T15:28:54.266Z</atom:updated>
            <content:encoded><![CDATA[<h4>Rotor now supports user-defined routes, custom domains, and gRPC</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/1*Op3XMMcH4lPUpjuBSKyHTg.png" /></figure><p>Today we’ve released an important update to <a href="http://github.com/turbinelabs/rotor">Rotor</a>. In addition to its existing integrations with Kubernetes, Consul, AWS, and more, Rotor now supports fully custom domains, routes, and protocols from a central configuration file. This means you can take full advantage of Envoy’s dynamic service discovery integrations without re-writing any of your existing Envoy listeners or routes. Simply put, Rotor is now the only full-featured control plane for Envoy with a full suite of service discovery integrations.</p><p>Why do you need a control plane? In modern environments, hosts don’t live for months or years. Cloud VMs may live for days, and Kubernetes pods may only live for minutes. Envoy’s dynamic configuration allows it to keep up-to-date on what infrastructure is available and healthy. Modern apps at any scale need a standardized approach for dealing with this ever-changing infrastructure. Rotor easily integrates with service discovery into a control plane for Envoy to act on.</p><p>But what about everything else? To support the full power of Envoy, Rotor’s new static configuration supports four specific use cases:</p><ul><li>Custom domains and routes</li><li>Additional static clusters</li><li>gRPC and other cluster options</li><li>A path to a more dynamic control plane</li></ul><h3>Domains and Routes</h3><p>Envoy defines the routes it can serve via the Listener Discovery Service (what host and port each domain is served on) and the Route Discovery Service (what paths within that domain map to each upstream). Previously, Rotor defined a default set of domains and routes, but these defaults rarely matched what customers had in production. Particularly when migrating from a web server like NGINX or HAProxy, there are pre-existing routes that need to be ported over to make everything work.</p><p>Rotor now picks up a set of routes in the same format as Envoy’s static config files with the ROTOR_XDS_STATIC_RESOURCES_FILENAME=&lt;config file&gt; environment variable, which can contain a set of listeners and routes. A simple set of routes might look like:</p><pre>listeners:<br>- address:<br>    socket_address:<br>      address: 0.0.0.0<br>      port_value: 80<br>  filter_chains:<br>  - filters:<br>    - name: envoy.http_connection_manager<br>      config:<br>        codec_type: AUTO<br>        stat_prefix: ingress_http<br>        route_config:<br>          virtual_hosts:<br>          - name: backend<br>            domains:<br>            - &quot;example.com&quot;<br>            routes:<br>            - match:<br>                prefix: &quot;/service/1&quot;<br>              route:<br>                cluster: service1<br>            - match:<br>                prefix: &quot;/service/2&quot;<br>              route:<br>                cluster: service2<br>        http_filters:<br>        - name: envoy.router<br>          config: {}</pre><p>The clusters’ names (service1 and service2) would have to match the label on those hosts (e.g. tbn_cluster: service1). See the <a href="http://github.com/turbinelabs/rotor">Rotor documentation</a> for more information on connecting your service discovery regime.</p><h3>Additional Static Clusters</h3><p>In an ideal world, all services use the same template and same format and behave the same way. In a real app, sometimes there are exceptions that must be hardcoded. Maybe there’s a service that still lives in a different cloud provider, or one that hasn’t been added to the service discovery yet. Rotor makes it easy to add a set of clusters that are statically defined. In the same way, add a set of Envoy <a href="https://www.envoyproxy.io/docs/envoy/latest/api-v2/clusters/clusters">clusters</a> with the ROTOR_XDS_STATIC_RESOURCES_FILENAME=&lt;config file&gt; environment variable, under clusters:</p><pre># A cluster with a hardcoded DNS <br>clusters:<br>- name: service1<br>  connect_timeout: 0.25s<br>  type: LOGICAL_DNS<br>  lb_policy: ROUND_ROBIN<br>  http2_protocol_options: {}<br>  hosts:<br>  - socket_address:<br>      address: olddomain.example.com<br>      port_value: 8888</pre><p>By default, static clusters are merged with any clusters read from service discovery. In the case of conflicts, the static file wins.</p><h3>gRPC and other options</h3><p>Sometimes it’s best to let all clusters be dynamically defined, but with certain overrides. But, how do you define certain aspects of a cluster without knowing the cluster beforehand?</p><p>Templates! By setting a cluster_template in the static file defined by the ROTOR_XDS_STATIC_RESOURCES_FILENAME=&lt;config file&gt; variable, you can set options. For instance, setting http_protocol_options: {} enables gRPC:</p><pre>cluster_template:<br>  connect_timeout: 0.25s<br>  lb_policy: RING_HASH<br>  type: EDS<br>  eds_cluster_config:<br>    eds_config:<br>      api_config_source:<br>        api_type: GRPC<br>        cluster_names:<br>        - tbn-xds<br>        refresh_delay: 120.000s</pre><p>Now, Rotor will override the dynamic clusters with these values, allowing you to use dynamic EDS with pre-set cluster configuration. In this example, it configures all the clusters to use Rotor for EDS, while overriding the connection timeout, load balancing policy, and refresh delay.</p><h3>A Path to a Dynamic Control Plane</h3><p>While specifying routes in a config file is a great place to start, there are three paths to make your routing definitions as dynamic and flexible as your endpoint definitions:</p><ol><li>Add your config file to version control and periodically re-deploy Rotor with config changes. Envoy will pick up the change and reload the configuration with no downtime.</li><li>Use Rotor as a starting point for your own dynamic control plane. For instance, you could store the routes in a MySQL database and provide an API to change them.</li><li>Get the best of both worlds and use <a href="http://turbinelabs.io/product">Houston</a>, Turbine Labs’ management plane. Just add a Houston API key and Rotor will serve the routing configuration you define in Houston. You can ever run multiple Rotors to bridge multiple types of infrastructure, which helps you with projects like migrating safely to Kubernetes.</li></ol><p>Check out the project on <a href="http://github.com/turbinelabs/rotor">Github</a>, or sign up for <a href="http://turbinelabs.io/contact">Houston</a> today!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=294acb3fc92" width="1" height="1" alt=""><hr><p><a href="https://medium.com/turbine-labs/rotor-a-complete-control-plane-for-envoy-294acb3fc92">Rotor: A Complete Control Plane for Envoy</a> was originally published in <a href="https://medium.com/turbine-labs">Turbine Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Basic Service Mesh with Envoy]]></title>
            <link>https://medium.com/turbine-labs/a-basic-service-mesh-with-envoy-71d16bb7347d?source=rss----352a8f053fef---4</link>
            <guid isPermaLink="false">https://medium.com/p/71d16bb7347d</guid>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[TR Jordan]]></dc:creator>
            <pubDate>Tue, 26 Jun 2018 12:46:01 GMT</pubDate>
            <atom:updated>2018-06-26T17:01:32.054Z</atom:updated>
            <content:encoded><![CDATA[<h4>Using sidecars for only outbound traffic</h4><p>The recent popularity of microservices has made the need for safe, reliable service-to-service communication more apparent than ever. Envoy’s lightweight footprint, powerful routing constructs, and flexible observability support make it a great proxy to build a service mesh on. In this configuration, Envoy acts as the primary load balancer for requests between internal services. Managing this public traffic — also called “East/West” traffic — is trivial in an environment with a small number of services, but with tens or hundreds of services making calls to one another, it’s combinatorially more complex.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-a9IHFWRbgTkDV77W-xnog.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/z6CcN8rlftY?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Ricardo Gomez Angel</a> on <a href="https://unsplash.com/search/photos/mesh?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>That’s where the Envoy service mesh comes in. This is a complementary deployment to a <a href="https://blog.turbinelabs.io/deploying-envoy-as-a-front-proxy-5b7e0a453f65">Front Proxy</a>, where Envoy handles traffic from the outside world (aka North/South traffic).</p><p>A basic Service Mesh uses Envoy sidecars to handle outbound traffic for each service instance. This allows Envoy to handle load balancing and resilience strategies for all internal calls, as well as providing a coherent layer for observability. Services are still exposed to the internal network, and all network calls pass through an Envoy on `localhost`. Rolling out a basic Service Mesh can be done one service at a time, making it a practical first step for most Envoy deployments.</p><h3>Envoy as a Sidecar</h3><h4>Kubernetes</h4><p>Kubernetes makes adding Envoy sidecars easy. You’ll need to do two things:</p><ul><li>Add an Envoy container to your Pod spec</li><li>Modify your services to send all outbound traffic to this sidecar</li></ul><p>How you want to configure Envoy will vary depending on your environment — more on that below. If you want to use fully dynamic configuration, you can use a container like <a href="https://github.com/turbinelabs/envoy-simple">envoy-simple</a> and set the location of the <a href="https://blog.turbinelabs.io/connect-your-service-discovery-to-envoy-6fbcd1dd0d2a">various</a> <a href="https://www.learnenvoy.io/articles/routing-configuration.html">configuration services</a> with environment variables.</p><pre>- image: turbinelabs/envoy-simple:0.17.2<br>  imagePullPolicy: Always<br>  name: envoy-simple<br>  ports:<br>    - containerPort: 8000<br>  env:<br>  - name: ENVOY_XDS_HOST<br>    value: &quot;rotor.default.svc.cluster.local&quot;<br>  - name: ENVOY_XDS_PORT<br>    value: &quot;50000&quot;</pre><p>Since Pods share the same notion of localhost, you can simply change your service to have them call localhost on port 8000 with the correct Host header set, instead of calling the remote service. If you’re using Kubernetes’ Services, you can override the environment variables (e.g. SERVICENAME_SERVICE_HOST and SERVICENAME_SERVICE_PORT) or Kubernetes’ DNS with your Envoy’s listener value.</p><h4>Other Environments</h4><p>Outside of Kubernetes, you have much more flexibility in how you deploy Envoy. You can run either the Envoy container or the binary on your hosts. Similar to Kubernetes, by running Envoy on localhost, you only have to change your services to communicated with Envoy on the port you specify.</p><p>Docker, listening on port 8000:</p><pre>$ docker run -d \<br>  -e &#39;ENVOY_XDS_HOST=127.0.0.1&#39; \<br>  -e &#39;ENVOY_XDS_PORT=50000&#39; \<br>  -p 8000:8000 \<br>  turbinelabs/envoy-simple:0.17.2</pre><p>Next, you should modify your services to route through Envoy. If changing their configuration or code isn’t possible, you can force all outbound traffic through Envoy with something like /etc/hosts or iptables.</p><h3>Sidecar Configuration</h3><p>The easiest way to get started is to have Envoy handle traffic just the same as your internal network does. Practically, this means three things:</p><ul><li><strong>Expose a single listener for your services to send outbound traffic to. </strong>This matches with the port exposed on your container, e.g. 8000 in the example configs above. Inbound traffic skips Envoy and continues to talk directly to the services. Adding a listener to handle incoming traffic will be covered in Advanced Service Mesh (coming soon!).</li><li><strong>Serve the full route table in all sidecars. </strong>By exposing all services to all other services, you’ll ensure nothing breaks on the first iteration. If you have a <a href="https://blog.turbinelabs.io/deploying-envoy-as-a-front-proxy-5b7e0a453f65">Front Proxy</a>, re-using these routes can save time. If not, it’s straightforward to create a <a href="https://www.learnenvoy.io/articles/routing-basics.html">basic set of routes and listeners</a> in a static Envoy configuration file. Once that’s working in production, it may make sense to limit the routes available for each service. The explicit routing between services helps service teams understand where their internal traffic is coming from, helping them define mutual SLOs.</li><li><strong>Consider using dynamic configuration for instance discovery in the first iteration</strong>. Specifically, using <a href="https://www.learnenvoy.io/articles/service-discovery.html">EDS to update Envoy’s notion of available hosts</a> with an EDS server like <a href="https://github.com/turbinelabs/rotor">Rotor</a> keeps Envoy’s routing tables in sync with the underlying infrastructure. Envoy can use static configuration for listeners and routes, so it’s simple and valuable to set up a control plane to manage instance availability.</li></ul><p>If you’ve been following the examples above, you can set up <a href="https://github.com/turbinelabs/rotor">Rotor</a>, an Envoy control plane and service discovery bridge, to implement xDS. Remember that Envoy can mix static and dynamic configuration, so if you want to statically configure listeners, routes, and clusters (LDS / RDS / CDS), you can use your own Envoy container with a static config file while still using a dynamic EDS control plane. Eventually, there are <a href="https://www.learnenvoy.io/articles/routing-configuration.html">good reasons</a> to move to a fully dynamic system.</p><h3>Observability</h3><p>One of the biggest benefits of a service mesh is that it provides a uniform view over your services. Each service will certainly have metrics and tooling unique to it, but Envoy provides a simple way to get the same high-level metrics for all services. Keep the following principles in mind when deciding which metrics to look at:</p><ul><li><strong>Pick metrics that relate to customer experience.</strong> In particular, Envoy can generate request volume, request rate, and latency histograms. Resource metrics like number of connections or amount of network traffic can mean different things on different services. See how <a href="https://blog.envoyproxy.io/lyfts-envoy-dashboards-5c91738816b1">Lyft does it here</a>.</li><li><strong>Segment simple metrics instead of picking many types of metrics.</strong> Envoy can produce a stunning number of metrics. Teams with lots of services tend to get more value out of a small set of metrics, segmented by service, instance, and region.</li><li><strong>Add tracing in Envoy.</strong> Since Envoy is present at every network hop, it’s guaranteed to capture all intra-instance communication. This means that a single configuration can produce complete traces across the entire mesh. That’s a powerful framework to add more detailed custom instrumentation. Note: you will have to propagate headers through each service to create full traces.</li></ul><h3>Multiple Regions</h3><p>As described in <a href="https://blog.turbinelabs.io/deploying-envoy-as-a-front-proxy-5b7e0a453f65">Front Proxy</a>, you should have one front proxy per datacenter. When setting up a mesh, it’s generally safer to send intra-data center traffic to the remote front proxy, instead of exposing all of the internals to all datacenters. This can simplify incident management as well, because changes to a single region are less likely to affect other regions.</p><p>This also means you should split up the configs. Generally you can do this by running a different control plane in each data center. If you want to run a single control plane, check out the discussion of Locality in Advanced Service Mesh (coming soon!).</p><p>In this setup, you would still map each service to a single Envoy cluster, but instead of including the remote instances, you’d include the remote front proxy as the out-of-zone instance in the cluster.</p><h3>Next Steps</h3><p>While this article has focused on how to handle traffic between services, it’s also possible for Envoy to handle traffic from the public internet (“North/South” traffic) as a <a href="https://blog.turbinelabs.io/deploying-envoy-as-a-front-proxy-5b7e0a453f65">Front Proxy</a>. The service mesh and front proxy have a lot of overlapping features, so it can be useful to consider how to roll them both out.</p><p>Beyond that, you can set up Envoy to also handle incoming traffic on each node within your service mesh. This gives better isolation between services and better observability.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=71d16bb7347d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/turbine-labs/a-basic-service-mesh-with-envoy-71d16bb7347d">A Basic Service Mesh with Envoy</a> was originally published in <a href="https://medium.com/turbine-labs">Turbine Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying Envoy as a Front Proxy]]></title>
            <link>https://medium.com/turbine-labs/deploying-envoy-as-a-front-proxy-5b7e0a453f65?source=rss----352a8f053fef---4</link>
            <guid isPermaLink="false">https://medium.com/p/5b7e0a453f65</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[data-center]]></category>
            <category><![CDATA[ssl]]></category>
            <category><![CDATA[load-balancing]]></category>
            <dc:creator><![CDATA[TR Jordan]]></dc:creator>
            <pubDate>Tue, 12 Jun 2018 12:31:01 GMT</pubDate>
            <atom:updated>2018-06-12T20:03:21.189Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to handle SSL, multiple data centers, and more when running Envoy at the edge</h4><p>One of the most powerful ways to deploy Envoy is as a front proxy. In this configuration, Envoy acts as the primary load balancer for customer requests from the public internet. Managing this public traffic — also called north-south traffic — is something that every web app must do, so it’s a natural place to start with Envoy. This kind of deployment style is also called an edge proxy or simply a load balancer.</p><p>Starting with a front proxy is straightforward, because the first step is simply to port whatever existing configuration you have to Envoy. When you deploy your Envoy-based front proxy, you can set it up in parallel to your current setup, giving you the change to test and evaluate Envoy before sending production traffic through it. Once Envoy is stable and tested, it provides a high-leverage place to start enabling Envoy’s more advanced features around resilience and observability for your whole organization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*1FTPQpWIot540Ca9" /></figure><h3>Deploying a Front Proxy</h3><p>Though Envoy is capable enough to be deployed right at the edge of your network, most public cloud providers expose layer 3/4 load balancers with more capabilities than you need. The typical deployment will use<a href="https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html"> AWS’ NLB</a>, <a href="https://cloud.google.com/compute/docs/load-balancing/network/">GCP’s Regional NLB</a>, or a minimal configuration of a more feature-rich load balancer like AWS ALB. These baked-in load balancers can handle regional or global traffic balancing, which is impossible for a set of VMs running Envoy (which are necessarily constrained to a single availability zone) to handle.</p><p>A simple, robust deployment of an Envoy front proxy uses an autoscaling group of Envoys based on network traffic. Based on where your routing rules currently live (NGINX config files, AWS ALB configuration, etc.), you will need to port these to a set of Envoy routes. See <a href="https://www.learnenvoy.io/articles/routing-configuration.html">Routing Configuration</a> for more details.</p><p>Most modern apps will want to implement a dynamic control plane, since the instances within each service are dynamic. A static configuration can only point to a fixed set of instances, while a dynamic control plane can keep Envoy up-to-date on the state of your environment, typically by reading from your service discovery registry. See <a href="https://blog.turbinelabs.io/integrating-service-discovery-with-envoy-19d871e6d0ad">Integrating Service Discovery with Envoy</a> for implementations that will do this for you, like <a href="http://github.com/turbinelabs/rotor">Rotor</a>.</p><p>For an example of how this would work in AWS, see <a href="https://github.com/turbinelabs/examples/tree/master/rotor-nlb">this repository, which uses AWS, CloudFormation, and Rotor</a>.</p><h3>Deploying Envoy in Kubernetes</h3><p>If you’re in Kubernetes, you can point NLBs directly to a an exposed Kubernetes service in front of an Envoy deployment.</p><p>Here’s what that Deployment might look like. It uses <a href="https://github.com/turbinelabs/envoy-simple">envoy-simple</a>, a Docker container that allows Envoy to be fully dynamically configured by specifying a control plane implementation via environment variables. Simply change ENVOY_XDS_HOST to the network location of your control plane (typically another Kubernetes service).</p><pre>apiVersion: extensions/v1beta1<br>kind: Deployment<br>metadata:<br>  name: envoy-front-proxy<br>spec:<br>  replicas: 2<br>  template:<br>    spec:<br>      containers:<br>      - image: turbinelabs/envoy-simple:0.17.2<br>        imagePullPolicy: Always<br>        name: envoy-simple<br>        ports:<br>          - containerPort: 80<br>        env:<br>          - name: ENVOY_XDS_HOST<br>            value: &quot;control-plane.svc.cluster.local&quot; # Wherever your xDS implementation lives<br>          - name: ENVOY_XDS_PORT<br>            value: &quot;50000&quot;</pre><p>Assuming you’ve saved this file as envoy.yaml, you can create a load balancer in your cloud provider with kubectl.</p><pre>$ kubectl create -f envoy.yaml<br>$ kubectl expose deployment --type=LoadBalancer --port=80 envoy-front-proxy</pre><p>You can also use an ingress controller like <a href="https://github.com/heptio/contour">Contour</a> if you want to manage everything through Kubernetes. These expose Envoy’s configuration as Kubernetes <a href="https://kubernetes.io/docs/concepts/services-networking/ingress/">Ingress Resources</a>. This is simple, but less expressive than configuring Envoy through a general-purpose control plane, as the Kubernetes ingress controller spec is lowest-common-denominator by design, with support for only a subset of Envoy’s capabilities. There are also Kubernetes-native API Gateways like <a href="https://github.com/datawire/ambassador">Ambassador</a> that offer expanded functionality by using other, more expressive Kubernetes resources for configuration.</p><p>Note that while running Envoy in Kubernetes is simpler if you’re committed to Kubernetes, many people start by running Envoy outside of Kubernetes in order to manage the migration. Running Envoy outside of Kubernetes ensures that any cluster failures doesn’t take down your Envoy front proxy. If you leave Envoy running outside the cluster after completing you migration, it also gives you flexibility to keep running other orchestrators, or even migrate to the thing that replaces Kubernetes!</p><h3>SSL and Metrics</h3><p>Front proxies are a natural place to terminate SSL, to ensure that individual services don’t have to. You can either do in your cloud’s load balancer (e.g. AWS ELB) or in Envoy itself. To configure SSL, see <a href="https://blog.turbinelabs.io/setting-up-ssl-with-envoy-f7c5aa06a5ce">this article</a>.</p><p>As a common choke point for all traffic, front proxies are a great place to generate high-level metrics for your site. Make sure to at least send request health metrics to your dashboards: latency, volume, and success rate. Coarsely, this gives you a simple dashboard for “is the site up?” and partitioning the traffic by upstream cluster can give you health metrics for each service that will be more relevant to the service teams.</p><p>If you generate logs from your current front proxy, Envoy can be configured to send request-level access logs to a centralized server. Be aware that the volume of traffic at the edge can make logging expensive or slow, so this may not be cost-effective if it’s not a critical capability.</p><p>Similarly, Envoy provides drop-in configuration for tracing, and and the front proxy is a great place to generate those tracing headers.</p><h3>Multiple Data Centers Made Simple</h3><p>There are two strategies for running services in multiple zones (availability zones, regions, data centers, etc.). You can pick a single strategy, or you can run both hierarchically.</p><p>If you have <strong>a single network space</strong>, your best bet is to use Envoy’s <strong>Zone Aware Routing</strong> to handle any underlying discrepancies in the topology seamlessly. This strategy is best used with closely connected zones like AWS Availability Zones, where multiple zones are operationally a single unit, but cross-zone traffic incurs a cost. In this setup, your network load balancer would balance across a front proxy that spans multiple zones, and Envoy would keep each request in the zone it was first load balanced to. This will minimize cross-zone traffic (which is expensive and slow), while maintaining the ability to incrementally fall back to out-of-zone instances if the local ones are failing.</p><p>Setting up zone-aware load balancing requires two options to be set:</p><ol><li>Each cluster (statically defined or returned by the control plane via CDS) must enable zone-aware routing.</li><li>Each Envoy must set `local_cluster_name` to one of the named clusters.</li></ol><p>Statically defined, this looks like:</p><pre>local_cluster_name: service1<br>...<br>clusters:<br>  - name: service1<br>    zone_aware_lb_config:<br>      min_cluster_size: 3</pre><p>On the other hand, <strong>fully isolated zones</strong> should be configured with <strong>entirely separate service discovery integrations</strong>. Instead of giving each Envoy the full routing table with all instances, it’s better to only enable each Envoy to route to instances in its own zone. To enable these zones to fail over gracefully, you can add remote front proxies to each cluster with a low weight. This “backhaul” path will allow automatic failover while allowing zones to maintain different internal configurations. For example, if you are doing a release in the EU region, having the US region fail to the EU’s front proxy keeps the US region from inadvertently routing too much traffic to the new version if there’s a failure mid-release.</p><h3>Best Practices</h3><p>Once you have a front proxy up and running, there are several ways to make operating it easier for you, your team, and the rest of the company.</p><ul><li><strong>Make routing self-serve. </strong>Microservices mean a lot more changes, and service teams should be part of the process. Add guardrails (either people process or RBAC-style rules) to prevent things from breaking.</li><li><strong>Centralize authentication at the edge. </strong>Adding a single authentication call at the edge prevents every service from re-verifying the user each time. Take the cookie from the user, verify it with the authentication service, and store the result (including any commonly used data like a user ID) in a header that gets passed to internal services and trusted without further verification.</li><li><strong>Add protections from bad actors</strong> <strong>and unexpected traffic spikes</strong>: <a href="https://blog.turbinelabs.io/automatic-retries-f5dc3ff2e700">retries</a>, <a href="https://blog.turbinelabs.io/healthy-hosts-in-envoy-ccb051f99b9c">health checks</a>, etc. What this looks like depends strongly on your infrastructure and the types of issues you’re looking to mitigate.</li></ul><h3>Next Steps</h3><p>While this article has focused on how to handle traffic coming from outside your network, it’s also possible for Envoy to handle traffic between services (or “east-west” traffic). For a lightweight way to set up this internal mesh, you can route internal requests through this front proxy (or a similarly configured proxy pool specifically for east-west).</p><p>Beyond that, you can take better advantage of Envoy’s unique features as a lightweight sidecar by setting up a Basic Service Mesh — post coming soon!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5b7e0a453f65" width="1" height="1" alt=""><hr><p><a href="https://medium.com/turbine-labs/deploying-envoy-as-a-front-proxy-5b7e0a453f65">Deploying Envoy as a Front Proxy</a> was originally published in <a href="https://medium.com/turbine-labs">Turbine Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Connecting Rotor to Houston]]></title>
            <link>https://medium.com/turbine-labs/connecting-rotor-to-houston-f9514a4686ad?source=rss----352a8f053fef---4</link>
            <guid isPermaLink="false">https://medium.com/p/f9514a4686ad</guid>
            <dc:creator><![CDATA[Mark McBride]]></dc:creator>
            <pubDate>Thu, 07 Jun 2018 12:31:01 GMT</pubDate>
            <atom:updated>2018-06-07T12:31:01.010Z</atom:updated>
            <content:encoded><![CDATA[<h3>Overview</h3><p>In a <a href="https://blog.turbinelabs.io/replacing-aws-application-load-balancers-with-envoy-and-rotor-5441f540b1da">previous post</a> we showed how to replace an ALB with the <a href="https://envoyproxy.io/">Envoy proxy</a> and <a href="http://github.com/turbinelabs/rotor">Rotor</a>, our lightweight service discovery bridge. Rotor provides a simple, straightforward way to get started with Envoy. Connecting it to <a href="https://www.turbinelabs.io/product">Houston</a> gives you a full-featured, collaborative traffic management solution.</p><h3>Before You Start</h3><p>This post builds on the <a href="https://blog.turbinelabs.io/replacing-aws-application-load-balancers-with-envoy-and-rotor-5441f540b1da">replacing ALB with Envoy + Rotor</a> post, so make sure you’ve completed that. You’ll also need a working Golang installation, and a <a href="https://www.turbinelabs.io/contact">Houston account</a>. <a href="https://github.com/turbinelabs/examples/tree/master/rotor-nlb">This repository</a> contains all the CloudFormation (and Terraform) files you’ll need.</p><h3>Preparing Houston</h3><p>Rotor can be connected to Houston by simply adding an API key, but we’ll need to prepare Houston’s data model. First, install tbnctl, the CLI for configuring Houston, as specified in the <a href="https://github.com/turbinelabs/tbnctl">README here</a>. Next, log in and create an access token:</p><pre>tbnctl login</pre><pre>tbnctl access-tokens add “nlb-rotor key”</pre><p>Save the value of signed_token.</p><p>Next, we’ll create a zone, a domain and routes to route traffic to your services. Find the domain name of your CloudFormation network load balancer, and save it in an environment variable. Then use `tbnctl init-zone` to create a Zone for your service.</p><pre>export TBN_DOMAIN=yourdomain.example.com</pre><pre>tbnctl init-zone --routes=$TBN_DOMAIN/=client --routes=$TBN_DOMAIN/api=server --proxies=default-cluster=$TBN_DOMAIN default-zone</pre><h3>Connecting to Houston</h3><p>Now you can modify our CloudFormation stack to provide Rotor with a Houston API key.</p><ul><li>Click on your Envoy CloudFormation stack &gt; Update Stack</li><li>Select “upload a template to Amazon S3” and choose `cloud-formation-houston-envoy.yaml`</li><li>Click Next</li><li>Provide the signed token you saved as the value of RotorApiKey and enter “default-zone” for RotorZoneName</li><li>Click Next twice, then Update.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6U7UnqHkJTuOujejktRcdQ.png" /><figcaption>Updating the stack to provide your Houston API Key and zone name</figcaption></figure><p>AWS will replace Rotor with a new, Houston-enabled version.</p><p>You should be able to visit your NLB domain name now and see our blinking lights demo. Logging into Houston and navigating to “default-zone” will show you customer-centric metrics for your application, as well as a log of configuration changes. Feel free to experiment with the full breadth of our traffic management controls, or try your own blue-green deploy by launching a new client autoscaling group that returns a different color!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/534/1*rfRLlXuvRCrGaxkqcaoITA.png" /></figure><p>If you’ve set up this project in a VPC with other services, you can also expose them to the public internet via NLB and Envoy, too! By adding a tag to the instance, Houston will collect your existing services that are running in EC2. From there, you just have to define the URLs you want to service via NLB/Envoy by creating routes in Houston.</p><h4><strong>Wrap up</strong></h4><p>By adding an API key, we’ve taken Rotor from a great CDS &amp; EDS server to a full traffic management solution. To collect your services and keep setting up Houston, see our documentation on labelling <a href="https://docs.turbinelabs.io/advanced/ec2.html#labeling-your-ec2-services">EC2 instances</a>, then follow our <a href="https://i.chmln.co/c/AAu62pMdK9eH1Thf">domain setup guide in Houston</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f9514a4686ad" width="1" height="1" alt=""><hr><p><a href="https://medium.com/turbine-labs/connecting-rotor-to-houston-f9514a4686ad">Connecting Rotor to Houston</a> was originally published in <a href="https://medium.com/turbine-labs">Turbine Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>