<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Morgan Peat on Medium]]></title>
        <description><![CDATA[Stories by Morgan Peat on Medium]]></description>
        <link>https://medium.com/@mogpeat?source=rss-ba9105556107------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Ji-bOKmcNp-rK5jMaYgWFA.jpeg</url>
            <title>Stories by Morgan Peat on Medium</title>
            <link>https://medium.com/@mogpeat?source=rss-ba9105556107------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 18:07:44 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mogpeat/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Why IaC stalls at scale: six advanced principles for technical leaders]]></title>
            <link>https://itnext.io/why-iac-stalls-at-scale-six-advanced-principles-for-technical-leaders-cb6a96d37436?source=rss-ba9105556107------2</link>
            <guid isPermaLink="false">https://medium.com/p/cb6a96d37436</guid>
            <category><![CDATA[compliance]]></category>
            <category><![CDATA[iac]]></category>
            <category><![CDATA[governance]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Morgan Peat]]></dc:creator>
            <pubDate>Thu, 16 Apr 2026 14:24:45 GMT</pubDate>
            <atom:updated>2026-04-16T14:24:45.733Z</atom:updated>
            <content:encoded><![CDATA[<h3>Standardise the workflow, not just the code: six principles for scaling IaC</h3><figure><img alt="Move from chaos to a well-defined paved road" src="https://cdn-images-1.medium.com/max/1024/1*e_wM-mvsM6uFJ1IhQcz81Q.png" /></figure><p>If every infrastructure update still waits for a change board, Infrastructure as Code (IaC) has not transformed your enterprise. It has only moved click-ops into code. The real bottleneck remains the governance workflow around it.</p><p>That is the pattern I keep seeing in customer workshops across EMEA. Teams adopt IaC, but leave the old governance and release processes in place: ticket hand-offs, manual approvals, and case-by-case reviews of routine changes. The result is predictable. Deployment frequency remains low, platform teams become bottlenecks, and compliance teams are forced to inspect the same low-risk pattern again and again.</p><p>The answer is not less governance. The answer is governance applied at the right level. If a reusable infrastructure template is well-designed, tightly controlled, and deployed through a governed path, you do not need to subject every instantiation to the same depth of review. Put the <strong>template</strong>, the <strong>policy rules</strong>, the <strong>identity model</strong>, and the <strong>deployment path</strong> through governance once. Then fast-track changes that stay inside that paved road.</p><p>Introducing IaC alone will not get you there. You need to optimise the workflow end-to-end. For a platform lead, head of cloud, or enterprise architect, the goal is clear: industrialise delivery without weakening control. These six principles show how to change the technology, automation, and release process so that the workflow is secure enough, observable enough, and constrained enough for change and compliance teams to trust.</p><p>That matters even more as enterprises bring AI into infrastructure delivery. AI can draft changes faster than your governance model can absorb them. If you have not already optimised your governance workflow, AI will exacerbate the bottleneck. It will just create an unmanageable queue for change and compliance teams to review.</p><h3><strong>1. Codify the whole change path, not just the infrastructure</strong></h3><p>If you want governance teams to trust a workflow, they need to be able to inspect the whole thing in code. That means the template, the approval logic, the policy checks, the deployment path, and the exception model — not just the infrastructure definition itself.</p><p>Many enterprises stop halfway. They provision with IaC, but still rely on manual tickets, runbooks, or portal changes to finish the job. That breaks the chain of evidence. Reviewers can no longer see the real control path, so they fall back to reviewing each change as if it were new. If you want faster approvals, codify the end-to-end workflow so it can be reasoned about, tested, and audited.</p><h3><strong>2. Make one governed route to production</strong></h3><p>Fast-tracked governance only works when there are no side doors. If administrators can deploy outside the standard route, or release managers can push through changes by exception, compliance teams cannot trust the path enough to streamline it.</p><p>Choose one promotion route from pull request to deployment, and enforce it. Use source control protections, automated checks, time-bound breakglass access, and mandatory reconciliation back into source. This is not bureaucracy for its own sake. It is what makes the workflow predictable. One route means one place to inspect controls, one set of evidence, and one model that change teams can understand.</p><h3><strong>3. Treat templates as governed products</strong></h3><p>You cannot approve a reusable template once if the template behaves like an unpredictable framework. Reusable modules need a clear purpose, opinionated defaults, a narrow input surface, versioning, and one accountable owner. Otherwise, every deployment becomes a bespoke risk assessment in disguise.</p><p>This is where many platform teams create friction unintentionally. They publish highly flexible templates, expose too many variables, and leave consumers to interpret the safe way to use them. Governance teams then have no choice but to keep reviewing each use. Good template design is not just about developer experience. It is a control strategy. A well-governed template reduces cognitive load, creates a secure-by-default Golden Path, and makes reuse safer and approvals easier.</p><h3><strong>4. Standardise the control model around the workflow</strong></h3><p>Governance slows down when every team uses a different IaC language, scanning pattern, and release process. That fragmentation is especially expensive in brownfield, hybrid, and multi-cloud estates. Reviewers cannot build trust in a model they have to relearn every time.</p><p>Pick one primary IaC language and one policy model for most work. Standardise the evidence pack around that workflow: peer review, policy results, identity boundaries, and deployment records. The goal is not purity. The goal is predictability. When the controls look the same across teams, change and compliance functions can approve the pattern rather than conducting a bespoke assessment for each deployment.</p><h3><strong>5. Approve the paved road, not every deployment</strong></h3><p>This is the core shift. Stop asking governance boards to review every routine deployment that uses an approved template in an approved path. Instead, govern the reusable path itself.</p><p>Review the template, the policy-as-code rules, the identity boundaries, and the release workflow. Then classify changes. If a deployment stays inside that approved pattern, fast-track it. If it strays outside the template, the guardrails, or the exception model, send it through deeper review. This gives governance teams more signal, not less control. Instead of spending time on 100 repetitive approvals, they focus on the few changes that actually alter risk.</p><h3><strong>6. Instrument the workflow so control teams can trust it</strong></h3><p>Fast-tracked governance depends on evidence. Change and compliance teams need to know which template version was used, which checks passed, who approved the change, which identity executed it, and whether the deployment stayed inside the approved route.</p><p>This is where automation matters most. Use central audit logs, granular role-based access control (RBAC), immutable records, and policy results that are easy to inspect. As an aside, the same instrumentation also helps with Day 2 concerns such as ongoing maintenance and drift detection. But that’s not the main point here. Its primary job is to make the workflow transparent enough that control teams can approve the system without inspecting every individual change by hand.</p><h3><strong>A note on AI</strong></h3><p>Many leaders are looking to AI to speed up infrastructure delivery. That instinct is sound. However, AI only helps when it operates inside a governed system. If engineers use AI to generate bespoke infrastructure, bypass approved modules, or introduce changes outside the standard release path, you amplify risk and increase review load. You do not fix the bottleneck. You feed it.</p><p>When used properly, AI should follow the same secure-by-default Golden Paths as humans. Constrain it to approved templates, consistently apply policy as code, and follow one governed route to production. In practice, that means AI can help draft low-risk changes, documentation, and testable updates within the paved road, while humans maintain their scrutiny of template changes, exceptions, and anything that alters the risk model.</p><h3><strong>Why this matters in numbers</strong></h3><p>These are not vanity metrics. They point in the same direction as the operating model in this article:</p><ul><li>A 2025 review of Terraformization research cites a 2023 analysis of regulated industries showing <strong>62% less effort required for compliance audits</strong> and <strong>83% fewer change management audit findings</strong> for organisations with mature Terraform implementations. Source: <a href="https://gjeta.com/sites/default/files/fulltext_pdf/GJETA-2025-0120.pdf">Global Journal of Engineering and Technology Advances</a>.</li><li>The same review cites a 2023 Gartner study showing that organisations implementing <strong>policy as code</strong> experienced <strong>78% fewer security incidents related to misconfiguration</strong> and resolved vulnerabilities <strong>3.4 times faster</strong>. It also cites HashiCorp’s 2023 State of Cloud Security Report, which found that organisations using Sentinel with Terraform Enterprise prevented an average of <strong>24.3 high-severity misconfigurations per week</strong> from reaching production. Source: <a href="https://gjeta.com/sites/default/files/fulltext_pdf/GJETA-2025-0120.pdf">Global Journal of Engineering and Technology Advances</a>.</li></ul><h3><strong>What good looks like</strong></h3><p>When these principles are in place, the workflow changes in a meaningful way:</p><ul><li>Dedicated central teams own the templates, the guardrails, and the release path, instead of being inserted into every deployment.</li><li>Delivery teams consume secure-by-default Golden Paths and self-service environments, rather than repeatedly negotiating the same approvals.</li><li>Change and compliance teams review new patterns, exceptions, and template changes instead of routine instantiations.</li><li>Leaders can track lead time, deployment frequency, IaC coverage, and audit readiness using a single operating model.</li></ul><p>That is the real opportunity. IaC does not transform the enterprise on its own. It only starts to pay off when you redesign the workflow around reuse, evidence, and trusted automation.</p><h3><strong>Final thought</strong></h3><p>If you want faster delivery without weakening control, challenge the process around the code. Govern the template, not every instantiation. Govern the path, not every ticket. Optimise the workflow end-to-end, and make <em>reuse</em> a metric you track.</p><p>The same rule applies to AI. If you add AI to a weak workflow, you will only create more changes waiting at the same gates. Fix the workflow first, and AI becomes a useful accelerator inside clear guardrails.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cb6a96d37436" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/why-iac-stalls-at-scale-six-advanced-principles-for-technical-leaders-cb6a96d37436">Why IaC stalls at scale: six advanced principles for technical leaders</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Guardrails at scale]]></title>
            <link>https://itnext.io/guardrails-at-scale-25e27a0601b3?source=rss-ba9105556107------2</link>
            <guid isPermaLink="false">https://medium.com/p/25e27a0601b3</guid>
            <category><![CDATA[cloud-governance]]></category>
            <category><![CDATA[platform-strategy]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[cloud-platform]]></category>
            <category><![CDATA[policy-as-code]]></category>
            <dc:creator><![CDATA[Morgan Peat]]></dc:creator>
            <pubDate>Wed, 22 Jan 2025 16:47:59 GMT</pubDate>
            <atom:updated>2025-01-22T22:46:49.349Z</atom:updated>
            <content:encoded><![CDATA[<p>Best practices for IaC platforms</p><p>Almost every large enterprise seeks to build a platform that allows application teams to spin up infrastructure independently, unencumbered by the organizational bottlenecks of security and compliance teams. They dream that this self-service nirvana will increase agility, accelerate adoption, and give them the kind of competitive edge known only to <a href="https://www.dictionary.com/e/acronyms/faang/">FAANG</a>. <em>“We are now a technology company, not a &lt;insert industry here&gt; company”</em> is a phrase repeated by CTOs aspiring to such a digital transformation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5VJZdwqJMWGnSqtw" /><figcaption>Photo by <a href="https://unsplash.com/@nickpage?utm_source=medium&amp;utm_medium=referral">Nick Page</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>These platforms often work well as the first few teams start development. They can point-and-click their infrastructure, write reusable Terraform modules, and rapidly develop their applications. However, cracks open in the platform as applications move closer to production. Governance, risk, and compliance (GRC) teams start to get involved and must ensure the company stays safe. They need time to make sure that every release passes muster because the consequences are too high not to. Release velocity slows, especially for companies subject to heavy regulations.</p><p>As every platform architect knows, the answer to this problem is guardrails. These come in many forms but have the same aim: to permit developer freedom and agility within the organization’s security boundaries.</p><p>This article discusses different guardrail styles and their pros and cons and offers recommendations on selecting the most appropriate ones. We also provide some pragmatic ways to introduce guardrails into your organization, which will help you achieve the elusive goal of <em>safe self-service</em>.</p><h3>Prepare the foundations — pre-requisites for solid guardrails</h3><h4>Everything as code</h4><p>Humans have bad days. They are rushed, tired, careless, and inconsistent. Asking an infosec expert to sit next to a DevOps engineer and double-check everything they do does not scale well. Instead, organizations use coded checks executed by computers. Computers don’t get tired or become careless; they consistently do what they are asked to. Problems (the ones that you can control) don’t come from the computer, but from the humans who write the code and create bugs.</p><p>But that’s where the guardrails come in. Guardrails are like the automated equivalent of that security colleague, double-checking every change. As more operations are codified, fewer misconfiguration errors will occur. Peer reviews, automated testing, and iterative development will iron out bugs.</p><h4>A single route to production</h4><p>Automated guardrails will faithfully enforce whatever checks you want, but only when they execute. If your guardrails are implemented in <a href="https://developer.hashicorp.com/terraform/cloud-docs/policy-enforcement">Terraform Enterprise</a> but users can still log directly into the cloud provider UI, directly point-and-click to create infrastructure, and bypass Terraform altogether, guardrails won’t run and are effectively useless.</p><p>If you have the time and resources to create and maintain duplicate sets of guardrails, great! Otherwise, it’s best to form a platform-wide opinion on the single route teams can take to deploy infrastructure. Using a single technology stack, like HCP Terraform/Terraform Enterprise, means that guardrails can be implemented within that stack and can’t be avoided.</p><h4>Secure pipelines</h4><p>CI/CD pipelines are nothing new, and it’s rare to find platform teams that don’t use them. But it’s not often that pipelines are <em>genuinely</em> secured. In designing their information security, developers frequently forget to guard against their worst enemy: themselves. When confronted with frustrating guardrails that hinder progress, developers have been known to skip checks or leak credentials to implement a quick, tactical fix. If guardrails are going to be successful, they <em>need</em> to be enforced. If a guardrail is incorrect, it should be confronted directly, not circumvented.</p><h3>Different guardrail styles</h3><p>Choosing <em>when</em> a guardrail is enforced changes the developer experience and an organization’s compliance posture. The most significant difference comes when deciding to enforce guardrails <em>before</em> or <em>after</em> deployment.</p><h4>Inherent controls</h4><p>These guardrails are built directly into a platform; there is no way to circumvent them. They form a very coarse control where platform functionality is simply turned on or off. Examples include:</p><ul><li><strong>IAM permissions</strong> — the robot account used by a CI/CD pipeline only has permission to change the necessary resources</li><li><strong>Organization policy</strong> — centralized controls applied at the organization level in a cloud platform which apply to everything</li><li><strong>Hardcoding</strong> — centrally-controlled IaC scripts where specific parameters are not exposed to the end user, or the inputs are checked to ensure they are valid</li></ul><h4>Golden modules</h4><p>Platform teams often enforce inherent controls using the “Golden module” pattern. Here, an opinionated Terraform module hardcodes the organization’s view on resource configuration. Parameter validation ensures all inputs are within bounds, and hardcoding prevents application teams from deviating from the “golden path”. Use of these golden modules can be enforced by Sentinel policy, or through control of the Terraform module private registry.</p><p>This pattern provides a straightforward, quick way to apply guardrails on application teams and helps them to accelerate development by reusing shared code. But when scaling the platforms, there are some considerations to bear in mind:</p><ul><li>Not every application team may fall into the “golden” pattern. Even if the pattern fits 80% of use cases, how should the platform team enforce guardrails on the rest?</li><li>Adding too many parameters to the module (to cater for all use cases) could make the module hard to use, and reduce the developer experience.</li><li>Supporting multiple “golden paths” leads to complexity, a maintenance burden, and confusion for new application teams (which golden path do I choose?)</li><li>Unable to leave the golden path, application teams may perceive the centrally-owned module as a bottleneck that hinders agility.</li></ul><h4>Preventative controls</h4><p>These are often called “policy as code” and typically execute as part of an automated pipeline before deployment. If the guardrails are not met, the pipeline will fail and prevent infrastructure from being deployed. They can provide more nuance than inherent controls since they are custom scripts: platform teams can apply conditionals and logic to a rule. Preventative controls also provide an excellent fast feedback mechanism, improving the developer experience during the code-commit-deploy cycle.</p><p>Policy as code controls can be set to “warn” mode, where they advise of control breaches but do not prevent deployment. This can help platform teams introduce new controls by gradually ramping up enforcement, notifying and nudging application teams into conformance before they are forced. However, as a long-term strategy, advisory guardrails have no real “teeth” and become background noise, ignored by the application teams they are meant to help.</p><h4>Detective controls</h4><p>As the name suggests, these controls will detect misconfiguration in infrastructure <em>already deployed</em>. Detection is done against deployed infrastructure, either through scheduled scans or upon triggers fired by changes. There are many well-established vendors in this space (e.g., Palo Alto, Qualys, Wiz) that support the major cloud providers plus on-premises infrastructure. Most come with pre-defined rules that help to define a strong security posture quickly and reduce the burden of maintaining a comprehensive set of rules.</p><p>Detective controls are essential for catching any suspicious changes made when the official deployment routes have been circumvented, like when a preventative control was somehow bypassed or a malicious actor gained access to your infrastructure. However, since they execute post-deployment, there is a delay in alerting against non-compliance, which makes them unsuitable as the sole guardrail mechanism for high-risk deployments.</p><h4>Corrective controls</h4><p>Detective controls alert against misconfigured infrastructure and prompt human operators to investigate. Some organizations take this further: they implement automatic intervention to forcefully correct any deployed infrastructure perceived as non-compliant. The aim is to automatically identify and achieve compliance without introducing delays caused by human intervention. However, this can be counterproductive: the effort required to build corrective controls is unlikely to be worth the cost of creating and maintaining them, and the risk of a misconfigured corrective control is exceptionally high. Instead, it’s better to “shift left” and ensure compliance before any infrastructure is deployed by using inherent and preventative controls. Sensible use of shared IaC modules, peer reviews, environment parity, and promotion through non-production environments should be sufficient to catch misconfigured infrastructure. Enforcing guardrails in lower environments gives more opportunity for non-compliance to be identified and acted upon.</p><h3>Choosing the right guardrail</h3><p>No single guardrail type or tool will suffice for every situation, and guardrails may need to change with an organization’s security posture. But these broad approaches can be helpful.</p><h4>Turn off everything that isn’t needed</h4><p>The <a href="https://en.wikipedia.org/wiki/Principle_of_least_privilege">principle of least privilege</a> states that access should only be granted to the minimum resources needed to perform an intended task. Resources that aren’t used can be disabled without impacting consumers and remove any need to provide fine-grained guardrails or controls.</p><h4>Choose an appropriate response to each threat</h4><p>Threat modeling techniques will pinpoint areas of a platform process that could introduce risk or security issues. Each risk can be measured against the likelihood of occurrence and its impact on the organization.</p><p>Measured against your organization’s risk appetite, you may choose to ignore risks where the impact is low or to implement detective controls that will prompt you to take retrospective action. Where the impact is high, you may decide to implement a preventative control that prevents a risk from happening at all.</p><p>For risks with an extremely low likelihood of happening, you may decide that the burden of maintaining a complex control is too high and instead rely upon an off-the-shelf alert from a commercial security scanning tool.</p><h4>Design for your developers</h4><p>Guardrails impact developers’ experience of your platform by providing feedback on what they are and aren’t allowed to do. Inherent and preventative controls provide much earlier feedback than detective ones, which only execute post-deployment. Corrective controls alter infrastructure outside the development lifecycle and cause drift versus the checked-in IaC, confusing and frustrating developers.</p><h3>Applying guardrails to your organization</h3><h4>Assign clear ownership</h4><p>A defined <a href="https://en.wikipedia.org/wiki/Responsibility_assignment_matrix">RACI</a> determines who owns and maintains guardrails. Although an InfoSec governance body may be accountable for guardrails, platform engineers likely have some responsibility for maintenance. Individual PaaS offerings could have a dedicated internal product owner who is accountable for guardrails, as well as the infrastructure resource. Platform consumers must know what guardrails have been enforced and how to influence them, so should be informed or consulted.</p><h4>Applying guardrails to existing platforms</h4><p>Imposing strict new guardrails on your consumers, like a decree from on high, is unlikely to be met with open arms. Thankfully, few developers <em>want</em> to flout the rules and cause risk to their organization! Positioning guardrails as an <em>enabler</em> of a secure platform rather than as a blocker of progress helps to win developers over. Guardrails should nudge developers into compliance and free them of much of the burden of evidencing it. A clear implementation roadmap, good communication, and a gradual “tightening of the screw” will help to improve the perception of any new guardrails.</p><h4>Consider the maintenance burden</h4><p>Out-of-date guardrails are counterproductive and will quickly be circumvented and ignored, so reduce the burden by seeking to minimize the number of custom guardrails and policy-as-code. Coarse-grained controls that (like disabling a whole PaaS product) are simple to maintain. Terraform modules that enforce compliance through hard coding and variable validation can remove the need for complex preventative controls.</p><p>Many third-party vendors maintain suites of detective controls that alert against standard security benchmarks, enabling you to outsource much of the maintenance burden.</p><h3>Realizing the benefits</h3><p>Guardrails against high impact / high likelihood risks will understandably increase an organization’s security posture. However, the most significant benefit comes from improved time-to-market for applications. Most large enterprises have compliance gates that must be passed on the route to live, where application teams must provide evidence that their deployment complies with standards of governance, risk, and compliance (GRC). By codifying the GRC rules into guardrails, the platform automatically and continuously attests that an application meets standards. Instead of attending meetings, filling in forms, and attending approval boards, an application team only needs a “green” CI/CD pipeline!</p><p>Subject matter experts in GRC policy can be used more effectively when they concentrate on a set of central guardrails rather than repetitively examining applications on each release. Any change in GRC rules can be quickly rolled out to all applications, enabling a more nimble, responsive security posture.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=25e27a0601b3" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/guardrails-at-scale-25e27a0601b3">Guardrails at scale</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why developers avoid your platform]]></title>
            <link>https://itnext.io/why-developers-avoid-your-platform-d9aa4a0eae65?source=rss-ba9105556107------2</link>
            <guid isPermaLink="false">https://medium.com/p/d9aa4a0eae65</guid>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[people-process-technology]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[thought-leadership]]></category>
            <dc:creator><![CDATA[Morgan Peat]]></dc:creator>
            <pubDate>Thu, 25 Apr 2024 10:56:10 GMT</pubDate>
            <atom:updated>2024-04-25T11:38:24.862Z</atom:updated>
            <content:encoded><![CDATA[<p>You’ve assembled a platform team and built a self-service, internal developer platform (IDP). The first version is up, but nobody is using it. What’s wrong? This article discusses common problems and ways to address them and offers a way forward that works for everyone.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7EDe4jc8Gc-Zjyb0" /><figcaption>Photo by <a href="https://unsplash.com/@hudsonhintze?utm_source=medium&amp;utm_medium=referral">Hudson Hintze</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In the 1989 movie “Field of Dreams,” Kevin Costner built a baseball diamond into his cornfield because a mysterious voice promised him, <em>“If you build it, they will come.”</em> This phrase became a popular way to describe ambitious IT projects where people believe the outcome is so valuable it will draw users toward it. But what happens if they don’t come? What happens when you invest a vast amount of time and resources, but your users simply stay away and don’t use what you built?</p><p>Here, we talk about this disappointing scenario in the context of cloud platforms. Let’s say you recently built a new self-service platform for your organization. It deploys secure, compliant infrastructure, allowing developers to self-service everything they need. Yet they refuse to use it. Instead, they prefer to follow a DIY approach, managing their own tools and workflows. Worse, maybe the developers choose to avoid the cloud entirely and continue with legacy technologies rather than be forced to use the organization’s platform.</p><p>Fortunately, several ways forward, discussed here, can improve the situation and help you build a platform that works for everyone.</p><h3>They will come if there is no alternative</h3><p>Regardless of the merits of your platform, some developers will not use it if given a choice. They may want to target a different cloud provider or feel that a DIY approach would permit faster delivery times, allowing them to realize the benefits of public cloud more quickly. Multiple, smaller platforms might be manageable for a small organization or a single team with a niche requirement. But at scale, this means no central visibility, reinventing the wheel each time, and a fragmented IT landscape. Maintaining a patchwork of tools and workflows loses economies of scale and makes it harder to achieve the gains that every organization seeks.</p><p>Many platform teams start life in this kind of fragmented technology landscape with a remit to introduce standardization and bring control. Pragmatism dictates a phased approach: it is often not feasible to expect the entire technology organization to move to a new way of working in a single “big bang.” Instead, the platform team works in parallel, providing shared services (like VCS, CI/CD, or observability) that help developers offload non-functional concerns they don’t want to manage independently. If the central platform can offer easy onboarding and promise a low-friction developer experience, teams will soon find that the benefits of using the centralized platform outweigh the cost of running their tools themselves, and they will start integrating into the platform.</p><p>At this point, the platform team could begin to make the platform mandatory. Sometimes enforcement is done bit by bit: first, the VCS system (which everyone uses anyway), then the CI/CD tool (which developers are comfortable with and aligns with industry best practices). Teams that started on their own might choose to onboard to the platform later, as the burden of running tools on their own becomes untenable. It’s important to remember that a mandatory platform shouldn’t mean “my way or the highway.” Developers may need to opt out of specific tools or workflows they can’t use, and they should be allowed to. But if this happens a lot, it’s a sign that your platform doesn’t meet the needs of your users, and you may need to rethink how it works. More generally, if your mandatory platform doesn’t address the needs of your users, you’ll hear soon enough. A feedback loop of developers’ grumbles and gripes should be used positively, with their requirements feeding into a roadmap to ensure the platform team focuses on the most demanded features.</p><h3>Features of a good platform</h3><p><a href="https://www.hashicorp.com/customer-success/professional-services">HashiCorp Professional Services</a> engages with customers to help them use HashiCorp products as best they can. In our line of work, we see many platforms and different operating models. We know employing the right technology is only part of the platform equation. Platform teams must consider people and processes equally, too. Increased adoption is any platform’s first and immediate aim, so let’s dig deeper using the <a href="https://open.ncl.ac.uk/theories/1/technology-acceptance-model/">Technology Acceptance Model</a>, which defines successful adoption as a function of 1/ perceived usefulness and 2/ perceived ease of use.</p><h4>People</h4><p>The focus should initially be on the platform consumers: developers. A platform should streamline their workflows and enable them to self-service the infrastructure they need without unnecessary process pain. Everything they require should “just work,” with a paved road guiding the way. They should also be able to leave that paved road and blaze their own trail, should the need arise.</p><p>Developer experience is a vast topic, and it isn’t possible to do it justice in a short blog post. But some aspects make a big difference. A low barrier to entry is essential, as is maintaining a low “cognitive load” for the platform. Your platform should be simple to understand and use. Do sensible default values mean users can start in a few clicks or lines of code? Is there good (read: up-to-date) documentation with clear examples and samples? Is there a <a href="https://engineering.atspotify.com/2020/08/how-we-use-golden-paths-to-solve-fragmentation-in-our-software-ecosystem/">Golden Path</a> they can follow to get them there?</p><p>Providing targeted, specific education will make it easier for developers to use your platform. Are developers aware of the platform, its capabilities, how to deploy their applications, and how to get help when needed? How can your developers become platform advocates who can spread the word and encourage others? Often, winning the hearts and minds of developers can tip the organization into a better perception of your platform and, from there, into making it a success.</p><h4>Process</h4><p>However easy a process may be to follow, no amount of manual, repetitive work can be called “useful” for your users. Good platforms seek to reduce developers’ <a href="https://sre.google/sre-book/eliminating-toil/">toil</a> through automation. <a href="https://devops.com/devops-shift-left-avoid-failure/">Shifting left</a> in this way helps shorten the software development feedback loop so developers can spend more time discovering errors during development time.</p><p>It’s not just infrastructure provisioning that platform teams can shift left. They can embed rules and checks into the code to secure the infrastructure by default. Compliance and governance checks can be performed against infrastructure patterns (like Terraform modules), significantly reducing the go-live processes that developers must follow. By exploiting the economies of scale that common patterns bring, the processes developers must follow become less invasive and more valuable.</p><p>Automation also increases the ease of use. Platforms can reduce the number of hand-offs between teams. Gates are bottlenecks are removed, giving developers a faster time to market and an easier route to production for their applications.</p><h4>Technology</h4><p>Many tools could prove helpful for your platform, but how well do they fit holistically in the end-to-end developer experience of your platform? You shouldn’t immediately accept re-use of the incumbent pre-platform ones, nor the default ones that come with your cloud provider. Do the tools integrate well, allowing you to compose multiple technologies into a cohesive platform? Do the tools help you automate and reduce developer toil? Can you stack the tooling to encapsulate a single golden path to solve a problem?</p><p>Sometimes, industry-standard tooling can be the easiest to use. User experience aside, <a href="https://nesslabs.com/shiny-toy-syndrome">shiny toy syndrome</a> is real in IT. You’ll find it easier to locate and attract suitable candidates, and they will be able to be productive quickly.</p><h3>Finding a way forward</h3><p>A common theme runs through all these platform features: they help developers do their jobs more efficiently and effectively. Consistent tooling and workflows will enable scale, but platforms need flexibility for those who can’t adopt the standards. Forcing developers to work in a certain way alienates those who can’t, resulting in less adoption of your platform. Platform flexibility and differing consumption models (like No-Ops and DevOps-as-a-Service) deserve a blog article on their own, so I’ll aim to write one soon.</p><p>Clear and consistent two-way communication with developers is vital. What do they need? What blockers prevent them from doing their jobs? What repetitive, manual tasks would they like to automate? Listen to your developers and find out what they need. Then, use their priorities to drive the roadmap.</p><p>As with all major IT projects, it’s best to start small. Create an MVP that addresses a big pain point or significantly benefits your developers. Iterate on the MVP with frequent, small releases, each adding a new feature developers need.</p><p>Developers are customers of your platform, so treat them as such by focusing on the customer experience. Help them to learn and understand your platform through training aids and education. Consider embedding platform engineers into development teams to help them get started. Working closely with developers will aid you in understanding their needs and support them in getting up to speed quickly. A few key people often drive opinions and sentiments and can influence how others perceive your platform. Work to identify these people, engage with them, and get them excited and on board.</p><p>Make an effort to establish continuous feedback from developers. Find out what works for them (or doesn’t) and address their burning issues. Build metrics to measure this success and overall sentiment (e.g., by using <a href="https://en.wikipedia.org/wiki/Net_promoter_score">net promoter score</a>) and employ them as part of the platform team’s KBOs or performance targets. By rewarding and incentivizing the platform team to serve your customers, you build a culture that works to create a platform that genuinely addresses customer needs.</p><p>Finally, consider bringing in outside help who has experience building good platforms. The HashiCorp professional services team can advise you and help develop a platform that addresses developer needs, enables scale, and keeps your organization safe and secure.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d9aa4a0eae65" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/why-developers-avoid-your-platform-d9aa4a0eae65">Why developers avoid your platform</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Credential-less authentication to HashiCorp Vault from Azure DevOps Pipelines]]></title>
            <link>https://itnext.io/credential-less-authentication-to-hashicorp-vault-from-azure-devops-pipelines-9e92c65fbf82?source=rss-ba9105556107------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e92c65fbf82</guid>
            <category><![CDATA[hashicorp-vault]]></category>
            <category><![CDATA[azure-devops]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <dc:creator><![CDATA[Morgan Peat]]></dc:creator>
            <pubDate>Wed, 19 Jul 2023 15:29:50 GMT</pubDate>
            <atom:updated>2023-07-21T12:36:12.888Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_0chJoMRyJpCltaa" /><figcaption>Photo by <a href="https://unsplash.com/@quinten149?utm_source=medium&amp;utm_medium=referral">Quinten de Graaf</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>CI/CD pipelines often need to access external services, like a cloud provider, to deploy and configure resources. Some credential (a password or access token) is needed to access the external service, which is often stored in configuration within the CI/CD tool.</p><p>Static credentials like these need to be issued by a human, securely stored, and regularly rotated. There is a risk that secrets could leak, giving access to malicious actors. <a href="https://www.reversinglabs.com/blog/secure-your-development-secrets-3-essential-steps">In 2022</a>, 10 million secrets were leaked on GitHub and one in 10 GitHub code authors exposed a secret.</p><p>A better approach is to use a secrets manager like <a href="https://developer.hashicorp.com/vault/docs/use-cases">HashiCorp Vault</a> which can issue short-lived, just-in-time credentials that are automatically revoked when their time expires. If a secret leaks it will be useless to anyone trying to exploit it.</p><p>Using Vault makes sense, but the CI/CD pipeline must authenticate with Vault to read any of these secure, dynamic secrets. This is the so-called “secret zero” problem: that first secret that grants access to all the other secrets.</p><p>The answer here is similar: most CI/CD tools use OIDC and JWT and issue short-lived access tokens that are valid just for the lifetime of a pipeline. Vault can be configured to trust the tokens issued by a CI/CD tool, allowing the token itself to be the means of authentication.</p><p><a href="https://jwt.io/introduction">JWT tokens</a> are more secure than stored, static credentials. The lifetime of a token is very short and is bound to the pipeline which issues it. The token is cryptographically signed by the CI/CD tool to ensure it cannot be tampered with or spoofed. The token contains “claims” which allow the authenticating system (Vault) to authorize which secrets the token should grant access to.</p><p>Instructions on how to use OIDC authentication to Vault for <a href="https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-hashicorp-vault">GitHub Actions</a> and <a href="https://docs.gitlab.com/ee/integration/vault.html">GitLab pipelines</a> are already published and widely used. I couldn’t find anything similar for Azure DevOps, so I found out what to do and wrote it up!</p><p>The authentication flow is outlined below, with more description on how to set it up.</p><ul><li>Each Azure DevOps project has a Service Connection which is backed by a Service Principal and an App registration</li><li>Each Service Connection is configured as a “role” in Vault</li><li>An Azure Pipeline uses the Service Connection to log in to Azure as the configured Service Principal</li><li>The pipeline can retrieve a JWT access token for the Service Principal and present it to Vault</li><li>Vault will call the configured Azure tenant to validate the token</li><li>Once validated, Vault will ensure that any configured claims match those in the token</li><li>If the token is valid and matches the configured claims, authentication will succeed and Vault will return a Vault token</li></ul><h3>Azure DevOps Service Connection</h3><p>Azure Pipelines documentation says that <a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints">Service Connections</a> are used to <em>“… create a connection from Azure Pipelines to external and remote services for executing tasks in a job.”</em></p><p>Azure Resource Manager service connections are the ones I used to authenticate my pipeline with Vault. <a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints#azure-resource-manager-service-connection">The documentation</a> says you should <em>“Use this approach when you need to connect to an Azure account using different credentials from the credentials you’re currently signed in with in Azure Pipelines”</em>. This is exactly what I need: the pipeline should have a consistent identity that can be used to authenticate with Vault. It should not change, regardless of how the pipeline is triggered. The pipeline will use a Service Principal, and Vault can be configured to trust it.</p><p>Instructions on how to create an Azure Resource Manager Service Connection can be found <a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/library/connect-to-azure">here</a>. I chose the simpler ‘automated security’ since this is just a PoC and I’m concerned only with authentication, not what permissions my service principal has in my subscription.</p><p>Once created, the Service Connection Overview page has a link to <em>“Manage Service Principal”</em> which leads to the Azure app registration page. The application (client) ID can be retrieved: this is the ID that will bind the Service Connection to Vault.</p><p>To prove that the Azure pipeline does indeed have a consistent connection to a service principal, I used the <a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/azure-cli-v2">Azure CLI task</a>. This lets the pipeline seamlessly authenticate with Azure and gives the ability to execute commands using the service principal’s identity.</p><p>Azure can issue JWT tokens for a logged-in account(4) which can be used by external systems to verify identity. This is the token I will use to authenticate with Vault.</p><pre>- task: AzureCLI@2<br>  displayName: Azure CLI<br>  inputs:<br>    azureSubscription: &lt;Name of the Azure Resource Manager service connection&gt;<br>    scriptType: ps<br>    scriptLocation: inlineScript<br>    inlineScript: |<br>      az --version<br>      az account show</pre><h3>HashiCorp Vault Configuration</h3><p>Configuring JWT authentication to Vault is straightforward. Vault simply needs the Azure tenant ID so it knows how to validate the tokens it is presented with. Then, for each Service Connection, I authenticate to Vault using the Azure App ID.</p><p>Following the <a href="https://developer.hashicorp.com/vault/docs/auth/jwt#configuration">HashiCorp documentation</a> I used the Vault CLI to configure the JWT backend.</p><pre>$ vault auth enable jwt<br>Success! Enabled jwt auth method at: jwt/<br><br>$ vault write auth/jwt/config oidc_discovery_url=&quot;https://sts.windows.net/&lt;azure-tenant-id&gt;/&quot;<br>Success! Data written to: auth/jwt/config</pre><p>A Service Connection’s App registration ID can be used to configure a Vault JWT role.</p><p>There is a bug in Vault which means the <em>bound_claims</em> map is not recognized on the command line, so a heredoc has to be used: <a href="https://github.com/hashicorp/vault-plugin-auth-jwt/issues/68">https://github.com/hashicorp/vault-plugin-auth-jwt/issues/68</a></p><pre>$ vault write auth/jwt/role/pipeline-abc -&lt;&lt;EOF<br>heredoc&gt; {<br>    &quot;role_type&quot;: &quot;jwt&quot;,<br>    &quot;policies&quot;: [<br>        &quot;default&quot;,<br>        &quot;some-policy-document&quot;<br>    ],<br>    &quot;bound_audiences&quot;: [<br>        &quot;https://management.core.windows.net/&quot;<br>    ],<br>    &quot;user_claim&quot;: &quot;appid&quot;,<br>    &quot;bound_claims_type&quot;: &quot;string&quot;,<br>    &quot;bound_claims&quot;: {<br>        &quot;appid&quot;: &quot;&lt;application-client-id&gt;&quot;<br>    }<br>}<br>heredoc&gt; EOF<br>Success! Data written to: auth/jwt/role/pipeline-abc</pre><h3>Azure Pipeline Configuration</h3><p>It can all now be pieced together by writing an Azure pipeline that can authenticate to Vault using the JWT. I used a short shell script in a pipeline that has both the Azure CLI and Vault CLI installed.</p><p>Note that, on the first run, permissions need to be granted from the Azure Pipelines run.</p><pre>JWT=$(az account get-access-token --query accessToken --output tsv)<br><br>export VAULT_ADDR=https://my-vault-address<br>export VAULT_NAMESPACE=admin/vault-namespace-with-jwt-backend<br>vault write auth/jwt/login role=pipeline-abc jwt=&quot;${JWT}&quot;</pre><p>And Hey Presto! It works!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e92c65fbf82" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/credential-less-authentication-to-hashicorp-vault-from-azure-devops-pipelines-9e92c65fbf82">Credential-less authentication to HashiCorp Vault from Azure DevOps Pipelines</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build a custom Terraform Run Task using Python and AWS Lambda]]></title>
            <link>https://awstip.com/build-a-custom-terraform-run-task-using-python-and-aws-lambda-4a1558ba903b?source=rss-ba9105556107------2</link>
            <guid isPermaLink="false">https://medium.com/p/4a1558ba903b</guid>
            <category><![CDATA[terraform-cloud]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Morgan Peat]]></dc:creator>
            <pubDate>Thu, 11 May 2023 13:42:33 GMT</pubDate>
            <atom:updated>2023-07-13T13:16:39.837Z</atom:updated>
            <content:encoded><![CDATA[<h4>A walk-through explaining how Run Tasks that execute arbitrary code can be integrated to the Terraform workflow</h4><figure><img alt="Image of Run Task workflow is copyright of HashiCorp" src="https://cdn-images-1.medium.com/max/1024/1*wu_bdBHsgIc0aGfIEmgrug.png" /></figure><p>I recently wrote a <a href="https://en.wikipedia.org/wiki/Proof_of_concept">proof of concept</a> run task to find out how they worked and how arbitrary code can be interleaved into the terraform workflow. My PoC is in the below GitHub repository which should be viewed along with this article.</p><p><a href="https://github.com/MorganPeat/run-task-poc">GitHub - MorganPeat/run-task-poc: PoC of a Terraform Cloud custom run task</a></p><h3>What is a Run Task?</h3><p>The image above shows where run tasks can fit into the terraform workflow. Before or after a terraform plan has completed, Terraform (Enterprise or Cloud) can call an arbitrary piece of code and ask it <em>“Is this terraform plan OK to apply?”</em>.</p><p>The run task will execute some logic and decide whether the terraform run passes or fails its validation rules. The terraform run cannot proceed (optionally, depending on config) if the run task says so.</p><p>This can be really useful where governance, risk or compliance rules have to pass before infrastructure can be provisioned. The <a href="https://registry.terraform.io/browse/run-tasks">public terraform registry</a> contains many off-the-shelf run tasks that can be used to enforce code quality, cost management, security analysis, and more.</p><p>But what if you have a requirement that isn’t present in an off-the-shelf run task? Then read on.</p><h3>HashiCorp docs</h3><p>The HashiCorp developer site does a good job of explaining <a href="https://developer.hashicorp.com/terraform/cloud-docs/integrations/run-tasks">how run tasks can be written</a> and integrate to Terraform Cloud (TFC). The associated <a href="https://developer.hashicorp.com/terraform/cloud-docs/api-docs/run-tasks/run-tasks-integration">API documentation</a> explains how the json payloads are formatted and what HTTP status codes to expect.</p><p>I took these docs and implemented my PoC using AWS Lambda.</p><h3>Run Task Workflow</h3><h4>Request Handler</h4><p>The TFC run task logic will send a POST request to an endpoint when a run task is triggered. In my PoC I have a request handler lambda that receives the request, validates the payload, then forwards the request on to a callback handler.</p><p>TFC expects the run task handler to return a 200 OK to confirm that the run task is being executed. Any further updates that the run task wants to report are sent to TFC via PATCH calls.</p><h4>Callback Handler</h4><p>This lambda performs the custom run task checks. Status updates and / or final results can be sent to TFC via PATCH requests. The URL and API token for the PATCH are supplied to the run task in the request payload.</p><h4>Configuring the run task</h4><p>The run task is configured against a TFC Organization or an individual workspace. I use terraform (in tfe.tf) in this PoC, and this is highly recommended.</p><h3>HMAC Validation</h3><p>To add a layer of security to the run task and detect any tampering or interception of the request payload, I am providing a HMAC Key in my run task. TFC will calculate a sha512 sum of the request payload using the HMAC key and provide it to the run task in the X-Tfc-Task-Signature HTTP header.</p><p>I configure my request handler lambda with the same key and use the python <a href="https://docs.python.org/3/library/hmac.html">hmac</a> library to verify the signature.</p><h3>Custom validation</h3><p>In my PoC run task I pull the terraform plan from TFC. The URL for this comes in the request payload. I don’t use the plan here — it’s just to show what is possible. Instead, my run task uses the terraform files themselves.</p><p>For every remote terraform run (plan or apply), files are uploaded to TFC as a <a href="https://developer.hashicorp.com/terraform/cloud-docs/workspaces/configurations">configuration version</a>. This is stored as a binary blob and contains all the terraform files present when the run was started.</p><p>These can be downloaded as a .tar.gz archive via the <a href="https://developer.hashicorp.com/terraform/cloud-docs/api-docs/configuration-versions#configuration-versions-api">configuration versions API</a>.</p><p>My custom run task simply executes a terraform fmt against the run&#39;s terraform files. <a href="https://developer.hashicorp.com/terraform/cli/commands/fmt">The command</a> will return a non-zero error code if the files aren&#39;t formatted correctly, which I then feed back to TFC as the result of my run task via a PATCH HTTP call.</p><p>I don’t pass anything more complicated as the result of my run task other than a status (passed or failed) and a short message (the output of terraform fmt if it failed). It is possible to pass a URL for more details: a possible future enhancement for my PoC. I could create a S3 bucket, stuff the details in there, then pass the bucket&#39;s URL to TFC.</p><p>And there you have it! It should be simple to take this example and execute any arbitrary code against the plan output or terraform files. Obviously, as a PoC, there is a huge amount of “industrialisation” omitted from the code. I have not addressed security concerns, monitoring, or any similar concern. For more help on that, look at the below links.</p><h3>Further reading</h3><ul><li><a href="https://developer.hashicorp.com/terraform/cloud-docs/integrations/run-tasks">Run Tasks Integration</a> on the HashiCorp site explains how to build a custom run task. It’s where I got most of my info.</li><li><a href="https://developer.hashicorp.com/terraform/cloud-docs/api-docs/run-tasks/run-tasks-integration">Run Tasks Integration</a> also on the HashiCorp site details the API for a custom run task — json payload, HTTP status codes.</li><li>AWS have produced a <a href="https://github.com/aws-ia/terraform-aws-runtask-iam-access-analyzer">IAM Access Analyzer</a> custom run task that also uses AWS Lambdas but is much more industrialised than my simple PoC (it uses things like Step Functions, Secrets Manager, WAF integration). My PoC is useful for anyone wanting to know the workings of a custom run task, but this repository provides a way, way better production offering.</li><li>This <a href="https://www.youtube.com/watch?v=1eSiIxVE9Vw">HashiTalks YouTube video</a> goes with the above AWS repo.</li><li>Not forgetting, of course, the excellent AWS docs on <a href="https://docs.aws.amazon.com/lambda/latest/dg/lambda-python.html">Building Lambda functions with Python</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4a1558ba903b" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/build-a-custom-terraform-run-task-using-python-and-aws-lambda-4a1558ba903b">Build a custom Terraform Run Task using Python and AWS Lambda</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploy to GCP from the command line using Terraform Cloud]]></title>
            <link>https://medium.com/@mogpeat/deploy-to-gcp-from-the-command-line-using-terraform-cloud-9e28ce321023?source=rss-ba9105556107------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e28ce321023</guid>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Morgan Peat]]></dc:creator>
            <pubDate>Fri, 04 Feb 2022 17:52:25 GMT</pubDate>
            <atom:updated>2022-02-04T17:52:25.166Z</atom:updated>
            <content:encoded><![CDATA[<h4>Set the foundations of an “enterprise safe” platform that allows developer freedom while enforcing compliance</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m5cbnPkXJBdILPWl-Q3Siw.png" /></figure><p>There are many tutorials and blogs that describe how GCP resources can be deployed using Terraform, and several that describe how to use Terraform Cloud. But I wanted to create a short article that summarises how the familiar command-line syntax terraform apply can be used with Terraform Cloud.</p><p>This simple deployment method can form the basis of something “enterprise safe”: allowing developers complete freedom to develop and deploy infrastrucure while maintaining control and compliance through automated “guardrails”. Terraform Cloud can keep GCP credentials away from developers, ensuring that only “permitted” changes are allowed. It can also enforce preventative controls through it’s <a href="https://www.terraform.io/cloud-docs/sentinel">Sentinel</a> policy-as-code framework, ensuring that all terraform “complies” with any rules set by the organisation.</p><p>So, on with the show…</p><h3>Set up the environment</h3><h4>Create a GCP Project where resources will be deployed</h4><p><a href="https://learn.hashicorp.com/tutorials/terraform/google-cloud-platform-build">This great Hashicorp Learn tutorial</a> describes how to get a GCP Project ready, which means:</p><ul><li>Creating a new project</li><li>Creating a service account with suitable permissions (the broad-brush “editor” role is used)</li><li>Generating a json key for the service account (which Terraform Cloud will use to authenticate with GCP)</li></ul><p>My sample terraform code below creates a GCS bucket, which requires the Resource Manager API to be enabled. That can be done <a href="https://console.cloud.google.com/apis/api/cloudresourcemanager.googleapis.com">in the GCP console</a> (or you can simply let your first terraform run fail and follow the instructions).</p><h4>Create a Terraform Cloud account and workspace</h4><p><a href="https://learn.hashicorp.com/tutorials/terraform/cloud-sign-up">Another good Hashicorp Learn tutorial</a> talks through how to create a Terraform Cloud account and a new organization. I’m using the CLI-driven workflow here, not the VCS-driven workflow that the tutorial describes. So when it comes to creating a new workspace make sure to choose that option.</p><figure><img alt="Choose the CLI-driven workflow for your new workspace" src="https://cdn-images-1.medium.com/max/1024/1*4Rzd1LnIyvbkU4pHGDpDnQ.png" /></figure><h4>Configuring the workspace</h4><p>The <a href="https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/getting_started#using-terraform-cloud-as-the-backend">google terraform provider docs</a> explain how to set up GCP credentials in Terraform Cloud (look for “Using Terraform Cloud as the Backend”). Effectively, you store your <em>one-line</em> json service account key (created earlier) as an environment variable named GOOGLE_CREDENTIALS in the workspace. The google provider will pick up the environment variable and use that SA key to authenticate with GCP.</p><h3>Let’s write some terraform</h3><p>I deliberately kept this terraform snippet simple so it’s clear to see what is happening. There is <a href="https://www.terraform-best-practices.com/">a whole website</a> devoted to terraform best practices, so go there for advice on production-ready terraform code!</p><p>This snippet of code simply: authenticates with GCP (magically, via the GOOGLE_CREDENTIALS environment variable set in the workspace) and deploys a Storage bucket in my project. The terraform cloud block tells terraform that my Terraform Cloud workspace should be used to remotely execute the terraform, rather than running it on my own machine. That’s how terraform knows how to retrieve the GCP credentials.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cbc2735864801d8b8a7b7200b61b4ba2/href">https://medium.com/media/cbc2735864801d8b8a7b7200b61b4ba2/href</a></iframe><h3>Time to spend some money</h3><p>Executing this from the command line won’t go so well the first time you run it. That is because your CLI needs to authenticate itself with Terraform Cloud.</p><figure><img alt="terraform init will fail if you haven’t connected to terraform cloud" src="https://cdn-images-1.medium.com/max/1024/1*hiODIwdaMiu3vkd0YEuYFw.png" /></figure><p>The terraform login command will fix this by generating a <a href="https://www.terraform.io/cloud-docs/users-teams-organizations/users#api-tokens">User API token</a> and saving it to your machine.</p><figure><img alt="terraform login will save a User API token to your machine for future use" src="https://cdn-images-1.medium.com/max/544/1*PfgPI08cb0KSJ0IxFDowOw.png" /></figure><p>Once your CLI is authenticated with Terraform Cloud you should be able to terraform apply and spin up some tin!</p><figure><img alt="terraform apply now works" src="https://cdn-images-1.medium.com/max/671/1*Whb2Z2deXR51beqJeF_i-Q.png" /></figure><h3>Next up…</h3><p>In future articles I will build out this “enterprise ready” approach, describing how to</p><ul><li>Use Hashicorp Vault to generate short-lived credentials</li><li>Use Identity (not secrets) to authenticate in CI/CD pipelines</li><li>Use Hashicorp Sentinel to enforce before-the-fact guardrails</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e28ce321023" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>