<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Leon Wright on Medium]]></title>
        <description><![CDATA[Stories by Leon Wright on Medium]]></description>
        <link>https://medium.com/@techman83?source=rss-d2e3898d5b4d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*UYORsqrQkRlihO8pE0ShZg.jpeg</url>
            <title>Stories by Leon Wright on Medium</title>
            <link>https://medium.com/@techman83?source=rss-d2e3898d5b4d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 12:32:57 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@techman83/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Cloud Developer Kits gloss over their best feature!]]></title>
            <link>https://medium.com/@techman83/cloud-developer-kits-gloss-over-their-best-feature-bb7c896848a7?source=rss-d2e3898d5b4d------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb7c896848a7</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Leon Wright]]></dc:creator>
            <pubDate>Fri, 06 Dec 2024 06:49:28 GMT</pubDate>
            <atom:updated>2024-12-06T08:06:21.045Z</atom:updated>
            <content:encoded><![CDATA[<p>Between the little over two years since I wrote ‘No one should write Terraform’, rewriting large parts of our Internal Developer Platform, and starting on ‘<a href="https://cally.callyco.io/">Cally</a>’; there was something myself and most of the commenters missed. Cloud Development Kits often ignore what I would consider one of the strongest selling points. Decoupling yourself and your engineers from your infrastructure!</p><p>This perspective is certainly biased towards my use case, which is centred around providing an Internal Developer Platform for our internal engineering team, a tool which has been responsible for well over 150,000 deployments since then; and has contributed to the design philosophies around the aforementioned ‘<a href="https://cally.callyco.io/">Cally</a>’.</p><p>The comments and feedback to my last article most frequently were “Have you thought about using X?”, or “have you seen this new tool Y?”. Doing my due diligence, and opening the intro page to find what is code block showing you how you can create an AWS service using your favourite language..</p><figure><img alt="Screenshot of SSTs opening page" src="https://cdn-images-1.medium.com/max/1024/1*ojBmj5aUNRkdFaqgXxJhTA.png" /><figcaption><a href="https://sst.dev/">https://sst.dev/</a></figcaption></figure><p>Now there are some tools that bring the promise of being able to generate infrastructure with your code, and that certainly sounds interesting. If you have a small team / startup, fully responsible for your infrastructure, with team members who are all infrastructure experts as well as software engineers, great! That might be the right tool for the job, but I suspect that in a lot of cases the opinion buried in my last article still applies</p><blockquote>Someone whose day-to-day is working on an application, is not going to have the bandwidth to learn a new tool just to create a bucket. Let alone understand the implications of those IAM binding copy/pasted from Stack Overflow resulting in a configuration making a bunch of private data publicly available.<br> — <a href="https://medium.com/@techman83/no-one-should-write-the-terraform-8ee8e53dedc0">No one should write Terraform</a></blockquote><p>Ultimately all the tools look quite robust, and appear to have converged around either primarily using, or offering the ability to use Terraform/OpenTofu providers. Which realistically is quite sensible, especially if you want to <a href="https://ndmckinley.github.io/terraform-provider-dominos/">order pizza</a> after a long day, or maybe use one of many <a href="https://www.hashicorp.com/blog/hashicorp-terraform-ecosystem-passes-3-000-providers-with-over-250-partners">thousand</a> equally useful providers available. But in my experience, once you have a pattern your Internal Developer Platform is not your bottle neck; it’s understanding how to package one or several of the 100s of services available to you, in a secure, robust, and sensible manner for your team to consume.</p><p><strong>Contracts and Configuration</strong></p><p>The most important aspect of our tooling has been the robust and rich multi layered configuration interface. This provides a consistent, simplified contract for all services managed by our tooling.</p><p>This is an example from Cally:</p><pre>defaults:<br>  providers:<br>    google:<br>      default_labels:<br>        git-repo: my-repo<br>development:<br>  defaults:<br>    providers:<br>      google:<br>        project: my-development-project<br>  services:<br>    test-service:<br>      stack_type: CallyStack<br>      stack_vars:<br>        example: variable</pre><p><strong>The Config YAML File:</strong></p><ul><li>Serves as a contract between engineers and our code</li><li>It is verified using JSONschema, which also generates our documentation.</li><li>Enforces a strict configuration interface, enabling simpler and less defensive code.</li></ul><p><strong>CDK for Terraform</strong></p><ul><li>Establishes a contract between our configuration and Terraform.</li><li>Guarantees consistent output for a given input.</li></ul><p><strong>Terraform</strong></p><ul><li>Acts as a contract between Terraform and the infrastructure.</li><li>Is a robust state management tool when paired with well-written providers.</li><li>OpenTofu can be used as an <a href="https://github.com/opentofu/opentofu/issues/601">alternative</a> to Terraform if preferred.</li></ul><p>The benefits of this approach are that a configuration problem is a configuration problem, a code problem is a code problem, and a terraform problem is a terraform problem. You aren’t chasing your tail trying to figure out if you’ve done something wrong, the config is incorrect, or if the provider has a <a href="https://github.com/GoogleCloudPlatform/magic-modules/pull/7160">bug/limitation</a>.</p><figure><img alt="Spiderman point meme, software engineers pointing at devops engineers pointing at HCL" src="https://cdn-images-1.medium.com/max/560/1*SQC5dLNPRC4c_304qNRjxQ.jpeg" /><figcaption>Who’s at fault??</figcaption></figure><p>When it comes to a tool, pick whichever suits your team and build around it. Whilst I’ve chosen the CDK for Terraform, I spend very little time thinking about it. Most of it has been abstracted away, so that we can focus on how we can provide useful guard-railed access to the infrastructure, because sometimes the raw terraform can be quite dangerous.</p><p>Here’s an example from Google’s IAM docs:</p><blockquote><strong>Be careful! </strong>You can accidentally lock yourself out of your project using this resource. Deleting a google_project_iam_policy removes access from anyone without organization-level access to the project. Proceed with caution. It’s not recommended to use google_project_iam_policy with your provider project to avoid locking yourself out, and it should generally only be used with projects fully managed by Terraform. If you do use this resource, it is recommended to import the policy before applying the change.</blockquote><p>I can assure you, missing that statement can make for an exciting morning. Shoutout to Google’s audit logs though!</p><p><strong>Maintenance Burden</strong></p><p>One thing often glossed over is the maintenance burden of your IaC. The big cloud vendors are releasing often weekly, with big breaking changes every year or so. And that’s without even considering the scenario of being stuck on an <a href="https://developer.hashicorp.com/terraform/language/v1.1.x/upgrade-guides/0-13">older version</a> of the tooling.</p><p>Mid 2021, at version 0.5.0 was when I started experimenting with the CDK for Terraform, and we had a decent amount of production deployments by 0.7.0. Though the provider layout changed in version 0.8.0, we were able to keep using the previous layout until mid last year and 0.17.0 dropped support for the legacy layout. By this stage we had 1000s of resources managed, 50+ different pre-built ‘Stack’ types, and this felt like a daunting task to address. However the only real breaking change that mattered was how Terraform IDs were handled, as they were much less restrictive than they used to be! But with the test coverage we had, it was easy enough to write a filter to ensure any generated Terraform identifiers remained consistent. The rest of the time was spent improving interactions with the underlying CDK for Terraform code, as the original goal of generating IaC for VM based Microservices and the related infrastructure for a single vendor had long since been exceeded.</p><p>The above refactor was preceded by a complete overhaul of the configuration parser. The bespoke multi layered parser had served us well, but there were edge cases that seemed trivial, and weren’t. Largely due to the organic development of an internal experiment, which had become a fundamental tool driving 1000s of deployments a week.</p><p>The changes to the tooling were completed, rolled out, and largely unnoticed by the consumers of the tooling. We’d kept our promise with those consuming it and with Terraform, even though we’d essentially gutted and re-written the internals.</p><p>I largely covered in the <a href="https://medium.com/@techman83/no-one-should-write-the-terraform-8ee8e53dedc0">prior article</a> the benefits of using an imperative language like Python over a declarative one like HCL, so here are some of the tools that ensure we can rapidly iterate our Internal Developer Platform and keep our focus on delivering useful and easy to consume infrastructure.</p><ul><li><a href="https://docs.astral.sh/ruff/">ruff</a> has been an absolute game changer in terms of linting and code quality. It’s extremely fast, and flexible in terms of what rule sets you want to configure.</li><li><a href="https://github.com/psf/black">black</a> allows us to set the standards, and keeps us honest. No more unnecessary discussions about layout or format.</li><li><a href="https://mypy-lang.org/">mypy</a> because sometimes being a little strict can be a good thing!</li><li><a href="https://coverage.readthedocs.io/">coverage</a> whilst 100% coverage isn’t the be all and end all, often unexpected lines not covered reveals bugs that would have gone unnoticed!</li></ul><p>Though Terraform tools do exist, they’re very domain specific. Being able to lean on decades of existing tools, and broadly understood programming concepts to provide strong guarantees that your Internal Developer Platform is robust and reliable, whilst being able to iterate rapidly.</p><p><strong>TL;DR</strong></p><p>Ultimately the decision is yours, pick a tool that suits your use case and language proficiency. Be it sticking with raw HCL or using a CDK like Pulumi, SST, the CDK for Terraform, or one of the many others I’m sure to learn about in the comments.</p><p><a href="https://cally.callyco.io/">Cally</a> is a project I’ve been working on in my spare time, it aims to provide a foundational layer for your Internal Developer Platform, with the goals of addressing what I’ve discussed in this article.</p><p>Key Features</p><ul><li>Simple but rich and <a href="https://cally.callyco.io/configuration">documented configuration</a> layer</li><li>Documentation with <a href="https://cally.callyco.io/concepts">examples</a> to make getting started easier</li><li>Built in <a href="https://cally.callyco.io/api#testing">test harness</a> to provide a starting point for ensuring your IaC has robust test coverage</li><li>The <a href="https://github.com/CallyCo-io/Cally/blob/main/LICENSE">license</a> is deliberately permissive as the goals are to lower the barrier to entry for building an Internal Developer Platform</li></ul><p>It’s still in the early stages, but if you’re interested, check it out! I’d love to find all the bugs that I’ve missed 😀</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb7c896848a7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[‘No one should write Terraform’ — A Follow Up]]></title>
            <link>https://medium.com/@techman83/no-one-should-write-terraform-a-follow-up-49ff92909c0f?source=rss-d2e3898d5b4d------2</link>
            <guid isPermaLink="false">https://medium.com/p/49ff92909c0f</guid>
            <category><![CDATA[infrastructure-as-code]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Leon Wright]]></dc:creator>
            <pubDate>Wed, 02 Nov 2022 07:30:39 GMT</pubDate>
            <atom:updated>2022-11-02T07:30:39.109Z</atom:updated>
            <content:encoded><![CDATA[<h3>‘No one should write Terraform’ — A Follow Up</h3><p>It’s been a few months since I wrote the <a href="https://medium.com/@techman83/no-one-should-write-the-terraform-8ee8e53dedc0">aforementioned</a> article, and the response has been extremely positive and a bit overwhelming!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dbjL9SvCRtXsjKZYhbnj8w.jpeg" /><figcaption>It’s just everything!</figcaption></figure><p>Whilst I have a follow up in the works, life outside has had some challenges, leaving spare energy to write without a spark of ADHD hyperfocus somewhat challenging. The joyful thing being, that in spite of that, iterating and enhancing our <a href="https://developer.hashicorp.com/terraform/cdktf">CDK for Terraform</a> tooling has not slowed down.</p><p>I have had numerous questions via email, linked in, and on the article itself. So as an interim, throw some questions in the comments! I’ll pick through the common themes from this and the previous, collate them up with some short responses over the next week or two.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49ff92909c0f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[No one should write Terraform]]></title>
            <link>https://medium.com/@techman83/no-one-should-write-the-terraform-8ee8e53dedc0?source=rss-d2e3898d5b4d------2</link>
            <guid isPermaLink="false">https://medium.com/p/8ee8e53dedc0</guid>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Leon Wright]]></dc:creator>
            <pubDate>Sat, 13 Aug 2022 09:36:25 GMT</pubDate>
            <atom:updated>2022-08-16T14:49:19.127Z</atom:updated>
            <content:encoded><![CDATA[<p>Reflecting upon <a href="https://zwischenzugs.com/">zwischenzugs</a> insightful and thought provoking article “<a href="https://zwischenzugs.com/2022/08/08/who-should-write-the-terraform/">Who should write the Terraform</a>?”, the TL;DR conclusion I’ve come to is “No One”.</p><p>That’s a pretty simple statement for me to make from the outset, but stick with me and we can go on a journey. Having spent nearly two decades in roles ranging from a Technician fixing student laptops, Sys Admin working with a team managing the operations of a 24/7 Enterprise, being the sole developer building a hosted VoIP solution at a startup; To now working with dozens of software engineers and empowering them to work more efficiently. All of that and I’m a blue haired guy with opinions!</p><figure><img alt="Two Engineers, one with blue hair" src="https://cdn-images-1.medium.com/max/238/1*_Mfpo3x0ct9IIuEhsRqiSg.png" /><figcaption>Colleague: “What’s the difference between a Software Engineer and a DevOps Engineer”</figcaption></figure><h3>The Problem</h3><p>Over the years I’ve used many configuration automation tools. From my days working with Zenworks Configuration Management repackaging windows applications for corporate deployments, writing ansible maintaining Linux boxes, CloudFormation for deploying large swathes of AWS resources, or Terraform to tie resources together in a platform independent way. Regardless of which tool I’ve picked up, I’ve always come up against a fairly similar set of challenges.</p><figure><img alt="Look! He bought a box of DevOps by @acloudguru" src="https://cdn-images-1.medium.com/max/900/1*hH2yZlxeOlFiF_2pFpJZwA.jpeg" /><figcaption>Look, He bought a box of DevOps! — <a href="https://medium.com/u/c73bfd51b440">A Cloud Guru</a></figcaption></figure><p>Now that comic is very tongue in cheek and undersells what can be achieved when embracing DevOps as not just tooling, but as a fundamental way of working with technology. However it neatly sums up the common problem with all of the aforementioned tools, they are purely declarative. Each one of them does offer a domain specific way to introduce ways to be programmatic, but someone whose day-to-day is working on an application, is not going to have the bandwidth to learn a new tool just to create a bucket. Let alone understand the implications of those IAM binding copy/pasted from Stack Overflow resulting in a configuration making a bunch of private data publicly available.</p><p>So what’s the answer here? Do we abandon all these tools and go back to a world of Devs doing Dev things and Sys Admins being grumpy gatekeepers?</p><p>So I must confess, as someone with ADHD I often take things literally. For me, and I don’t mean this as a way of gate keeping or to look down upon someone whose a wizard in their tool of choice; because all those things take considerable skill to do well. But in my brain that was source controlled configuration and I always saw it as a stepping stone to true the end goal of using “Code” via a functional or OO language. Now this is not because I’m some opinionated developer. Rather, I grew up and spent the early part of my career hating programming. As time went on I realised not knowing how to program was holding me back as an Engineer, so I spent hundreds of hours teaching myself to do just that.</p><p>Getting back on track, in a former role we were leaning quite hard on CloudFormation. Even though we were only a team of two, we still faced challenges that I have encountered in bigger teams</p><p><strong>Consistency<br></strong>Though we had similar ways of thinking, we still found it hard to ensure we were both doing the same things. For me personally, ADHD can mean that the Leon that did the thing in the morning is a different person to the Leon doing it later in the day.</p><p><strong>Large Declarative Files<br></strong>Ensuring a couple of thousand lines of YAML/JSON is correctly formatted, with the correct pieces in the right places is hard. That is before you want to start writing large sets of commands as your startup script!</p><p><strong>Duplication of Effort<br></strong>We were spending considerable time doing the same work over and over, which also meant as we got busy, changes were being left off a services due to being nested deep in a YAML files.</p><p><strong>Slow Feedback Loops<br></strong>Changes meant applying updates, updates could be quite consuming. Especially if you made a typo or mistake, meaning you had to wait for some internal timeout. Too many times I just packed up for the day, because some change screwed up the stack and would be stuck for hours.</p><p><strong>Context Switching<br></strong>It was very challenging to spend all day buried in some obscure Asterisk dial-plan issue, to then switch over to figuring out a tooling problem. Now that problem might impact someone with ADHD more so, it is still something everybody faces.</p><p><strong>Configuration vs Code Repositories<br></strong>Due to the large amount of unrelated changes and commits that often come with purely declarative configuration, modules, environment files, permission control, and so on. There is a tendency to separate Configuration from Code. This leads to needing to go to multiple places to understand how your project is deployed, and can lead to configuration items being missed when code changes. Or if your team is really inefficient, developers working around the configuration problem and wedging it in code or a database table.</p><h3>The Solution</h3><p>One day my boss at job[-1] came to me, having gone down a rabbit hole with <a href="https://github.com/cloudtools/troposphere">Troposphere</a>. For those not in the know, this is a library for generating valid CloudFormation via Python. This was it, finally I could do what the industry was talking about all over the place, I could start writing Infrastructure as Code!</p><p>In the following weeks I hyper-focused on solving the challenges that had really been frustrating me. I added AutoScaling, Load Balancers, logging sidecars by default, and way more. The core platform went from having some disparate tools for logging, to automatically getting it as I’d added it to the one of the other services that had a far greater need for centralised logging.</p><p>We slowly but surely spent less time managing an gargantuan collection of YAML files and instead found ways to optimise our workflows, increase our security, tighten up our IAM controls, and get back to focusing on the primary parts of jobs as being the only 2 developers in a fast growing startup.</p><p>When it was time to move on from that role, I happened to be pinged by an internal recruiter for a DevOps role that would involve bringing their Developers and the Platform on a journey towards more efficient processes, allowing to support the growth of E-Commerces version of the ‘<a href="https://en.wikipedia.org/wiki/Eternal_September">Eternal September</a>’ in March of 2020.</p><p>Now as part of that, I was switching from Amazon Web Services to Google Cloud. Troposphere being very much CloudFormation specific and noting that even AWS were starting to invest outside non-vendor specific tools, Terraform seamed like the obvious choice.</p><p>However when I went searching around for even something like Troposphere for GDM that was actively maintained, nothing. And then it hit me, I’d spent the better part of a decade thinking “Infrastructure as Code” was just that, using something like Python to automate infrastructure changes. I was in quite an existential crisis, as having experienced the burdens of purely declarative IaC, I didn’t know what to do!</p><p>At some point I stumbled across the right set of keywords, I found out that the industry had coined the term ‘CDK’ or Cloud Development Kit for the process of using Functional or OO programming languages that developers are already familiar with. So after talking it over with colleagues, vendors, and friends, I got a variety of opinions. Ranging from not really understanding the point, to outright suggesting it was too early to look at the CDK for Terraform, but most importantly support from my boss at the time, to come up with an MVP concept to get us going.</p><h3>Summary</h3><p>It’s been a little over 12 months since I made that first commit, but since then we’ve gone from one type of “Stack” to several dozen. It’s performed &gt; 5000 automated deployments, and has gone from something I envisioned wrapping up our MicroServices, to supporting all sorts of uses cases required from the developers. A moment that really stood out recently was a developer in their second or third week wanted experiment with a new service, and I didn’t even know they were building the configuration until they had a question! I answered and got back to the thing I was working on. I checked back later and their service was built, deployed, with a load balancer, auto-scaling, auto-healing, completely driven via GitHub Actions and no direct Involvement from Ops. The comment I got from them “Yeah, I was a bit confused for about a day, and then it all made sense”, which as someone who was less than a month at the company being able have a brand new service deployed into our test environment. It was likely quite refreshing for them and really rewarding for me.</p><p>In a future article I will delve into the specifics, but three things come to mind when I’ve talked about it with people who I have introduced the concept to</p><ul><li><strong>Fully Unit Tested IaC</strong> — We can change any part of the code that generates our resulting configuration and be confident that what we give to Terraform to apply is correct.</li><li><strong>State Files</strong> — As we don’t need to try and reduce the amount of Stacks we manage, breaking them up into discrete stacks is extremely easy. Any state problems don’t create any wide scale impacts to operations.</li><li><strong>Provider Versions and Availability</strong>— The tools ensure the provider is available with the correct version, migrating them is just part of the tools responsibility. As we can thoroughly test these pathways, applying them wider is as simple as releasing a new version of our internal CDK wrapper.</li></ul><h3>The Real TL;DR</h3><p>In the whole time I’ve been working with Terraform I’ve written &lt; 100 lines of HCL. No one should write Terraform, not even me.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8ee8e53dedc0" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>