<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Eric Sorenson on Medium]]></title>
        <description><![CDATA[Stories by Eric Sorenson on Medium]]></description>
        <link>https://medium.com/@ahpook?source=rss-d2f9578e48a7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*z-64OzuNMmyuBapNMZa_vQ.jpeg</url>
            <title>Stories by Eric Sorenson on Medium</title>
            <link>https://medium.com/@ahpook?source=rss-d2f9578e48a7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 01:39:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ahpook/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A Quick Guide to Developing Steps for Relay]]></title>
            <link>https://medium.com/relay-sh/a-quick-guide-to-developing-steps-for-relay-7da542ec857a?source=rss-d2f9578e48a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/7da542ec857a</guid>
            <category><![CDATA[development]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[Eric Sorenson]]></dc:creator>
            <pubDate>Tue, 13 Apr 2021 18:18:43 GMT</pubDate>
            <atom:updated>2021-04-13T18:18:43.406Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="90s video-game style art showing an epic dais in a beam of light" src="https://cdn-images-1.medium.com/max/1024/1*eZQmuREZAXXacMmJAvsPdw.png" /></figure><p>Relay has a substantial library of external services and tools — as of March 2021 there are 60 integrations in <a href="https://github.com/relay-integrations">our Github organization</a>. Each integration repo can contain multiple <strong>triggers</strong>, containers that receive webhook payloads from other services, and <strong>steps</strong>, which Relay executes to get stuff <em>done</em> in your workflow. Some quick work with fd (see our <a href="https://relay.sh/blog/command-line-ux-in-2020/">post on CLI design</a> for more on the awesome Rust *nix tool replacements!) shows this means there are about 150 different &quot;atoms&quot; of functionality in Relay&#39;s current ecosystem, from <a href="https://relay.sh/steps/jira/issue-create">creating JIRA issues</a> to <a href="https://relay.sh/steps/microsoft-teams/microsoft-teams-step-send-message">sending MS Teams notifications</a>.</p><p>But what if that’s not enough? What if there’s a new service that’s not currently supported or a new API for an existing service that’s key to the workflow you’re trying to build? This post will walk you through the process of developing a new step, with an emphasis on using the same tooling and workflow that the Relay team uses. We’ll focus on step containers here since most triggers can use the <a href="https://relay.sh/triggers/stdlib/json">generic webhook receiver</a> in our standard library; if you need a custom trigger, check out the <a href="https://relay.sh/docs/developers/integrating-with-relay/#writing-entrypoint-code">full developer documentation</a></p><p>Before you start, it’s worth asking whether you need to do anything at all. Laziness, after all, is one of the three main virtues of a programmer, <a href="https://en.wikiquote.org/wiki/Larry_Wall#Other">according to Perl’s Larry Wall</a> (the other two: impatience and hubris). Laziness in this case means:</p><ul><li>Asking around to see if anyone else is working on a similar step — maybe you can combine forces! Check on the <a href="https://slack.puppet.com/">Puppet Community slack</a> and <a href="https://github.com/puppetlabs/relay/issues">Relay Github issues</a>. Feel free to <a href="https://github.com/puppetlabs/relay/issues/new">file a new issue</a> with your request if you don’t see a pre-existing one; it’s a great way to coordinate work and make your plans more widely visible.</li><li>Even if the answer’s no, there’s one more escape hatch for the lazy. The <a href="https://hub.docker.com/repository/docker/relaysh/core">relaysh/core step containers</a> accept an inputFile parameter, so you can pass in a shell or Python script. The <a href="https://relay.sh/docs/developers/integrating-with-relay/#using-relayshcore">integration documentation</a> goes into this method in more detail. It might not be suitable if you have complex requirements, but if you can get started with a minimum of effort, it could save you time and energy.</li></ul><p>If laziness fails, it’s time to dive in. The steps we’ll follow are:</p><ul><li>Setting up — creating the working repository to contain your code and associated metadata</li><li>Developing and testing — getting your code working in isolation as part of a workflow</li><li>Publishing — making your step available to the world</li></ul><p>Fair warning: this section will get pretty far into the deep end of the techie pool, assuming you’re familiar with git/Github, Dockerfiles, and running containers.</p><h3>Setting up</h3><p>For existing integrations which you want to extend with a new step, for example to perform an additional action against a new API endpoint, the starting point is to fork-and-clone the existing repo from the <a href="https://github.com/relay-integrations/">relay-integrations namespace</a> on Github. For new integrations, use the <a href="https://github.com/relay-integrations/">relay-integrations/template</a> as your starting point — you can use the <strong>Clone this repo as template</strong> feature in Github to make this super easy.</p><p>In either case, make a new subdirectory under steps/ to contain your code. Our naming convention is that a step should be named <strong>noun-verb</strong>, like <strong>action-create</strong>. This maps to a container image that&#39;s named, for an example step in the <a href="https://puppet.com/bolt">Bolt</a> integration that runs a plan, bolt-step-plan-run. The initial contents can come from the <a href="https://github.com/relay-integrations/template/tree/master/steps/template">step template directory</a>, and should look like this:</p><pre>├── steps # subdirectory for containerized steps <br>│ └── noun-verb # rename this to your own step&#39;s name <br>|   ├── README.md # detail about how to use this step<br>|   ├── Dockerfile # needed to build the container<br>│   ├── step.sh # entrypoint (plus any additional code)<br>│   └── step.yaml # step metadata</pre><h3>Developing and testing</h3><p>The code you’re writing will act as the entrypoint to a container that the Relay service executes as part of a workflow. As such, there are really two parts to the task: to get variable data from the workflow, and then use that data do to whatever you’re trying to accomplish in the step.</p><p>Relay has an internal API called the metadata service, which presents a REST endpoint to step containers. It serves up dynamic information from workflow steps’ spec sections and receives outputs from steps as well as events from trigger containers. If you&#39;re working in shell, use the , which should be automatically available on your $PATH. We also have SDKs available for <a href="https://relay-sdk-python.readthedocs.io/en/latest/">Python</a> and <a href="https://github.com/puppetlabs/relay-sdk-go">Go</a>.</p><p>In order to shorten the development loop for writing, testing, and debugging step code, the relay CLI tool has a built-in version of the metadata service that you can run locally. This will allow your step entrypoint code to run unmodified from the shell, rather than requiring a container build, push, and execute cycle up to the Relay service.</p><p>See the <a href="https://relay.sh/docs/developers/step-authoring/">detailed developer documentation</a> for the details of the YAML mock-up data. After you’ve constructed the YAML input, start up the metadata service with:</p><pre>relay dev metadata --input test-metadata.yaml --run 1 --step first-step</pre><p>The last line of output contains an environment variable that you’ll need to copy-and-paste into the shell where you’re debugging your entrypoint:</p><pre>No command was supplied, awaiting requests. Set environment with: export METADATA_API_URL=&#39;http://:VeRyLoNgJWT[::]:59025&#39;</pre><p>This is the environment variable that the Python and Go SDKs, as well as the ni shell tool, use to communicate with the metadata service in Relay; setting it locally will point your code at the mock service. Running your code should result in HTTP GETs against the /spec path for each parameter and HTTP PUTs against the /output path if the step sets output variables.</p><p>Once you’re successfully interacting with the metadata service, the rest is… kind of up to you, really! There are some great examples of step code in the <a href="https://github.com/relay-integrations">relay-integrations</a> repos; I’m especially proud of the ones we’ve built for <a href="https://github.com/relay-integrations/relay-pagerduty/tree/master/steps/change-event-send">change events to PagerDuty (Python)</a> and <a href="https://github.com/relay-integrations/relay-kubernetes/tree/master/steps/kubernetes-step-kubectl">running kubectl from Relay</a>.</p><p>As you’re developing your code, keep in mind that the <a href="https://github.com/relay-integrations/template/blob/master/steps/template/step.yaml">step metadata</a> should stay in sync with it. In particular, it’s important to add <a href="https://github.com/puppetlabs/relay-rfcs/blob/master/content/0006-integration-layout/rfc.md#step-metadata">json schema</a> for the spec section of workflows that use your step. This schema documents what&#39;s effectively the &quot;API&quot; for your container, because it defines the names and data types of the inputs that it accepts. The Relay service uses this metadata to validate workflows and build the Library UI for users, so incomplete or inaccurate schema can prevent people from making use of your hard work!</p><h3>Publishing</h3><p>Now that your step code is running on your laptop, the next step is getting into production. A simple Dockerfile that sets your code as the entrypoint could <a href="https://github.com/relay-integrations/relay-firehydrant/blob/master/steps/timeline-update/Dockerfile">look like this one</a>, which updates a <a href="https://firehydrant.io/">FireHydrant.io</a> incident timeline:</p><pre>FROM relaysh/core:latest-python COPY timeline-update.py /relay/timeline-update.py CMD [&quot;python3&quot;, &quot;/relay/timeline-update.py&quot;]</pre><p>If docker build . works, and using the same relay dev metadata service works when you point a local container execution at it, it&#39;s probably time to publish. Steps in the relay-integrations Github organization use <a href="https://docs.docker.com/docker-hub/builds/">Dockerhub autobuilds</a> to build and push new container images when commits land in the Github repo - so if you&#39;ve forked an existing integration, just send in a pull request and we&#39;ll get your new step added. For completely new integrations, you&#39;ll need to push your image to a public container registry. Dockerhub is our default, but Relay workflows can pull from any registry as long as the image: field in the workflow includes its url.</p><p>Once your image is published, try it out in a minimalist workflow on <a href="https://app.relay.sh/">the Relay service</a>. Create a new workflow with <em>only</em> your new step and <em>only</em> the parameters your step expects. For the <a href="https://firehydrant.io/">FireHydrant.io</a> example earlier, this could look like:</p><pre>description: Minimal example to update a FireHydrant incident timeline<br>parameters:<br>  incidentID:<br>    description: &quot;The numerical FH incident ID to update (default: 1)&quot;<br>    default: 1<br>steps:<br>- name: timeline-update<br>  image: relaysh/firehydrant-step-timeline-update<br>  spec:<br>    apiKey: !Secret apiKey<br>    incidentID: !Parameter incidentID<br>    message: &quot;Relay ran a workflow and fixed the problem&quot;</pre><p>In order for this to run successfully, Relay will prompt you to <a href="https://relay.sh/docs/using-workflows/managing-secrets/">fill in the Secret value</a> for the API key, which will need to come from the FireHydrant app:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Gpn65Ror5-hxyYSL.png" /></figure><p>Once you enter the API key value, the workflow should run successfully:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/0*QmA5ExpSaGd2NcsR.png" /></figure><p>If you need to debug further, note that Relay saves any output from STDOUT or STDERR to the logs that are visible in the workflow run, so feel free to go wild with the print statements! On the back end, our engineers can also trace step execution from container pull to user execution, so if you get stuck, <a href="https://slack.puppet.com/">don’t hesitate to ping us on Slack</a>.</p><p>The final step is to let us know that your new step is working so we can add it to the <a href="https://relay.sh/library">Library on the website</a> and the app itself. If the step does what it says on the tin and its metadata is set up correctly, this is a pretty lightweight process: just file a pull request against the repository for existing integrations or a <a href="https://github.com/puppetlabs/relay/issues/new">top-level Github issue</a> for new integrations and we’ll turn ourselves inside out to get things working.</p><p>If you’ve made it this far — congratulations! And profuse thanks from the Relay team. Nurturing an ecosystem is a delicate task, and our goal is to make working with Relay as simple and rewarding as possible. We built Relay because we think it’s worth the effort it takes to make automation more accessible. Spreading knowledge that once was tribal, democratizing data that used to be hidden — these are goals that not only feel right, they produce better outcomes for the business.</p><p><em>Originally published at </em><a href="https://relay.sh/blog/developing-steps-with-cli/"><em>https://relay.sh</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7da542ec857a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/relay-sh/a-quick-guide-to-developing-steps-for-relay-7da542ec857a">A Quick Guide to Developing Steps for Relay</a> was originally published in <a href="https://medium.com/relay-sh">relay-sh</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Roll back failed Kubernetes deployments automatically via FireHydrant Runbooks]]></title>
            <link>https://medium.com/relay-sh/roll-back-failed-kubernetes-deployments-automatically-via-firehydrant-runbooks-5ba829810f7a?source=rss-d2f9578e48a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/5ba829810f7a</guid>
            <category><![CDATA[incident-management]]></category>
            <category><![CDATA[incident-response]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Eric Sorenson]]></dc:creator>
            <pubDate>Tue, 02 Mar 2021 21:16:56 GMT</pubDate>
            <atom:updated>2021-03-02T21:16:56.391Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/995/0*0PiAlpjMaiTutleC.png" /></figure><p><a href="https://firehydrant.io/">FireHydrant</a> has a sophisticated set of response actions for coordinating communications, activities, and retrospectives for incidents that affect your services. <a href="https://relay.sh/">Relay</a> helps by automating remediations that involve orchestrating actions across your infrastructure. In this example workflow, an incident that affects an application deployed on Kubernetes can trigger a rollback to a previous version automatically.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/310/1*BXfoVgreT_85c-JRc9Wygw.png" /><figcaption>Relay’s visualization of the workflow DAG</figcaption></figure><p>The workflow makes a couple of assumptions about your infrastructure that will need to be true to work out of the box; we’d <a href="https://github.com/puppetlabs/relay/issues/new">love to work with you</a> if you need additional flexibility! Specifically, it maps FireHydrant “Services” to Kubernetes <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/deployment">deployments</a>, and FireHydrant Environments to Kubernetes <a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/">namespaces</a>. Your deployment and rollback process is likely different from the one modelled here, but this should provide a good starting point for automating incident response activity with Relay.</p><h3>Connecting the Services</h3><p>In order to enable two-way communication between FireHydrant and Relay, we’ll need to do some work on both sides: Relay posting to FireHydrant requires a FireHydrant API key, and FireHydrant triggering Relay workflows uses a dynamically-generated Relay webhook URL.</p><p>First, add the Relay workflow to your account <a href="https://app.relay.sh/create-workflow?workflowName=firehydrant-rollback&amp;initialContentURL=https%3A%2F%2Fraw.githubusercontent.com%2Fpuppetlabs%2Frelay-workflows%2Fmaster%2Ffirehydrant-rollback%2Ffirehydrant-rollback.yaml">using this link</a>. When you click “Save”, Relay will both create the webhook URL you’ll need and prompt that you’re missing a Secret and a Connection — we’ll get to those in a moment.</p><p>In FireHydrant, we’ll create a Runbook that will trigger the workflow by sending a webhook to Relay. Create a new Runbook and add a <strong>Send a Webhook</strong> step. For the <strong>Endpoint URL</strong>, paste the webhook address from the Relay’s <strong>Settings</strong> sidebar. The <strong>HMAC Secret</strong> field is an arbitrary string (not currently used). For the <strong>JSON Payload</strong> field, paste the following template:</p><pre>{<br>   &quot;incident_id&quot;: &quot;{{ incident.id }}&quot;,<br>   &quot;name&quot;: &quot;{{ incident.name }}&quot;,<br>   &quot;summary&quot;: &quot;{{ incident.summary }}&quot;,<br>   &quot;service&quot;: &quot;{{ incident.services[0].name | downcase }}&quot;,<br>   &quot;environment&quot;: &quot;{{ incident.environments[0].name | downcase }}&quot;,<br>   &quot;channel_id&quot;: &quot;{{ incident.channel_id }}&quot;,<br>   &quot;channel_name&quot;: &quot;{{ incident.channel_name }}&quot;<br>}</pre><p>Next, create a FireHydrant API key for Relay to post information back into the incident timeline. Under <strong>Integrations</strong> — <strong>Bot users</strong> in FireHydrant, create a new <strong>Bot user</strong> with a memorable name and description. Save the resulting API token into a Relay secret on the Relay workflow’s <strong>Settings</strong> sidebar named apiKey (case-sensitive).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*p_L0awG6dYzkv3KPwwvNJA.png" /><figcaption>Integrations — Bot Users — Add a new one</figcaption></figure><h3>GCP Authentication Setup</h3><p>This workflow uses a GCP Connection type on Relay’s end, which requires a <a href="https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens">service account</a> configured on your cluster. Follow the GCP guide to API Server Authentication’s <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/api-server-authentication#service_in_other_environments">“Service in other environments”</a> section to set one up. This workflow will require the service account have the role roles/container.developer attached to it; if you re-use the connection for other workflows it may require additional permissions. Once you&#39;ve gotten the service account JSON file downloaded, add a GCP Connection in Relay, name it relay-service-account and paste the contents of the JSON file into the dialog. Under the hood, Relay stores this securely in our Vault service and makes the contents available to workflow containers through the <a href="https://relay.sh/docs/using-workflows/managing-connections/">!Connection custom type</a> in the workflow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*6sV032CFuJtMgpfx9FDHNg.png" /><figcaption>Adding a GCP Connection in Relay</figcaption></figure><p>For non-GCP clusters, you can use Relay’s Kubernetes connection type, which requires less setup. GCP rotates access tokens every hour, making them unsuitable for automated use. The Kubernetes connection type needs an access token, the cluster URL, and the CA certificate for the cluster; there are more detailed instructions accompanying <a href="https://github.com/puppetlabs/relay-workflows/tree/master/kubectl-apply-on-dockerhub-push">this deployment workflow example</a>.</p><h3>Configuring Services and Environments</h3><p>One of the FireHydrant’s big benefits is its awareness of your infrastructure. It takes a little bit of up-front work, but if you invest the time to map out your services and environments, you can dramatically streamline your incident response. In this example, we’ve enumerated the microservices that make up our Sock Shop application and associated them with different runbooks. Fortunately for the demo, the sockshop-frontend service is a simple stateless Deployment in GKE, which makes new releases easy to manage with the kubectl rollout command.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/949/1*TlWy_T-TJaMQ1uluwFmsqw.png" /><figcaption>Infrastructure — Services in FireHydrant</figcaption></figure><p>Similarly, the Environments section lets you enumerate the instances of your service, to better characterize the impact of an incident, help assign owners for remediation actions, and message outage information to the appropriate audiences. Check out this FireHydrant <a href="https://help.firehydrant.io/en/articles/4192249-inventory-management-functionalities-services-and-environments">helpdesk article on inventory management</a> for more details on infrastructure organization. For our purposes, the goal of defining environments is to map them onto Kubernetes namespaces where our application is running. (For production workloads, it’s more likely that your environments map to distinct clusters; that’s totally possible to handle in Relay but is beyond the scope of this introduction!)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/1*ESEx0ZBjttDqcZmOSEsotA.png" /><figcaption>Infrastructure — Environments in FireHydrant</figcaption></figure><h3>Incident Creation and Response</h3><p>Now for the exciting part. Let’s say an update bumped the image on the frontend pods from a pinned version to latest and everything broke.</p><pre>% kubectl set image deployment/sockshop-frontend nginx-1=nginx:latest --record --namespace production</pre><pre>deployment.apps/sockshop-frontend image updated </pre><pre>% kubectl rollout history deployment sockshop-frontend --namespace production deployment.apps/sockshop-frontend <br>REVISION CHANGE-CAUSE 1 kubectl set image deployment/sockshop-frontend nginx:1.18.0=nginx:latest --record=true --namespace=production <br>2 kubectl set image deployment/sockshop-frontend nginx-1=nginx:latest --record=true --namespace=production</pre><p>But unbeknownst to the deployer, all was not well. After some troubleshooting, we determined that the application was degraded and rollout was to blame. In FireHydrant, the on-call person declares an incident and indicates the service and environment that were affected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2KA65Vyvg1_hUxnJx6Zmxg.png" /><figcaption>Declaring a new incident in FireHydrant</figcaption></figure><p>Once the incident is created, we can attach the rollback-via-relay Runbook that contains our webhook to the incident. The benefit of doing it this way is that the credentials are stored in Relay rather than needing a comamnd-line kubectl setup as above, and you don&#39;t have to remember the exact syntax to type if something&#39;s broken at 4AM! The correct steps are stored in the workflow, reducing the possibility for an error.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*DCgXZv5uJSY75F6XtcAL6A.png" /></figure><p>The incident timeline and associated Slack channel show these action taking place, and in Relay we can see the webhook come in, the workflow kick off, and ultimately post back into the timeline with the output of the rollback command. Thanks to FireHydrant’s <a href="https://help.firehydrant.io/en/articles/2862753-integrating-with-slack">awesome Slack integration</a>, the updates roll into the channel in real time and chat messages are mirrored back into the incident so teams can coordinate their activities and keep a record of what happened. In this case, the rollback worked and we can resolve the issue quickly!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/861/1*Z2tCNI7meGUiJDbD0CxlmQ.png" /><figcaption>FireHydrant Slack Integration</figcaption></figure><h3>Conclusion and Next Steps</h3><p>FireHydrant’s Runbook based system for coordinating actions in response to incidents is extremely powerful. The more you teach it about your infrastructure, the faster you’ll be able to respond when something goes wrong. And linking it to Relay via Runbooks enables another level of automated response and remediation. In this example, we were able to roll back a deployment without someone needing to run manual commands and potentially making things worse!</p><p>There are lots of <a href="https://relay.sh/workflows/">existing Relay workflows</a> that can act as building blocks or examples to construct your own incident response workflow. By combining them with clear processes codified in FireHydrant, responders can solve issues more quickly, reduce downtime, and get back to higher-value work.</p><p>To try this out for yourself, sign up for free <a href="https://firehydrant.io/">FireHydrant</a> account and <a href="https://relay.sh/">Relay</a> accounts, and get started automating!</p><p><em>Originally published at </em><a href="https://relay.sh/blog/firehydrant-rollback-runbook/"><em>https://relay.sh</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5ba829810f7a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/relay-sh/roll-back-failed-kubernetes-deployments-automatically-via-firehydrant-runbooks-5ba829810f7a">Roll back failed Kubernetes deployments automatically via FireHydrant Runbooks</a> was originally published in <a href="https://medium.com/relay-sh">relay-sh</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learn how to link GitHub and Relay for versioned, collaborative workflow authoring]]></title>
            <link>https://medium.com/relay-sh/learn-how-to-link-github-and-relay-for-versioned-collaborative-workflow-authoring-c6f323277cc6?source=rss-d2f9578e48a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6f323277cc6</guid>
            <category><![CDATA[github]]></category>
            <category><![CDATA[gitops]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Eric Sorenson]]></dc:creator>
            <pubDate>Tue, 02 Mar 2021 21:12:15 GMT</pubDate>
            <atom:updated>2021-03-02T21:12:15.440Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SAmGQPWd51jp0r4qve7pmw.png" /><figcaption>Github + Relay</figcaption></figure><p>The emergence of GitOps has been an interesting and important recent development in the Infrastructure as Code (IaC) space. Alexis Richardson of <a href="https://weave.works/">Weaveworks</a> coined the GitOps term and defined it in an <a href="https://www.weave.works/technologies/gitops/">introductory blog post</a> as these two things:</p><blockquote>1. An operating model for Kubernetes and other cloud native technologies, providing a set of best practices that unify deployment, management and monitoring for containerized clusters and applications.</blockquote><blockquote>2. A path towards a developer experience for managing applications; where end-to-end CICD pipelines and Git workflows are applied to both operations, and development.</blockquote><h3>The Source of Truth</h3><p>Over the past couple of years, people have seen the value in these principles and applied them to additional contexts, while maintaining the initial motivation: to bring CI-styled automation to operational changes, making Git the source of truth for both application deployments and the underlying infrastructure.</p><p>While this sounds familiar to users of IaC tools like <a href="https://terraform.io/">Terraform</a>, <a href="https://pulumi.com/">Pulumi</a> and <a href="https://puppet.com/">Puppet</a>, the “best practices” identified in the first bullet point go quite a bit further than simply keeping your infrastructure code in source control and running a test suite before pushing changes. The key difference is that the <em>rollout</em> of those changes is also automated. A combination of pull request code review and testing provides safeguards; the underlying platform (particularly Kubernetes) provides commit/revert rollback of infrastructure changes.</p><h3>Why GitOps for Relay?</h3><p>With all that context in mind, the next question is: Where does Relay fit in? Relay runs workflow automation as events come in from external sources, both humans and other services. The workflows handle tough-to-automate tasks that combine information from different sources, make changes across your infrastructure, and report out by sending notifications. In addition to the GitOps-friendly work you can do <em>inside</em> of the workflows (check out <a href="https://relay.sh/workflows/terraform-continuous-deployment/">this Terraform example</a>), since workflows are represented as individual YAML files, they’re great candidates for IaC best practices.</p><p>Internally, the Relay service maintains a versioned history of every workflow run: both the code and the output. This is great because even though workflows change over time, you can go back to a previous point in time and revisit the logs, or restore an earlier version if you encounter a regression. It’s also built in to the service, so edits via the <a href="https://app.relay.sh/">web app</a> or the <a href="https://github.com/puppetlabs/relay">Relay CLI</a> will stay in sync.</p><p>But this ease of use comes with some limitations: multiple people editing a workflow might not be working on the latest changes, it’s hard to get code review from your teammates, and the service doesn’t store a rich revision history with commit logs. GitHub solves all of these problems, so if you’re getting into advanced Relay usage, a natural step is to link the workflows stored on the service to GitHub. Thus, GitOps with Relay.</p><h3>Walkthrough</h3><p>In the rest of this post, I’ll walk through:</p><ul><li>Setting up a GitHub repo to contain your Relay workflows.</li><li>Configuring GitHub Actions to push new commits to the Relay service.</li><li>Updating a sample workflow with a pull request review/commit cycle.</li></ul><p>To follow along, you’ll need free accounts on <a href="https://github.com/">GitHub</a> and <a href="https://relay.sh/">Relay</a>. You should also <a href="https://relay.sh/docs/getting-started/#install-the-cli">install the Relay CLI</a>. Some familiarity with Git terminology (clone, push, pull) will also be helpful.</p><h3>Add the repository</h3><p>GitHub has a friendly <a href="https://github.com/new">Create a new repository</a> page; I’ve used it as shown below to create the repository <a href="https://github.com/ahpook/relay-gitops">ahpook/relay-gitops</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/775/0*oJmIfmOnTIpkMjqW.png" /></figure><p>After creating the repo, I made a <a href="https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/cloning-a-repository">local clone</a>, and created a directory called relay-workflows containing this hello-world.yaml workflow from <a href="https://relay.sh/docs/getting-started">the Getting Started with Relay guide</a> as an example.</p><pre># relay-workflows/hello-world.yaml<br>parameters:<br>  message:<br>    description: &#39;The message to output from this step&#39;<br>steps:<br>- name: hello-world<br>  image: relaysh/core<br>  input: - echo &quot;Hello world. Your message was $(ni get -p {.message})&quot;</pre><p>I’m also going to add this workflow to my Relay account so there’s a starting point for the automation. I’m in the relay-workflows directory I created earlier and I have the <a href="https://github.com/puppetlabs/relay/">Relay CLI</a> installed, so I can run:</p><pre>relay auth login <br>relay workflow add hello-world -f hello-world.yaml</pre><p>Alternatively, I could do this from the web app.</p><p>After adding the workflow to Relay, I’ll commit and push up my changes, so my repo looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/986/0*c2xROMfjXgtQDotX.png" /></figure><p>Note that my default branch is named main, which the rest of this example will use. We&#39;ve posted <a href="https://puppet.com/blog/removing-harmful-terminology-from-our-products/">Puppet&#39;s rationale for that change</a> on our blog; Github&#39;s <a href="https://github.com/github/renaming/blob/main/README.md">renaming guide is here</a>.</p><h3>Configure GitHub Actions</h3><p><a href="https://docs.github.com/en/actions/getting-started-with-github-actions">GitHub Actions</a> are a very cool way to run CI/CD workflows right on GitHub. They’re complementary to Relay because they can perform sequenced actions when things change in repositories and have a lot of GitHub-specific context available in their execution environment, whereas Relay is made to respond to external events from lots of different sources. To avoid any confusion from the name overlap, this post will always provide context to disambiguate “GitHub workflows” from “Relay workflows”.</p><p>To start setting up the GitHub Actions workflow that will push to Relay, you can either go to the “Actions” tab from the repository’s home page on GitHub and click the “Set up a workflow yourself” link, or add a file in your local repository clone that lives at .github/workflows/update-relay.yaml. Either way you go, the contents should look like this:</p><pre>name: update-workflow-on-commit</pre><pre># Controls when the action will run. Triggers the workflow on push or pull request<br># events but only for the master branch<br>on:<br>  push:<br>    branches: [main]</pre><pre># A workflow run is made up of one or more jobs that can run sequentially or in parallel<br>jobs:<br>  update:<br>    runs-on: ubuntu-latest<br>    steps:<br>      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it<br>      - uses: actions/checkout@v2<br>        with:<br>          fetch-depth: 0<br>      - uses: puppetlabs/relaysh-docker-update-workflow@main<br>        with:<br>          RELAY_USERNAME: ${{ secrets.RELAY_USERNAME }}<br>          RELAY_PASSWORD: ${{ secrets.RELAY_PASSWORD }}<br>          RELAY_WORKFLOW_FILE: &#39;relay-workflows/hello-world.yaml&#39;</pre><p>There are two Actions in play here: first, the workflow will run <a href="https://github.com/actions/checkout">actions/checkout@v2</a>, which checks out the repository that houses this GitHub workflow. Then the relay-specific workflow <a href="https://github.com/puppetlabs/relaysh-docker-update-workflow">puppetlabs/relaysh-docker-update-workflow</a>, which connects to Relay using the CLI and runs the relay workflow replace command, which replaces the existing workflow code with the contents of the last commit.</p><p>In this case I’m updating a single workflow, in the relay-workflows/hello-world.yaml file, which matches a workflow on the service named &#39;hello-world&#39;. If the repo ends up with multiple workflows, this GitHub action supports adding a workflow_mappings.yaml file in the root of the repository, which enables you to update multiple workflows at once and supports workflow names on the service whose labels differ from the path- and extension-stripped basename of the file in the repository. There are some caveats as below; see the <a href="https://github.com/puppetlabs/relaysh-docker-update-workflow/">README for the Relay action</a> for the latest details, because this blog might be out of date!</p><ul><li>You must set up repo secrets RELAY_USERNAME and RELAY_PASSWORD with your login credentials to Relay</li><li>You need to provide <em>at a minimum</em> a RELAY_WORKFLOW_FILE value in the GitHub Action workflow; if the name on the service for that single workflow differs from the file&#39;s basename, provide a RELAY_WORKFLOW key as well. For more advanced use-cases, add a workflow_mappings.yaml configuration to map files onto workflow names on the service.</li><li>If a workflow_mappings.yaml file exists, only the files/names enumerated in it will be updated.</li><li>If a workflow is listed in the workflow or mappings, but does not exist yet on the service, the GitHub action will create it.</li><li>Workflows will not be run automatically after being updated.</li></ul><p>To try this out, we’ve got to fulfill the last of those requirements in this repo, and then we should be good to go! In the GitHub repository page, navigate to <strong>Settings</strong> then <strong>Secrets</strong>. Click <strong>New Secret</strong> and add one named RELAY_USERNAME containing your Relay account&#39;s email address and RELAY_PASSWORD which, naturally, should have your password. GitHub <a href="https://docs.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets">protects these secrets well</a> so I feel pretty comfortable putting them in; please check with your security team if you have any doubts about your organization&#39;s security posture, though!</p><p>Once you’re done you should see both secrets on this screen.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/897/0*Sp65GQJxNtSRdHMN.png" /></figure><h3>Try it out!</h3><p>To test it out, we need to make a commit that modifies the workflow in some visible (but safe!) way. I’ve just made a change to the output line that shows we’re running version 2:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/0*dRdby7Wzhkoq7MXl.png" /></figure><p>Because this is a trivial change I’m going to YOLO it and push directly to the main branch. For production usage, a pull request with some code review would probably be a good idea. The GitHub workflow is agnostic to <em>how</em> the commit comes in, and the on selector at the top of the workflow will ignore activity on all non- main branches, so it&#39;s safe to experiment and push revisions onto other branches until you&#39;re ready to go.</p><p>The GitHub Actions section of the repository page shows the success or failure of each run; if something’s wrong, this is your debugging lifeline. The logs of a successful run should show the message from the relay CLI that your workflow was successfully replaced, and the version on the Relay app should match the contents of the Git repository. It took me several tries to get the workflow and Action entrypoint working (turns out I had pasted the wrong password into my RELAY_PASSWORD secret), but that’s what tags are for!</p><p>Hopefully this quick spin helped introduce the concepts behind GitOps as well as demonstrate a concrete implementation for Relay. There’s a lot of work to do, both in improving the resiliency of this specific GitHub Action (pull requests gladly accepted!) and deepening a built-in integration between Relay and git-based sources of truth.</p><p><em>Originally published at </em><a href="https://relay.sh/blog/gitops-with-relay/"><em>https://relay.sh</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6f323277cc6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/relay-sh/learn-how-to-link-github-and-relay-for-versioned-collaborative-workflow-authoring-c6f323277cc6">Learn how to link GitHub and Relay for versioned, collaborative workflow authoring</a> was originally published in <a href="https://medium.com/relay-sh">relay-sh</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Relay and the open source community]]></title>
            <link>https://medium.com/relay-sh/relay-and-open-source-c43cf3ef682f?source=rss-d2f9578e48a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/c43cf3ef682f</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[development]]></category>
            <dc:creator><![CDATA[Eric Sorenson]]></dc:creator>
            <pubDate>Wed, 08 Jul 2020 18:07:50 GMT</pubDate>
            <atom:updated>2020-07-08T18:07:50.608Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YpGmttPubmHnN9ydqL6wlw.png" /><figcaption>Source code for the Relay CLI</figcaption></figure><p>Relay is deliberately built to be easy to extend. The more <a href="https://relay.sh/docs/how-it-works/#definitions">steps</a> there are in the world the better it is for everyone who wants to write a workflow. It’s sort of like fax machines or MySpace: you’ll want to use the same technology that the people you’re communicating with use. OK, maybe the fax machine isn’t not a great analogy… the point is, nourishing a healthy open-source ecosystem and community around Relay is super important to its success.</p><p>In this post I’ll talk about the different aspects of Relay’s open source DNA: the ecosystem model, the codebase itself, and upstream projects.</p><h3>The Relay Ecosystem</h3><p>First and foremost, Relay is about <a href="https://relay.sh/docs/how-it-works/#workflow-execution">workflows</a>. Relay workflows are a well-defined YAML dialect that’s readily sharable. The first batch of workflows we’ve been writing to exercise the system and demonstrate its capabilities are on the <a href="https://relay.sh/workflows/">Workflows section of the website</a> with READMEs for example usage and a one-click button to add them to your account. The code for all of them is <a href="https://github.com/puppetlabs/relay-workflows">collected in the puppetlabs/relay-workflows repo</a>. We’d love to see pull requests with bug fixes or improvements, new workflows you’ve written to solve problems with Relay, or <a href="https://github.com/puppetlabs/relay/issues/new">requests for features</a>. The workflow syntax is <a href="https://relay.sh/docs/reference/relay-workflows/">documented at relay.sh/docs</a> along with instructions on how to <a href="https://relay.sh/docs/setting-up-editor/">set up VSCode</a> to do automatic syntax validation and completion.</p><p>Relay workflows make use of containers to both respond to incoming events and to run as steps to accomplish the task you’re trying to automate. Step and Trigger container definitions that interact with a particular tool or service can be bundled up into what we call an Integration. The <a href="https://relay.sh/integrations/">Integrations library on the website</a> has a browsable, taxonomized list of the currently supported integrations.</p><p>We expect the current list will cover a lot of common Relay use cases, but there’s no way it’s going to cover <em>all</em> of them. If you run into the situation where there’s an integration with a service you use, but not a specific Step or Trigger that does what you want, we’d love to hear about it! Many of the features marked “Coming soon” on the integration pages just need some prioritization and tweaking to get out the door, and your vote counts to help get them on the list. Plus, all of the integrations are aggregated together under the <a href="https://github.com/relay-integrations">relay-integrations Github organization</a>, and by now it should go without saying that we welcome any PRs or issues on them.</p><p>To make a new integration, or if you want to go further than tweaking the existing containers, there’s a long-form <a href="https://relay.sh/docs/integrating-with-relay/">Integrating with Relay</a> guide that breaks down the different container types, the <a href="https://pypi.org/project/relay-sdk">Python SDK</a>, and best practices for integrating with the internal Relay service APIs.</p><h3>The Relay Codebase</h3><p>Speaking of documentation, the entire <a href="https://github.com/puppetlabs/relay-docs/">Relay documentation site</a> is available on Github. If you run across anything in the docs that doesn’t seem right, from typos to examples that don’t work, we’d welcome corrections in the form of PRs or issues. Sending pull requests into the docs can be a great way to get involved with a project. Docs are generally on a quicker release cycle than the codebase and everybody benefits from having improved documentation.</p><p>If you’re a Go developer, whether aspirational (like me!) or expert, the <a href="https://github.com/puppetlabs/relay">Relay CLI tool</a> could present a fun opportunity to dig into a relatively constrained codebase and improve the Relay development experience. I’ve written at length about our <a href="https://relay.sh/blog/command-line-ux-in-2020/">philosophy of CLI design</a>, something I personally am pretty passionate about, and it’s been really gratifying to see those design principles come to life in the Relay CLI. There are a few issues <a href="https://github.com/puppetlabs/relay/labels/cli">tagged with CLI</a> to get started, and of course we welcome your ideas too.</p><p>Going deeper into the architecture, the Kubernetes execution backend for the service is published at <a href="https://github.com/puppetlabs/relay-core">puppetlabs/relay-core</a>. There are some intriguing possibilities for this codebase, like adding a local executor that could use minikube to run workflows locally. The metadata service would also be useful to have as a self-hosted version, because it would allow better validation and testing of step definitions while you’re developing workflows.</p><p>Of potential interest to front-end folks, the <a href="https://github.com/puppetlabs/design-system">Puppet Design System</a> is an open source collection of <a href="https://reactjs.org/">React components</a> that the Relay web app (among other Puppet products) uses to provide gorgeous, modern look-and-feel.</p><h3>Upstream</h3><p>Relay makes heavy use of upstream open source technology. We try to be good citizens of these projects by contributing bug fixes and enhancements back upstream. This helps everybody because we don’t end up maintaining forks or carrying our own patches, the code gets more eyes on it, and the rest of the community can make use of the features too.</p><p>The projects we’re most heavily involved in are <a href="https://tekton.dev/">Tekton pipelines</a>, which we’ve <a href="https://relay.sh/blog/whats-going-on-with-tekton-part-1/">blogged about</a> <a href="https://relay.sh/blog/whats-going-on-with-tekton-part-2/">extensively</a> and <a href="https://www.getambassador.io/">Ambassador</a>, with whom we’ve written a <a href="https://relay.sh/blog/user-defined-webhooks-with-knative-and-ambassador/">detailed technical blog post</a> describing how Relay uses Knative and Ambassador together to run your Trigger containers.</p><h3>Conclusion</h3><p>To wrap things up, I’d like to acknowledge that open source is tough to get right. Communities are valuable, fragile things, which need lots of love and attention to flourish. We’re deeply committed to investing in community for Relay, but acknowledge up front that we won’t always get it right. We pledge to listen and improve based on your feedback. You can email the team at <a href="/service/mailto:relay@puppet.com">relay@puppet.com</a>or jump on <a href="http://slack.puppet.com/">slack.puppet.com</a> to chat in #relay. Please keep the lines of communication open and let’s build some awesome automation together!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c43cf3ef682f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/relay-sh/relay-and-open-source-c43cf3ef682f">Relay and the open source community</a> was originally published in <a href="https://medium.com/relay-sh">relay-sh</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[3 Commandments for CLI Design]]></title>
            <link>https://medium.com/relay-sh/command-line-ux-in-2020-e537018ebb69?source=rss-d2f9578e48a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/e537018ebb69</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[design]]></category>
            <category><![CDATA[ux]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Eric Sorenson]]></dc:creator>
            <pubDate>Wed, 08 Apr 2020 20:52:57 GMT</pubDate>
            <atom:updated>2020-04-17T17:31:58.678Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Close crop of the Sistine Chapel ceiling painting with God and Adam’s fingers" src="https://cdn-images-1.medium.com/max/550/1*aRLC1LnS2UB-sCnU_-Sqrw.png" /><figcaption>photo: Wikimedia Commons (public domain)</figcaption></figure><h3>In the beginning…</h3><p>In Neal Stephenson’s 1999 essay <a href="https://web.archive.org/web/20180218045352/http://www.cryptonomicon.com/beginning.html">“In the Beginning Was The Command Line”</a>, he describes how using Command-Line Interface (CLI) utilities is simpler and requires less code than a Graphical User Interface (GUI). They’re tiny, they do one thing really well, and they are meant to work in conjunction with other small, single-purpose tools to accomplish complicated tasks quickly.</p><p>However, the old-school Unix utilities are not without their problems. The passage of time and inevitable feature creep have led to situations like this:</p><pre>usage: ls [-ABCFGHLOPRSTUWabcdefghiklmnopqrstuwx1] [file ...]</pre><p>The humble ls command now has more options than there are letters in the alphabet! Plus, despite the proliferation of features, many advanced and useful options are off by default to preserve backwards compatibility. Other core utilities are in similar situations: find and tar are stuck with arcane, user-hostile syntaxes, and the man pages for modern distributions are littered with comments like this one from the BSD-derived macOS 10.15 grep manual: “This implementation supports those options; however, their use is strongly discouraged.” (Discouraged? By whom? <em>Why?</em>)</p><h3>Fast Forward</h3><p>Fortunately, though much remains constant in the world of command-line tools, much has changed. There’s a buzz of activity in the open-source world around re-examining the core user experience of CLIs with fresh eyes. Not only are the tiny, single-purpose tools getting a refresh, but a new generation of CLIs are emerging that provide entry points into complex cloud services and system tools. The best of these enable power users while retaining the user interaction paradigms that make the small tools great.</p><p><a href="https://relay.sh/">Relay.sh</a> is an automation platform for event-driven devops workflows and we’re deeply committed to building awesome command-line experiences. I started writing down these principles as we began rethinking the command line tool for Relay. To me, the key UX principles that underpin the best of the new breed of projects in this vein are:</p><ol><li><strong>Enable Progressive Discovery</strong> — users are trying to solve a problem, but might start out knowing very little about the tool. The tool should guide them to a solution in iterative steps and provide plain-language help along the way. Users resorting to Google or StackOverflow is an anti-pattern here.</li><li><strong>Go Upscale, Intelligently</strong> — the best tools infer the context they’re running in take advantage of modern capabilities if they can. For example, colorized text can greatly add to usability and aesthetics, but if the tool is being run in a pipeline, colors should be off to avoid polluting the output stream.</li><li><strong>Be a Good CLI Citizen</strong> — there are a number of hard-fought CLI best practices that good tools follow, both to match users’ expectations and to peacefully co-exist with the rest of the ecosystem. It’s important for tool authors to be aware of these conventions, comply with as many as possible, and be deliberate about any significant divergence.</li></ol><p>In the rest of the post I’ll provide some examples of each of these principles using CLI tools I’ve fallen in love with.</p><h3>Enable Progressive Discovery</h3><p><a href="http://pulumi.io/">Pulumi</a> is an infrastructure-as-code tool that allows developers to create and manage cloud infrastructure using general-purpose programming languages. While Pulumi programs can be written in Python, Typescript, .Net and so forth, the user interaction loop centers on using the pulumi CLI tool to interface between the local code repository and the Pulumi service. It’s a fantastic example of modern CLI design that exemplifies all of the principles I mentioned, but I’m going to focus on the way it enables progressive discovery.</p><p>First, installing it is a breeze because the website has helpful instructions for all the major OSes with a one-click copy-to-clipboard command to get you started.</p><figure><img alt="Instructions on how to install the Pulumi command for different operating systems" src="https://cdn-images-1.medium.com/max/670/1*weICV9gULn8iIA3b-rnmUg.png" /></figure><p>Once you’ve got it installed, simply running the pulumi command with no arguments gives you helpful hints on what to do next in order to begin working with it. I’ve annotated the output of running pulumi with no arguments to illustrate this principle.</p><figure><img alt="Annotated output of the ‘pulumi’ command, indicating that users do not need to know how to use it in order to be successful." src="https://cdn-images-1.medium.com/max/1024/1*cYk9xsnvw53cijyhAUm4YA.png" /></figure><p>The output not only shows, right at the top, the first thing you ought to run if you’re new, it provides some hints as to what the second and third steps might be after you’ve begun. It links to the detailed docs on the website, invites exploration and discovery by giving the complete list of available commands, and has a reference section to show the flags which modify the program’s behavior in consistent ways, regardless of what subcommand you’re using.</p><p>Once you’ve gotten started and are working with a “stack” (a collection of resources that make up one application instance), the idea of progressive discovery shows up again. Running the pulumi stack subcommand without additional arguments doesn’t error out; instead it provides a reasonable guess as to what you meant: “show me information about the current stack”.</p><figure><img alt="Running ‘pulumi stack’ outputs info about the current stack and suggests additional commands you might want to run." src="https://cdn-images-1.medium.com/max/1024/1*idW9hXxRqW7hMcIk68XymA.png" /></figure><p>In addition to providing metadata about the stack and a nicely-formatted hierarchical representation of its resources, the output has links to the web app for more info and again shows hints about what to do next if this isn’t what you were looking for.</p><p>All in all, this experience is pretty delightful. It makes working withpulumi positive and enjoyable, and rather than feeling like a cut-down version of a web app it feels like a powerful first-class way of interacting with the service.</p><h3>Go Upscale, Intelligently</h3><p>My former colleague from Danger and all-around good human <a href="https://medium.com/u/8dbafa28ca85">C J Silverio</a> tweeted recently:</p><h3>Ceej is sheltering under cats on Twitter</h3><p>The rustlang ecosystem scene that&#39;s making me happy right now: the retakes on ancient unix cli tools, with new approaches to info display. bat dua exa</p><p>This sent me down a rabbit hole of investigation and discovery. There’s indeed a vibrant community working in this space, and the ones CJ mentions are replacements for (respectively) cat, du, and ls. (There’s even <a href="https://github.com/palash25/wc-rs">a rewrite of the </a><a href="https://github.com/palash25/wc-rs">wc command</a> from the Stephenson essay!) I came out the other side of the rabbit hole with a new set of aliases for<a href="https://github.com/ahpook/dotfiles/blob/master/.zshrc#L84-L92"> my .zshrc</a>:</p><pre># if rust stuff is found, use it<br>RBIN=$HOME/.cargo/bin<br>if [[ -d $RBIN ]]; then<br>  [[ -f $RBIN/bat ]] &amp;&amp; for f in less more cat ; do <br>     alias $f=bat ; <br>  done<br>  [[ -f $RBIN/dua ]] &amp;&amp; alias du=dua<br>  [[ -f $RBIN/rg  ]] &amp;&amp; alias grep=rg<br>  [[ -f $RBIN/exa ]] &amp;&amp; alias ls=exa<br>  [[ -f $RBIN/fd  ]] &amp;&amp; alias find=fd<br>fi</pre><p>These tools vary in their implementation but all of them exemplify the “Go Upscale” principle: they are aware of the affordances that modern terminal programs have and make use of them to provide upgraded experiences. To focus on one example, the lowly cat program is one of the oldest and most useful Unix utilities. Its name is short for “concatenate” and it can be used for all kinds of input and output stream manipulations, but it’s pretty primitive. bat (<a href="https://github.com/sharkdp/bat">here’s its homepage on github</a>) retains the usefulness of the original but adds colorized and line-numbered output, syntax highlighting for over a hundred file types, and cool features like git awareness. So bat README.md in my <a href="https://github.com/lyraproj/hiera">local hiera repository</a> shows me:</p><figure><img alt="a README.md file showing markdown-aware syntax highlighting, line numbers, and git-aware annotations" src="https://cdn-images-1.medium.com/max/726/1*UX_JhvilmBkGTPUGNbxzDA.png" /></figure><p>The subtly shaded line numbers and line-drawn formatting, the + markers for uncommitted lines, and the helpful Markdown syntax highlighting are all super cool. But the thing that allowed me to alias cat=bat in my shell without fear of breaking scripts is that bat detects when it’s not printing to a terminal and turns all of that pretty-printing off so it retains compatibility.</p><h3>Be a Good CLI Citizen</h3><p>There’s a set of conventions and practices that high quality CLI tools share, which enable them to interact seamlessly with each other and, more importantly, allow a user to extrapolate lessons and habits learned from one tool into new contexts. This leads to a quicker sense of mastery and user delight because things feel “intuitive” (in reality, nothing about computers is truly intuitive to humans — there’s just experiences that confirm or confound our expectations based on previous encounters).</p><p>It’s tough to focus on a single tool as an example. Some of these conventions are defined in <a href="https://medium.com/u/8fcea8539bad">Jeff Dickey</a>’s great post <a href="https://medium.com/@jdxcode/12-factor-cli-apps-dd3c227a0e46">“12 Factor CLI Apps”</a> about the Heroku CLI design philosophy, others from the GNU coreutils doc “<a href="https://www.gnu.org/software/coreutils/manual/html_node/Opening-the-software-toolbox.html">Opening the software tool box”</a>, and some are arguably matters of taste. <strong>Please comment if you think I’ve gotten something wrong or missed an important one!</strong></p><ul><li>Expect to operate as part of a pipeline and adjust behavior accordingly. Like the bat example above, turn off any fancy formatting if output isn’t a terminal so downstream tools see plain-text output for easy processing.</li><li>Don’t cross the streams! Closely related to pipeline operation, output should go to stdout and errors to stderr, so errors won’t be swallowed by redirection or intermingled with actual output.</li><li>Provide tabular and structured output formats for lists of data. For extracting only parts of a record, make it easy for users to split on whitespace to get just single fields. And no modern tool that outputs a lot of records in a table ought to be without a json output option.</li><li>For sophisticated CLIs, such as those which interact with remote services like Pulumi, think deeply about your command’s information architecture, then stick with it. A complex command invocation is like a grammatical sentence: it’s usually got a subject, a verb, and an object. For nontrivial commands there’s a hierarchy of these nouns and verbs that users put together to solve a problem. Either use a verb-object construction like kubectl get pods , where get is a top-level verb and the targets vary, or subject-verb construction like gcloud container clusters list (“clusters” is a subcommand of “container” and “list” is the verb).</li><li>Observe control conventions. Users frequently get things wrong, and sometimes that happens after they’ve hit the enter key on a command. The convention is that ^C (control-C) interrupts execution and exits the program, cleanly if possible. Similarly, ^Z (control-Z) suspends a program so that it can be paused or put in the background. Good CLI commands should honor these signals and behave predictably when they come in.</li><li>Don’t litter the filesystem. Use package managers and follow their conventions for the target operating system wherever possible. Similarly, follow conventions like <a href="https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html">XDG-Spec</a> for guidance on where to install configuration files, docs, and supporting libraries. This kind of consistent packaging may take some additional work up front but it pays off in the long run when your software can be installed, upgraded, and config-managed alongside the rest of the user’s toolchain.</li></ul><h3>END OF FILE</h3><p>When you’re done inputting a file into cat or its equivalents, a ^D (control-D) character indicates to the terminal that the end of the file’s been reached. Similarly, when ending a blog post it’s customary to summarize the points you’ve made and provide a stirring call to action for the reader. So, to recap the three top level design principles for modern command line interaction design:</p><ol><li>Enable progressive discovery to lead users down a low-friction path to solving their problem with your tool.</li><li>Upgrade capabilities to take advantage of modern terminal emulation, so users get the advantage of colors, graphics, and interaction.</li><li>Understand and implement the conventions for well-behaved tools in your space, users can get to a sense of mastery quicker and avoid unpleasant surprises.</li></ol><p>As I mentioned at the top, I’m working on a product called <a href="https://relay.sh/">Relay</a> that hopefully embodies these principles. Check out the <a href="https://medium.com/relay-sh/building-the-future-of-devops-automation-c25d6cf45ed1">introductory blog post</a> to learn more and follow along as we develop our open-source CLI at <a href="https://github.com/puppetlabs/relay">puppetlabs/relay</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e537018ebb69" width="1" height="1" alt=""><hr><p><a href="https://medium.com/relay-sh/command-line-ux-in-2020-e537018ebb69">3 Commandments for CLI Design</a> was originally published in <a href="https://medium.com/relay-sh">relay-sh</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What’s Going on With Tekton? (Part 2)]]></title>
            <link>https://medium.com/relay-sh/whats-going-on-with-tekton-part-2-ab39176c9c45?source=rss-d2f9578e48a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/ab39176c9c45</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[continuous-delivery]]></category>
            <dc:creator><![CDATA[Eric Sorenson]]></dc:creator>
            <pubDate>Wed, 25 Mar 2020 20:37:35 GMT</pubDate>
            <atom:updated>2020-04-01T06:12:55.021Z</atom:updated>
            <content:encoded><![CDATA[<h3>What’s Going on with Tekton? (Part 2)</h3><h3>The Story So Far…</h3><p>In<a href="https://medium.com/relay-sh/whats-going-on-with-tekton-part-1-9c3e006d39fc"> Part 1</a> of this two-part blog, I talked about how we’re using Tekton Pipelines in building <a href="https://relay.sh/">relay.sh</a>, how <a href="https://jenkins-x.io/">Jenkins X</a> is using it, and a brief history of the project and its terminology. In this part, I’ll dig into some of the cool advancements and features in the core Pipelines project and the broader ecosystem.</p><h3>The ‘tkn’ CLI tool</h3><p>There’s been a ton of work lately going into the tkn command-line interface. I wanted to introduce it first, not just because there’ve been a number of very cool improvements in it recently, but also because we at Puppet are heavily invested in awesome command-line user experiences for our users, and tkn exhibits some great UX patterns that are worth looking at, even if you’re not using it directly.</p><p>You can install tkn via Homebrew or tarball; check out the <a href="https://github.com/tektoncd/cli">README in the tektoncd/cli repo</a> for the details. It uses the kubernetes client API, so if kubectl get pods --namespace=tekton-pipelines works for you, so will tkn. The first cool thing about the CLI that it supports both bash and zsh command completion, so typing tkn &lt;Tab&gt; will show you the list of subcommands it supports:</p><pre>&gt; tkn &lt;Tab&gt;<br> -- command --<br>clustertask      -- Manage clustertasks<br>completion       -- Prints shell completion scripts<br>condition        -- Manage conditions<br>eventlistener    -- Manage eventlisteners<br>help             -- Help about any command<br>pipeline         -- Manage pipelines<br>pipelinerun      -- Manage pipelineruns<br>resource         -- Manage pipeline resources<br>task             -- Manage tasks<br>taskrun          -- Manage taskruns<br>triggerbinding   -- Manage triggerbindings<br>triggertemplate  -- Manage triggertemplates<br>version          -- Prints version information</pre><p>Many of these subcommands map directly back to the terminology and underlying CRDs I mentioned in the opening. It’s important to note that, by design, the tkn CLI is primarily a <em>read-only</em> interface, not a <em>read-write</em> one, so creation and editing of the objects still needs to go through kubectl. That’s because the creation of Tekton objects boils down to feeding YAML descriptions into the Kubernetes cluster API; modulo introducing a higher-order language like Relay or Jenkins X, there’s not much tkn can add over raw kubectl apply -f mypipeline.yaml. There are, however, several commands for controlling existing objects that are substantially nicer to use than their kubectl equivalents. For example, tkn pipeline start will kick off a new run of an existing pipeline, without you needing to craft a new YAML file for each one.</p><p>In addition to execution, the detailed output you get from tkn is vastly improved over thekubectl describe equivalents. For example, after running an <a href="https://github.com/tektoncd/pipeline/blob/master/examples/v1beta1/pipelineruns/conditional-pipelinerun.yaml">example pipeline using conditions</a> (more on those in a moment), I can get a compact list of recent runs and see useful information about the one I’m most interested in:</p><pre>&gt; tkn taskrun list<br>NAME                                               STARTED          DURATION     STATUS<br>condtional-pr-then-check-gtp96                     27 minutes ago   7 seconds    Succeeded<br>condtional-pr-then-check-gtp96-file-exists-v9m2t   27 minutes ago   14 seconds   Succeeded<br>condtional-pr-first-create-file-4n9zh              27 minutes ago   22 seconds   Succeeded<br>steps-run-in-order-p6v7q                           5 hours ago      9 seconds    Succeeded<br>echo-hello-world-task-run                          1 day ago        9 seconds    Succeeded</pre><pre>&gt; tkn taskrun describe condtional-pr-first-create-file-4n9zh             [0/3547]<br>Name:        condtional-pr-first-create-file-4n9zh<br>Namespace:   default<br>Task Ref:    create-readme-file<br>Service Account:   default</pre><pre>🌡️  Status</pre><pre>STARTED       DURATION     STATUS<br>4 hours ago   22 seconds   Succeeded</pre><pre>[ ... ]</pre><pre>🦶 Steps</pre><pre>NAME                                STATUS<br> ∙ write-new-stuff                   Completed<br> ∙ create-dir-workspace-868lz        Completed<br> ∙ source-copy-pipeline-git-bpt22    Completed<br> ∙ source-mkdir-pipeline-git-4lg5b   Completed</pre><pre>🚗 Sidecars</pre><pre>No sidecars</pre><p>(Yes, the emoji are part of the output! 😺)</p><p>The equivalent kubectl describe commands would spew out some verbose, unhelpful output and require lots of awk-ward manipulation in order to see the same level of usable info we see here.</p><h3>Pipeline Improvements: Conditions</h3><p>I mentioned above that the sample pipeline run we’re looking at makes use of Conditions. Conditions landed recently and are a welcome addition to Pipeline capabilities. Conditions extend Tekton’s core concept — launching task-specific containers and managing their success or failure — to enable decision-making in the middle of a pipeline run. Normally, a container’s entrypoint exiting with a non-zero exit code indicates that something went horribly wrong — it couldn’t clone a git repository, or unit tests failed, perhaps — but a container flagged as a condition failing just means that other Tasks marked as dependent on that container’s success will not be executed.</p><p>The <a href="https://github.com/tektoncd/pipeline/blob/master/examples/v1beta1/pipelineruns/conditional-pipelinerun.yaml">pipeline run that generated the output above</a> contains a condition that shows the utility and potential power of the feature.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/96511ce9865ebdd02b3578df7cef8a27/href">https://medium.com/media/96511ce9865ebdd02b3578df7cef8a27/href</a></iframe><p>This Condition uses the alpine image to run a really simple inline script to make sure a given file exists.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8d39559110ce68b868b83cc8e314a9aa/href">https://medium.com/media/8d39559110ce68b868b83cc8e314a9aa/href</a></iframe><p>The Pipeline then creates a README.md file on line 14, then uses file-exists condition (starting on line 22) to test whether the file was created successfully. If so, it executes the echo-hello task. Both of these individual tasks are defined as separate Task objects to enable reuse:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/54ad96d38a1dfeb739b939ed8e08dbe5/href">https://medium.com/media/54ad96d38a1dfeb739b939ed8e08dbe5/href</a></iframe><p>There’s an elegance to this design that, to me anyway, expresses one of the coolest parts of working with Kubernetes. It distills a problem down to a set of declarative resources: has this container run? If not, run it. Then builds upwards: Did it exit successfully? If so, proceed with the next task.</p><p>The specification is <a href="https://github.com/tektoncd/pipeline/blob/master/docs/conditions.md">described in the Conditions doc</a> and there are several more conditional <a href="https://github.com/tektoncd/pipeline/tree/master/examples/v1beta1/pipelineruns">examples in the tektoncd/pipeline repo</a>. Make sure you’re running the latest version as some of the implementation details are still solidifying as Tekton moves towards its beta release in early 2020.</p><h3>Dashboard and Trigger Deliciousness</h3><p>One of the best things about the Tekton community is the weekly working group calls. They’re open to anyone who joins the tekton-dev Google group (check out the details <a href="https://github.com/tektoncd/community/blob/master/working-groups.md">in the tektoncd/community repo</a>), stay on-topic more than 95% of meetings I’ve seen, and present a really powerful and positive example of how multi-vendor open source ought to work. On any given Wednesday, there’ll be representatives from Google, VMware, IBM, Red Hat (well, a different part of IBM, anyway!), eBay, Apple, D2iQ, and many other vendor and end user organizations. While some of these companies’ agendas might diverge, the team overall works in a remarkably healthy and cooperative manner to get stuff done: from collaborative comments on design docs to encouraging messages on pull requests, from my perspective Tekton is a burgeoning open-source success story.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*bYeir6qqqDqQvxWhOFyFLw.png" /><figcaption>Contributor encouragement … and kittens!</figcaption></figure><p>On a recent working group call, Andrea Frittoli from IBM did a two-fer demo, showing not only how <a href="https://dashboard.dogfooding.tekton.dev/">Tekton dogfoods Tekton for its own build and deployment pipeline</a>, but how far the Tekton Dashboard has come in displaying the inner workings of the infrastructure.</p><p>Dashboard is an graphical add-on to Tekton, primarily developed by IBM folks as an upstream component of the <a href="https://kabanero.io/">Kabanero.io</a> project, which (as you might expect) provides a graphical view into what’s going on across your Tekton deployment. I’ll give a brief rundown of Dashboard’s capabilities here; for an in-depth look, check out Adam Roberts’ blog post on the IBM developer blog <a href="https://developer.ibm.com/blogs/why-nows-a-great-time-to-use-the-tekton-dashboard/">“Why Now’s a Great Time to Use the Tekton Dashboard”</a> (for whatever value of “now” happens to be true for you!)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W65Bj746jObgefl9nSv6jg.png" /></figure><p>As you can see from the sidebar, the Dashboard has kept pace with the new CRDs that the platform implements. One that’s particularly interesting is the highlighted EventListeners resources — the ones in the dogfooding repo are “raw” resources, but there’s an <a href="https://github.com/tektoncd/experimental/blob/master/webhooks-extension/docs/GettingStarted.md">extension to the Dashboard</a> that allows you to easily configure webhook events from external services so you can (for example) send Tekton an event when a Github PR is merged.</p><p>The Dashboard’s webhooks extension provides a friendly interaction layer into the Triggers system, but even without it the <a href="https://github.com/tektoncd/triggers">Tekton Triggers</a> project is very cool. It allows Tekton to become responsive to events that originate outside of your Kubernetes cluster. This makes it much easier to integrate Tekton into your broader infrastructure, instead of requiring explicit requests against the k8s cluster API to trigger task and pipeline runs. The EventListeners , as in the screenshot above, provide an endpoint for external systems to talk to. As events come in, TriggerBindings transform fields form them into parameters forTriggerTemplates. The templates, in turn, generate resources like the Task and Pipeline runs that we’ve already seen. This level of dynamic instantiation truly levels Tekton’s capabilities up to the point where it can handle modeling the complexity of modern deployment scenarios.</p><h3>The End</h3><p>As you can tell by the length of this post, I’m really excited about all of the advancement in Tekton. As part of Relay, we’ve been trying to both keep abreast of the new features and contribute back wherever we can (in fact the cute kitten pic was from one of our pull requests!). The tkn CLI is an awesome example of using a familiar shell-based paradigm to interact with an API-driven service, Conditions bring a much-needed level of expressiveness to pipeline definitions, and we believe deeply in the <a href="https://medium.com/relay-sh/building-the-future-of-devops-automation-c25d6cf45ed1">future of event-driven automation</a>.</p><p>Here are some links to follow up with the Tekton community and the related projects I mentioned in this series:</p><ul><li>The <a href="https://github.com/tektoncd/community">tektoncd/community</a> repo has links to our Slack workspace and mailing lists.</li><li>The <a href="https://jenkins-x.io/community/">Jenkins X community page</a> has office hours and contact info.</li><li>The <a href="https://kabanero.io/">Kabanero project</a> is a spicy way to get teams deploying Kubernetes apps faster.</li><li>We’d love to get your input on which <a href="https://docs.google.com/forms/d/e/1FAIpQLSdwFjdpkX_DtAZveqY7uEPAsKhroutJXVSbfXRtVzYDmmpSRQ/viewform?usp=send_form">services and tools you’d like to connect with Relay</a>.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ab39176c9c45" width="1" height="1" alt=""><hr><p><a href="https://medium.com/relay-sh/whats-going-on-with-tekton-part-2-ab39176c9c45">What’s Going on With Tekton? (Part 2)</a> was originally published in <a href="https://medium.com/relay-sh">relay-sh</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What’s Going on With Tekton? (Part 1)]]></title>
            <link>https://medium.com/relay-sh/whats-going-on-with-tekton-part-1-9c3e006d39fc?source=rss-d2f9578e48a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/9c3e006d39fc</guid>
            <category><![CDATA[continuous-delivery]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Eric Sorenson]]></dc:creator>
            <pubDate>Wed, 25 Mar 2020 20:35:44 GMT</pubDate>
            <atom:updated>2020-05-18T17:26:21.416Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*F4MYnD7LzXbLPzAVr2QNzg.png" /></figure><h3>Introduction</h3><p><a href="http://tekton.dev/">Tekton</a> is a powerful tool for building continuous delivery pipelines using modern infrastructure. The project has been moving very quickly and has added a lot of awesome features recently, but due to the (necessary and wise) cancellation of conferences through the spring and summer of 2020 they have not gotten the attention they deserve. We’ve been working with Tekton as an upstream component of our devops automation service <a href="https://relay.sh/">relay.sh</a>, so I thought our new blog be a great chance to get the word out.</p><p>This 2-part blog series aims to give an overview of some core concepts for background and then dive into some more advanced technical topics and updates for Tekton. If you’re brand new to Tekton, check out Christie Wilson and Kim Lewandowski’s talk on <a href="https://www.youtube.com/watch?v=TQJ_pdTxZr0&amp;t=1s">“Next Generation CI/CD with GKE and Tekton”</a> from Google Cloud Next 2019. If you want to skip ahead, you can <a href="https://medium.com/relay-sh/whats-going-on-with-tekton-part-2-ab39176c9c45">go straight to Part 2</a>.</p><h3>Who’s Using Tekton?</h3><p>It’s important to note that Tekton is more of a platform for building CD tools around, rather than a user facing product. The most advanced example of products built on Tekton is probably <a href="http://jenkins-x.io/">Jenkins X</a>, a re-imagining of the ubiquitous Jenkins CI tool for cloud native applications. Jenkins X provides the user-facing GUI, a higher-level YAML dialect for describing pipelines, and a powerful GitOps workflow. To borrow git’s terminology, Jenkins X is the “porcelain”, while Tekton provides the “plumbing”: the pipeline execution engine that’s responsible for ordering the steps in a pipeline, launching pods that get the build/test/deploy work done, passing state and secrets around, etc. Ethan Jones at Cloudbees <a href="https://jenkins-x.io/blog/2020/03/11/tekton/">wrote a great blog post</a> describing the history of the projects and the rationale behind the Jenkins X team’s decision to consolidate on Tekton as the one-and-only execution engine.</p><p>Similarly, here at Puppet, we’ve been using Tekton as the plumbing under the service formerly known as Project Nebula — now out of incubation and renamed <a href="http://relay.sh/">Relay</a> — for the last year. Relay lets you define workflows of operations tasks and trigger them by linking Relay to external services that you already use, like Github or PagerDuty. The goal is to fill in what we see as an emerging category of <a href="https://medium.com/relay-sh/rise-of-the-apis-1d13d6968e73"><em>event-driven </em>devops automation</a>, now that desired-state and ad-hoc automation are well understood. When an event comes in to the service, Relay launches a container to figure out how to handle it (we’re calling this a trigger action). It can extract values from the incoming event and use them to fill in parameters in a workflow, which is launched as a Tekton pipeline on GCP. Similar to Jenkins X, workflows are <a href="https://github.com/puppetlabs/relay-workflows/">written in simple YAML</a> that the service translates into Tekton CRDs; Tekton then handles dependencies, ordering, and state management between workflow steps.</p><h3>A Brief History of the (Tekton) Universe</h3><p>Tekton is a project that evolved from an internal Google tool that used <a href="https://knative.dev/">Knative</a> to build and deploy software. In 2018, it was spun out as an independent project and donated to the Continuous Delivery Foundation, similar to Kubernetes’ donation to the CNCF.</p><p>The core component, Tekton Pipelines, runs as a controller in a Kubernetes cluster. It registers several custom resource definitions which represent the basic Tekton objects with the Kubernetes API server, so the cluster knows to delegate requests containing those objects to Tekton. These primitives are fundamental to the way Tekton works, and once you’ve got Tekton and the Tekton Dashboard up and running, you can view them like this:</p><pre>&gt; kubectl get crd | rg tekton | cut -f 1 -d &#39; &#39;<br>clustertasks.tekton.dev<br>conditions.tekton.dev<br>extensions.dashboard.tekton.dev<br>pipelineresources.tekton.dev<br>pipelineruns.tekton.dev<br>pipelines.tekton.dev<br>taskruns.tekton.dev<br>tasks.tekton.dev</pre><p>If the nomenclature here feels confusing, don’t feel bad — it is complicated! Each tool in the space uses slightly different terms; this is something we’re working on standardizing in the <a href="https://github.com/cdfoundation/sig-interoperability">CDF Interoperability SIG</a>. (We’d love your input!) Tekton’s usage of these terms — and the CRDs which implement the hierarchy of objects — maps closely to this list of CRDs (via the <a href="https://github.com/cdfoundation/sig-interoperability/blob/master/docs/vocabulary.md">sig-interop Vocabulary definitions doc</a>):</p><ul><li><strong>Step</strong>: a specific function to perform.</li><li><strong>Task</strong>: is a collection of sequential steps you would want to run as part of your continuous integration flow. A task will run inside a pod on your cluster.</li><li><strong>ClusterTask</strong>: Similar to Task, but with a cluster scope.</li><li><strong>Pipeline</strong>: stateless, reusable, parameterized collection of tasks. Tasks are linked together in a Pipeline, which describes the end-to-end deployment for an application.</li><li><strong>PipelineRun</strong>: an instantiation of a Pipeline definition, filling in the Pipeline’s parameters with concrete values</li><li><strong>Pipeline Resource</strong>: objects that will be input to or output from tasks</li><li><strong>Trigger</strong>: Triggers is a Kubernetes Custom Resource Definition (CRD) controller that allows you to extract information from event payloads (a “trigger”) to create Kubernetes resources.</li></ul><p>Notable omissions from this list are “Steps”, which don’t have their own CRD because they’re the smallest unit of execution which are always contained inside a Task; plus Conditions and Dashboard Extensions, which are still optional and experimental — but very exciting! We’ll talk about these more in part 2.</p><p>If you’re working on an in-house CD platform, and especially if you’re looking at ways to deliver cloud-native applications with a toolchain that looks and feels like the apps it’s managing, Tekton could be a good fit. <a href="https://medium.com/relay-sh/whats-going-on-with-tekton-part-2-ab39176c9c45">Read on if these capabilities sound intriguing…</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9c3e006d39fc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/relay-sh/whats-going-on-with-tekton-part-1-9c3e006d39fc">What’s Going on With Tekton? (Part 1)</a> was originally published in <a href="https://medium.com/relay-sh">relay-sh</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>