<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Susovan Panja on Medium]]></title>
        <description><![CDATA[Stories by Susovan Panja on Medium]]></description>
        <link>https://medium.com/@susovanpanja?source=rss-6042739db668------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*GcMkgasHM4SjKoRiSa4R0g.jpeg</url>
            <title>Stories by Susovan Panja on Medium</title>
            <link>https://medium.com/@susovanpanja?source=rss-6042739db668------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 00:18:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@susovanpanja/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Don’t live in the shadow of others]]></title>
            <link>https://medium.com/@susovanpanja/dont-live-in-the-shadow-of-others-abcfb44a8774?source=rss-6042739db668------2</link>
            <guid isPermaLink="false">https://medium.com/p/abcfb44a8774</guid>
            <category><![CDATA[self-realization]]></category>
            <dc:creator><![CDATA[Susovan Panja]]></dc:creator>
            <pubDate>Fri, 17 Apr 2026 11:01:32 GMT</pubDate>
            <atom:updated>2026-04-17T11:01:32.471Z</atom:updated>
            <content:encoded><![CDATA[<h4>Are you a Honker?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rf9m1OYcGthAbf3wYYwEEg.png" /><figcaption>AI Generated</figcaption></figure><p>I am on the highway. On the flyover. Driving my Alto K10. Going to the office for 1 day/week, compulsory RTO (Return To Office) ritual. Listening to the music. My speed is around 40 km/h.</p><p>I suddenly had a light bulb moment. Why am I driving so slowly? I generally drive at 90 km/h on highways. I realised that I am driving behind a car, unintentionally. In a way, following the car. The car is slow, and it made me slow down too.</p><p>I changed the lane. Picked up the pace. Overtook the car. Reached the office in my own way.</p><h4>Eureka Moment:</h4><p>Same day. Back at home. Listening to some music. Thinking about my morning drive to the office. I suddenly made a connection. The highway is my career. The car in front is my mentor/manager. If my manager is slow or less skilled, I should overtake them instead of following them. Otherwise, I will be slow like them.</p><p>If you feel you are stuck with a slow manager, change your lane. It can be anything, a person, a habit, an object, or a location that is constraining you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=abcfb44a8774" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GitHub Advanced Security (GHAS)— Part 1]]></title>
            <link>https://medium.com/@susovanpanja/github-advanced-security-ghas-part-1-d9754742b403?source=rss-6042739db668------2</link>
            <guid isPermaLink="false">https://medium.com/p/d9754742b403</guid>
            <category><![CDATA[dependencies]]></category>
            <category><![CDATA[code]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[code-security]]></category>
            <category><![CDATA[sdlc]]></category>
            <dc:creator><![CDATA[Susovan Panja]]></dc:creator>
            <pubDate>Mon, 06 Oct 2025 07:50:43 GMT</pubDate>
            <atom:updated>2025-10-06T07:50:43.896Z</atom:updated>
            <content:encoded><![CDATA[<h3>GitHub Advanced Security (GHAS)-Part 1</h3><h4>Fast-track your code security if you are on GitHub</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jPL4uhfe4D-ga-zsPNl3Dw.png" /><figcaption>Image Credit: https://github.blog</figcaption></figure><blockquote>Public repos on GitHub.com → GHAS is free (all features).</blockquote><blockquote>Private/internal repos → GHAS is paid, based on active committers.</blockquote><blockquote>GitHub Enterprise Server → GHAS is always paid, regardless of repo visibility.</blockquote><h4>What is GHAS?</h4><p>GHAS is a collection of security tools (CodeQL, Dependabot, and Secret Scanning) that is available directly from GitHub. We just need to toggle some buttons or check some boxes to make it work for our code.</p><blockquote>GHAS solution can be broken down into two products in terms of what can be purchased as bundle:<br><strong>1.</strong> <strong>Secret Protection </strong>-&gt; Enables Secret scanning on the repos as well as pro-active mitigations. Tool: Secret Scanning<br><strong>2. Code Security</strong> -&gt; Enables code scanning and dependency scanning for dependent library/packages. Tools: CodeQL, Dependabot</blockquote><p><strong><em>What would be covered in this multi-part series?</em></strong><br><strong><em>1. </em></strong>Understanding these tools and what they can do for our code security journey.<br><strong><em>2.</em></strong> How to get started? Which boxes to check or not check?<br><strong><em>3.</em></strong> Proactively stop security vulnerabilities from creeping up in our code.</p><p>I would try to put all the links and references in between so that for advanced or custom setups, those can be consulted.</p><h4>What are the tools?</h4><p><strong><em>1) CodeQL:</em></strong><em> </em>This is the code scanning tool. It scans the code and detects any security vulnerabilities in the code. It can run on push to a particular branch OR on opening a new PR, OR on a scheduled time (cron job). It can essentially run on any GitHub event, but the most common are the above three.</p><blockquote><strong>CodeQL</strong> is geared more towards the security of the code and not to the code clean/health best practices which <a href="https://www.sonarsource.com/products/sonarqube/"><strong>SonarQube</strong></a> is more inclined. For eg. hardcoded variables like Country Name/Code, SonarQube can detect but not CodeQL (yet).</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YYYrxHd54ERUTgCQqN5wzg.png" /><figcaption>CodeQL Open Alerts in main Branch</figcaption></figure><p>For <strong>compiled languages</strong> like Java, we need to build the artifact to get the complete code vulnerability report.</p><p>For <strong>interpreted languages </strong>like JS (JavaScript), it runs the scan on the source code.</p><p><strong><em>2) Dependabot:</em> </strong>This tool scans package manifest files (<em>pom.xml, package.json</em>) and package lock files (<em>package-lock.json</em>) and creates a <strong><em>dependency graph </em></strong>to understand the direct as well as transitive (dependency of your dependency) dependencies of the source code. Then it checks these packages along with the exact version against the <a href="https://github.com/advisories"><strong><em>GitHub Advisory Database</em></strong></a><strong><em> </em></strong>for security vulnerabilities. And alerts us for the package along with version and severity (critical, high, medium, moderate, and low).</p><blockquote>There are many terms like CVE, CWE, GHSA ID. I will try to cover those in next part, as it’s important to understand these terms and how they can be used to track and mitigate package related issues.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PG5BYqKNUxGxXU7T_Sy6gQ.png" /><figcaption>Dependabot Open Alerts.</figcaption></figure><blockquote>Dependabot shows alerts for default branch only.</blockquote><p>Dependabot not only alerts us, but it also gives us <strong>recommendations</strong>, where possible, for a <strong>newer secure version</strong> of a vulnerable package, in which the vulnerability has been fixed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y_zP_FVvGJpmgTX4j3Lc9A.png" /><figcaption>Dependabot recommendation for version bump to resolve vulnerability issue.</figcaption></figure><p>Apart from the vulnerability recommendation, it can also recommend a <strong>newer version of packages when available</strong>, so that we are using latest versions in our code.</p><p>Dependabot can also automatically create PRs for <strong><em>Security</em></strong> <strong><em>Updates</em></strong> and<em> </em><strong><em>Version Updates</em></strong>. We will look into how to set these up in the next part.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wmJjJcx_10KSpVTRRfYTYw.png" /><figcaption>Dependabot create a PR to resolve a vulnerability issue.</figcaption></figure><p><strong><em>3) Secret Scanning:</em> </strong>This tool can detect known types of secrets in our source code repo. It not only scans the code but also titles, descriptions, and comments in pull requests, issues raised, and in discussions.</p><p>The example of secrets is any type of password, AWS/Azure/GCP access keys or GitHub PAT (Personal Access Token) key.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tKqim6Xb-xHD-13Vc4P26A.png" /><figcaption>Secret Scanning Open Alerts</figcaption></figure><blockquote>It also supports <strong>custom pattern</strong>, where we can define our own regex expression for secret scanning. The pattern of the secret should be specified as <strong>Hyperscan regex</strong>.</blockquote><h4><strong><em>Dependency Graph:</em></strong></h4><p>It is a built-in feature in GitHub that creates a list of all the libraries, packages, and modules that a project/code repo depends upon. It works with many languages, like Java, JavaScript, Go, Python, etc.</p><blockquote><strong>Dependabot</strong> uses <strong>Dependency Graph</strong> for its package scanning.</blockquote><blockquote>It can also help us to generate <strong>SBOM (Software Bill of Materials)</strong> for our project/ code repo.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cj0iwv0Pm5KrdC62bZkjMg.png" /><figcaption>Dependency Graph showing all the direct/transitive dependencies.</figcaption></figure><p>If you’ve enjoyed this story, please click the 👏 button and share it, so that others can find it as well! Also, feel free to leave a comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d9754742b403" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Checkov — IaC Scanner]]></title>
            <link>https://medium.com/@susovanpanja/checkov-iac-scanner-db14c30529dd?source=rss-6042739db668------2</link>
            <guid isPermaLink="false">https://medium.com/p/db14c30529dd</guid>
            <category><![CDATA[iac]]></category>
            <category><![CDATA[scanning-tool]]></category>
            <category><![CDATA[checkov]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[devsecops]]></category>
            <dc:creator><![CDATA[Susovan Panja]]></dc:creator>
            <pubDate>Mon, 14 Oct 2024 06:00:30 GMT</pubDate>
            <atom:updated>2024-10-14T06:08:52.302Z</atom:updated>
            <content:encoded><![CDATA[<h3>Checkov — IaC Scanner</h3><h4>Tool Opinion</h4><p>My first interaction with Checkov was a few years back, and it resulted from one of the conversations I had with our Solutions Architect and my good friend Anirudh. A few people across the globe were contributing to the IaC code (Terraform), and we need to ensure misconfigurations do not creep into infrastructure via the IaC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*fMYYwvwBOY8Ms33Udloevw.png" /></figure><h3>Infrastructure Misconfiguration</h3><p>It basically means we have created/coded our infrastructure in such a way that may cause data breach, compliance issue, hacking etc. unintentionaly</p><p><strong>Examples of misconfiguration:</strong><br>1. Storage bucket made public.<br>2. Database opened to public (0.0.0.0/0)<br>3. Un-restricted access on SSH ports (0.0.0.0/0)<br>4. pgaudit not enabled for Cloud Postgresql</p><h3>What is Checkov?</h3><p>Think <a href="https://www.checkov.io/1.Welcome/What%20is%20Checkov.html">of Checkvo</a> as a SAST (Static Code Analysis) scanner for IaC code. It can find misconfigurations in IaC code that may lead to security or compliance problems. It comes with more than 750 predefined policies to check for common misconfiguration issues, which I find sufficient for most projects. It also supports <a href="https://checkov.io/3.Custom%20Policies/Custom%20Policies%20Overview.html">custom policies</a>, which is outside the scope of this article but may be in the future.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CkU77YOVkPEZMnaSmcaapg.gif" /><figcaption><a href="https://www.checkov.io/">https://www.checkov.io/</a></figcaption></figure><h3>IaC types supported</h3><p>Checkov supports a wide range of IaC types. Our scope will be Terraform on GCP.</p><ul><li>Terraform (for AWS, GCP, Azure and OCI)</li><li>CloudFormation (including AWS SAM)</li><li>Azure Resource Manager (ARM)</li><li>Serverless framework</li><li>Helm charts</li><li>Kubernetes</li><li>Docker</li></ul><h3>Installation</h3><p><em>{a} Install via pip =&gt;</em></p><pre>$ pip3 install checkov --user</pre><p><em>{b} Install via homebrew =&gt;</em></p><pre>$ brew tap bridgecrewio/checkov https://github.com/bridgecrewio/checkov<br>$ brew update<br>$ brew install checkov</pre><h3>Running scan locally</h3><p>For this article, we are going to use the terraform code that is “<em>Vulnerable by Design</em>” and provided by Checkov itself — <a href="https://github.com/bridgecrewio/terragoat"><strong><em>TerraGoat</em></strong></a></p><ol><li>Clone the repo.</li></ol><pre>$ git clone https://github.com/bridgecrewio/terragoat.git</pre><p>2. Switch to terraform folder.</p><pre>$ cd terragoat/terraform/</pre><p>3. Execute the scan.</p><pre># For a single file we can use -f &lt;file_name&gt; and <br># for a directory -d &lt;directory_name&gt;<br>$ checkov -d gcp</pre><h3>Analyze the result</h3><pre>➜  terraform git:(master) checkov -d gcp<br>[ terraform framework ]: 100%|████████████████████|[7/7], Current File Scanned=gcp/variables.tf<br>[ secrets framework ]: 100%|████████████████████|[7/7], Current File Scanned=gcp/gcs.tf<br>       _               _<br>   ___| |__   ___  ___| | _______   __<br>  / __| &#39;_ \ / _ \/ __| |/ / _ \ \ / /<br> | (__| | | |  __/ (__|   &lt; (_) \ V /<br>  \___|_| |_|\___|\___|_|\_\___/ \_/<br><br>By Prisma Cloud | version: 3.2.257<br><br>terraform scan results:<br><br>Passed checks: 17, Failed checks: 56, Skipped checks: 0<br><br>Check: CKV_GCP_56: &quot;Ensure PostgreSQL database &#39;log_temp_files flag is set to &#39;0&#39;&quot;<br> PASSED for resource: google_sql_database_instance.master_instance<br> File: /big_data.tf:1-19<br> Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/google-cloud-policies/cloud-sql-policies/bc-gcp-sql-7<br>Check: CKV_GCP_55: &quot;Ensure PostgreSQL database &#39;log_min_messages&#39; flag is set to a valid value&quot;<br> PASSED for resource: google_sql_database_instance.master_instance<br> File: /big_data.tf:1-19<br> Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/google-cloud-policies/cloud-sql-policies/bc-gcp-sql-6</pre><p>If we analyze the above result, 17 passed and 56 failed.</p><p>Let&#39;s analyze one of the failed checks.</p><pre>Check: CKV_GCP_11: &quot;Ensure that Cloud SQL database Instances are not open to the world&quot;<br> FAILED for resource: google_sql_database_instance.master_instance<br> File: /big_data.tf:1-19<br> Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/google-cloud-policies/google-cloud-networking-policies/bc-gcp-networking-4<br><br>  1  | resource &quot;google_sql_database_instance&quot; &quot;master_instance&quot; {<br>  2  |   name             = &quot;terragoat-${var.environment}-master&quot;<br>  3  |   database_version = &quot;POSTGRES_11&quot;<br>  4  |   region           = var.region<br>  5  |<br>  6  |   settings {<br>  7  |     tier = &quot;db-f1-micro&quot;<br>  8  |     ip_configuration {<br>  9  |       ipv4_enabled = true<br>  10 |       authorized_networks {<br>  11 |         name  = &quot;WWW&quot;<br>  12 |         value = &quot;0.0.0.0/0&quot;<br>  13 |       }<br>  14 |     }<br>  15 |     backup_configuration {<br>  16 |       enabled = false<br>  17 |     }<br>  18 |   }<br>  19 | }</pre><p>The check is <strong><em>CKV_GCP_11: “Ensure that Cloud SQL database Instances are not open to the world”. </em></strong>This check will look for any Cloud SQL resource which are made public as part of our terraform code.</p><p>It will show the violating file name with line number “<strong><em>/big_data.tf:1–19</em></strong>” and the code snippet.</p><blockquote>Also the link to the guide “<a href="https://docs.prismacloud.io/en/enterprise-edition/policy-reference/google-cloud-policies/google-cloud-networking-policies/bc-gcp-networking-4"><strong><em>https://docs.prismacloud.io/en/enterprise-edition/policy-reference/google-cloud-policies/google-cloud-networking-policies/bc-gcp-networking-4</em></strong></a>”, so that we can understand the vulnerability itself and also how to fix it.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*BJWZPLSw4bEEUjAGQnzsIw.png" /><figcaption><a href="https://docs.prismacloud.io/">https://docs.prismacloud.io/</a></figcaption></figure><h3>Exclusions</h3><p>There maybe times, when we have written a resource in such a way that it is against best practices for infrastructure, but we have a clear use case for that. Example can be a public bucket that contains all the consumer facing documentation.</p><p>In those situations we need to exclude that resource or to suppress the check itself for the scanning job.</p><p>There are two ways of exclusion:</p><ol><li><strong>Skip certain checks for the entire scan</strong></li></ol><pre># This will skip the above check failure (public Cloud SQL), multiple <br># &lt;check_id&gt; allowed<br>$ checkov -d gcp --skip-check CKV_GCP_11<br>OR<br>$ checkov -d gcp --skip-check CKV_GCP_11,CKV_GCP_12</pre><p>2. <strong>Skip certain resources from the scan</strong></p><p>To skip a check on a given Terraform definition block, apply the following comment pattern inside its scope: checkov:skip=&lt;check_id&gt;:&lt;suppression_comment&gt;</p><pre>resource &quot;google_sql_database_instance&quot; &quot;master_instance&quot; {<br>  name             = &quot;terragoat-${var.environment}-master&quot;<br>  database_version = &quot;POSTGRES_11&quot;<br>  region           = var.region<br><br>  #checkov:skip=CKV_GCP_11:The Cloud SQL is part of honeypot setup<br><br>  settings {<br>    tier = &quot;db-f1-micro&quot;<br>    ip_configuration {<br>      ipv4_enabled = true<br>      authorized_networks {<br>        name  = &quot;WWW&quot;<br>        value = &quot;0.0.0.0/0&quot;<br>      }<br>    }<br>    backup_configuration {<br>      enabled = false<br>    }<br>  }<br>}</pre><blockquote>There are different ways for skipping/suppressing checks, depending upon the IaC type we are scanning. More examples <a href="https://www.checkov.io/2.Basics/Suppressing%20and%20Skipping%20Policies.html">here</a>.</blockquote><h3>Integrations</h3><p>Checkov can be integrated with the code repository and with CI/CD providers, like Jenkins, and GitHub Actions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/277/1*3Rwvs1-08X1VCfW-zfCbXA.png" /></figure><p><em>Sample GitHub Workflow:</em></p><pre>---<br>name: Checkov<br>on:<br>  push:<br>    branches:<br>      - master<br>jobs:<br>  build:<br><br>    runs-on: ubuntu-latest<br>    steps:<br>      - uses: actions/checkout@v2<br>      - name: Set up Python 3.8<br>        uses: actions/setup-python@v4<br>        with:<br>          python-version: 3.8<br>      - name: Test with Checkov<br>        id: checkov<br>        uses: bridgecrewio/checkov-action@master<br>        with:<br>          directory: example/examplea<br>          framework: terraform </pre><blockquote><em>For more integrations </em><a href="https://www.checkov.io/4.Integrations/GitHub%20Actions.html"><em>https://www.checkov.io/4.Integrations/GitHub%20Actions.html</em></a></blockquote><h3>When to scan</h3><ol><li><strong>Run the scan locally,</strong> before committing and pushing to git.</li><li>Run it as part of <strong>MR</strong> (merge request) or <strong>PR</strong> (pull request).</li><li>Run it as part of <strong>Terraform Build </strong>(plan, deploy).</li></ol><blockquote>Just to clarify, the above points are not <strong>OR</strong> but <strong>AND</strong>. Scan wherever and whenever possible.</blockquote><h3>Conclusion</h3><p>As we rapidly move from on-prem to cloud infrastructure, the attack surface has increased exponentially. Infrastructure is the base of any application we build and run; it must be protected. If infrastructure is not secured, our application, and user data are also not secured.</p><p>Since we are treating infrastructure as code (IaC), we must embrace all the scanning, testing, and securing practices used by developers in SDLC and start introducing them in our IDLC (Infrastructure Development Life Cycle). Using Checkov or any other similar tool is step 1.</p><h3>Bonus</h3><p><a href="https://www.checkov.io/4.Integrations/Kubernetes.html"><strong><em>Kubernetes Cluster Scanning (Realtime)</em></strong></a></p><p>Checkov can be used to scan the Kubernetes cluster resources during runtime. It just needs read-only access. We need to apply the provided manifest and the scanning job will start on the cluster.</p><pre>### List all the scan variables:<br>$ checkov -l --framework kubernetes<br><br>### k8s manifest file scanning:<br>$ checkov -f deployment.yaml<br><br>### Run checov against live k8s cluster:<br>$ kubectl apply -f https://raw.githubusercontent.com/bridgecrewio/checkov/main/kubernetes/checkov-job.yaml<br>$ kubectl get jobs -n checkov<br>$ kubectl logs job/checkov -n checkov</pre><p><a href="https://www.checkov.io/7.Scan%20Examples/Helm.html"><strong><em>Helm Chart Scanning</em></strong></a></p><p>Checkov can also scan helm charts. Helm charts are basically templates, so we need to convert them to actual deployable manifest files. The process is somewhat the same for third-party helm charts and private helm charts.</p><p><strong>The steps are:</strong></p><p>1. Add the helm repo to local helm registry. We can also fetch and extract the helm charts. So that we can examine the template/code and we know exactly what resources we are creating on k8s cluster, which images we are pulling etc. I prefer the second approach as I know exactly what I am doing with the helm chart and its easier to debug any issues.</p><p>2. Create chart configuration file i.e. <strong><em>gocd.yaml. </em></strong>Every helm chart have a values.yaml file, it contains the default config values for the chart. The approach should be create copy of this file, it can be named anything, in our case it is <strong><em>gocd.yaml</em></strong>. Update this new config file with all the changes we want and use this to apply the helm chart.</p><p>3. Use <strong><em>helm template</em></strong> command to convert the helm chart to deployable manifest file.</p><p>4. Run C<strong><em>heckov</em></strong> scan against the generated manifest file i.e.<strong><em> k8s-template.yaml</em></strong></p><p>5. Analyze the result and adjust the helm chart accordingly.</p><pre>### Download the helm chart and create the config file<br>$ helm repo add gocd https://gocd.github.io/helm-chart<br>$ helm inspect values gocd/gocd &gt; gocd.yaml<br><br>### Run checov against helm charts:<br>$ helm template gocd/gocd -f gocd.yaml &gt; k8s-template.yaml<br>$ checkov -f k8s-template.yaml --framework kubernetes</pre><p>If you’ve enjoyed this story, please click the 👏 button and share it, so that others can find it as well! Also, feel free to leave a comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=db14c30529dd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Git-bash in Windows Terminal]]></title>
            <link>https://medium.com/@susovanpanja/git-bash-inwindows-terminal-fc198cc15c8e?source=rss-6042739db668------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc198cc15c8e</guid>
            <category><![CDATA[windows]]></category>
            <category><![CDATA[setup]]></category>
            <category><![CDATA[terminal-windows]]></category>
            <category><![CDATA[git-bash]]></category>
            <category><![CDATA[setup-guide]]></category>
            <dc:creator><![CDATA[Susovan Panja]]></dc:creator>
            <pubDate>Mon, 23 Sep 2024 10:29:22 GMT</pubDate>
            <atom:updated>2024-09-23T10:35:29.610Z</atom:updated>
            <content:encoded><![CDATA[<p>So that we don’t have to open Git-bash as separate window</p><p>This one is a quick writeup. I am creating this so that my future me can refer to it later 😛.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E6aaHCSJSI1vATtg5-GCFg.jpeg" /><figcaption>AI Generated</figcaption></figure><h3>Problem 🚨</h3><p>So after using Linux and MacOS (company provided) as my daily driver for more than a decade, I am forced to use Windows.</p><p>To do, what I do, I need to install {bash, git, WSL, minikube, Docker CLI} etc. I started with git installation and it comes with git-bash. The only irritation for me is it opens up a separate window.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CuJV5LgTkTW5DzOJAEcQdA.png" /></figure><h3>Desired State 😀</h3><p>What I want is to integrate git-bash in Windows Terminal as Dropdown, open new terminal under tabs, use shortcut keys etc.</p><h3>Solution ✅</h3><p><strong>Step 1:</strong> Get the git-bash installation folder. For me it was “<em>C:\Users\susovan.panja\AppData\Local\Programs\Git</em>”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mCe99MOrE6m2yW_AUi5Qtw.png" /><figcaption>Git installation location</figcaption></figure><p><strong>Step 2:</strong> We need the complete path for “<strong>bash.exe</strong>” and not “<strong>git-bash.exe</strong>” i.e. “<em>C:\Users\susovan.panja\AppData\Local\Programs\Git\bin\bash.exe</em>”.</p><p><strong>Step 3:</strong> Open <strong>Windows Terminal </strong>&gt; <strong>Settings</strong> and click on “<strong>Add New Profile</strong>”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Kdo149EY2QpHqkETPUe7zw.png" /></figure><p><strong>Step 4:</strong> Fill in the below details</p><p><strong># It can be anything, whatever you fancy<br>Name:</strong> <em>Git Bash</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tcJFK5uHSzEt3YpsMyeBtg.png" /></figure><p><strong>Command line:</strong> <em>C:\Users\susovan.panja\AppData\Local\Programs\Git\bin\bash.exe</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XmekpRR1EfJ1hR0o70YYLQ.png" /></figure><p><strong># It is recomended to use your git repos local folder<br>Starting directory: </strong>C:\work</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*biga8imGkRRJ0Y1XiCrt4Q.png" /></figure><p><strong># This is optional, but like the icon to display in the tab<br>Icon:</strong> C:\Users\susovan.panja\AppData\Local\Programs\Git\mingw64\share\git\git-for-windows.ico</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8rI7Gn2-tHCa4yJ7H8m0Ug.png" /></figure><p><strong>Step 5:</strong> Voilà!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*DLgata-6Cc1XgB_-s0-60A.png" /></figure><p>If you’ve enjoyed this story, please click the 👏 button and share it, so that others can find it as well! Also, feel free to leave a comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc198cc15c8e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build your own Technical Documentation Portal using BackStage — Part 4]]></title>
            <link>https://medium.com/@susovanpanja/build-your-own-technical-documentation-portal-using-backstage-part-4-aa2ffc1579af?source=rss-6042739db668------2</link>
            <guid isPermaLink="false">https://medium.com/p/aa2ffc1579af</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[backstage]]></category>
            <category><![CDATA[devops-tool]]></category>
            <category><![CDATA[azure]]></category>
            <dc:creator><![CDATA[Susovan Panja]]></dc:creator>
            <pubDate>Tue, 20 Aug 2024 14:25:59 GMT</pubDate>
            <atom:updated>2024-08-20T14:25:59.303Z</atom:updated>
            <content:encoded><![CDATA[<h3>Build your own Technical Documentation Portal using BackStage — Part 4</h3><h4>Publish documents on Azure Cloud via TechDocs</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1yOgDdbhttOd1P1pd3NBVA.png" /></figure><p>Previously, Part 3 of this series shows a working Backstage instance running on top of Azure Cloud. Let&#39;s separate our document repo(s) from the Backstage Repo and start publishing.</p><p><a href="https://medium.com/@susovanpanja/build-your-own-technical-documentation-portal-using-backstage-part-1-343b6dcf1dd7">Build your own Technical Documentation Portal using BackStage — Part 1</a></p><p><a href="https://medium.com/@susovanpanja/build-your-own-technical-documentation-portal-using-backstage-part-2-e314bdcc9aff">Build your own Technical Documentation Portal using BackStage — Part 2</a></p><p><a href="https://medium.com/@susovanpanja/build-your-own-technical-documentation-portal-using-backstage-part-3-811993103e88">Build your own Technical Documentation Portal using BackStage — Part 3</a></p><p><a href="https://github.com/susovanpanja/backstage-app-medium/tree/main">Source code on GitHub — Backstage App Code</a></p><p><a href="https://github.com/susovanpanja/backstage-docs-medium">Source code on GitHub — Document Repo</a></p><h3>Preparing the document repo:</h3><p>For the document repo, we need the ‘<strong>docs’ </strong>folder, <strong><em>mkdocs.yaml</em></strong> and <strong><em>catalog-info.yaml</em></strong> (same as <strong><em>sample_document.yaml</em></strong>).</p><blockquote>Refer to part 3 of this series for more explanation. The source code for this repo can be found at <a href="https://github.com/susovanpanja/backstage-docs-medium">https://github.com/susovanpanja/backstage-docs-medium</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/1*6a5wDmMY3Y7C3YcNKTMnuw.png" /><figcaption>Sample structure for Document Repo</figcaption></figure><h3>GitHub PAT token</h3><p>In part 3, we have generated a PAT token on GitHub to be used by Backstage. Update that token (convert into base64 first ) in the ‘<a href="https://github.com/susovanpanja/backstage-app-medium/blob/main/aks_deploy_manifest/backstage-secrets.yaml">https://github.com/susovanpanja/backstage-app-medium/blob/main/aks_deploy_manifest/backstage-secrets.yaml</a>’ file and apply on the AKS cluster.</p><blockquote>Do this on local machine and do not push that secret to Github.</blockquote><h3>On-board the Document Repo as a Component on Backstage UI</h3><p><strong>Step 1:</strong> Open the BackStage UI. Click on ‘<strong><em>Create</em></strong>’ in the right-hand top corner.</p><p><strong>Step 2:</strong> On the new page, click ‘<strong><em>Register Existing Component</em></strong>’.</p><p><strong>Step 3: </strong>In the<strong> </strong>new page paste the full URL of ‘<strong><em>catalog-info.yaml’. </em></strong>Backstage will fetch the file and ask to import. Click on ‘<strong><em>Import</em></strong>’ and then <strong><em>‘View Component’</em></strong></p><blockquote>The full URL in our case is ‘<a href="https://github.com/susovanpanja/backstage-docs-medium/blob/master/catalog-info.yaml">https://github.com/susovanpanja/backstage-docs-medium/blob/master/catalog-info.yaml</a>’</blockquote><blockquote>If we go to the techdocs section, it will show us error as we have connected the backstage instance to ‘<strong>Azure Cloud Container</strong>’ but did not published the documents. This will be our next task.</blockquote><blockquote>If there is some issue with fetch, look in the logs, it might be github PAT issue.</blockquote><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F_NUbiZNNtb0%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D_NUbiZNNtb0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F_NUbiZNNtb0%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e1152ace274e7810cb27cf5f0caea214/href">https://medium.com/media/e1152ace274e7810cb27cf5f0caea214/href</a></iframe><h3>Create Azure Storage Container</h3><pre>techdocs:<br>  builder: external<br>  generator:<br>    runIn: local<br>  publisher:<br>    type: &#39;azureBlobStorage&#39;<br>    azureBlobStorage:<br>      containerName: &#39;backstage-techdocs&#39;<br>      credentials:<br>        accountName: ${TECHDOCS_AZURE_BLOB_STORAGE_ACCOUNT_NAME}<br>        accountKey: ${TECHDOCS_AZURE_BLOB_STORAGE_ACCOUNT_KEY}</pre><p>We have already seen this snippet of config yaml (<strong><em>app-config.yaml</em></strong>) in part 3. All we need is an <strong><em>Azure Storage Container</em></strong> (our is <strong><em>backstage-techdocs</em></strong>). We also need a <strong><em>Storage Account Name</em></strong> and <strong><em>Account Key </em></strong>with writer permission on the<strong><em> Azure Storage Container.</em></strong></p><blockquote>Both <strong><em>Storage Account Name</em></strong> and <strong><em>Account Key </em></strong>values need to be converted into base64 and updated in file ‘<a href="https://github.com/susovanpanja/backstage-app-medium/blob/main/aks_deploy_manifest/backstage-secrets.yaml">https://github.com/susovanpanja/backstage-app-medium/blob/main/aks_deploy_manifest/backstage-secrets.yaml</a>’ and apply from local machine.</blockquote><h3>Setup CI/CD (Jenkins)</h3><p>Jenkins is used to build the static files from the Markdown(.MD) documents and push them to Azure Storage Container.</p><p>We will be using the same <strong><em>Storage Container</em></strong> to host multiple documents/components separated by component-named folders.</p><p>Sample Jenkins File.</p><blockquote>The container image we are using is from DockerHub, we cal build our image also the minimum requirements are ‘<strong>python 3</strong>’ and ‘<strong>nodejs 22</strong>’ binaries in the same container.<br>Rest is self-explanatory. Sample code URL:</blockquote><pre>pipeline {<br>    agent {<br>        docker {<br>            image &#39;nikolaik/python-nodejs:python3.12-nodejs22-slim@sha256:ff897d2b27bb714f3485ff118ba4ed9de5540dc5300158ea9a069a88039207c1&#39;<br>        }<br>    }<br>    options {<br>        disableConcurrentBuilds()<br>        buildDiscarder(logRotator(numToKeepStr: &#39;5&#39;))<br>    }<br>    environment {<br><br>    BACKSTAGE_COMP_NAME = &quot;sample-document&quot; // It should be same as our component name<br><br>    // Replace by actual azure storage account name, this is not a secret.<br>    AZURE_STORAGE_ACC_NAME = &quot;example_account&quot; <br><br>    }  <br><br>    stages{<br>        stage(&#39;Publish and Push&#39;) {<br>            steps{<br>                    // Install @techdocs/cli, mkdocs and mkdocs plugins<br>                    sh &#39;&#39;&#39;<br>                    node -v<br>                    python3 --version<br>                    export PATH=$PATH:/opt/jenkins/.local/bin<br><br>                    npm install @techdocs/cli<br>                    pip3 install &quot;mkdocs-techdocs-core==1.*&quot; &quot;mkdocs-exclude&quot;<br><br>                    npx techdocs-cli generate --no-docker --verbose<br><br>                    &#39;&#39;&#39;<br>                    println &quot;Component name is ${BACKSTAGE_COMP_NAME}&quot;<br><br>                    // Store Azure Service Princliple Credentials in Jenkins Credential store<br><br><br><br>                    withCredentials([azureServicePrincipal(credentialsId: &quot;AzureSP&quot;)]) {  <br>                    sh &#39;&#39;&#39;<br>                        # Publish<br>                        npx techdocs-cli publish --publisher-type azureBlobStorage --storage-name backstage-techdocs --entity &quot;default/Component/${BACKSTAGE_COMP_NAME}&quot; --azureAccountName ${AZURE_STORAGE_ACC_NAME}<br>                    &#39;&#39;&#39;<br>                    }<br>                }<br>        }    <br>    }    <br>}       </pre><blockquote>Recommended way of setting up the Jenkins pipeline will be webhook, so that any changes made in the Github repo syncs in neat real-time.</blockquote><h3>Using GitHub Actions</h3><p>If we don’t use Jenkins in our setup, Backstage also provides a way to utilize GitHub Actions for build and publish. Please follow the below instructions:</p><p><a href="https://backstage.io/docs/features/techdocs/configuring-ci-cd/#example-github-actions-ci-and-aws-s3">Configuring CI/CD to generate and publish TechDocs sites | Backstage Software Catalog and Developer Platform</a></p><blockquote>The above example is for AWS S3, just replace AWS variables with Azure variables.</blockquote><h3><strong>Verify on Azure Storage Bucket</strong></h3><p>Execute the above pipeline and everything goes fine, we should be able to see the files inside the Azure Storage Container.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/986/1*iX-QMGO9y8pMqg_XzCsuJg.png" /></figure><p>And we will also be able to see the documents in Backstage UI.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fh3LLnHKa59I%3Flist%3DPLNOyuXJ-e83_F1MnVcVYxWk4_8GoS88Md&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dh3LLnHKa59I&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fh3LLnHKa59I%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ae11d5ec6f2dad378dea1ca8e6fa8a6c/href">https://medium.com/media/ae11d5ec6f2dad378dea1ca8e6fa8a6c/href</a></iframe><h3>On-boarding a new component</h3><p>To onboard a new-component follow the below steps:</p><ol><li><strong><em>Prepare the Documents Repo: </em></strong><br>Create <strong>docs folder</strong> and start putting all the documents in <strong>.MD</strong> format. Create <strong>mkdocs.yaml </strong>for navigation. <br>Create <strong>catalog-info.yaml</strong> for defining the type and configuration of the component. <br>Create the <strong>JenkinsFile</strong>.</li><li><strong><em>Create the Jenkins/Github Pipeline.</em></strong></li><li><strong><em>Execute the Pipeline and verify on Azure Portal.</em></strong></li><li><strong>On-board the component/document repo on Backstage UI and verify techdocs.</strong></li></ol><p>In next article, we will work with “<strong><em>Setting up basic Software Templates on Backstage.io</em></strong>”</p><p>If you’ve enjoyed this story, please click the 👏 button and share it, so that others can find it as well! Also, feel free to leave a comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aa2ffc1579af" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build your own Technical Documentation Portal using BackStage — Part 3]]></title>
            <link>https://medium.com/@susovanpanja/build-your-own-technical-documentation-portal-using-backstage-part-3-811993103e88?source=rss-6042739db668------2</link>
            <guid isPermaLink="false">https://medium.com/p/811993103e88</guid>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[backstage]]></category>
            <dc:creator><![CDATA[Susovan Panja]]></dc:creator>
            <pubDate>Thu, 20 Jun 2024 15:07:31 GMT</pubDate>
            <atom:updated>2024-06-20T15:07:31.674Z</atom:updated>
            <content:encoded><![CDATA[<h3>Build your own Technical Documentation Portal using BackStage — Part 3</h3><h4>Deployment on Azure Cloud</h4><p>In previous parts of this series, we have talked about how to get started with backstage using a local developer environment, how to create documentation and publish those on backstage, using the tech-docs plugin.</p><p>In this part, we will talk about deploying backstage on Azure Cloud.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SEgYd1XdqLPOPXo6rTQXQg.png" /><figcaption>Backstage setup on Azure Cloud</figcaption></figure><p><a href="https://medium.com/@susovanpanja/build-your-own-technical-documentation-portal-using-backstage-part-1-343b6dcf1dd7">Build your own Technical Documentation Portal using BackStage — Part 1</a></p><p><a href="https://medium.com/@susovanpanja/build-your-own-technical-documentation-portal-using-backstage-part-2-e314bdcc9aff">Build your own Technical Documentation Portal using BackStage — Part 2</a></p><p><a href="https://github.com/susovanpanja/backstage-app-medium/tree/main">Source code on GitHub</a></p><h4>Azure Infrastructure Resources:</h4><ol><li><strong>Azure Container Registry </strong>-&gt; To store a single Backstage application container image.</li><li><strong>Azure Kubernetes Service (AKS)</strong>-&gt; To deploy the Backstage application.</li><li><strong>Azure Postgresql Database</strong> -&gt; For Backstage to save application configurations. A minimal/shared instance will work.</li><li><strong>Azure Postgresql Credentials</strong> -&gt; For Backstage to connect to the Postgresql Instance. <br><em>Backstage currently does not support Entra (token ) based authentication for Postgresql.</em></li><li><strong>Azure Storage Account</strong> -&gt; To store technical documentation rendered as html files. Required for tech-docs.</li><li><strong>Azure Service Principal </strong>-&gt; For Backstage application to connect to Storage(rw).</li></ol><blockquote>These resources can be created manually or using IaC like <a href="https://www.terraform.io/">terraform</a>, <a href="https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview?tabs=bicep">biceps</a>, etc. Please make sure all the resources are inside a private network and are not publicly exposed.</blockquote><blockquote>We are not going to cover the resource creation portion here. If you are interested let me know and I can share the terraform code for the same.</blockquote><h3>Github PAT (Personal Access Token):</h3><p>The PAT will be used by backstage to communicate with Github for repo creation (software template), read <strong><em>catalog-info.yaml </em></strong>files (software catalog, software template catalog), etc.</p><p><a href="https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic">Instructions to create PAT on GitHub.</a></p><blockquote>Start will least privilege on GitHub, only check repo level access for this PAT, like creating repo, pushing to repo. Do not enable repo deletion for this PAT unless we want to manage that also via Backstage.</blockquote><blockquote>For production deployment recommendation is to use <a href="https://backstage.io/docs/integrations/github/github-apps/">GitHub App Credentials</a> instead of PAT.</blockquote><h3>Deployment on AKS</h3><h4>Step 1: Prepare the app-config.yaml</h4><p>For production setup <strong><em>app-config.yaml</em></strong> file is used for configuring Backstage. The example file is already in the sample GitHub repo.</p><blockquote>Add <strong><em>app-config-local.yaml </em></strong>in <strong><em>.gitignore</em></strong>, if not already there, and remove it from git tracking. This file is only for local environment setup and should not be in the git repo. I have this file in the sample GitHub repo just as an example.</blockquote><p><strong>The sections we need to update:<br><em>database: </em></strong>The database connectivity information.<strong><em> </em></strong>The variables will be picked up from environment variables. We will talk more about it during Kubernetes Secrets creation.</p><pre>  database:<br>    client: pg<br>    pluginDivisionMode: schema<br>    connection:<br>      host: ${POSTGRES_HOST} # Azure Postgresql instance connection url<br>      port: ${POSTGRES_PORT}<br>      user: ${POSTGRES_USER}<br>      password: ${POSTGRES_PASSWORD}<br>      database: backstage # Database will be created if not exist<br>      ssl:<br>        require: true<br>        rejectUnauthorized: true</pre><p><strong><em>techdocs: </em></strong>Instead of building locally, we will build it externally (Jenkins pipeline) and publish it to Azure Blob Storage. This we will cover in detail, in the next article on this series.</p><pre>techdocs:<br>  builder: external<br>  generator:<br>    runIn: local<br>  publisher:<br>    type: &#39;azureBlobStorage&#39;<br>    azureBlobStorage:<br>      containerName: &#39;backstage-techdocs&#39;<br>      credentials:<br>        accountName: ${TECHDOCS_AZURE_BLOB_STORAGE_ACCOUNT_NAME}<br>        accountKey: ${TECHDOCS_AZURE_BLOB_STORAGE_ACCOUNT_KEY}</pre><p><strong><em>locations: </em></strong>In the local environment we have picked up relative paths for component, entity, and system definition files. For production, we want to change them to GitHub URLs, so that if we make any changes or updates to the files, we don’t have to rebuild the Backstage container image again.</p><pre>  locations:<br>    - type: url<br>      target: https://github.com/susovanpanja/backstage-app-medium/blob/main/examples/entities.yaml <br>    - type: url<br>      target: https://github.com/susovanpanja/backstage-app-medium/blob/main/examples/template/template.yaml<br>      rules:<br>        - allow:<br>            - Template<br>    - type: url<br>      target: https://github.com/susovanpanja/backstage-app-medium/blob/main/examples/org.yaml<br>      rules:<br>        - allow:<br>            - User<br>            - Group</pre><h4>Step 2: Build the Container Image</h4><p>Since we have built this Azure Infrastructure Setup in a private network, none of the resources can be accessed outside that network. So, to go ahead with the next steps:</p><p>a) Create a VM inside the same network on Azure. We need docker/<a href="https://podman.io/">podman</a> and Azure CLI installed on this machine.</p><p>b) SSH into the VM and execute the below commands.</p><pre># Build the container image from the Dockerfile<br># We are building with the actual ACR repo url, so that extra step of tagging<br># is not required<br><br># Dockerfile is in the sample repo<br><br>DOCKER_BUILDKIT=1 docker image build -t centralusacr.azurecr.io/backstage/backstage:1.21.1 .</pre><h4>Step 3: Push to ACR (Azure Container Registry)</h4><pre># Login to Azure and ACR<br><br>az login<br>az acr login --n centralusacr<br><br># Push the dockr image to ACR<br><br>docker push centralusacr.azurecr.io/backstage/backstage:1.21.1</pre><h4>Step 4: Prepare the Kubernetes spec files</h4><p>All the spec files are in the ‘<strong><em>aks_deploy_manifest</em></strong>’ folder in the sample repo, just update the secrets with their <strong><em>base64</em></strong> version and apply.</p><p>We have four YAML files for deployment:</p><p><strong><em>backstage-secrets.yaml: </em></strong>It contains GitHub PAT token, and Azure Blob Storage Credentials (Account_Name, Account_Key).</p><p><strong><em>postgres-secrets.yaml:</em></strong> It contains Postgresql information and static credentials.</p><p><strong><em>backstage.yaml: </em></strong>This is the backstage deployment spec file.</p><p><strong><em>backstage-service.yaml:</em></strong> This will create a service for backstage deployment.</p><blockquote>If we look closely in the <strong><em>backstage.yaml </em></strong><em>file, we are exporting values from</em><strong><em> backstage-secret </em></strong><em>and </em><strong><em>postgres-secret </em></strong><em>as environment values to the</em><strong><em> Backstage POD. </em></strong><em>These will be replaced in </em><strong><em>app-config.yaml </em></strong><em>during runtime and will be available for the backstage app to use. That’s how backstage will be able to connect to Github, Postgresql and Azure Blob Storage.</em></blockquote><h4>Step 5: Deployment on AKS</h4><pre># Fetch AKS credentials<br><br>$ az aks get-credentials --resource-group  centralus-rg --name aks-sbx <br><br># apply all the manifest file<br><br>$ kubectl apply -f backstage-secrets.yaml<br>$ kubectl apply -f postgres-secrets.yaml<br>$ kubectl apply -f backstage.yaml<br>$ kubectl apply -f backstage-service.yaml<br><br># Check the status of pods, it should be in running state<br><br>$ kubectl get pods -n backstage</pre><h4>Step 6: Expose the service and Access</h4><p>To expose the service outside of the Kubernetes cluster we will be using <a href="https://learn.microsoft.com/en-us/azure/application-gateway/ingress-controller-expose-service-over-http-https"><strong>Azure Application Gateway</strong></a>. The sample ingress file is located in the ‘<strong><em>aks_deploy_manifest</em></strong>’ folder. The service will be exposed over private IP.</p><p>Access the IP over the private network and we will be welcomed by the same Backstage UI we saw in our local environment.</p><pre>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br>  name: backstage-ing<br>  annotations:<br>    kubernetes.io/ingress.class: azure/application-gateway<br>    appgw.ingress.kubernetes.io/use-private-ip: &quot;true&quot;<br>spec:<br>  rules:<br>  - http:<br>      paths:<br>      - backend:<br>          serviceName: backstage<br>          servicePort: 80</pre><p>Part 4 of this series will contain:</p><p><em>Publishing the documents from GitHub -&gt; Azure Storage -&gt; Backstage.</em></p><p>If you’ve enjoyed this story, please click the 👏 button and share it, so that others can find it as well! Also, feel free to leave a comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=811993103e88" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build your own Technical Documentation Portal using BackStage — Part 2]]></title>
            <link>https://medium.com/@susovanpanja/build-your-own-technical-documentation-portal-using-backstage-part-2-e314bdcc9aff?source=rss-6042739db668------2</link>
            <guid isPermaLink="false">https://medium.com/p/e314bdcc9aff</guid>
            <category><![CDATA[techdoc]]></category>
            <category><![CDATA[documentation]]></category>
            <category><![CDATA[backstage]]></category>
            <category><![CDATA[idp]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Susovan Panja]]></dc:creator>
            <pubDate>Fri, 07 Jun 2024 11:08:00 GMT</pubDate>
            <atom:updated>2024-06-07T11:08:00.098Z</atom:updated>
            <content:encoded><![CDATA[<h3>Build your own Technical Documentation Portal using BackStage — Part 2</h3><h4>powered by Tech-Docs Plugin</h4><p>In this series&#39;s first part, we have set up the Backstage application on the local development environment. In this part, we will configure the tech-docs plugin and render documentation using docker.</p><p><a href="https://medium.com/@susovanpanja/build-your-own-technical-documentation-portal-using-backstage-part-1-343b6dcf1dd7">Build your own Technical Documentation Portal using BackStage — Part 1</a></p><p><a href="https://github.com/susovanpanja/backstage-app-medium/tree/main">Source code on GitHub</a></p><h3>Prepare the documentation:</h3><p>In the local setup, we will use the same folder/repo for the documents as the backstage app. When we deploy on <strong>Azure</strong> we will be using separate repos for the backstage app and the documents repo (<em>We can onboard as many repos on Backstage as we need</em>).</p><blockquote>There is one-to-one mapping of <strong>repo to Backstage component</strong>. Components in Backstage are software components, it can be a service, website or library. <a href="https://backstage.io/docs/features/software-catalog/descriptor-format/#kind-component">More info</a>.</blockquote><p><strong>Step 1:</strong> Create a <strong>docs folder</strong> in the root of your source code repo. Since we are on local setup we will create this in the <strong>examples folder</strong>.</p><p><strong>Step 2:</strong> Create the documents inside the <strong>docs folder</strong>, in markdown format. We are going to use backstage document files for this tutorial. There should be an <strong><em>index.md </em></strong>file, it becomes the default entry point to the documents site.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*NBSKJZqOGi7pkQ34IdKOZw.png" /><figcaption>docs folder will contain all the documents</figcaption></figure><blockquote><strong><em>Tech-docs</em></strong> uses <strong><em>mkdocs</em></strong> in the backend to render the documents written in markdown format (.MD). <a href="https://www.markdownguide.org/basic-syntax/">Markdown Syntax</a></blockquote><p>Step 3: Create <strong><em>mkdocs.yaml </em></strong>file at the root of the source code repo, outside the docs folder. In our case the <strong>example folder</strong>.</p><blockquote><strong><em>mkdocs.yaml</em></strong> file is used to create the navigation menu for our documents, it gives the structure to the documents.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lrtrjFGyxlGDNbUaJx3pqw.png" /><figcaption>Sample mkdocs.yaml format</figcaption></figure><h3>Configure tech-docs:</h3><p><strong>Step 1:</strong> Create a component file (<strong><em>sample_document.yaml</em></strong>) at the root of the source code, <strong>examples</strong> <strong>folder</strong> in our case. The file name can be anything.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sGDzaPPcx2foPGjsw7XwxQ.png" /><figcaption>Sample component example</figcaption></figure><blockquote>Do not put space in between the name, it might not get picked up. ‘sample document’ might not get picked up.</blockquote><blockquote>The annotation enables the tech-docs feature for the documents (repo).</blockquote><pre>  annotations:<br>    backstage.io/techdocs-ref: dir:.</pre><p><strong>Step 2: </strong>Backstage uses <strong><em>app-config.yaml</em></strong> to pick the configuration, but on the local system, it also provides <strong><em>app-config.local.yaml </em></strong>file to override the configuration for local development. We are going to use this file for our configurations. Make the same changes to <strong><em>app-config.yaml, </em></strong>if you want the same configuration in your deployment environment.</p><p>Add configurations for tech-docs.<br>Add the location of <strong><em>sample_document.yaml </em></strong>in the catalog section, we should go two levels up to reach the examples folder<strong><em> ‘../../examples/sample_document.yaml’.</em></strong></p><blockquote>The <strong><em>app-config.local.yaml</em></strong> file is ignored by <strong><em>.gitignore</em></strong>, do not push this file to your git, as it might end up on your deployment environment. Included in the sample repo just for example.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AyWBLFR1vGFx5ZTEAYiChA.png" /><figcaption>Connecting the component to the Backstage using app-config.local.yaml</figcaption></figure><blockquote>The above image only shows the difference. For actual file content refer to the source code. Link provided at the top of this article.</blockquote><h3>Start the Backstage UI:</h3><p>Run the below command and it will open up a new tab, wait for a few minutes to load. It should look like the following video.</p><pre>yarn dev</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gjAfKcWKjGxvIi3L8yiMwg.png" /><figcaption>Component list in Backstage app</figcaption></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fh3LLnHKa59I%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dh3LLnHKa59I&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fh3LLnHKa59I%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/45d5243308d8e48dbd5caece7a8233d1/href">https://medium.com/media/45d5243308d8e48dbd5caece7a8233d1/href</a></iframe><h3>Debugging tech-docs:</h3><p>There might be various reasons why tech-docs might not build the documents. The reasons might be that docker is not installed on the system or it is not able to pull the required image.</p><p><strong>First</strong>, make sure the component is visible as above.<br><strong>Second</strong>, view the build logs as shown below.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FHZcb1hxU2-g%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DHZcb1hxU2-g&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FHZcb1hxU2-g%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/c493a9dfd9c66fd7083559d4db4b96a1/href">https://medium.com/media/c493a9dfd9c66fd7083559d4db4b96a1/href</a></iframe><p>Part 3 will contain:</p><p><em>Deploying backstage application on Azure AKS and publishing the documents from GitHub -&gt; Azure Storage -&gt; Backstage.</em></p><p>If you’ve enjoyed this story, please click the 👏 button and share it, so that others can find it as well! Also, feel free to leave a comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e314bdcc9aff" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build your own Technical Documentation Portal using BackStage — Part 1]]></title>
            <link>https://medium.com/@susovanpanja/build-your-own-technical-documentation-portal-using-backstage-part-1-343b6dcf1dd7?source=rss-6042739db668------2</link>
            <guid isPermaLink="false">https://medium.com/p/343b6dcf1dd7</guid>
            <category><![CDATA[technical-documentation]]></category>
            <category><![CDATA[backstage]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[idp]]></category>
            <category><![CDATA[azure]]></category>
            <dc:creator><![CDATA[Susovan Panja]]></dc:creator>
            <pubDate>Fri, 31 May 2024 10:46:11 GMT</pubDate>
            <atom:updated>2024-06-07T11:17:08.732Z</atom:updated>
            <content:encoded><![CDATA[<h3>Build your own Technical Documentation Portal using BackStage — Part 1</h3><h4>powered by Tech-Docs Plugin</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wje_BzgrQEhX5Y0k94Cm1w.png" /><figcaption>Credit: https://backstage.io</figcaption></figure><p>Backstage is a platform to build custom <a href="https://www.cortex.io/post/what-is-an-internal-developer-portal">IDP</a> (Internal Developer Portal). Spotify created it to give developers a single pane of glass to manage, develop, and explore the internal software ecosystem. It was then donated to CNCF and made open source. It is now out of the CNCF sandbox and is an incubating project.</p><p>Out of the box, Backstage includes:</p><ul><li><strong>Backstage Software Catalog</strong> for managing all your software (microservices, libraries, data pipelines, websites, ML models, etc.)</li><li><strong>Backstage Software Templates</strong> for quickly spinning up new projects and standardizing your tooling with your organization’s best practices</li><li><strong>Backstage TechDocs</strong> for making it easy to create, maintain, find, and use technical documentation, using a “<strong>docs-like-code</strong>” approach</li><li><strong>Backstage Kubernetes </strong>helps monitor all our service’s deployments at a glance, even across clusters.</li><li><strong>Backstage Search</strong> is<strong> </strong>a universal search for backstage instances that can search against documentation, software templates, software catalogs, and APIs.</li><li>Plus, a growing ecosystem of <strong>open-source plugins</strong> that further expand Backstage’s customizability and functionality</li></ul><p>We will discuss Techdocs in this article and how we can leverage this to write, generate, view, and search your documents in the Backstage app.</p><h3>Setup Backstage Local Development machine</h3><h4>Pre-Requisites:</h4><ol><li><a href="https://nodejs.org/en/download/package-manager">node</a> (my local version is v20.9.0)</li><li>npm (my local version is 10.4.0)</li><li><a href="https://classic.yarnpkg.com/lang/en/docs/install/#mac-stable">yarn</a> (my local version is 1.22.19)</li><li>docker (my local version is 20.10.9){this is needed for tech-docs, there are other ways to build the documents also in the local system but it makes life easier}</li><li>git (my local version is 2.39.3)</li></ol><blockquote><em>I am on macOS Sonoma 14.4.1.</em></blockquote><h4>Creating Backstage App:</h4><ol><li>Go to your working folder for me it is <em>/work/backstage</em>.</li></ol><pre>cd /work/backstage</pre><p>2. Run the below command and give the app a name. I have given <em>athena. </em>It will create a folder named <em>athena</em> in side <em>/work/backstage, copy all the backstage code,</em> and install dependencies inside the <em>athena</em> folder. The output should look like the following:</p><pre>npx @backstage/create-app@latest</pre><pre>########### Output of the command ################<br>➜ npx @backstage/create-app@latest<br>Need to install the following packages:<br>@backstage/create-app@0.5.15<br>Ok to proceed? (y) y<br>? Enter a name for the app [required] athena # Name of the Backstage(IDP) App<br><br>Creating the app...<br><br> Checking if the directory is available:<br>  checking      athena ✔<br><br> Creating a temporary app directory:<br><br> Preparing files:<br>  copying       .dockerignore ✔<br>  copying       .eslintignore ✔<br>  templating    .eslintrc.js.hbs ✔<br>  templating    .gitignore.hbs ✔<br>  copying       .prettierignore ✔<br>  copying       README.md ✔<br>  copying       app-config.production.yaml ✔<br>  copying       app-config.local.yaml ✔<br>  templating    app-config.yaml.hbs ✔<br>  templating    backstage.json.hbs ✔<br>  templating    catalog-info.yaml.hbs ✔<br>  copying       lerna.json ✔<br>  templating    package.json.hbs ✔<br>  copying       playwright.config.ts ✔<br>  copying       tsconfig.json ✔<br>  copying       yarn.lock ✔<br>  copying       README.md ✔<br>  copying       entities.yaml ✔<br>  copying       org.yaml ✔<br>  copying       template.yaml ✔<br>  copying       catalog-info.yaml ✔<br>  copying       index.js ✔<br>  copying       package.json ✔<br>  copying       README.md ✔<br>  templating    .eslintrc.js.hbs ✔<br>  copying       Dockerfile ✔<br>  copying       README.md ✔<br>  templating    package.json.hbs ✔<br>  copying       index.ts ✔<br>  copying       .eslintignore ✔<br>  templating    .eslintrc.js.hbs ✔<br>  templating    package.json.hbs ✔<br>  copying       app.test.ts ✔<br>  copying       android-chrome-192x192.png ✔<br>  copying       apple-touch-icon.png ✔<br>  copying       favicon-16x16.png ✔<br>  copying       favicon-32x32.png ✔<br>  copying       favicon.ico ✔<br>  copying       index.html ✔<br>  copying       manifest.json ✔<br>  copying       robots.txt ✔<br>  copying       safari-pinned-tab.svg ✔<br>  copying       App.test.tsx ✔<br>  copying       App.tsx ✔<br>  copying       apis.ts ✔<br>  copying       index.tsx ✔<br>  copying       setupTests.ts ✔<br>  copying       LogoFull.tsx ✔<br>  copying       LogoIcon.tsx ✔<br>  copying       Root.tsx ✔<br>  copying       index.ts ✔<br>  copying       SearchPage.tsx ✔<br>  copying       EntityPage.tsx ✔<br><br> Moving to final location:<br>  moving        athena ✔<br>  fetching      yarn.lock seed ✔<br>  init          git repository ◜<br> Installing dependencies:<br>  init          git repository ✔<br>  determining   yarn version ✔<br>  executing     yarn install ✔<br>  executing     yarn tsc ✔<br><br>🥇  Successfully created athena # All show be green checkmark<br><br><br> All set! Now you might want to:<br>  Run the app: cd athena &amp;&amp; yarn dev<br>  Set up the software catalog: https://backstage.io/docs/features/software-catalog/configuration<br>  Add authentication: https://backstage.io/docs/auth/</pre><p>3. Launch the Backstage application.</p><pre>cd athena &amp;&amp; yarn dev</pre><pre>########### Output of the command ################<br>➜ cd athena &amp;&amp; yarn dev<br>yarn run v1.22.19<br>$ concurrently &quot;yarn start&quot; &quot;yarn start-backend&quot;<br>$ yarn workspace backend start<br>$ yarn workspace app start<br>$ backstage-cli package start<br>$ backstage-cli package start<br>[0] Loaded config from app-config.yaml<br>[0] &lt;i&gt; [webpack-dev-server] Project is running at:<br>[0] &lt;i&gt; [webpack-dev-server] Loopback: http://localhost:3000/, http://[::1]:3000/<br>[0] &lt;i&gt; [webpack-dev-server] Content not from webpack is served from &#39;/Users/susovanpanja/work/medium/backstage/athena/packages/app/public&#39; directory<br>[0] &lt;i&gt; [webpack-dev-server] 404s will fallback to &#39;/index.html&#39;<br>[0] &lt;i&gt; [webpack-dev-middleware] wait until bundle finished: /<br><br>... # Redacted full log</pre><p>4. It should open up a new tab in the browser and should look like this (it will take some time to load the UI):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fKaqgWRbBcNK333X1vlklQ.png" /><figcaption>Backstage UI | localhost:3000</figcaption></figure><p>The local development environment setup is done. In the later parts of this series, we will be :<br><em>a) Configuring tech-docs and integrating our documents with Backstage (using docker).</em><br><em>b) Deploying backstage application on Azure AKS and publishing the documents from GitHub -&gt; Azure Storage -&gt; Backstage.</em></p><p><a href="https://medium.com/@susovanpanja/build-your-own-technical-documentation-portal-using-backstage-part-2-e314bdcc9aff">Build your own Technical Documentation Portal using BackStage — Part 2</a></p><p>If you’ve enjoyed this story, please click the 👏 button and share it, so that others can find it as well! Also, feel free to leave a comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=343b6dcf1dd7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CloudWatch Agent Configuration on Windows EC2 Instance]]></title>
            <link>https://medium.com/@susovanpanja/cloudwatch-agent-configuration-on-windows-ec2-instance-cbcb377b9917?source=rss-6042739db668------2</link>
            <guid isPermaLink="false">https://medium.com/p/cbcb377b9917</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[sms]]></category>
            <category><![CDATA[systems-manager]]></category>
            <category><![CDATA[cloudwatch-agent]]></category>
            <category><![CDATA[cloudwatch]]></category>
            <dc:creator><![CDATA[Susovan Panja]]></dc:creator>
            <pubDate>Fri, 12 Jan 2024 08:54:32 GMT</pubDate>
            <atom:updated>2024-01-12T08:54:32.025Z</atom:updated>
            <content:encoded><![CDATA[<h4>Using Systems Manager (SSM)</h4><h4>What is CloudWatch {Agent}?</h4><p><a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html">Amazon CloudWatch</a> is an AWS service that helps in monitoring your AWS resources like EC2 instances, RDS (Database) Instances, S3 buckets/objects, Application Signals, X-ray tracing, etc.</p><p><a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html">CloudWatch Agent</a> is a small tool that can collect metrics as well as logs from EC2 instances and on-prem instances running either Linux or Windows Operating Systems.</p><p><strong>Note: </strong><em>The default namespace for metrics collected by the CloudWatch agent is </em><em>CWAgent, although we can specify a different namespace when configuring the agent.</em></p><h4>What is Systems Manager (SSM)?</h4><p><strong>As per AWS documentation: </strong><a href="https://docs.aws.amazon.com/systems-manager/latest/userguide/what-is-systems-manager.html">AWS Systems Manager</a> is the operations hub for your AWS applications and resources and a secure end-to-end management solution for hybrid and multicloud environments that enables secure operations at scale.</p><p>Explaining all the capabilities of SSM is a topic for another article 😝.</p><p>In this article, we will be leveraging the “<a href="https://docs.aws.amazon.com/systems-manager/latest/userguide/run-command.html">Run Command</a>” capability with a small glimpse of “<a href="https://docs.aws.amazon.com/systems-manager/latest/userguide/fleet.html">Fleet Manager</a>”</p><h4>Pre-Requisites:</h4><ol><li>Active AWS account.</li><li>Permission to launch EC2, create IAM roles, SSM admin permissions, etc.</li></ol><h4>Step 1: Create an IAM role for the EC2 instance</h4><ol><li>Login to your AWS console and go to the IAM (Identity and Access Management) service.</li><li>On the left side menu, <strong><em>Access management</em></strong> -&gt; <strong><em>Roles</em></strong>, click on ‘<strong><em>Create Role</em></strong>’ button on the top right-hand side.</li><li>Select ‘<strong><em>AWS service</em></strong>’ as the <strong><em>Trusted entity type</em> </strong>and<strong> ‘<em>EC2 </em></strong>’ as the <strong><em>Use Case</em></strong>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JAVbwy_mJqTU6gvVf1521g.png" /><figcaption>Pick the AWS service</figcaption></figure><p>4. In the next window select these three policies and click next; <strong><em>AmazonSSMManagedInstanceCore</em></strong>, <strong><em>CloudWatchAgentAdminPolicy, </em></strong>and<strong><em> CloudWatchAgentServerPolicy.</em></strong></p><p>5. In the next window give the role a name, ‘<strong><em>Cloudwatch_Agent_Manage_by_SSM</em></strong>’, and give a <strong>tag</strong> to this role for traceability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3iBQU1KY_6D7pcVMynAi_A.png" /><figcaption>Policies to select for the role</figcaption></figure><h4>Step 2: Launch Windows EC2 Instance</h4><ol><li>Go to the EC2 dashboard, and click on ‘<strong><em>Launch Instance</em></strong>’.</li><li>Give the instance a name, ‘<strong><em>Cloudwatch-SSM-POC</em></strong>’. Select ‘<strong><em>Windows 2022</em></strong>’ as <strong>AMI</strong> (Amazon Machine Image).</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SndFjRMpHnCQcFy0iAcFCQ.png" /><figcaption>Select the AMI</figcaption></figure><p>3. Pick an instance type, ‘<strong><em>t2.micro</em></strong>’ (free tier) in this case. And generate a <strong>key-pair </strong>(<strong><em>RSA, .pem</em></strong>) or re-use an existing one. Save this key (<strong><em>ssm-poc</em></strong>) <strong>securely</strong>, this will be needed when we will <strong>retrieve</strong> (decrypt) the <strong><em>Windows administrator password</em></strong> later in this article.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*odunZOmHr5kJ4ymbiSvrfA.png" /><figcaption>Select instance type and key pair</figcaption></figure><p>4. In ‘<strong><em>Network setting</em></strong>’, select a <strong><em>VPC Network</em></strong>, it can be default VPC or custom VPC. ‘<strong><em>Auto-assign public IP</em></strong>’ can be enabled to assign a random public IP to the instance automatically or we can disable it here and assign an <strong><em>Elastic IP</em></strong> <strong><em>(EIP)</em></strong> in later stages also.</p><p><strong>Note:</strong> <em>Just assigning a public IP, does not mean that the instance is now open to everyone, it depends upon the open ports in the </em><strong><em>Firewall (security group)</em></strong><em>, so be careful.</em></p><p>In the <strong><em>Firewall (security group)</em></strong> section, create a new <strong>SG</strong> <strong><em>(security group) </em></strong>or re-use an existing one. Make sure <strong>only</strong> ‘<strong><em>Allow RDP traffic from</em></strong>’ is checked <strong><em>(port 3389)</em></strong> and <strong>only</strong> ‘<strong><em>MyIP</em></strong>’ is allowed in the firewall.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lXt71-MURDJswai0nK88rg.png" /><figcaption>Select the firewall rule</figcaption></figure><p>5. Expand ‘<strong><em>Advanced details’</em></strong>, and select the role created in <strong>step 1</strong>(<strong><em>Cloudwatch_Agent_Manage_by_SSM</em></strong>), from the ‘<strong><em>IAM instance profile</em></strong>’ dropdown.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eGeOIuvOoZk8te7dGFXhcA.png" /><figcaption>Select the IAM role</figcaption></figure><p>6. Keep the rest of the settings as default and click on ‘<strong><em>Launch instance</em></strong>’</p><h4>Step 3: Connect to the Windows EC2 Instance</h4><ol><li>Go to the<strong> Instances</strong> tab and look for the instance name we created in <strong>Step 2</strong>. All the ‘<strong><em>Status check</em></strong>’ should be passed and green. Also, make sure the <strong>IAM Role</strong> attached is the correct one, <strong><em>Cloudwatch_Agent_Manage_by_SSM</em></strong>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u0tz-e5Oe--vSZ8qIcO3-g.png" /></figure><p>2. Search for ‘<strong>Systems Manager</strong>’ in the search bar and go to the <strong><em>Systems Manager</em></strong> -&gt; <strong><em>Fleet Manager</em></strong>. The instance will be visible under ‘<strong><em>Managed Nodes</em></strong>’. The <strong>node ID</strong> is the same as the <strong>Instance ID</strong> in the ‘<strong><em>EC2 Dashboard’</em></strong>. This verifies that now SSM can manage this instance and run commands.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*le9cQfQHBMtuLpVIHVUPaw.png" /></figure><p>3. In the <strong>EC2 Dashboard</strong>, select the instance, and from the ‘<strong><em>Actions</em></strong>’ dropdown select ‘<strong><em>Connect</em></strong>’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jHIrVHCEMffhX1yYhmborg.png" /><figcaption>Select the instance and click connect</figcaption></figure><p>4. Click on the ‘<strong><em>RDP client</em></strong>’ tab in the next window. Click on ‘<strong><em>Download remote desktop file</em></strong>’ to download the <strong><em>.rdp</em></strong> file on your local machine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S-mXh-aVL4KsXb14e9VBEw.png" /><figcaption>Download the remote desktop file</figcaption></figure><p>5. To get the Windows Administrator user password for this machine, click on ‘<strong><em>Get password</em></strong>’. This will open up a new window for ‘<strong><em>Get Windows password</em></strong>’, and upload the ‘<strong><em>ssm-poc.pem</em></strong>’ file that we downloaded in <strong>Step 2</strong>. The file name is also mentioned under ‘<strong><em>Key pair associated with this instance</em></strong>’. Click on ‘<strong><em>Decrypt password</em></strong>’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CHmNJ_GRmIRV-jn0UsSX5Q.png" /><figcaption>Decrypt windows password</figcaption></figure><p>6. In the last line, the <strong><em>password</em></strong> can be seen, save it securely.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZTLak0IsdtTAlkH-Dcpu5g.png" /><figcaption>Save the password</figcaption></figure><p>7. Launch the RDP (Remote Desktop) client. I am using <a href="https://learn.microsoft.com/en-us/windows-server/remote/remote-desktop-services/clients/remote-desktop-mac"><strong>Microsoft Remote Desktop</strong></a> on macOS.</p><p>8. Click ‘<strong><em>Import from rdp file…</em></strong>’, select the downloaded <strong><em>.rdp</em></strong> file, and ‘<strong><em>import</em></strong>’. Launch the RDP session and enter the password we decrypted earlier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1dR-LsHif4JoOUtRWJxkOQ.png" /><figcaption>Import rdp file</figcaption></figure><p>9. Once inside the instance, launch the <strong><em>PowerShell (PS) terminal</em></strong> and execute the below command to verify if the ‘<strong><em>CloudWatch agent</em></strong>’ is installed or not.</p><pre>Get-Service -Name &quot;AmazonCloudWatchAgent&quot;</pre><p>10. The output verifies, it is not installed yet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DeSEvlid_F9VQ2_5y7rLqw.jpeg" /></figure><h4>Step 4: Install CloudWatch Agent on the Windows EC2 Instance via Systems Manager</h4><ol><li>Go to<strong> the Systems Manager</strong> screen. Select the instance/node, click on <strong><em>Node</em></strong> <strong><em>actions</em></strong> -&gt; <strong><em>Tools</em></strong> -&gt; <strong><em>Execute run command</em></strong>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*28MzKkXGQ6p6gek1Wkgvew.png" /><figcaption>Select the node and Execute run command</figcaption></figure><p>2. This will open the <strong>Run a command</strong> page. In the <strong>Command document</strong>, section select ‘<strong><em>AWS-ConfigureAWSPackage</em></strong>’ and ‘<strong><em>Latest version at runtime</em></strong>’ from the <strong>Document version</strong> drop-down list.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XoGFArF5U4b4hSQ4EIqjgg.png" /><figcaption>Selecting command documents to execute</figcaption></figure><p>3. In the Command parameters section select the below options:<br> a) Action -&gt; <strong><em>Install<br> </em></strong>b) Installation Type -&gt; <strong><em>Uninstall and reinstall</em></strong><br> c) Name -&gt;<strong><em> AmazonCloudWatchAgent </em></strong>(Paste this as it is)<br> d) Version -&gt; <strong><em>Latest </em></strong>(Paste this as it is)<br> e) Additional Arguments -&gt; <strong><em>{} </em></strong>(Paste this as it is)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YnHA9zZqEDcfXnED6IZcJA.png" /><figcaption>Configure command parameters</figcaption></figure><p>4. In the <strong>Target selection</strong> section, click on ‘<strong><em>choose instances manually</em></strong>’ and select the instance i.e. ‘<strong><em>Cloudwatch-SSM-POC</em></strong>’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G73DpZFF23dNuvL6jP40Qg.png" /><figcaption>Selecting command targets manually</figcaption></figure><p>5. In <strong>Output options</strong>, ‘<strong><em>Enable an S3 bucket</em></strong>’ and ‘<strong><em>Enable CloudWatch logs</em></strong>’ can be used to send the run command output logs to either the<strong> S3 bucket</strong> and/or the <strong>Cloudwatch log</strong>. If we don’t want to save any logs just <strong>uncheck</strong> both the boxes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g7HKFsUgpsfiaUQsVMvTyg.png" /><figcaption>Enabling command log output destination</figcaption></figure><p>6. Click on the ‘<strong><em>Run</em></strong>’ button and it will start executing the selected commands in the selected instances. We can check the progress of the commands and verify that the commands get executed successfully.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z4ZnKJZ6nubXIqdKOU5Wbw.png" /><figcaption>Command execution in progress</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nikfi-WXSPG-OcZ6LY3Ahw.png" /><figcaption>Command execution successful</figcaption></figure><p>7. To verify that the <strong>CloudWatch Agent</strong> installation is a success, login to the Windows EC2 instance via RDP and re-execute the below command.</p><pre>Get-Service -Name &quot;AmazonCloudWatchAgent&quot;</pre><p>8. This time we can verify that the <strong>CloudWatch Agent</strong> is indeed installed on the instance. Do note that the status of the <strong>CloudWatch Agent</strong> is ‘<strong><em>Stopped</em></strong>’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y0AGyXRLiPD-QTk8R-FCgg.png" /><figcaption>Cloudwatch Agent Service Status</figcaption></figure><h4>Step 5: Configure the CloudWatch Agent</h4><ol><li>Log in to the <strong>Windows EC2 Instance</strong> using the <strong>RDP client</strong> and launch <strong>PowerShell</strong>. Execute the below commands:</li></ol><pre>&gt; cd &quot;C:\Program Files\Amazon\AmazonCloudWatchAgent&quot;<br><br>&gt; .\amazon-cloudwatch-agent-config-wizard.exe</pre><p>2. This will start the CloudWatch Agent configuration wizard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lNdMvfpzFy7F5CGV4u4ccw.png" /><figcaption>Launch the wizard</figcaption></figure><p>3. Answer the questions as per requirement.</p><pre>PS C:\Program Files\Amazon\AmazonCloudWatchAgent&gt; .\amazon-cloudwatch-agent-config-wizard.exe<br>================================================================<br>= Welcome to the Amazon CloudWatch Agent Configuration Manager =<br>=                                                              =<br>= CloudWatch Agent allows you to collect metrics and logs from =<br>= your host and send them to CloudWatch. Additional CloudWatch =<br>= charges may apply.                                           =<br>================================================================<br>On which OS are you planning to use the agent?<br>1. linux<br>2. windows<br>3. darwin<br>default choice: [2]:<br>2<br>Trying to fetch the default region based on ec2 metadata...<br>I! imds retry client will retry 1 timesAre you using EC2 or On-Premises hosts?<br>1. EC2<br>2. On-Premises<br>default choice: [1]:<br>1<br>Do you want to turn on StatsD daemon?<br>1. yes<br>2. no<br>default choice: [1]:<br>2<br>Do you have any existing CloudWatch Log Agent configuration file to import for migration?<br>1. yes<br>2. no<br>default choice: [2]:<br>2<br>Do you want to monitor any host metrics? e.g. CPU, memory, etc.<br>1. yes<br>2. no<br>default choice: [1]:<br>1<br>Do you want to monitor cpu metrics per core?<br>1. yes<br>2. no<br>default choice: [1]:<br>1<br>Do you want to add ec2 dimensions (ImageId, InstanceId, InstanceType, AutoScalingGroupName) into all of your metrics if the info is available?<br>1. yes<br>2. no<br>default choice: [1]:<br>1<br>Do you want to aggregate ec2 dimensions (InstanceId)?<br>1. yes<br>2. no<br>default choice: [1]:<br>1<br>Would you like to collect your metrics at high resolution (sub-minute resolution)? This enables sub-minute resolution for all metrics, but you can customize for specific metrics in the output json file.<br>1. 1s<br>2. 10s<br>3. 30s<br>4. 60s<br>default choice: [4]:<br>4<br>Which default metrics config do you want?<br>1. Basic<br>2. Standard<br>3. Advanced<br>4. None<br>default choice: [1]:<br>2<br>Current config as follows:<br>{<br>        &quot;metrics&quot;: {<br>                &quot;aggregation_dimensions&quot;: [<br>                        [<br>                                &quot;InstanceId&quot;<br>                        ]<br>                ],<br>                &quot;append_dimensions&quot;: {<br>                        &quot;AutoScalingGroupName&quot;: &quot;${aws:AutoScalingGroupName}&quot;,<br>                        &quot;ImageId&quot;: &quot;${aws:ImageId}&quot;,<br>                        &quot;InstanceId&quot;: &quot;${aws:InstanceId}&quot;,<br>                        &quot;InstanceType&quot;: &quot;${aws:InstanceType}&quot;<br>                },<br>                &quot;metrics_collected&quot;: {<br>                        &quot;LogicalDisk&quot;: {<br>                                &quot;measurement&quot;: [<br>                                        &quot;% Free Space&quot;<br>                                ],<br>                                &quot;metrics_collection_interval&quot;: 60,<br>                                &quot;resources&quot;: [<br>                                        &quot;*&quot;<br>                                ]<br>                        },<br>                        &quot;Memory&quot;: {<br>                                &quot;measurement&quot;: [<br>                                        &quot;% Committed Bytes In Use&quot;<br>                                ],<br>                                &quot;metrics_collection_interval&quot;: 60<br>                        },<br>                        &quot;Paging File&quot;: {<br>                                &quot;measurement&quot;: [<br>                                        &quot;% Usage&quot;<br>                                ],<br>                                &quot;metrics_collection_interval&quot;: 60,<br>                                &quot;resources&quot;: [<br>                                        &quot;*&quot;<br>                                ]<br>                        },<br>                        &quot;PhysicalDisk&quot;: {<br>                                &quot;measurement&quot;: [<br>                                        &quot;% Disk Time&quot;<br>                                ],<br>                                &quot;metrics_collection_interval&quot;: 60,<br>                                &quot;resources&quot;: [<br>                                        &quot;*&quot;<br>                                ]<br>                        },<br>                        &quot;Processor&quot;: {<br>                                &quot;measurement&quot;: [<br>                                        &quot;% User Time&quot;,<br>                                        &quot;% Idle Time&quot;,<br>                                        &quot;% Interrupt Time&quot;<br>                                ],<br>                                &quot;metrics_collection_interval&quot;: 60,<br>                                &quot;resources&quot;: [<br>                                        &quot;*&quot;<br>                                ]<br>                        }<br>                }<br>        }<br>}<br>Are you satisfied with the above config? Note: it can be manually customized after the wizard completes to add additional items.<br>1. yes<br>2. no<br>default choice: [1]:<br>1<br>Do you want to monitor any customized log files?<br>1. yes<br>2. no<br>default choice: [1]:<br>2<br>Do you want to monitor any Windows event log?<br>1. yes<br>2. no<br>default choice: [1]:<br>2<br>Do you want the CloudWatch agent to also retrieve X-ray traces?<br>1. yes<br>2. no<br>default choice: [1]:<br>2<br>Existing config JSON identified and copied to:  C:\Users\Administrator\AppData\Roaming\Amazon\CloudWatchAgent\etc\backup-configs<br>Saved config file to config.json successfully.<br>Current config as follows:<br>{<br>        &quot;metrics&quot;: {<br>                &quot;aggregation_dimensions&quot;: [<br>                        [<br>                                &quot;InstanceId&quot;<br>                        ]<br>                ],<br>                &quot;append_dimensions&quot;: {<br>                        &quot;AutoScalingGroupName&quot;: &quot;${aws:AutoScalingGroupName}&quot;,<br>                        &quot;ImageId&quot;: &quot;${aws:ImageId}&quot;,<br>                        &quot;InstanceId&quot;: &quot;${aws:InstanceId}&quot;,<br>                        &quot;InstanceType&quot;: &quot;${aws:InstanceType}&quot;<br>                },<br>                &quot;metrics_collected&quot;: {<br>                        &quot;LogicalDisk&quot;: {<br>                                &quot;measurement&quot;: [<br>                                        &quot;% Free Space&quot;<br>                                ],<br>                                &quot;metrics_collection_interval&quot;: 60,<br>                                &quot;resources&quot;: [<br>                                        &quot;*&quot;<br>                                ]<br>                        },<br>                        &quot;Memory&quot;: {<br>                                &quot;measurement&quot;: [<br>                                        &quot;% Committed Bytes In Use&quot;<br>                                ],<br>                                &quot;metrics_collection_interval&quot;: 60<br>                        },<br>                        &quot;Paging File&quot;: {<br>                                &quot;measurement&quot;: [<br>                                        &quot;% Usage&quot;<br>                                ],<br>                                &quot;metrics_collection_interval&quot;: 60,<br>                                &quot;resources&quot;: [<br>                                        &quot;*&quot;<br>                                ]<br>                        },<br>                        &quot;PhysicalDisk&quot;: {<br>                                &quot;measurement&quot;: [<br>                                        &quot;% Disk Time&quot;<br>                                ],<br>                                &quot;metrics_collection_interval&quot;: 60,<br>                                &quot;resources&quot;: [<br>                                        &quot;*&quot;<br>                                ]<br>                        },<br>                        &quot;Processor&quot;: {<br>                                &quot;measurement&quot;: [<br>                                        &quot;% User Time&quot;,<br>                                        &quot;% Idle Time&quot;,<br>                                        &quot;% Interrupt Time&quot;<br>                                ],<br>                                &quot;metrics_collection_interval&quot;: 60,<br>                                &quot;resources&quot;: [<br>                                        &quot;*&quot;<br>                                ]<br>                        }<br>                }<br>        }<br>}<br>Please check the above content of the config.<br>The config file is also located at config.json.<br>Edit it manually if needed.<br>Do you want to store the config in the SSM parameter store?<br>1. yes<br>2. no<br>default choice: [1]:<br>1<br>What parameter store name do you want to use to store your config? (Use &#39;AmazonCloudWatch-&#39; prefix if you use our managed AWS policy)<br>default choice: [AmazonCloudWatch-windows]<br><br>Trying to fetch the default region based on ec2 metadata...<br>I! imds retry client will retry 1 timesWhich region do you want to store the config in the parameter store?<br>default choice: [us-east-1]<br><br>Which AWS credential should be used to send json config to parameter store?<br>1. ASIAUR4XXXXXXXXXX(From SDK)<br>2. Other<br>default choice: [1]:<br>1<br>Successfully put config to parameter store AmazonCloudWatch-windows.<br>Please press Enter to exit...<br><br>Program exits now.</pre><p>4. Verify the CloudWatch Agent configuration file by going to <strong>Systems Manager</strong> -&gt; <strong><em>Parameter store</em></strong> -&gt; ‘<strong><em>AmazonCloudWatch-windows’</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*evQNqxlDB8jLQQnfvIVj2A.png" /><figcaption>Verify config in the SSM parameter store</figcaption></figure><h4>Step 6: Start the CloudWatch Agent via the Systems Manager</h4><ol><li>Go to <strong><em>Systems Manager</em></strong> -&gt; <strong><em>Fleet manager</em></strong>, select the instance/node, and click on ‘<strong><em>Execute run command</em></strong>’.</li></ol><p>2. In the <strong>Command document</strong>, section select ‘<strong><em>AmazonCloudWatch-ManageAgent</em></strong>’ and ‘<strong><em>Latest version at runtime</em></strong>’ from the <strong>Document version</strong> drop-down list.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q7jadKo_dlVPSdL_-l7CFg.png" /><figcaption>Select <strong><em>AmazonCloudWatch-ManageAgent </em></strong><em>from the command documents</em></figcaption></figure><p>3. In the <strong>Command parameters</strong> section select the below options:<br> a) Action -&gt; <strong><em>configure<br> </em></strong>b) Mode-&gt; <strong><em>ec2</em></strong><br> c) Optional Configuration Source -&gt; <strong><em>ssm </em></strong><em>(to pick up the config file from ssm parameters)</em><br> d) Optional Configuration Location -&gt; <strong><em>AmazonCloudWatch-windows </em></strong>(paste as it is, name of the ssm parameter we created in<strong> Step 5</strong>)<br> e) Optional Restart -&gt; <strong><em>yes</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mw4YAkgxB8Ia9gFGBk4m7A.png" /><figcaption>Select the appropriate options</figcaption></figure><p>4. Select the instance in the <strong>Target selection</strong>.</p><p>5. Click on the ‘<strong><em>Run</em></strong>’ button to execute the command and make sure it gets executed successfully.</p><h4>Step 6: Verify that CloudWatch Agent is sending metrics to the CloudWatch Console</h4><ol><li>Log in to the Windows EC2 instance via RDP and re-execute the below command. The status should be ‘<strong><em>Running</em></strong>’.</li></ol><pre>Get-Service -Name &quot;AmazonCloudWatchAgent&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DHtuhHgn8A1rp7Ie1RjMPw.png" /><figcaption>CloudWatch Agent is running</figcaption></figure><p>2. Go to AWS CloudWatch Console, on the left-hand side, <strong><em>Metrics</em></strong> -&gt; <strong><em>All metrics</em></strong>. We can see <strong>CWAgent</strong> under ‘<strong><em>Custom namespaces</em></strong>’, click on this and we can see all the metrics that are being collected by the <strong>CloudWatch Agent</strong> and sent to the <strong>CloudWatch console</strong> for the <strong>Windows EC2 instance</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z9L3Sj7wMyD07V3pdIwS0A.png" /><figcaption>Metrics collected by the CloudWatch Agent</figcaption></figure><p>3. Select the metrics for which we want to see the data, it should look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QN9PumJhmDDDw4ufJMXxEQ.png" /><figcaption>Metrics data</figcaption></figure><h4>Conclusion</h4><p>We have created a <strong><em>Windows EC2 instance</em></strong>, installed and configured <strong><em>CloudWatch Agent</em></strong> on that instance using the <strong><em>SSM console</em></strong>, and viewed the system-level metrics on the <strong><em>CloudWatch Dashboard</em></strong>.</p><p>Using SSM to manage our instance is super easy and in upcoming articles, we will explore more capabilities around SSM.</p><p>If you’ve enjoyed this story, please click the 👏 button and share it, so that others can find it as well! Also, feel free to leave a comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cbcb377b9917" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Handle Special Characters in your GitHub Secret]]></title>
            <link>https://medium.com/@susovanpanja/how-to-handle-special-characters-in-your-github-secret-ec7492d1bbca?source=rss-6042739db668------2</link>
            <guid isPermaLink="false">https://medium.com/p/ec7492d1bbca</guid>
            <category><![CDATA[secrets]]></category>
            <category><![CDATA[debugging]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[special-character]]></category>
            <dc:creator><![CDATA[Susovan Panja]]></dc:creator>
            <pubDate>Mon, 08 Jan 2024 13:29:54 GMT</pubDate>
            <atom:updated>2024-01-08T13:29:54.175Z</atom:updated>
            <content:encoded><![CDATA[<h4>In GitHub Actions Workflow</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z1urmN9ajSrTl1413d_8Pw.png" /></figure><p>The use case is simple, I have a secret stored in GitHub that contains <a href="https://gist.github.com/susovanpanja/c113f035b93fc7a2abf5b5e7713a817e">special characters</a>. Let&#39;s say it’s <strong>Aoiu$98!</strong></p><p>I need to store this secret as a GitHub secret and use it for another command. Now we can add an escape character before special characters while creating a secret so that when using GitHub Workflows they render correctly i.e. <strong>Aoiu\$98\!</strong></p><p>But I like the natural way of creating environment variables, just use it as it is. So we will be implementing the same in this article and also we need to somehow verify that it is rendering properly inside GitHub Workflow. That is a bit challenging as GitHub will actively try to redact/mask any secret values from being printed to the GitHub Action Workflow log.</p><h4><strong>Steps:</strong></h4><ol><li>Create a Github Repo or use an existing one.</li><li>Create a GitHub Secret with special characters. Go to <strong><em>Repo</em></strong> -&gt; <strong><em>Settings</em></strong>. On the left-hand side <strong><em>Secrets and variables </em></strong>-&gt; <strong><em>Actions</em></strong> -&gt; <strong><em>New Repository secret</em></strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QnxEDphqAOTRnzF5lwpWaQ.png" /></figure><p>3. Create a GitHub Action, we will be using the sample <strong>Simple Workflow</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a0X90CNb1i10LrUdsoaKfw.png" /><figcaption>Pick the sample Simple Workflow</figcaption></figure><p>It will create a <strong><em>blank.yml</em></strong> file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ruLbCJWg4r62K3AwihhPJA.png" /><figcaption>GitHub Action Workflow file</figcaption></figure><p>Edit this file and remove the last two steps:</p><pre>      # Runs a single command using the runners shell<br>      - name: Run a one-line script<br>        run: echo Hello, world!<br><br>      # Runs a set of commands using the runners shell<br>      - name: Run a multi-line script<br>        run: |<br>          echo Add other actions to build,<br>          echo test, and deploy your project.</pre><p>Add this step:</p><pre>      # Runs a single command using the runners shell<br>      - name: Run a one-line script<br>        env:<br>          SUPER_SECRET: ${{ secrets.SuperSecret }}<br>        run: echo &quot;$SUPER_SECRET&quot; | sed &#39;s/./&amp; /g&#39;</pre><p>Commit and Save.</p><p>Another way of calling the secret is:</p><pre>      # Runs a single command using the runners shell<br>      - name: Run a one-line script<br>        run: echo &#39;${{ secrets.SuperSecret }}&#39; | sed &#39;s/./&amp; /g&#39;</pre><p><strong>Note: </strong><em>Do not use double quotation </em><strong><em>“ ”</em></strong><em> in the above method as it will try to resolve the special characters.</em></p><p>Since GitHub redacts any string that resembles a secret, we are using <strong><em>sed</em></strong> to add a <strong><em>space</em></strong> after each character in the secret and now we can view the secret as plain text in the output.</p><p><strong>Note: </strong><em>This is good for debugging but not for active workload. Once we verify that the secrets are rendering correctly and the workflow is working as expected rotate the test secret.</em></p><p>4. Click on the <strong><em>Actions</em></strong> -&gt; <strong><em>Workflow Name (CI in our case) </em></strong>-&gt; <strong><em>Latest workflow run</em></strong> -&gt; <strong><em>Build</em></strong> -&gt; <strong><em>Expand ‘ Run a one-line script’</em></strong>. Here we can verify the special characters are rendered correctly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ehsnOtp1oWdbYNxHslJLXg.png" /></figure><p>5. Once verified use the secret with your actual command i.e.</p><pre># Runs a single command using the runners shell<br>      - name: Run a one-line script<br>        env:<br>          SUPER_SECRET: ${{ secrets.SuperSecret }}<br>        run: PGPASSWORD=&quot;$SUPER_SECRET&quot; psql -h host_name -d database_name \<br>             -U user_name</pre><p>If you’ve enjoyed this story, please click the 👏 button and share it, so that others can find it as well! Also, feel free to leave a comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ec7492d1bbca" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>