<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sanyam Sharma on Medium]]></title>
        <description><![CDATA[Stories by Sanyam Sharma on Medium]]></description>
        <link>https://medium.com/@sanyamsharma940?source=rss-df1317ec09b0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*YTBtTj19ttVDooGnzEvygg@2x.jpeg</url>
            <title>Stories by Sanyam Sharma on Medium</title>
            <link>https://medium.com/@sanyamsharma940?source=rss-df1317ec09b0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 04:55:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sanyamsharma940/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How We Unified CI/CD for 20+ Repos with a Single Orchestrator]]></title>
            <link>https://medium.com/@sanyamsharma940/how-we-unified-ci-cd-for-20-repos-with-a-single-orchestrator-18f68bc7a01e?source=rss-df1317ec09b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/18f68bc7a01e</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[bitbucket]]></category>
            <category><![CDATA[bitbucket-pipelines]]></category>
            <dc:creator><![CDATA[Sanyam Sharma]]></dc:creator>
            <pubDate>Tue, 03 Feb 2026 13:09:02 GMT</pubDate>
            <atom:updated>2026-02-03T13:09:02.698Z</atom:updated>
            <content:encoded><![CDATA[<p>In a growing <strong>microservices ecosystem</strong>, every service needs a consistent way to <strong>build, package, and deploy, </strong>But with 20+ repositories, duplicating the same pipeline logic in each repo becomes painful:</p><ul><li>Different teams may use inconsistent build steps.</li><li>Updating something simple (like upgrading NodeJS or Docker version) means touching multiple repos.</li><li>Managing secrets and environment configs becomes messy.</li></ul><p>The solution? A <strong>Centralized CI/CD Orchestrator</strong>.</p><p>In this article, I’ll share how we built a <strong>single orchestrator repo</strong> that:</p><ul><li>Builds services across multiple repos</li><li>Pushes Docker images to <strong>AWS ECR</strong></li><li>Updates Helm values automatically</li><li>Deploys apps with <strong>ArgoCD</strong></li></ul><h3>The Problem with Per-App Pipelines</h3><p>A typical app repo looks like this:</p><pre>my-service/<br> ├── src/<br> ├── Dockerfile<br> └── bitbucket-pipelines.yml</pre><p>Each bitbucket-pipelines.yml contained almost the same steps:</p><ul><li>Install dependencies</li><li>Build</li><li>Docker build &amp; push</li><li>Update Helm values</li><li>Trigger deployment</li></ul><p>That meant <strong>every repo duplicated pipeline logic</strong>.<br> Let’s assume if we had to update from Node 16 → Node 20, we had to fix every app repo.</p><h3>The Centralized Orchestrator</h3><p>Instead, we created a dedicated repo:</p><pre>ci-cd-orchestrator/<br> ├── bitbucket-pipelines.yml   # Main centralized pipeline<br> ├── build-scripts/            # App-specific build scripts<br> │    ├── frontend-app-business.sh<br> │    ├── backend-app-jobs.sh<br> │    └── ...<br> └── README.md</pre><p>Let me explain the flow:</p><ol><li>Developers commit code → in an app repo (e.g., backendapp-repo)</li><li>The app repo doesn’t build itself — instead, it triggers the orchestrator pipeline (via Bitbucket API).</li></ol><p>Functionality of orchestrator pipeline:</p><ul><li>Clones the app repo.</li><li>Runs the app-specific build script.</li><li>Builds &amp; pushes a Docker image to AWS ECR.</li><li>Updates Helm values in the <strong>Helm repo.</strong></li><li>ArgoCD auto-syncs the change and deploys.</li></ul><h3>Workflow Diagram</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uPqbW7lK2Bh8jLC0hgJ5Hg.png" /><figcaption>Workflow of Bitbucket centralised pipeline</figcaption></figure><p><strong>Developer → App Repo → Orchestrator Pipeline → AWS ECR → Helm Repo (Helm values) → ArgoCD → Kubernetes Cluster</strong></p><h3>The Pipeline (Bitbucket)</h3><p>Here’s a simplified version of our orchestrator pipeline:</p><pre>pipelines:<br>  default:<br>    - step:<br>        name: Deploy Application<br>        image: node:20.16-alpine3.20<br>        size: 2x<br>        script:<br>          # Install tools<br>          - apk add --no-cache git bash curl jq docker-cli<br># Clone the target app repo<br>          - git clone -b &quot;$BRANCH&quot; @bitbucket.org/abcgrou/$APP_NAME.git&quot;&gt;https://${PIPELINE_USERNAME}:${BITBUCKET_CLONE_TOKEN}@bitbucket.org/sanyam/$APP_NAME.git<br>          - cd $APP_NAME<br>          # Run app-specific build script<br>          - bash ../build-scripts/${APP_NAME}.sh<br>          # Get latest commit ID from app repo<br>          - export APP_COMMIT_ID=$(git rev-parse HEAD)<br>          - export APP_COMMIT_SHORT=$(echo $APP_COMMIT_ID | cut -c1-7)<br>          # Build &amp; tag Docker image<br>          - export IMAGE_TAG=${BRANCH}-${APP_COMMIT_SHORT}<br>          - docker build . -t sanyam/$APP_NAME:$IMAGE_TAG<br>          - docker push sanyam/$APP_NAME:$IMAGE_TAG<br>          # Update Helm values in CAC repo<br>          - git clone https://x-token-auth:$CAC_HTTPS_TOKEN@bitbucket.org/sanyam/abc-cac.git<br>          - cd abc-cac<br>          - yq -i &quot;.image.tag = env(IMAGE_TAG)&quot; ./helm/abc-values/k8s-${ENV}/$APP_NAME.yaml<br>          - git commit -am &quot;$APP_NAME updated to $IMAGE_TAG&quot;<br>          - git push<br>          # Trigger ArgoCD sync<br>          - pipe: atlassian/aws-eks-kubectl-run:3.2.0<br>            variables:<br>              CLUSTER_NAME: $AWS_EKS_CLUSTER_NAME<br>              KUBECTL_COMMAND: &quot;apply&quot;<br>              RESOURCE_PATH: &quot;./argocd-apps/k8s-${ENV}/$APP_NAME.yaml&quot;</pre><h3>App-Specific Build Scripts</h3><p>Below is the app specific script for the NodeJs application. However, based on different application (Python, .Net, Java), similar scripts can be created.</p><p>Example: build-scripts/frontend-webapp-business.sh</p><pre>#!/bin/bash<br>set -euo pipefail</pre><pre># Install dependencies<br>npm install --no-audit --no-fund</pre><pre># Security audit (won’t fail pipeline, just warn)<br>npm audit --audit-level critical || true</pre><pre># Prisma<br>npm run prisma:generate</pre><pre># Build app<br>npm run build</pre><pre># (Optional) keep dist/gen artifacts for debugging<br>ls -lh dist/ || trueThis keeps the orchestrator generic, while apps stay flexible.</pre><h3>Key Benefits</h3><p><strong>No duplication</strong> since CI/CD is maintained at a single place <br><strong>Consistent deployments</strong> across services<br><strong>Commit-based versioning</strong> — Docker tags = branch-commitID<br><strong>GitOps ready:</strong> – ArgoCD handles app deployments/upgradation<br><strong>Easy Scalability</strong>: New repos can be seamlessly</p><p>This is how we approached <strong>Centralized CI/CD with Bitbucket Pipelines, AWS ECR, and ArgoCD</strong>. This has improved our operational efficiency, cutting hours of redundant effort and enhancing the stability of our deployments.</p><p>I’d love to hear how your team handles CI/CD in a multi-repo setup.<em>Connect with me on </em><a href="http://www.linkedin.com/in/sanyam-sharmaa"><em>linkedin</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18f68bc7a01e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Self hosted Bitbucket-Runners on Google Kubernetes Engine]]></title>
            <link>https://medium.com/@sanyamsharma940/self-hosted-bitbucket-runners-on-google-kubernetes-engine-6c61a05f76ff?source=rss-df1317ec09b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/6c61a05f76ff</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[bitbucket]]></category>
            <dc:creator><![CDATA[Sanyam Sharma]]></dc:creator>
            <pubDate>Wed, 14 Jun 2023 05:59:58 GMT</pubDate>
            <atom:updated>2023-06-14T07:58:05.979Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>In this article, we will be deploying Bitbucket runners on GKE on a dedicated node pool.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*654rh4e4oWMAJgEYVXPWQg.png" /></figure><p>Self hosted Runners allows us to run builds in Pipelines on your own infrastructure. Some key features of self-hosted runners are depicted in the diagram below.</p><h3>Key features of bitbucket self hosted runners:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KxK_SlSz92sP5Sy_zFAw8Q.png" /></figure><h4>Minimum system requirement to host a runner (linux-docker):</h4><ul><li>Minimum machine type required for GKE node is e2-standard-2.</li><li>Docker v19.03 and above</li></ul><h3>Let’s start with the deployment of Self-Hosted runners on GKE.</h3><h4>Step1:- Open the specific-repository and navigate to repository-setting &gt; runners.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZLhywWG5iaOOWhcAoVZRCw.png" /></figure><p>Click on add runner, choose system-architecture and give runner name. We also need to mention runners label.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f9KnMTjFNEgsqnZfc8hEcw.png" /></figure><p>Click on the next and it will give command output like the following</p><pre>docker container run -it -v /tmp:/tmp -v /var/run/docker.sock:/var/run/docker.sock <br>-v /var/lib/docker/containers:/var/lib/docker/containers:ro <br>-e ACCOUNT_UUID={xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx} <br>-e REPOSITORY_UUID={xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx} <br>-e RUNNER_UUID={xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx} <br>-e RUNTIME_PREREQUISITES_ENABLED=true -e OAUTH_CLIENT_ID=xxxxxxxxxxxxxxxx <br>-e OAUTH_CLIENT_SECRET=xxxxxxxxxxxxxxxxxx <br>-e WORKING_DIRECTORY=/tmp --name runner-xxxxxxxxxxxxxxxxxx docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner:1</pre><h4>Step2: Connect to GKE cluster and create environment variables as following</h4><p>Fetch the <strong>ACCOUNT_UUID</strong>, the <strong>RUNNER_UUID</strong>, the <strong>REPOSITORY_UUID</strong>, the <strong>OAUTH_CLIENT_ID</strong> and the <strong>OAUTH_CLIENT_SECRET</strong> variables from the command.</p><pre>export ACCOUNT_UUID=xxxxxxxx-xxxx-xxxx-xxxxxxxxxxx</pre><pre>export REPOSITORY_UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxx</pre><pre>export RUNNER_UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx</pre><pre>export OAUTH_CLIENT_ID=xxxxxxxxxxxxxxxxxxxx</pre><pre>Export OAUTH_CLIENT_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</pre><pre>export BASE64_OAUTH_CLIENT_ID=$(echo -n $OAUTH_CLIENT_ID | base64)</pre><pre>export BASE64_OAUTH_CLIENT_SECRET=$(echo -n $OAUTH_CLIENT_SECRET | base64)</pre><h4>Step3: Connect to cluster and Implementation on kubernetes cluster</h4><p>Run the gcloud command to connect with GKE cluster</p><pre>gcloud container clusters get-credentials cluster-name --zone specific-zone --project Project-ID</pre><p>Create separate namespace for runners with</p><pre>kubectl create namespace runner-ns</pre><p>The following bash scripts are going to generate the secret.yaml and job.yaml files in the current working directory.</p><p>Secret.yaml file</p><pre>cat &gt; ./secret.yaml &lt;&lt;EOF<br>apiVersion: v1<br>kind: Secret<br>Namespace: runner-ns<br>metadata:<br> name: runner-oauth-credentials<br> labels:<br>   accountUuid: ${ACCOUNT_UUID}<br>   runnerUuid: ${RUNNER_UUID}<br>   repository_Uuid: ${REPOSITORY_UUID}<br>data:<br> oauthClientId: ${BASE64_OAUTH_CLIENT_ID}<br> oauthClientSecret: ${BASE64_OAUTH_CLIENT_SECRET}<br>EOF</pre><p>Job.yaml file</p><pre>cat &gt; ./job.yaml &lt;&lt;EOF<br>apiVersion: batch/v1<br>kind: Job<br>Namespace: runner-ns<br>metadata:<br> name: runner<br>spec:<br> template:<br>   metadata:<br>     labels:<br>       accountUuid: ${ACCOUNT_UUID}<br>       runnerUuid: ${RUNNER_UUID}<br>       runnerUuid: ${REPOSITORY_UUID}<br>   spec:<br>     containers:<br>       - name: bitbucket-k8s-runner<br>         image: docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner<br>         env:<br>           - name: ACCOUNT_UUID<br>             value: &quot;{${ACCOUNT_UUID}}&quot;<br>           - name: RUNNER_UUID<br>             value: &quot;{${RUNNER_UUID}}&quot;<br>           - name: REPOSITORY_UUID<br>             value: &quot;{${REPOSITORY_UUID}}&quot;<br>           - name: OAUTH_CLIENT_ID<br>             valueFrom:<br>               secretKeyRef:<br>                 name: runner-oauth-credentials<br>                 key: oauthClientId<br>           - name: OAUTH_CLIENT_SECRET<br>             valueFrom:<br>               secretKeyRef:<br>                 name: runner-oauth-credentials<br>                 key: oauthClientSecret<br>           - name: WORKING_DIRECTORY<br>             value: &quot;/tmp&quot;<br>         volumeMounts:<br>           - name: tmp<br>             mountPath: /tmp<br>           - name: docker-containers<br>             mountPath: /var/lib/docker/containers<br>             readOnly: true<br>           - name: var-run<br>             mountPath: /var/run<br>       - name: docker-in-docker<br>         image: docker:20.10.7-dind<br>         securityContext:<br>           privileged: true<br>         volumeMounts:<br>           - name: tmp<br>             mountPath: /tmp<br>           - name: docker-containers<br>             mountPath: /var/lib/docker/containers<br>           - name: var-run<br>             mountPath: /var/run<br>     restartPolicy: OnFailure<br>     volumes:<br>       - name: tmp<br>       - name: docker-containers<br>       - name: var-run<br> backoffLimit: 6<br> completions: 1<br> parallelism: 1<br>EOF</pre><p>Run the above shell script and it will create job.yaml manifest file</p><h4>Step4: Apply the manifests to kubernetes</h4><p>Now its time to create runners using jobs.yaml and secret.yaml</p><pre>$ kubectl apply -f secrets.yaml -n=runner-ns</pre><pre>$ kubectl apply -f job.yaml -n=runner-ns</pre><p>We will soon see the runner with ONLINE status on the Repository Runners page</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zTx__FYcSAJ5GT_4" /><figcaption>Runners-activated</figcaption></figure><p>We are all set to use runners in a bitbucket pipeline. Add the runner’s labels to the pipeline step.</p><pre>pipelines:<br>  custom:<br>    pipeline:<br>      - step:<br>          name: Step1<br>          size: 8x # default 4gb, 8x for 32gb<br>          runs-on: <br>            - &#39;self.hosted&#39;<br>            - &#39;my.custom.label&#39;<br>          script:<br>            - echo &quot;This step will run on a self hosted runner with 32 GB of memory.&quot;;<br>      - step:<br>          name: Step2<br>          script:<br>            - echo &quot;This step will run on infrastructure as usual.&quot;;</pre><h4>Usage limits:</h4><p>Self-Hosted Runner has following usage limits.</p><p>∙ <strong>Max available memory</strong>: With Runners, you can configure up to 32GB (8x) of memory to run your builds.</p><p>∙ <strong>Total number of runners</strong>: You can configure up to 100 runners per workspace and 100 runners per repository.</p><p>∙ <strong>Step queue time</strong>: Steps can be queued for a maximum of 2.5 hours.</p><blockquote>Connect with me on <a href="http://www.linkedin.com/in/sanyam-sharmaa">linkedin</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6c61a05f76ff" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>