Kubernetes Canary Deployment with Istio and Visualizing Traffic Using the Kiali Dashboard

In the previous workshop, we deployed an application using Kubernetes Sealed Secrets to securely manage configuration data that would otherwise be stored in plain text. In this blog, we continue that workflow by introducing the Canary deployment process.

What is Canary deployment?

A canary deployment in Kubernetes is a risk-mitigation strategy that releases a new application version to a small, subset of users (or traffic) before rolling it out to the entire infrastructure.

In a canary deployment process, the entire new version of the application is not released to all users at once. Instead, the old version continues running while the new version is deployed in a separate environment. Only a small portion of traffic is routed to the new version at first, allowing us to test its stability before gradually rolling it out to everyone.

When users access the application, a small portion of their request traffic is redirected to the new version, while the remaining traffic continues to be handled by the old version. For example, 10% of the request traffic may be routed to the new version, and the remaining 90% sent to the old version.

In today’s project, we will deploy the application and gradually release the new version to users. If any issues or problems occur in the new version, we can detect them early and take the necessary actions to mitigate those issues.

First, create two files named deployment-canary.yaml and deployment-stable.yaml. Both files will contain similar configurations, but they will differ in their versions and container images.

deployment-canary.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: lms-static-canary
  labels:
    app: lms-static
    version: canary
  namespace: lms
spec:
  replicas: 1
  selector:
    matchLabels:
      app: lms-static
      version: canary
  template:
    metadata:
      labels:
        app: lms-static
        version: canary
    spec:
      containers:
      - name: lms-static-container
        image: k8sworkshopcertd.azurecr.io/lms-static:v1.0.20 # {"$imagepolicy": "flux-system:lms-static-canary"}
        ports:
        - containerPort: 80
      imagePullSecrets:
      - name: acr-secret

In the above file content, you can see that the version is set to canary, and the image section uses the new image from the ACR repository.

deployment-stable.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: lms-static-stable
  labels:
    app: lms-static
    version: stable
  namespace: lms
spec:
  replicas: 1
  selector:
    matchLabels:
      app: lms-static
      version: stable
  template:
    metadata:
      labels:
        app: lms-static
        version: stable
    spec:
      containers:
      - name: lms-static-container
        image: k8sworkshopcertd.azurecr.io/lms-static:v1.0.10    
        ports:
        - containerPort: 80
      imagePullSecrets:
      - name: acr-secret 

In the above file content, you can see that the version is set to stable, and the image section uses the older image from the ACR repository.

Next, create another file named destination-rule.yaml and add the following content to it:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: lms-static-destination-rule
  namespace: lms
spec:
  host: lms-static-svc.lms.svc.cluster.local
  subsets:
  - name: stable
    labels:
      version: stable
  - name: canary
    labels:
      version: canary

In the above file, we added a subset section containing two subsets: stable and canary.

Next, open the “virtual-service.yaml” file. We need to modify its content according to our traffic‑splitting requirements. Add the following part to the file:

- destination:
            host: lms-static-svc.lms.svc.cluster.local
            port:
              number: 80
            subset: stable
          weight: 90
- destination:
            host: lms-static-svc.lms.svc.cluster.local
            port:
              number: 80
            subset: canary
          weight: 10

The above content defines how the request traffic is distributed between the stable and canary deployments. In this configuration, 90% of the traffic is routed to the stable version, while the remaining 10% is directed to the canary version.

Then, commit and push the changes to the GitHub repository. After that, reconcile the cluster using the flux reconcile command.

flux reconcile source git -n flux-system flux-system

Then go to your application and refresh it multiple times. You will notice that your updated changes are applied correctly. Most of the time, the updated content will appear, and occasionally you will see the old version’s data. This confirms that the traffic is being split between the canary and stable deployments.

What is a Kiali dashboard?

Kiali is an observability console for Istio with service mesh configuration and validation capabilities. It helps you understand the structure and health of your service mesh by monitoring traffic flow to infer the topology and report errors.

Get Sasanka Ranawaka’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

In the Kiali dashboard, we can visually explore our Kubernetes cluster. It allows us to view the number of Pods, Namespaces, and Deployments running in the cluster, along with their health status and traffic flow.

Kiali is built on top of service mesh technology and operates within the Istio system. To enable it, navigate to the Istio folder and create a new file named kiali.yaml , then add the required content to it.

Since the content of this file is quite large and not suitable to include directly in a blog article, you can download it from the GitHub repository.

Next, create two more files: jaeger.yaml and prometheus.yaml , and add their respective configurations as well. (Refer to the GitHub repository for the complete file contents.)

jaeger.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jaeger
  namespace: istio-system
  labels:
    app: jaeger
spec:
  selector:
    matchLabels:
      app: jaeger
  template:
    metadata:
      labels:
        app: jaeger
        sidecar.istio.io/inject: "false"
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "14269"
    spec:
      containers:
        - name: jaeger
          image: "docker.io/jaegertracing/all-in-one:1.46"
          env:
            - name: BADGER_EPHEMERAL
              value: "false"
            - name: SPAN_STORAGE_TYPE
              value: "badger"
            - name: BADGER_DIRECTORY_VALUE
              value: "/badger/data"
            - name: BADGER_DIRECTORY_KEY
              value: "/badger/key"
            - name: COLLECTOR_ZIPKIN_HOST_PORT
              value: ":9411"
            - name: MEMORY_MAX_TRACES
              value: "50000"
            - name: QUERY_BASE_PATH
              value: /jaeger
          livenessProbe:
            httpGet:
              path: /
              port: 14269
          readinessProbe:
            httpGet:
              path: /
              port: 14269
          volumeMounts:
            - name: data
              mountPath: /badger
          resources:
            requests:
              cpu: 10m
      volumes:
        - name: data
          emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  name: tracing
  namespace: istio-system
  labels:
    app: jaeger
spec:
  type: ClusterIP
  ports:
    - name: http-query
      port: 80
      protocol: TCP
      targetPort: 16686
    # Note: Change port name if you add '--query.grpc.tls.enabled=true'
    - name: grpc-query
      port: 16685
      protocol: TCP
      targetPort: 16685
  selector:
    app: jaeger
---
# Jaeger implements the Zipkin API. To support swapping out the tracing backend, we use a Service named Zipkin.
apiVersion: v1
kind: Service
metadata:
  labels:
    name: zipkin
  name: zipkin
  namespace: istio-system
spec:
  ports:
    - port: 9411
      targetPort: 9411
      name: http-query
  selector:
    app: jaeger
---
apiVersion: v1
kind: Service
metadata:
  name: jaeger-collector
  namespace: istio-system
  labels:
    app: jaeger
spec:
  type: ClusterIP
  ports:
  - name: jaeger-collector-http
    port: 14268
    targetPort: 14268
    protocol: TCP
  - name: jaeger-collector-grpc
    port: 14250
    targetPort: 14250
    protocol: TCP
  - port: 9411
    targetPort: 9411
    name: http-zipkin
  - port: 4317
    name: grpc-otel
  - port: 4318
    name: http-otel
  selector:
    app: jaeger

Then, commit and push the changes to the GitHub repository. After that, reconcile the cluster using the flux reconcile command.

flux reconcile source git -n flux-system flux-system

Now you can verify whether Kiali has been created in your cluster by running the following command:

kubectl get svc -m istio-system

After that, you will be able to see that Kiali and the other configurations have been created successfully, as shown below:

You can view these insights visually through your web browser by using port forwarding.

kubectl port-forward svc/kiali -n istio-system 20001:20001

After that, go to your web browser and open http://localhost:20001. You will now see the Kiali dashboard, where you can view your cluster insights, health information, and other important configurations in a visual and interactive interface.

Conclusion:

In this blog, we explored how Canary Deployment can be used in Kubernetes to safely release a new application version by gradually routing traffic between stable and canary deployments using Istio. We also introduced the Kiali Dashboard, which helps visualize service communication, traffic flow, and the health of the cluster. Together, these tools help teams deploy applications more safely while improving observability in cloud-native environments.

Additionally, if you find this entire file and folder structure difficult to follow, you can refer to the GitHub repository below. It contains the complete project with all configurations used in this setup.

🔗 GitHub Repository:
https://github.com/tharinduk001/helm-istio-certmanager-v2

🤝 Let’s Connect

I’m documenting my Kubernetes and Cloud Native learning journey week by week through this workshop. If you’re also learning Kubernetes, DevOps, or Cloud technologies, feel free to connect with me and share knowledge.

🔗 LinkedIn: linkedin.com/in/sasankad101
💬 Always open to discussions, feedback, and collaboration opportunities.

Let’s learn and grow together in the Cloud Native ecosystem 🚀