[Part 4— Devops Journey] Dealing with Istio, Kiali and Jaeger

In our previous blogs:

Part 1: We used Terraform to provision our infrastructure — virtual machines, networking, and security groups.

[Part 1 — Devops Journey] Getting Started with Terraform for Microservices Infrastructure

Part 2: We automated the configuration of those virtual machines and deployed Kubernetes using Kubespray.

[Part 2 — Devops Journey] Infrastructure Automation with Terraform & Kubespray

Part 3: We implemented continuous deployment using ArgoCD and GitOps for our applications.

[Part 3— Devops Journey] Continuous Deployment with ArgoCD

Now, in Part 4, we plan to use Istio to manage communication, security, and observability between our microservices.

Why Istio?

When we create microservices in a cluster, there are several challenges to handle:

1. Service Communication: Each microservice needs to know the endpoints of the other services it interacts with. Adding a new service means updating all relevant connections.
2. Security: Once a request enters the cluster, it can potentially access any microservice. This becomes critical in sensitive applications like banking.
3. Reliability: If a microservice is temporarily unreachable, we need retry logic and proper error handling.
4. Observability: Monitoring is crucial — tracking the number of requests per service, error rates, and request latency to identify bottlenecks.

All of these require developers to write extra code and maintain it alongside the business logic of the microservices.

Wouldn’t it make more sense to extract all the non-business logic into a separate component? That’s where Istio comes in. Istio automatically injects sidecar proxies into each microservice. These sidecars handle:

• Routing and retries
• Security policies
• Monitoring and telemetry

This way, your microservices can focus entirely on business logic, while Istio takes care of the networking, security, and observability.

The image shows the Appointment Service running in a Kubernetes pod with an Envoy sidecar automatically injected by Istio. The sidecar handles all non-business logic such as routing, retries, security, and telemetry, allowing the main container to focus solely on the application’s core functionality.

Istio and Canary Deployments

Another powerful use of Istio is in canary deployments. In this strategy, you run two versions of a microservice simultaneously — the current stable version and a new version that you want to test. Instead of routing all traffic to the new version, Istio allows you to split the traffic, sending only a small percentage to the new version while the majority continues to hit the stable version. This is often called traffic splitting.

This approach provides several benefits:

1. Safe Testing: You can test new features in production with minimal risk since only a small portion of users experience the changes.
2. Quick Rollback: If something goes wrong with the new version, you can immediately redirect traffic back to the stable version without downtime.
3. Gradual Rollout: Traffic can be gradually increased to the new version as confidence grows, ensuring smooth adoption.
4. Observability: Istio collects metrics and logs for both versions, making it easy to compare performance, error rates, and latency before fully switching over.

The diagram shows a kiali dashboard visualization of a load generator sending traffic to a microservice, which is split between the stable version and the new version. Istio handles the routing automatically, allowing only a small percentage of traffic to reach the new version for safe canary testing.

Architecture Diagram of Istio Implementation

Istio Ingress Gateway

One of the key features of Istio is the Istio Ingress Gateway, which acts as the entry point into your Kubernetes cluster. All external traffic — HTTP or HTTPS — passes through the ingress gateway before reaching your microservices.

Here’s an example Istio Gateway configuration with HTTPS support:

# Updated Istio Gateway with HTTPS support
# TLS certificates should be created as Kubernetes secrets
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: salon-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
    # HTTP server - redirects to HTTPS when TLS is configured
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - "*"
    # HTTPS server for aurora-glam.com and subdomains
    - port:
        number: 443
        name: https
        protocol: HTTPS
      hosts:
        - "aurora-glam.com"
        - "*.aurora-glam.com"
      tls:
        mode: SIMPLE
        credentialName: aurora-glam-tls  # Secret containing TLS cert

How This Works

1. HTTPS Requests: When a user accesses https://aurora-glam.com, the request hits the Istio Ingress Gateway and is forwarded to the appropriate load balancer target group.
2. HTTP Requests: If the user sends http://aurora-glam.com, the gateway automatically redirects the browser to use HTTPS. This ensures all traffic is encrypted.
3. Load Balancer Interaction: When a load balancer is present, it terminates TLS and forwards the decrypted traffic to the cluster on HTTP port 80. Even without a load balancer, the Istio Ingress Gateway can handle HTTP → HTTPS redirection.
4. Internal Communication: Once inside the cluster, traffic between services is communicated over HTTP, while Istio sidecars can enforce mTLS if needed for service-to-service encryption.

AWS Load Balancer Rules and Istio Ingress Gateway Traffic Flow
The diagram shows how the AWS Load Balancer rules handle incoming HTTP and HTTPS requests. HTTP traffic is automatically redirected to HTTPS, and HTTPS traffic is forwarded to the Istio Ingress Gateway, which routes requests to the target microservices inside the cluster.

Securing Microservice Communication with mTLS

As mentioned earlier, security is a major concern when microservices communicate within a cluster. Without proper safeguards, any service could potentially access any other service freely, which is especially risky in sensitive applications like banking. This is where mTLS (mutual TLS) comes in.

Get Rithara Kithmanthie’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

In standard HTTPS, only the server’s certificate is verified, meaning the client (browser or service) doesn’t need to prove its identity. In mTLS, both sides — client and server — verify each other’s certificates. This ensures that only authorized services can communicate with each other.

In our cluster, Envoy sidecars handle mTLS automatically between microservices. This provides:

• Encryption: All traffic between services is encrypted.
• Authentication: Only verified services can communicate, preventing unauthorized access.
• Integrity: Messages cannot be tampered with in transit.

mtls

Let’s see how its implemented in the code

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: istio-system
spec:
  mtls:
    mode: STRICT

Istio allows two modes for mTLS:

1. Strict Mode: All traffic between services must use mTLS. This is recommended for production.
2. Permissive Mode: Both mTLS and plain HTTP traffic are allowed. Useful for testing or gradual rollout, but not recommended for production environments.

In our implementation, we have enabled mTLS in Strict mode to ensure that all inter-service communication is fully encrypted and authenticated.

Domain Verification with ACME

In addition to mTLS, our Istio setup includes ACME (Automatic Certificate Management Environment) to ensure that the domains we use in our cluster actually belong to us. This prevents unauthorized users from issuing certificates for our domain and adds an extra layer of security for HTTPS traffic.

In our repository, you can find the ACME configuration under the istio folder, which includes the setup for certificate secrets, challenges, and automated renewal. This ensures that our Istio Ingress Gateway always has valid, verified certificates, keeping traffic secure and trusted.

Kiali — Observing Service Mesh Traffic

Kiali is a visualization and observability tool for Istio service meshes. It provides a graphical overview of your microservices and how they interact, making it easy to understand complex service-to-service communication.

With Kiali, you can:

• Visualize Service Dependencies: See which services call which others.
• Monitor Traffic Flows: Track request rates, success/failure rates, and latency between services.
• Identify Bottlenecks: Quickly spot slow or failing services that could affect application performance.
• Observe Canary Deployments: Monitor traffic splitting between old and new versions of services.

For example, in our cluster, Kiali shows the traffic distribution between the stable version and the new version of the frontend service, helping us verify our canary deployment and routing rules.

Jaeger — Distributed Tracing

Jaeger is used for tracing requests as they flow through a distributed system. It helps developers and operators understand the latency and execution path of requests across multiple microservices.

With Jaeger, you can:

• Trace Requests End-to-End: Follow a single request from entry point to the backend services it touches.
• Identify Performance Bottlenecks: Pinpoint services or operations causing delays.
• Debug Failures: Determine where errors or timeouts occur in the request flow.
• Measure Latency Metrics: Get insights into average response times, helping optimize system performance.

In our project, Jaeger complements Kiali by showing detailed request traces, so if a request to the frontend takes longer than expected, we can see exactly which microservice or operation is slowing it down.

Author’s note

Yp! That’s the end guys.

I honestly had so much fun writing down these.

Feel free to let me know if I missed something or if you like to share some interesting stuff.

See you soon with more blogs :)

References

GitHub - WSO2-G02/salon-gitops: This repo contains GitOps flow and Manifests for all services