Service Mesh & Network Policies — Advanced Traffic Management

“We need mTLS between all services.” “How do we do canary deployments?” “Can we trace requests across microservices?”

Welcome to service mesh territory.

Get Mathumathiv’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

This is where networking gets sophisticated.

What Is a Service Mesh?

A dedicated infrastructure layer for service-to-service communication.

Without Service Mesh:
┌─────────┐         ┌─────────┐
│Service A│────────►│Service B│
└─────────┘  HTTP   └─────────┘
  (direct connection, no control)

With Service Mesh:
┌─────────────────┐         ┌─────────────────┐
│  ┌───────────┐  │         │  ┌───────────┐  │
│  │Service A  │  │         │  │Service B  │  │
│  └─────┬─────┘  │         │  └─────▲─────┘  │
│        │        │         │        │        │
│  ┌─────▼─────┐  │         │  ┌─────┴─────┐  │
│  │Sidecar    │◄─┼─────────┼─►│Sidecar    │  │
│  │Proxy      │  │  mTLS   │  │Proxy      │  │
│  └───────────┘  │         │  └───────────┘  │
└─────────────────┘         └─────────────────┘

The sidecar proxy handles:

• mTLS encryption
• Load balancing
• Retries
• Circuit breaking
• Observability
• Traffic routing

Why Service Mesh?

Problems it solves:

• Service-to-service encryption → Automatic mTLS
• Traffic routing (canary, A/B) → Traffic splitting
• Observability → Built-in metrics, traces
• Reliability → Retries, timeouts, circuit breakers
• Security policies → Authorization rules

When you need it:

• Many microservices (10+)
• Strict security requirements
• Complex traffic routing
• Detailed observability needs

When you don’t:

• Simple applications
• Few services
• Team can’t handle added complexity

Popular Service Meshes

Istio

Most feature-rich, most complex.

┌─────────────────────────────────────┐
│           Control Plane             │
│  ┌─────────┐ ┌─────────┐ ┌───────┐  │
│  │ Pilot   │ │ Citadel │ │ Galley│  │
│  │(routing)│ │ (certs) │ │(config│  │
│  └─────────┘ └─────────┘ └───────┘  │
└─────────────────────────────────────┘
              │
              ▼
┌─────────────────────────────────────┐
│           Data Plane                │
│  ┌─────────────────────────────┐    │
│  │  Envoy Sidecar Proxies      │    │
│  │  (one per pod)              │    │
│  └─────────────────────────────┘    │
└─────────────────────────────────────┘

Linkerd

Simpler, lighter, easier to operate.

- Written in Rust (proxy) + Go (control plane)
- Smaller resource footprint
- Easier to learn
- Less features than Istio

Cilium Service Mesh

eBPF-based, high performance.

- No sidecar (eBPF in kernel)
- Better performance
- L7 policies
- Integrated with Cilium CNI

Istio Basics

Installation

# Install istioctl
curl -L https://istio.io/downloadIstio | sh -
cd istio-*
export PATH=$PWD/bin:$PATH

# Install Istio on cluster
istioctl install --set profile=demo

# Enable sidecar injection
kubectl label namespace default istio-injection=enabled

# Verify
kubectl get pods -n istio-system

Automatic Sidecar Injection

# Label namespace for auto-injection
kubectl label namespace default istio-injection=enabled

# Or annotate specific pods
metadata:
  annotations:
    sidecar.istio.io/inject: "true"

Now every pod gets an Envoy sidecar automatically.

Traffic Management with Istio

VirtualService

Define how requests are routed.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-service
spec:
  hosts:
    - my-service
  http:
    - match:
        - headers:
            x-version:
              exact: "v2"
      route:
        - destination:
            host: my-service
            subset: v2
    - route:
        - destination:
            host: my-service
            subset: v1

DestinationRule

Define subsets (versions) and policies.

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: my-service
spec:
  host: my-service
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
  subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2

Canary Deployment

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-service
spec:
  hosts:
    - my-service
  http:
    - route:
        - destination:
            host: my-service
            subset: v1
          weight: 90
        - destination:
            host: my-service
            subset: v2
          weight: 10

90% of traffic → v1 (stable)
10% of traffic → v2 (canary)

Fault Injection (Testing)

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-service
spec:
  hosts:
    - my-service
  http:
    - fault:
        delay:
          percentage:
            value: 10
          fixedDelay: 5s
        abort:
          percentage:
            value: 5
          httpStatus: 500
      route:
        - destination:
            host: my-service

10% of requests delayed 5s, 5% return 500 error.

mTLS (Mutual TLS)

Automatic mTLS

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: istio-system
spec:
  mtls:
    mode: STRICT  # All traffic must be mTLS

Modes:

• PERMISSIVE: Accept both mTLS and plain text
• STRICT: Only mTLS allowed
• DISABLE: No mTLS

Check mTLS Status

# Check if mTLS is enabled
istioctl x authz check deploy/my-service

# View certificates
istioctl proxy-config secret deploy/my-service

Authorization Policies

Control who can access what.

Allow Policy

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: allow-frontend
  namespace: default
spec:
  selector:
    matchLabels:
      app: backend
  action: ALLOW
  rules:
    - from:
        - source:
            principals: ["cluster.local/ns/default/sa/frontend"]
      to:
        - operation:
            methods: ["GET", "POST"]
            paths: ["/api/*"]

Only frontend service account can access backend’s /api/* with GET/POST.

Deny Policy

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: deny-external
spec:
  selector:
    matchLabels:
      app: internal-service
  action: DENY
  rules:
    - from:
        - source:
            notNamespaces: ["internal"]

Kubernetes Network Policies (No Service Mesh)

For teams not ready for service mesh, Network Policies provide L3/L4 control.

Default Deny All

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: default
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

Allow Specific Traffic

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-backend
spec:
  podSelector:
    matchLabels:
      app: backend
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - port: 8080

Allow DNS

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
spec:
  podSelector: {}
  policyTypes:
    - Egress
  egress:
    - to:
        - namespaceSelector: {}
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - port: 53
          protocol: UDP

Complex Example

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-policy
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: api
  policyTypes:
    - Ingress
    - Egress
  ingress:
    # Allow from ingress controller
    - from:
        - namespaceSelector:
            matchLabels:
              name: ingress-nginx
      ports:
        - port: 8080
    # Allow from monitoring
    - from:
        - namespaceSelector:
            matchLabels:
              name: monitoring
      ports:
        - port: 9090
  egress:
    # Allow to database
    - to:
        - podSelector:
            matchLabels:
              app: postgres
      ports:
        - port: 5432
    # Allow DNS
    - to:
        - namespaceSelector: {}
      ports:
        - port: 53
          protocol: UDP

Service Mesh vs Network Policies

Network Policies:

• Layer: L3/L4 (IP, Port)
• mTLS: No
• Traffic routing: No
• Observability: No
• Retries/timeouts: No
• Resource overhead: Minimal
• Complexity: Low

Service Mesh:

• Layer: L7 (HTTP, gRPC)
• mTLS: Yes
• Traffic routing: Yes (canary, A/B)
• Observability: Yes (metrics, traces)
• Retries/timeouts: Yes
• Resource overhead: Significant
• Complexity: High

Use Network Policies when:

• You need basic traffic control
• L3/L4 is enough
• Team isn’t ready for service mesh

Use Service Mesh when:

• You need mTLS everywhere
• You need L7 traffic control
• You need advanced observability
• You have many microservices

Observability with Service Mesh

Metrics

Istio automatically generates metrics:

istio_requests_total
istio_request_duration_milliseconds
istio_tcp_connections_opened_total

# Port forward to Prometheus
kubectl port-forward svc/prometheus -n istio-system 9090:9090

# View in Grafana
kubectl port-forward svc/grafana -n istio-system 3000:3000

Distributed Tracing

# View traces in Jaeger
kubectl port-forward svc/jaeger-query -n istio-system 16686:16686

Traces show request flow across services:

frontend (50ms)
  └── api-gateway (30ms)
       ├── user-service (15ms)
       └── order-service (20ms)
            └── database (10ms)

Service Graph

# Kiali dashboard
kubectl port-forward svc/kiali -n istio-system 20001:20001

Visual map of service communication.

Debugging Service Mesh

Check Sidecar Status

# Is sidecar injected?
kubectl get pod my-pod -o jsonpath='{.spec.containers[*].name}'
# Should show: my-app istio-proxy

# Sidecar logs
kubectl logs my-pod -c istio-proxy

# Envoy config
istioctl proxy-config cluster deploy/my-service
istioctl proxy-config route deploy/my-service
istioctl proxy-config listener deploy/my-service

Check mTLS

istioctl authn tls-check my-pod.default

# Output shows if mTLS is working
HOST:PORT         STATUS     SERVER   CLIENT
my-service:8080   OK         mTLS     mTLS

Analyze Issues

istioctl analyze

# Shows configuration issues

Getting Started Path

Step 1: Network Policies
  - Start with default deny
  - Add allow rules as needed
  - L3/L4 security
  
Step 2: Linkerd (Simple Service Mesh)
  - Easy to install
  - Automatic mTLS
  - Basic observability

Step 3: Istio (Full Service Mesh)
  - When you need advanced features
  - Traffic management
  - Complex routing

Quick Reference

Network Policy

podSelector:  Which pods this applies to
policyTypes:  Ingress, Egress, or both
ingress.from: Who can connect in
egress.to:    Where pod can connect

Istio

VirtualService:   How to route traffic
DestinationRule:  Subsets and policies
PeerAuthentication: mTLS settings
AuthorizationPolicy: Access control

Traffic Splitting

# Canary
weight: 90  # v1
weight: 10  # v2

Conclusion

Service mesh adds complexity but provides:

• Automatic encryption
• Traffic control
• Observability
• Security policies

Start with Network Policies for basic control. Graduate to service mesh when you need L7 features.

The key is matching the tool to your needs. Not every cluster needs Istio.

Service mesh or network policies — which do you prefer? Share in the comments.

Checkout my portfolio to know me more — mathumathiv.com