The Green Paved Road

Why Efficiency is the New SRE Signal

In the world of Internal Developer Platforms (IDPs), we’ve spent years perfecting the “Golden Path.” We made deploying services so frictionless that developers are now deploying 10x more microservices than they did a year ago. We told them: “Don’t worry about the nodes or the data center location. Just give us your container, and the platform will handle the rest”.

But in creating this frictionless path, we inadvertently created Resource Blindness. When developers lose sight of the “metal,” they lose the connection to the physical energy required to run their code.

The Problem: Waste by Design

Industry data shows that average Kubernetes cluster utilisation often sits below 30%, yet we pay for — and power — 100% of the underlying hardware. This isn’t just a budget issue; it’s a “Ghost Pod” problem where we pay for “just-in-case” CPU cycles that never execute.

If your platform defaults are inefficient, your entire organisation is inefficient by default. It’s time to move toward GreenOps — where sustainability equals yield.

The “Idle Gap” Problem: The Silent Efficiency Killer

In the world of platform engineering, the Idle Gap is the chasm between the resources a developer requests and what the application actually uses.

When a developer sets a CPU request of 1000m "just to be safe," but the service only consumes 100m during average load, that 900m difference is a "reserved-but-unusable" wasteland. On a single pod, it looks negligible. Multiplied across a cluster of 500 microservices, it translates to thousands of dollars in "cloud tax" and megawatts of wasted energy powering empty cycles.

The Idle Gap usually stems from three main factors:

1. Over-Provisioning for Peaks: Provisioning for Black Friday traffic levels on a random Tuesday in March.
2. Resource Blindness: A lack of observability tools that show developers their actual utilisation vs. their YAML definitions.
3. The “Safety Margin” Culture: The fear that being efficient will lead to OOMKills (Out Of Memory) or CPU throttling, leading teams to default to massive, wasteful buffers.

To bridge this gap, modern “Watt-Aware” platforms are moving away from static configurations toward Dynamic Right-Sizing. By using tools like Vertical Pod Autoscaler (VPA) or Goldilocks, platforms can automatically suggest (or enforce) resource limits based on historical usage data, effectively shrinking the “ghost” footprint of the infrastructure without sacrificing performance.

Measuring the Unmeasurable: Kepler and the Power of eBPF

To build a truly “Watt-Aware” platform, you first need to solve a fundamental visibility problem: standard Kubernetes metrics like CPU and RAM usage do not actually measure energy draw. You can have two pods using 10% CPU, but depending on the instruction sets or hardware state, their actual power consumption in Watts can vary significantly.

Get Pronomita Dey’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

This is where Kepler (Kubernetes-based Efficient Power Level Exporter) comes in. Kepler uses eBPF to hook into the kernel’s power counters (such as RAPL or ACPI) to attribute real-time power consumption to specific PIDs and Pods.

Key Insights from Kepler Telemetry:

• Identifying “Ghost” Workloads: Kepler highlights the massive gap between resource requests and actual energy draw. For example, you might see CPU utilisation drop to 8%, but because Resource Requests are pinned at 60%, the infrastructure stays “hot”.
• The Cost of Idle Resources: A typical node (like an m5.2xlarge) can draw approximately 140 Watts per hour just for idle resources. When the infrastructure is “hot” but empty, your cost-per-request can effectively triple.
• Data-Driven Rightsizing: Instead of guessing resource limits, you can use the last 8 days of Kepler telemetry to inform your Vertical Pod Autoscaler (VPA).

The Efficiency Win:

By moving from static, “gut-feeling” limits to VPA-informed requests based on Kepler data, we were able to pinpoint waste with surgical precision. In one scenario, a service with a 1 CPU request was adjusted to a recommended 250m.

The result? We reclaimed 40% of cluster capacity, allowing us to host 40% more microservices without adding a single new node to the Auto Scaling Group (ASG).

3 Hard Lessons from the GreenOps Trenches

Building a “Watt-Aware” platform isn’t as simple as turning things off. Here are three real-world failures we encountered while trying to optimise for carbon and cost.

1. The “Scale-to-Zero” Trap

We attempted the “Holy Grail” of GreenOps: scaling an invoice generator to zero at 3:00 AM using KEDA.

• The Failure: When the 8:00 AM rush hit, the heavy Spring Boot app — with its 45-second startup time — couldn’t keep up. Timeouts led to retries, which led to a 10,000-message backlog and a “thundering herd” that crashed the pipeline.
• The Lesson: Efficiency must be App-Aware. For critical paths, use a “Pilot Light” strategy (keeping a minimum of one replica running) and implement Startup Probes to give apps a grace period before liveness checks kick in.

2. The “Chatty Services” Latency Storm

We tried Spatial Shifting, moving non-critical reporting services from a coal-heavy “US-East” grid to a renewable-powered “US-West” grid at night.

• The Failure: P99 latency spiked from 200ms to 1.5s. We forgot the service was “chatty,” making 50+ synchronous REST calls to a User-Service back in US-East. The speed of light (70ms round-trip) added 3.5 seconds of overhead, exhausting connection pools.
• The Lesson: GreenOps must be Topology-Aware. Keep “chatty” neighbours together and only shift “loner” workloads — like batch processing or image transcoding — that don’t need constant cross-region communication.

3. The “OOMKill” Efficiency Trap

To close “Idle Gaps,” we moved from static limits to aggressive Vertical Pod Autoscaling (VPA).

• The Failure: The VPA shrunk a Java service’s memory from 2GiB to 1GiB. It worked until a traffic burst caused a spike in Metaspace, exceeding the lean container limit and triggering a cluster-wide OOMKill spiral.
• The Lesson: GreenOps requires an Error Budget. Efficiency should never compromise stability. We now use a mandatory 20% Green Buffer on all VPA recommendations and sync Java heap scaling with container limits.

The “Right Away” Wins: How to Start

You don’t need to move mountains to start saving watts and dollars today:

1. Stop Guessing with VPA: Use VPA in Recommender mode. By analysing Kepler telemetry (which uses eBPF to measure real-time Power Consumption), we reclaimed 40% of the cluster's capacity without adding a single node.
2. Enforce Efficiency Policies: Use tools like Kyverno to reject deployments that exceed a healthy “Efficiency Ratio” (e.g., Limit < 2x Request).
3. Exploit the “Cheap & Green” Window: Categorise workloads into “Hot” (user-facing) and “Cold” (batch/CI). Delay heavy lifting — like security scans — to 2:00 AM when the grid is clean, and spot prices are at the floor.

The Bottom Line

A highly tuned, right-sized cluster is cheaper to run, more predictable in performance, and — by-productively — green. Sustainability isn’t just about saving the planet; it’s about building better engineering systems.