⚡ “Spring Boot at Warp Speed: My Journey to 1 Million Requests per Second 🚀🔥”

⚡ Spring Boot at Warp Speed: My Journey to 1 Million Requests per Second 🚀🔥

🧩 The Challenge

Let’s be real — most Spring Boot apps crumble under pressure.
At around a few thousand requests per second, CPU spikes, GC pauses increase, and response times explode like fireworks 🎇.

But what if I told you it’s completely possible to scale a Spring Boot app to handle 1 million+ requests per second — without rewriting your entire stack in Go or Rust?

That’s exactly what I set out to prove.
Here’s how I built, profiled, tuned, and battle-tested a Spring Boot service to survive the storm. 🌪️

🧠 Step 1: Ditch Tomcat — Go Reactive with Netty

Tomcat is great… until you start chasing ultra-low latency.
So the first move was switching from the traditional blocking servlet stack to a non-blocking WebFlux stack powered by Netty.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-webflux</artifactId>
</dependency>

Here’s what the reactive controller looks like:

@RestController
public class ReactiveController {
@GetMapping("/ping")
    public Mono<String> ping() {
        return Mono.just("pong");
    }
}

Why it matters:
Netty uses an event loop model — a handful of threads can handle thousands of concurrent connections, unlike the thread-per-request model in Tomcat.

⚙️ Step 2: Tune Your JVM Like a Formula 1 Engine 🏎️

Default JVM settings are designed for “safe mode,” not “scale mode.”

Here’s a battle-tested JVM config that made a huge difference:

JAVA_OPTS="
  -Xms4G -Xmx4G
  -XX:+UseG1GC
  -XX:MaxGCPauseMillis=100
  -XX:+ParallelRefProcEnabled
  -XX:+AlwaysPreTouch
  -XX:+DisableExplicitGC
  -XX:+UseStringDeduplication
"

✅ Why this works:

• G1GC reduces long pauses.
• PreTouch allocates memory upfront (no surprises later).
• String Dedup saves memory for high-throughput text-heavy services.

🌐 Step 3: Load Balance & Scale Horizontally with Nginx or Kubernetes

No matter how optimized your single instance is, real scale means distribution.

Get Karuna’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Here’s a simple Nginx load balancer config to fan out requests:

upstream spring_cluster {
    server 10.0.0.1:8080;
    server 10.0.0.2:8080;
    server 10.0.0.3:8080;
}
server {
    listen 80;
    location / {
        proxy_pass http://spring_cluster;
    }
}

Then, in Kubernetes, use an HPA (Horizontal Pod Autoscaler):

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: spring-boot-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: spring-boot-app
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

This ensures your app auto-scales when CPU usage spikes — no 3 AM panic wakeups ☕.

⚡ Step 4: Optimize Spring Boot’s Threading Model

In WebFlux (Netty-based), you can fine-tune event loops for better concurrency:

reactor:
  netty:
    ioWorkerCount: 8
    selectorCount: 4

Tip:

• ioWorkerCount: handles incoming requests
• selectorCount: multiplexes I/O events efficiently

You can calculate optimal values as:
👉 (number of cores * 2)

🧮 Step 5: Cache Everything That Moves (and Some That Don’t)

Caching is the secret weapon of scale.
Add Caffeine, an ultra-fast in-memory cache:

<dependency>
    <groupId>com.github.ben-manes.caffeine</groupId>
    <artifactId>caffeine</artifactId>
</dependency>

Usage:

@Service
public class UserService {
    private final Cache<String, User> cache = Caffeine.newBuilder()
            .maximumSize(10_000)
            .expireAfterWrite(10, TimeUnit.MINUTES)
            .build();
public User getUser(String id) {
        return cache.get(id, this::loadUserFromDb);
    }
    private User loadUserFromDb(String id) {
        simulateLatency();
        return new User(id, "John Doe");
    }
}

🧠 Step 6: Profile Everything, Guess Nothing

Before and after tuning, always measure performance with:

• 🔹 JMeter for load testing
• 🔹 Actuator metrics for response times
• 🔹 Java Flight Recorder (JFR) for GC and thread insights
• 🔹 Grafana + Prometheus for live dashboards

management:
  endpoints:
    web:
      exposure:
        include: health, metrics, prometheus

You can visualize throughput in Grafana using the /actuator/prometheus endpoint.

🔐 Step 7: Offload Heavy Work with Async Pipelines

For tasks like file processing, notifications, or reports, don’t block HTTP threads.
Use Spring’s Async + Executor pools:

@EnableAsync
@Configuration
public class AsyncConfig {
@Bean(name = "taskExecutor")
    public Executor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(20);
        executor.setMaxPoolSize(100);
        executor.setQueueCapacity(500);
        executor.initialize();
        return executor;
    }
}

Then use:

@Async("taskExecutor")
public void processReport() {
    // heavy I/O or computation
}

Your web thread stays clean, your throughput goes up 📈.

🧩 Step 8: Use a Gateway for Global Caching and Rate Limiting

To survive 1M+ RPS, use an API gateway like Spring Cloud Gateway or NGINX Plus to:

• Cache static responses
• Apply rate limits
• Offload SSL/TLS termination

🧠 The Final Architecture

🔥 Tech Stack That Survived 1M RPS:

• Spring Boot WebFlux (Netty)
• JVM tuned for low GC
• Redis + Caffeine caching
• Nginx load balancer
• Async worker pools
• Prometheus + Grafana monitoring
• Kubernetes autoscaling

📈 Results

After all optimizations, the test results blew my mind:

🏁 Final Thoughts

You don’t need magic frameworks or microservices buzzwords to scale — just understand your stack deeply and tune it like an engineer 🧑‍🔧.

Spring Boot can scale.
You just have to make it earn it. 💪

A message from our Founder

Hey, Sunil here. I wanted to take a moment to thank you for reading until the end and for being a part of this community.

Did you know that our team run these publications as a volunteer effort to over 3.5m monthly readers? We don’t receive any funding, we do this to support the community. ❤️

If you want to show some love, please take a moment to follow me on LinkedIn, TikTok, Instagram. You can also subscribe to our weekly newsletter.

And before you go, don’t forget to clap and follow the writer️!