From Browser to Backend: How Tomcat Processes HTTP Requests

Understand how connections, threads, and I/O work together to bring an HTTP request into your Java application.

You write a controller:

@GetMapping("/hello")
public String hello() {
    return "Hello, Java!";
}

You open the URL, and a response appears.

But your controller is only one part of the process.

Who accepts the connection? Who reads the request? Which thread runs the controller? What happens when hundreds of requests arrive together?

And if Tomcat uses non-blocking I/O, why can a slow database still exhaust its workers?

To understand the answers, we need to connect three ideas:

• Connections carry data.
• Threads execute code.
• I/O mechanisms determine how the server waits for data.

This article focuses on Tomcat’s Java NIO connector and a conventional Spring MVC application. We will also examine asynchronous processing, HTTP/2, and virtual threads because they change parts of the execution model.

The documentation references use Tomcat 11. Configuration and implementation details can differ between versions.

1. What Is Tomcat?

Apache Tomcat is a web server and Servlet container.

A Servlet is a Java component that handles requests and produces responses. The container manages its lifecycle and provides the environment in which it runs.

Tomcat accepts HTTP traffic, selects the appropriate web application, and invokes application processing.

Its main architectural components include:

ComponentResponsibilityServerRepresents the Tomcat serverServiceConnects one or more connectors to one engineConnectorHandles communication with clientsEngineProcesses requests for a serviceHostRepresents a virtual hostContextRepresents a web application

These are organizational components. They are not separate threads for every step.

Where Does Spring MVC Fit?

Tomcat and Spring MVC have different responsibilities.

Tomcat provides the HTTP and Servlet environment. Spring MVC uses a central Servlet called DispatcherServlet to coordinate request mapping, controller invocation, and response handling.

For example:

1. Tomcat receives traffic on port 8080.
2. The request enters the application.
3. Spring MVC maps /hello to your hello() method.
4. Your method returns application data.
5. The framework and container cooperate to send the response.

When using embedded Tomcat, the server runs inside your application’s JVM. You do not necessarily install and start it as a separate program.

2. Connection, Request, Thread, and Session Are Different

These terms are often mixed up.

TermMeaningExampleConnectionA communication channel between endpointsA TCP connection to port 8080RequestOne application-level messageGET /products/42ThreadAn execution path inside a processThe thread running your controllerSessionApplication state associated with a clientA shopping cart associated with a session ID

Imagine a restaurant:

• A connection is an open communication channel with a customer.
• A request is an order.
• A worker thread is a staff member processing an order.
• A session is information associated with that customer.

The restaurant does not need to reserve one staff member for every customer who remains seated.

Similarly, an open connection does not necessarily occupy a request-processing thread continuously.

Also, 1,000 users does not mean 1,000 threads.

Some users may be idle. Others may send multiple requests. A browser may open several connections, and a proxy may reuse connections for traffic from many users.

3. What Happens Before Your Controller Runs?

Imagine a client requests:

GET /products/42 HTTP/1.1
Host: localhost:8080
Accept: application/json

Assume a direct HTTP/1.1 connection to Tomcat.

Before application code runs:

1. The client determines the server address.
2. A TCP connection is established, unless a reusable connection already exists.
3. The client sends HTTP request bytes.
4. Tomcat reads and interprets those bytes.
5. The request reaches the application.

With HTTPS, TLS adds encryption and handshake processing.

In a deployed system, a reverse proxy or load balancer may terminate TLS and forward traffic to Tomcat. Therefore, the connection Tomcat sees might come from a proxy rather than directly from the browser.

TCP Delivers Bytes

TCP carries a byte stream. It does not deliver complete Java request objects.

A request can arrive in multiple pieces:

• Part of the request line.
• The remaining headers.
• Some of the body.
• More body data later.

This distinction matters because socket readiness does not mean the entire HTTP request has arrived.

Tomcat must interpret the bytes according to the HTTP protocol and manage incomplete input.

4. The Three Roles in Tomcat’s NIO Model

For understanding the NIO connector, focus on three roles:

RoleMain jobAcceptorAccept new connectionsPollerMonitor registered sockets for readinessWorkerExecute processing tasks

Tomcat’s NioEndpoint contains networking machinery, including its Poller and SocketProcessor components. A SocketProcessor runs processing work through an executor.

The Acceptor

The Acceptor accepts connections from the listening server socket.

Accepting a connection is different from executing a controller. It establishes the server-side connection that subsequent processing can use.

The Poller

The Poller registers sockets, checks readiness events, and hands relevant work to processing code.

It uses a Java Selector to monitor multiple channels. It does not execute all your application’s business logic.

The Worker

A worker executes a processing task.

For a normal synchronous request, this includes application Servlet processing. The same worker can later process unrelated requests.

A thread pool reuses execution resources. It does not permanently assign a worker to each user.

5. What Does a Selector Actually Do?

Suppose 5,000 clients have open connections, but only 30 currently have useful data available.

Assigning a dedicated platform thread to wait on every connection would be expensive.

A selector lets a thread wait for readiness across many registered channels.

Applications register interest in operations such as reading or writing. Selection reports which channels are ready for the relevant operations.

A selection call can itself block while waiting for events. This is compatible with a non-blocking networking design: one thread waits for many channels instead of one thread waiting for each channel.

Think of a receptionist watching a notification board.

The receptionist does not repeatedly visit every room. The board identifies where attention is needed.

However, a readiness notification is only a signal to attempt progress. It is not a promise that a complete application operation can finish immediately.

Figure 1 — How Tomcat separates connection monitoring from application execution. Idle connections do not each require a dedicated application worker.

6. Following One Request Through Tomcat

Let’s follow:

GET /products/42

Step 1: Accept and Register the Connection

Tomcat accepts the connection and makes it available to the NIO event-processing machinery.

For a reused connection, this acceptance step has already happened.

Step 2: Detect Readiness

The networking layer detects that processing can make progress.

For example, bytes may now be available to read.

Step 3: Execute a Processing Task

A worker executes the corresponding task.

The HTTP processor interprets the request line and headers. Request-body consumption may continue later when application code needs it.

Step 4: Enter the Container

Tomcat’s CoyoteAdapter bridges protocol processing and the container pipeline. Its post-parsing work prepares the request and response for container processing.

Step 5: Run Application Processing

The selected application’s filter chain and Servlet run.

In Spring MVC, DispatcherServlet coordinates the framework work. It uses mappings and adapters to invoke the appropriate controller.

Response handling may include converting a returned object into JSON.

Step 6: Produce the Response

The response contains a status, headers, and potentially a body:

HTTP/1.1 200 OK
Content-Type: application/json

{"id":42,"name":"Java Notebook"}

Returning a Java object from a controller does not itself transmit that object over the network. The response must be represented as bytes and written through the connector.

Step 7: Release Processing Resources

When synchronous processing finishes, the worker becomes available for another task.

The connection can remain open for another request.

Figure 2 — A synchronous request travels through Tomcat, Spring MVC, application code, and the database before the response returns to the client.

7. Does Each Application Layer Use a Different Thread?

Usually, ordinary synchronous method calls stay on the same thread.

Consider:

public Product findProduct(long id) {
    return productRepository.findById(id);
}

Calling a repository does not automatically create another thread.

A request might produce these log messages:

Controller: http-nio-8080-exec-4
Service:    http-nio-8080-exec-4
Repository: http-nio-8080-exec-4

The class changes. The execution thread does not.

A thread change requires a mechanism that schedules work elsewhere, such as an executor or an asynchronous framework feature.

Controller, service, and repository are responsibility boundaries. They are not automatically thread boundaries.

8. Blocking I/O: Waiting Still Occupies Capacity

Consider an illustrative request:

ActivityTimeValidate input2 msWait for a database result180 msPrepare the response8 msTotal190 ms

The database wait dominates.

In a conventional blocking execution model, the worker cannot process another request while that call is waiting.

It may consume little CPU during the wait, yet still occupy a worker slot.

This explains an apparently strange production symptom:

CPU usage is low, but requests are timing out.

Workers may be waiting for:

• Database connections.
• Database results.
• External services.
• Locks.
• Slow request bodies.
• Slow response consumers.

Not using CPU and being available to serve another request are different conditions.

Figure 3 — A blocking database call keeps the worker occupied, even when the thread consumes little CPU while waiting.

9. Non-Blocking Networking Does Not Make All Code Non-Blocking

Tomcat’s HTTP connector supports Java NIO and NIO2. Its normal synchronous request model still needs a processing thread while application processing is active.

For example:

@GetMapping("/slow")
public String slow() throws InterruptedException {
    Thread.sleep(2_000);
    return "Finished";
}

The NIO connector does not transform Thread.sleep() into asynchronous application execution.

The worker executing this method remains occupied for approximately two seconds.

Likewise, replacing the sleep with a blocking database call does not make that call non-blocking.

There are several independent questions:

QuestionWhat it tells youHow does the connector monitor sockets?Networking modelDoes application code wait inside calls?Blocking behaviorCan the request remain open after the initial dispatch exits?Servlet asynchronous processingAre tasks running on platform or virtual threads?Thread implementation

Do not answer all four questions with the single word “NIO.”

10. Keep-Alive: Reusing Connections

HTTP keep-alive allows a connection to be reused.

A client may request a product, receive its response, and later request another product over the same connection.

During the idle interval, Tomcat’s NIO machinery can monitor the connection without reserving an application worker for it.

This is why connection count and busy-thread count can be very different.

For example, a server might have:

• 2,000 established connections.
• 80 requests currently executing.
• Many remaining connections waiting for future traffic.

These numbers are illustrative, not capacity recommendations.

An idle connection still consumes resources, such as socket state and buffers. It is cheaper than dedicating a worker to it, but it is not free.

11. Thread Pools, Connection Limits, and Queues

Several configuration settings control different resources:

Tomcat settingControlsmaxThreadsRequest-processing threads in the internal platform-thread executormaxConnectionsAccepted connection capacityacceptCountRequested OS connection-backlog sizemaxQueueSizeExecutor tasks waiting for execution

The operating system can interpret the requested backlog differently. When connection capacity and the backlog are exhausted, clients may see connection refusal or timeouts.

These settings do not form one interchangeable request limit.

Tomcat can also use a shared executor. Its standard executor has its own thread and task-queue settings. Connector thread settings are ignored when that executor supplies the threads.

Why the Queue Distinction Matters

Imagine a restaurant with:

• Customers waiting outside.
• Customers already inside.
• Orders waiting in the kitchen.
• Cooks currently working.

These are different populations.

Get Imon Bhuiya’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Increasing the outdoor waiting area does not create more cooks.

Similarly, increasing a connection backlog does not increase application processing throughput.

Avoid this misleading formula:

Maximum requests = maxThreads + acceptCount

It mixes executing work with pending connections and ignores other limits.

A queue can absorb a temporary burst. Under sustained overload, it can also make requests wait longer before they fail.

12. A Small Experiment You Can Run

Create a Spring Boot Spring MVC application with embedded Tomcat.

Place this controller beneath the package scanned by your application:

package com.example.tomcatdemo;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class ThreadDemoController {
    @GetMapping("/demo/slow")
    public String slow() throws InterruptedException {
        String worker = Thread.currentThread().getName();
        System.out.println("START: " + worker);
        try {
            Thread.sleep(2_000);
            return "Handled by " + worker + "\n";
        } finally {
            System.out.println("END: " + worker);
        }
    }
}

The sleep deliberately simulates waiting. It is not a recommendation for production request handling.

Use this local experiment configuration:

spring.threads.virtual.enabled=false

server.tomcat.threads.max=2
server.tomcat.threads.min-spare=2

Spring Boot exposes these worker settings. Its documented platform-thread limits do not apply in the same way when virtual threads are enabled.

Send six requests concurrently from a terminal:

seq 1 6 | xargs -P 6 -I '{}' \
  curl -s http://localhost:8080/demo/slow

With no competing requests and the intended executor configuration, approximately two controller invocations should run at once.

Six requests should finish in roughly three groups:

GroupApproximate completion timeFirst two requests2 secondsNext two requests4 secondsFinal two requests6 seconds

Actual ordering and timing depend on scheduling and overhead.

This is an expected result, not a measured benchmark.

Try these variations:

1. Change the maximum to three workers.
2. Repeat the requests.
3. Compare the START and END messages.
4. Remove the sleep and compare again.

The experiment makes thread occupancy visible. It does not measure realistic database or production performance.

13. Why More Threads Are Not Always Better

Suppose your application has 200 workers but only 20 database connections.

If all 200 requests need the database simultaneously, only some can acquire connections immediately. Others wait.

Increasing Tomcat to 500 workers may simply create more waiting.

Possible consequences include:

• Greater memory consumption.
• More scheduling overhead.
• More contention.
• Longer queues.
• More pressure on downstream services.

For CPU-heavy work, additional threads beyond useful CPU parallelism do not create extra processing power.

For waiting-heavy work, more concurrency can help — but only while downstream systems remain healthy.

The tuning question is:

Which resource limits useful throughput, and what happens when we add more work?

Consider a slow database query. Increasing the worker count does not improve its execution plan or add an index. It may cause more copies of that slow query to compete for the same database resources.

14. Estimating Capacity Without Guessing

For an illustrative pool of 100 workers, if each request occupies a worker for 0.2 seconds:

100 workers / 0.2 seconds = 500 requests per second

This is an idealized ceiling based only on worker occupancy.

It assumes sufficient CPU, database capacity, network capacity, and continuous work. It excludes overhead and changing latency.

If average occupancy grows to one second:

100 workers / 1 second = 100 requests per second

The application code may be unchanged. A slower dependency can reduce throughput dramatically.

Little’s Law

A related relationship is:

Average in-flight work =
average throughput × average time in the system

If a stable system completes 300 requests per second with an average response time of 0.4 seconds:

300 × 0.4 = 120 requests in flight on average

“In flight” includes waiting and execution within the boundary you measured.

It does not automatically mean 120 occupied platform threads.

Use estimates to form hypotheses. Use measurements to validate them.

15. Servlet Asynchronous Processing

A synchronous request normally stays inside the application dispatch until processing completes.

Servlet asynchronous processing allows the initial dispatch to exit while the response remains open.

At the Servlet level, this involves startAsync() and an AsyncContext. The request can later be completed or dispatched again.

Separately, Servlet non-blocking body I/O uses readiness callbacks such as ReadListener and WriteListener.

These mechanisms are related but not identical.

An Asynchronous Spring MVC Example

A controller can return a Callable:

@GetMapping("/demo/async")
public Callable<String> async() {
    String initialThread = Thread.currentThread().getName();

    return () -> {
        String taskThread = Thread.currentThread().getName();
        Thread.sleep(2_000);
        return "Initial: " + initialThread
                + ", task: " + taskThread;
    };
}

Add:

import java.util.concurrent.Callable;

Spring submits the callable to an async executor. The initial container dispatch exits. When the result becomes available, a container dispatch resumes response processing.

Configure the async executor deliberately for real workloads. Do not treat an unspecified executor as an unlimited resource.

The sleep still occupies the thread executing the callable.

Moving blocking work to another executor changes where the waiting happens. It does not eliminate the waiting.

16. DeferredResult, CompletableFuture, and Non-Blocking Work

DeferredResult allows an application to supply a result later, potentially in response to an external event.

Spring MVC also supports other asynchronous return values. However, its response writes remain blocking, unlike the non-blocking response-writing model used by WebFlux.

When you see code like this:

CompletableFuture.supplyAsync(() -> blockingCall());

ask:

• Which executor runs it?
• How many tasks can wait?
• What limits downstream calls?
• Who handles timeout and failure?
• Does the caller immediately call join()?

If the request thread immediately waits with join(), it has not gained the intended opportunity to do other work.

Asynchronous and Non-Blocking Mean Different Things

Asynchronous describes coordination: an operation can complete later, and the caller does not necessarily wait for its result immediately.

Non-blocking describes whether execution must wait inside an operation.

A task can run asynchronously on another thread while performing blocking I/O.

For example, an executor thread can spend two seconds waiting for a blocking HTTP client. The original caller may be free, but a thread is still waiting.

17. Virtual Threads: A Different Cost Model

Virtual threads are Java threads scheduled by the JVM onto platform threads.

For supported blocking operations, a virtual thread can suspend while its underlying carrier thread becomes available for other work.

This can make large numbers of waiting tasks more affordable. It does not make CPU calculations faster or remove database limits.

Tomcat provides a virtual-thread executor that creates a virtual thread for each submitted task, rather than reusing a fixed platform-thread pool.

Consider 5,000 requests waiting for a service that permits only 50 concurrent calls.

Virtual threads may reduce the cost of representing those requests. They do not grant capacity for 5,000 simultaneous downstream calls.

You still need:

• Concurrency limits.
• Timeouts.
• Capacity planning.
• Cancellation and cleanup.
• Monitoring.

Virtual threads change thread economics. They do not remove overload.

18. What Changes with HTTP/2?

HTTP/2 multiplexes streams over a connection.

Several requests can therefore be active on the same connection. Counting TCP connections becomes an even weaker estimate of application concurrency.

Tomcat distinguishes the number of active streams from the number allowed to execute using container threads. HTTP/2 flow control can also delay response progress.

A blocking application remains blocking even when its transport uses HTTP/2.

Keep three things separate:

• The connection.
• The streams carried by it.
• The execution resources processing those streams.

A protocol can improve how data is transported without changing how your database driver waits for a query result.

19. Thread Safety: Shared Controllers Need Care

Spring beans use singleton scope by default. A typical controller instance is therefore shared within its application context.

This is unsafe:

@RestController
public class UnsafeController {

    private String currentUser;
    @GetMapping("/greet")
    public String greet(
            @RequestParam("name") String name) {
        currentUser = name;
        return "Hello, " + currentUser;
    }
}

Imagine this interleaving:

1. Request A sets currentUser to "Imon".
2. Request B sets it to "Sara".
3. Request A builds its response using "Sara".

The problem is shared mutable request-specific state.

Use a local variable instead:

@GetMapping("/greet")
public String greet(
        @RequestParam("name") String name) {

    String greeting = "Hello, " + name;
    return greeting;
}

Each invocation has its own local variable.

However, a local variable can still refer to a shared mutable object. Local references do not automatically make all operations thread-safe.

For shared state, choose a design appropriate to the requirement:

• Immutable values.
• Atomic operations.
• Carefully scoped synchronization.
• Database transactions and concurrency controls.

Adding volatile to the unsafe field would not solve the request-isolation problem. The field would still be shared between requests.

20. Timeouts Have Different Jobs

A single timeout setting does not protect every stage.

Timeout categoryProtects againstIncoming connection/read timeoutA peer that does not send expected data promptlyKeep-alive timeoutAn idle connection remaining open too longAsync request timeoutAsynchronous request processing that does not completeDatabase acquisition timeoutWaiting too long for a database connectionQuery timeoutExcessive database execution timeOutgoing HTTP timeoutA slow remote serviceProxy timeoutExcessive waiting at the intermediary

Tomcat’s connectionTimeout is not a universal deadline for controller execution. Its keep-alive timeout concerns waiting for another request on a connection.

Setting a two-second connector timeout does not mean every Java method is interrupted after two seconds.

Also, a client disconnect or an HTTP timeout does not guarantee that an already-running database operation has stopped.

Cancellation must be supported and handled through the relevant layers.

For example, if a proxy stops waiting after five seconds but a downstream call continues for thirty seconds, your application may keep consuming resources after the client has already received an error.

Timeouts should form a deliberate end-to-end policy.

21. How to Diagnose a Slow Application

Start with evidence.

ObservationUseful next investigationHigh CPU and rising latencyExpensive computation, allocation, serializationBusy workers and low CPUI/O waits, connection-pool waits, lock contentionIncreasing database acquisition timePool pressure or slow transactionsHigh tail latencyQueue growth, pauses, slow dependenciesConnection failuresNetwork path, connection limits, backlog pressure

Track throughput, error rate, and latency percentiles such as p95 and p99.

The p95 latency is the value at or below which approximately 95% of measured request durations fall. It helps reveal slow requests that an average can hide.

A practical investigation sequence is:

1. Identify which endpoints are slow.
2. Compare application and dependency timing.
3. Inspect thread dumps during the slowdown.
4. Check database pool usage and query duration.
5. Check CPU, memory, and garbage collection.
6. Identify queue growth.
7. Change one relevant limit.
8. Repeat the same load test.

For example, if many workers are waiting to acquire database connections, increasing Tomcat’s worker limit may make the queue longer. Investigating query duration and transaction boundaries is more useful than guessing a larger thread count.

Low CPU does not prove spare request capacity. High connection count does not prove high application activity.

The central lesson is that a backend request passes through several resource limits. Connections, workers, queues, database pools, and remote services each contribute to its behavior.

Once you can identify where a request is executing and where it is waiting, Tomcat’s behavior becomes much easier to explain — and much easier to diagnose.