JWT in Microservices: What Really Happens from API Gateway to 10 Microservices?

There is one question I keep seeing whenever JWT and microservices come together:

If I have 10 microservices and one API Gateway, where should I validate the JWT?

Should I validate it only at the API Gateway?

Or should I validate it again inside every microservice?

And then another question comes:

If every service validates the JWT, does every service need the private key?

And finally:

What actually happens when the user logs out?

These questions look complicated in the beginning.

But once you understand what the JWT, private key, public key, Gateway and microservice are each responsible for, the architecture becomes surprisingly simple.

Let’s understand it from the beginning.

First, let’s imagine a real system

Suppose we are building an HR platform.

We have one API Gateway and 10 microservices:

                         Client
                            |
                            v
                       API Gateway
                            |
       +--------------------+--------------------+
       |         |          |         |          |
       v         v          v         v          v
    User     Employee    Company   Payroll    Leave
   Service    Service    Service   Service   Service
       +--------------------+--------------------+
       |         |          |         |          |
       v         v          v         v          v
 Attendance   Audit     Document   Report   Notification
  Service    Service    Service    Service     Service

Now we need authentication.

We don’t want every service to implement username/password login.

So we introduce an Authorization Server.

                  Authorization Server
                            |
                            v
                         Client
                            |
                            v
                       API Gateway
                            |
              +-------------+-------------+
              |             |             |
              v             v             v
          Service 1     Service 2     Service 3
              ...

The Authorization Server is responsible for issuing access tokens.

In our case, that access token is a JWT.

Step 1: User logs in

The user enters their username and password.

User
 |
 | username + password
 v
Authorization Server

The Authorization Server verifies the credentials.

If everything is correct, it creates an access token.

For example:

{
  "iss": "https://auth.example.com",
  "sub": "user-123",
  "aud": "hrmis-api",
  "scope": "employee.read employee.write",
  "exp": 1799999999
}

These claims tell us things like:

iss   → Who issued this token?
sub   → Who is the user?
aud   → Who is this token intended for?
scope → What is the token allowed to do?
exp   → When does it expire?

RFC 9068 defines a standardized profile for JWT-formatted OAuth 2.0 access tokens and describes important claims and validation expectations.

But there is one very important thing happening here.

The Authorization Server signs this JWT using its private key.

Private key and public key:

Let’s stop here because this is where many people get confused.

There are two keys:

Private Key 
Public Key 

The private key belongs to the Authorization Server.

It must remain secret.

The Authorization Server uses it to sign tokens.

                Authorization Server
                         |
                    PRIVATE KEY 
                         |
                         | Sign
                         v
                        JWT

The public key is used by other components to verify that signature.

Authorization Server
        |
        | Public Key
        v
     Gateway
        |
        +------> Microservice 1
        |
        +------> Microservice 2
        |
        +------> Microservice 3

The important rule is:

The private key signs. The public key verifies.

The private key does not travel with the request.

The JWT does.

What does the JWT actually contain?

A JWT normally has three parts:

HEADER.PAYLOAD.SIGNATURE

Something like:

eyJhbGciOiJSUzI1NiJ9
.
eyJzdWIiOiJ1c2VyLTEyMyJ9
.
ABC123...

The payload contains claims.

The signature proves that the token was signed by someone holding the private key corresponding to the public key used for verification.

One important point:

JWT is not encrypted by default.

The payload can generally be decoded.

So don’t think:

“Nobody can see what’s inside my JWT.”

The important security property here is usually the signature, not secrecy.

Never put sensitive secrets or passwords inside a normal JWT payload just because it is encoded.

Step 2: Client calls the API Gateway

The user now wants employee information.

The client sends:

GET /employees/123

Authorization: Bearer <JWT>

The request goes to:

Client
   |
   | JWT
   v
API Gateway

Now the Gateway becomes the first security boundary.

Step 3: Gateway gets the public key

The Gateway does not need the private key.

It needs the public key of the trusted Authorization Server.

In modern OAuth/OIDC systems, public signing keys are commonly exposed through a JWKS endpoint.

Conceptually:

Authorization Server
        |
        | Public Key
        v
API Gateway

The Gateway can cache these keys.

It doesn’t need to download the public key every time a request arrives.

Step 4: Gateway validates the JWT

Now the Gateway has:

JWT
+
Trusted Public Key

It verifies the JWT signature.

Think about it like this:

Authorization Server

PRIVATE KEY
     |
     | signs
     v
    JWT
     |
     | travels through network
     v
API Gateway
     |
     | PUBLIC KEY
     v
  Verify

If the signature is valid:

Token was signed by the trusted issuer

If somebody changed the token:

Signature verification fails

For example, imagine an attacker changes:

scope = employee.read

to:

scope = employee.delete

The signature no longer matches the modified content.

The Gateway rejects it.

Gateway does more than checking the signature

A valid signature alone isn’t enough.

The Gateway should validate the token according to the expected issuer, audience, time validity, signing algorithm and authorization requirements.

For example:

Signature       
Issuer          
Audience        
Expiration      
Not Before      
Algorithm       
Scope          

Imagine the token says:

iss = auth.example.com
aud = hrmis-api
scope = employee.read
exp = future

The Gateway asks:

Did my trusted Authorization Server issue this?
             ↓
            YES
Is this token meant for this API?
             ↓
            YES
      Has it expired?
             ↓
             NO
Is the signature valid?
             ↓
            YES
Does it have the required permission?
             ↓
            YES

Only then does the Gateway forward the request.

Microsoft’s API gateway guidance recommends gateway-level validation while also emphasizing validation at the backend API.

Step 5: Gateway sends the request to the microservice

Suppose the request is for Employee Service.

The Gateway forwards the request.

Importantly, the Gateway can forward the access token as well:

Client
  |
  | JWT
  v
API Gateway
  |
  | same JWT
  v
Employee Service

So Employee Service receives something like:

GET /employees/123

Authorization: Bearer <JWT>

What does not get forwarded?

Private Key

The Gateway doesn’t send the private key.

And normally it doesn’t need to send the public key with every request either.

The microservice can already obtain/trust the Authorization Server’s public keys.

What about additional headers?

Now, the Gateway can also add some additional headers before sending the request to the microservice. For example, it may add X-User-Id, X-Tenant-Id, or X-Organization-Id. These values can be taken from the already validated JWT.

For example, if the JWT contains the user, tenant, and organization information, the Gateway can send:

Authorization: Bearer <JWT>
X-User-Id: user-789
X-Tenant-Id: tenant-123
X-Organization-Id: org-456

This makes it easier for the microservice to understand which user and tenant the request belongs to.

But there is one important rule: the microservice should not blindly trust these headers if the client can send them directly. The Gateway should validate the JWT first and then create or overwrite these headers using trusted information from the token.

So the flow becomes:

Client
   ↓
JWT
   ↓
API Gateway
   ↓
Validate JWT
   ↓
Add User/Tenant/Organization headers
   ↓
Microservice

These headers are not a replacement for JWT validation. They are simply additional context that the Gateway passes to the microservice.

Step 6: Microservice validates the JWT again

Now comes the part that creates the most discussion.

The Gateway already validated the token.

So why does Employee Service validate it again?

Because the service should not blindly trust the Gateway.

Employee Service has its own security boundary.

It receives:

JWT
+
Trusted Public Key

and verifies it.

Employee Service
                        |
                  PUBLIC KEY 
                        |
                        v
                    Verify JWT
                        |
                +-------+-------+
                |               |
              Valid           Invalid
                |               |
                v               v
          Business Logic       401

This is defense in depth.

The Gateway protects the system at the edge.

The microservice protects itself.

Microsoft explicitly recommends backend validation even when a gateway has already validated the access token.

Why is this actually important?

Today your architecture might look like:

Client
  |
Gateway
  |
Service

But six months later, someone may add another path:

Internal Service
      |
      v
Employee Service

Or:

Batch Job
    |
    v
Employee Service

Or perhaps somebody accidentally exposes the service internally.

Now someone can bypass the Gateway:

Attacker
   |
   X
Gateway
   |
   X
Employee Service

If Employee Service says:

“I don’t need to authenticate. Gateway already did it.”

you have a problem.

But if Employee Service says:

“I will validate the token myself.”

then:

Attacker
   |
   v
Employee Service
   |
   | Invalid / missing JWT
   v
  401

That is why the second validation is valuable.

Does every microservice need its own private key?

No.

This is probably the most important thing to remember.

If the Authorization Server is issuing the user JWT, then the Authorization Server owns the signing private key.

You don’t need:

Service 1 → Private Key
Service 2 → Private Key
Service 3 → Private Key
...
Service 10 → Private Key

That would be unnecessary and dangerous if all those services were given the ability to sign the same kind of tokens.

Instead:

                Authorization Server
                         |
                   PRIVATE KEY 
                         |
                       Signs
                         |
                         v
                        JWT
                         |
                         v
                       Client
                         |
                         v
                    API Gateway
                         |
                    PUBLIC KEY 
                         |
                       Verify
                         |
              +----------+----------+
              |          |          |
              v          v          v
          Service 1  Service 2  Service 3
              |          |          |
          PUBLIC KEY  PUBLIC KEY  PUBLIC KEY
              |          |          |
            Verify     Verify     Verify

The services only need the public key for verification.

Can someone steal the public key?

Yes.

And that’s okay.

It’s public.

If someone gets your public key:

Public Key
    |
    +---- Can verify tokens
    |
    +---- Cannot create legitimate tokens

But if someone gets the private key:

Private Key
    |
    +---- Can sign tokens
    |
    +---- Can potentially create tokens that validators accept

That’s why the private key is the crown jewel.

Get Himanshu Singour’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

It needs strong protection.

How do microservices get the public key?

You don’t necessarily hard-code a public key manually into every service.

A common approach is for the Authorization Server to publish its public keys through a JWKS endpoint.

Conceptually:

Authorization Server
        |
        | JWKS
        v
+-------------------+
| Public Keys       |
|                   |
| kid = key-1       |
| kid = key-2       |
+-------------------+
        |
        v
Gateway / Services

The services can discover the issuer’s keys and cache them.

This becomes especially useful when keys are rotated.

What happens during key rotation?

Suppose today the Authorization Server is using:

Private Key 1

and the corresponding public key is:

Public Key 1

Tomorrow you want to rotate the signing key.

You create:

Private Key 2
Public Key 2

The Authorization Server starts signing new tokens using Private Key 2.

The public keys can temporarily contain both keys:

Public Keys:

Key 1
Key 2

Why?

Because old tokens may still be valid.

So the Gateway and microservices can verify:

Old JWT → Public Key 1
New JWT → Public Key 2

Once all old tokens have expired and the old key is no longer needed, Key 1 can eventually be removed.

This is where the JWT kid header is useful: it helps a validator identify which published key should be used.

Now let’s talk about service-to-service calls

Suppose:

Client
   |
   | User JWT
   v
Gateway
   |
   v
Order Service
   |
   v
Payment Service

Now we have another question:

Should Order Service send the same JWT to Payment Service?

There isn’t one universal answer.

It depends on what identity and authorization model you want.

One approach is to propagate the user’s access token:

Client
   |
 User JWT
   v
Order Service
   |
 User JWT
   v
Payment Service

Payment Service then validates the token.

But you must make sure the token’s intended audience and permissions are appropriate.

Another approach is for Order Service to obtain a separate access token for calling Payment Service:

Order Service
      |
      | service identity
      v
Authorization Server
      |
      | new access token
      v
Payment Service

This gives you a clearer service-to-service trust boundary.

The important thing is not to blindly forward a user token everywhere without thinking about audience and permissions.

Authentication vs Authorization

Another simple but important distinction:

Authentication

Who are you?

Authorization

What are you allowed to do?

Suppose:

JWT valid

That only tells us that the token is acceptable according to our validation rules.

It doesn’t mean the user can do everything.

For example:

scope = employee.read

may allow:

GET /employees/123

But it may not allow:

DELETE /employees/123

because that could require:

employee.delete

So the flow becomes:

JWT
 ↓
Validate token
 ↓
Authenticate identity
 ↓
Check permission
 ↓
Authorize operation
 ↓
Business logic

This distinction is extremely important in microservices.

Now the big question: What happens during logout?

This is where JWT is different from a traditional server-side session.

With a traditional session, you might have:

Browser
   |
   | session ID
   v
Server
   |
   v
Session Store

When the user logs out:

Delete session

The server can immediately know:

This session no longer exists.

JWT access tokens are often designed differently.

The token is usually self-contained.

For example:

JWT
 |
 +-- user
 +-- issuer
 +-- audience
 +-- scope
 +-- expiration
 +-- signature

The server doesn’t necessarily maintain a database entry saying:

JWT-123 = logged-in

So you cannot always just “delete the JWT from the Authorization Server.”

Simple JWT logout

The simplest flow is:

User clicks Logout
        |
        v
Client deletes access token
        |
        v
       Done

Now the client no longer sends:

Authorization: Bearer <JWT>

So future requests fail because there is no access token.

But there is an important catch.

What if someone already copied the JWT?

Imagine:

10:00 AM
User logs in

Access Token:
Valid until 10:15 AM

At:

10:05 AM

the user logs out.

The client deletes the token.

But suppose an attacker copied the token at 10:04.

That copied token may still be cryptographically valid until 10:15 unless your architecture has a server-side revocation mechanism or some other way to reject it earlier.

This is one of the fundamental trade-offs of stateless access tokens.

That’s why access tokens are generally short-lived

Instead of:

Access Token
Valid for 30 days

you normally use a relatively short-lived access token.

Conceptually:

Login
  |
  +---- Access Token
  |
  +---- Refresh Token

The access token is used to access APIs.

The refresh token is used to obtain a new access token, according to the authorization server’s policy.

So when the access token expires:

Access Token
     |
     X
   expired

the client needs to obtain another one.

Logout with refresh tokens

A more complete logout can look like:

User
 |
 | Logout
 v
Client
 |
 +---- Delete Access Token
 |
 +---- Stop using Refresh Token
 |
 v
Authorization Server
 |
 +---- Revoke / invalidate refresh credential

The exact logout and revocation behavior depends on the authorization system you’re using.

The important idea is:

Deleting the access token from the client prevents the normal client from using it.

Revoking the refresh credential prevents the client from obtaining new access tokens through that credential.

And the old access token naturally becomes unusable when it expires.

So the complete architecture looks like this

Now let’s put everything together.

                         +-----------------------+
                         | Authorization Server  |
                         |                       |
                         | PRIVATE KEY           |
                         +-----------+-----------+
                                     |
                                     | Sign
                                     v
                                    JWT
                                     |
                                     v
                                  Client
                                     |
                                     | Bearer JWT
                                     v
                         +-----------------------+
                         |      API Gateway      |
                         |                       |
                         | PUBLIC KEY         |
                         | Validate JWT          |
                         +-----------+-----------+
                                     |
                              Valid token
                                     |
              +----------------------+----------------------+
              |                      |                      |
              v                      v                      v
       +-------------+       +-------------+       +-------------+
       | User        |       | Employee    |       | Payroll     |
       | Service     |       | Service     |       | Service     |
       |             |       |             |       |             |
       | PUBLIC KEY  |       | PUBLIC KEY  |       | PUBLIC KEY  |
       | Verify JWT  |       | Verify JWT  |       | Verify JWT  |
       +-------------+       +-------------+       +-------------+
              |                      |                      |
              +----------------------+----------------------+
                                     |
                              Other Services
                                     |
                              Verify JWT
                                     |
                                     v
                                  Database

The pattern continues for all 10 services.

The request lifecycle in one minute

Let’s make the whole thing extremely simple.

User logs in:

User
 ↓
Authorization Server
 ↓
JWT signed with PRIVATE KEY
 ↓
Client

Client calls the API:

Client
 ↓
JWT
 ↓
API Gateway

Gateway:

Gateway
 ↓
PUBLIC KEY
 ↓
Verify JWT
 ↓
Check issuer
 ↓
Check audience
 ↓
Check expiration
 ↓
Check permissions

If valid:

Gateway
 ↓
JWT
 ↓
Microservice

Microservice:

Microservice
 ↓
PUBLIC KEY
 ↓
Verify JWT again
 ↓
Check required permission
 ↓
Business logic
 ↓
Database

The private key never leaves the Authorization Server.

The four rules I would remember

If you forget everything else, remember these four things.

1. Authorization Server owns the private key

PRIVATE KEY 
      ↓
    SIGN
      ↓
     JWT

2. JWT travels with the request

Client
  ↓
Gateway
  ↓
Microservice

3. Gateway and protected services verify using the public key

PUBLIC KEY 
      ↓
   VERIFY
      ↓
     JWT

4. Logout is different from session logout

Logout
  ↓
Client deletes access token
  ↓
Refresh credential is revoked/inactivated when applicable
  ↓
Short-lived access token eventually expires

If you need immediate invalidation of an already-issued access token, you need an additional revocation/introspection or deny-list strategy; that is a deliberate trade-off because it makes the system less purely stateless.

The architecture I would actually keep in my head

Don’t think:

“JWT is just a token.”

Think about the complete chain:

             PRIVATE KEY
                   |
                   | Sign
                   ↓
                  JWT
                   |
                   | travels
                   ↓
                CLIENT
                   |
                   | JWT
                   ↓
             API GATEWAY
                   |
             PUBLIC KEY
                   |
                Verify
                   |
                   ↓
             MICRO SERVICE
                   |
             PUBLIC KEY
                   |
                Verify
                   |
                   ↓
             Authorization
                   |
                   ↓
              Business Logic
                   |
                   ↓
                Database

That’s the whole story.

The private key creates the trust.

The JWT carries the identity and permissions.

The public key verifies that trust.

The Gateway rejects bad requests early.

The microservice verifies the request again because it is its own security boundary.

And logout is handled differently from traditional server-side sessions because a JWT access token can remain cryptographically valid until it expires unless you introduce a mechanism for earlier revocation.

Once this mental model is clear, JWT in a 10-service architecture is not really complicated.

It is simply:

Issue → Sign → Send → Verify at Gateway → Forward → Verify at Service → Authorize → Execute.

Why I started reading the actual standards

While working on this, I had a lot of questions about what should actually happen in a microservices architecture. Instead of just following our old implementation, I wanted to see how this problem is handled in real systems.

So I started reading the actual standards and engineering documentation.

First, I read RFC 9068. It is an IETF standard for JWT access tokens. The important part for this discussion is that it explains how a resource server should validate a JWT. It talks about checking things like the issuer (iss), audience (aud), signature, and expiration (exp). It also explains how the Authorization Server can publish the keys that resource servers use to validate tokens.

Read RFC 9068 — JWT Access Token Profile

Then I looked at Microsoft’s API Gateway guidance to understand what changes when APIs are placed behind a Gateway. It helped me look at the Gateway as an additional security layer instead of assuming that the Gateway should become the only place responsible for security.

Read Microsoft’s API Gateway guidance

I also looked at Cloudflare’s documentation to see how a large production platform handles application authentication and tokens.

Read Cloudflare’s Application Token documentation

What I learned from this

The main lesson for me was simple:

Before building an important architecture, first check how the problem is solved by the standards and by companies that have already built systems at scale.

This doesn’t mean we should copy Microsoft, Cloudflare, AWS, or anyone else blindly. Our requirements may be different.

The idea is to understand why they are doing something, what problem they are solving, and then decide what makes sense for our own system.

So instead of:

Think → Code → Find problems → Redesign

I prefer:

Read the standard
       ↓
Study real-world implementations
       ↓
Understand why
       ↓
Design
       ↓
Then code

Sometimes reading the right RFC or engineering documentation for an hour can save days of implementing the wrong architecture.