You Aborted the LLM Request. Did You Actually Stop the AI?

13 min read1 day ago

A user closes the browser tab.

Your Node.js server notices the disconnect.

You propagate the AbortSignal to the upstream LLM request:

const signal = AbortSignal.any([
  timeoutSignal,
  c.req.raw.signal,
]);

await fetch(LLM_URL, { signal });

The fetch() is aborted.

So the model stopped generating.

Right?

Not necessarily.

This is an important distinction that is easy to miss when building streaming AI applications:

HTTP cancellation
       ≠
inference cancellation
       ≠
billing cancellation

Aborting a request proves something about your HTTP client. It does not, by itself, prove what happened inside the provider.

That boundary matters.

The Cancellation Chain

Consider the complete path:

Browser
   │
   │ disconnect
   ▼
Your Server
   │
   │ AbortSignal
   ▼
HTTP Client
   │
   │ connection closes
   ▼
LLM Provider
   │
   ├── request cancellation received?
   │
   ▼
Inference system
   │
   ├── generation stopped?
   │
   ▼
Usage / billing system
   │
   └── fewer tokens billed?

There are several independent claims here.

Claim 1: The browser disconnected

Your server can observe this.

Claim 2: Your server aborted the upstream request

You can observe this too:

signal.addEventListener('abort', () => {
  console.log('upstream request aborted');
});

Claim 3: The provider stopped inference

This is a provider-side semantic.

Claim 4: Billing stopped at the same point

This is a usage-accounting semantic.

The first two are things your application controls.

The last two are things you have to verify with the provider.

`AbortSignal` Only Controls What Holds the Signal

The standard cancellation pattern is still absolutely worth implementing.

const controller = new AbortController();

const response = await fetch(url, {
  signal: controller.signal,
});// Later:
controller.abort();

Your server can also combine independent cancellation conditions:

const signal = AbortSignal.any([
  AbortSignal.timeout(30_000),
  c.req.raw.signal,
]);

This is exactly the right pattern for a connection-bound streaming response.

The important qualification is what the signal actually guarantees.

It tells the operation holding the signal:

Stop this operation.

It does not give your process magical control over the provider’s internal scheduler.

AbortSignal propagates cancellation across async operations, and AbortSignal.any() lets a client disconnect and a server timeout share one signal.

But once the signal crosses the HTTP boundary, you need to know what the remote API promises.

A Better Diagram

Instead of:

Browser
   X
   │
   ▼
Server
   │
   X
   ▼
LLM

I prefer this:

Cancellation boundary
                        │
                        ▼
Browser ───────► Your Server ───────► Provider
   │                  │                  │
   │                  │                  │
closed             fetch aborted      ????
                                         │
                                         ├── inference stopped?
                                         │
                                         └── billing stopped?

Every arrow represents a different contract.

This is a distributed-systems problem disguised as a JavaScript API.

Why “The Connection Closed” Is Not Enough

Suppose your server logs:

12:00:00 request started
12:00:03 client disconnected
12:00:03 AbortSignal aborted
12:00:03 fetch rejected with AbortError

That proves your server stopped waiting for the upstream response.

It does not prove:

12:00:03 inference stopped
12:00:03 generation stopped
12:00:03 billing stopped

The provider might:

immediately stop generation;
stop generation asynchronously;
finish a small amount of already-buffered work;
continue the generation despite the client disconnect;
stop generation but still account for tokens already generated.

Only the provider’s semantics and usage data can tell you which one happened.

The Experiment

This article comes with a small experiment suite. The code is at aiengineering-book/llm-cancellation-experiment.

The important thing about the experiment is that it separates two questions.

Experiment A: Does my server cancel the HTTP operation?

This is purely a client/server question.

We start a long streaming request and abort it after a fixed number of milliseconds.

Expected result:

client:
  request started
  received N chunks
  AbortController.abort()
  fetch/stream terminates

This proves the local cancellation path.

Experiment B: Did the provider stop generating?

That requires provider-side evidence.

The experiment therefore records:

request ID
abort timestamp
last received chunk
provider completion/status
usage record

Do not infer the last two from the first two.

A Local Provider That Makes the Boundary Visible

The experiment repository includes a deliberately simple mock LLM provider.

It supports two modes:

HONOR_ABORT
    provider stops when the client disconnects

IGNORE_ABORT
    provider keeps generating even after the client disconnects

This is not pretending to be a real LLM provider.

It is a controlled experiment showing why the HTTP-layer observation cannot prove the provider-layer behavior.

Run:

pnpm install
pnpm experiment:honor

and:

pnpm experiment:ignore

The two runs use exactly the same client code.

Only the provider-side behavior changes.

What You Should See

In the HONOR_ABORT run, the provider will report something like:

provider:
  generated: 21
  billed:    21
  stopped:   client_disconnect

In the IGNORE_ABORT run:

provider:
  generated: 100
  billed:    100
  stopped:   completed

The exact numbers will vary because the experiment uses timing.

The important observation is:

Client observation:
  "I aborted the request."

Provider observation:
  "What happened to inference?"

Those are different measurements.

A Real Provider Result: DeepSeek

The mock provider shows that the two outcomes are possible. A real provider shows which one you have.

The experiment was run against DeepSeek’s streaming Chat Completions endpoint (deepseek-flash, thinking disabled). DeepSeek was chosen because it exposes an account balance endpoint, GET /user/balance, which can serve as an external meter. The test needs two properties from that meter: it must be independent of the stream your client reads, and it must be fine-grained enough to tell the two hypotheses apart.

The method:

Run one request to completion as a control, and record its usage and the balance change. This gives a rough cost per token.
Run several requests with a large max_tokens (12,000). Abort each one a fixed time after its first content chunk, so every abort lands mid-generation. Do this at two different abort points.
Wait for the balance to stop changing, then compare the drop with two predictions: the provider stopped near the abort, or the provider ran each request to max_tokens.

Two abort points matter because a single one cannot show that billing follows the abort. If the cost tracks how long the stream stayed open, the later abort should cost proportionally more than the earlier one.

What was measured

The control request finished all 8,000 tokens and dropped the balance by about 0.03–0.04 CNY. That is about 4.4 millionths of a CNY per output token, which is consistent with the provider’s published off-peak output price at typical exchange rates. The control also produced 41,892 characters, or about 5.2 characters per token, which is how the received tokens below are estimated from the characters the client saw.

Aborted mid-stream, 8 requests each, max_tokens = 12,000

                              3 s after       15 s after
                              first chunk     first chunk
received by client (tokens)       ~3,000         ~14,600
predicted cost if stopped       ~0.013 CNY      ~0.064 CNY
predicted cost if run to max    ~0.42  CNY      ~0.42  CNY
observed balance drop            0.01  CNY       0.06  CNY

Aborting five times later let the client receive about five times as many tokens, and the cost rose from 0.01 to 0.06 CNY. Both observations sit close to the “stopped near the abort” prediction and far below the 0.42 CNY that running every request to max_tokens would have cost.

An earlier run of 5 aborted requests, only 2 of which were mid-stream, showed the same pattern: the total balance drop for the whole run was 0.04 CNY, almost all of it explained by the 8,000-token control.

For this model and these settings, a client disconnect did not lead to billing out to max_tokens. Billing followed the point at which the client aborted.

What this does and does not establish

The result is a statement about billing, measured from outside. It is a proxy for inference: a provider that kept generating but did not charge would look identical. In practice that is an unusual arrangement, but the experiment cannot distinguish it.

Some limits apply to the numbers themselves:

Resolution: the balance has two decimals of CNY. The 3-second batch cost about 0.01 CNY, which is at the edge of what the meter can show, so that point alone is weak evidence. The 15-second batch is clearer, and the comparison with 0.42 CNY holds for both. A precise per-request token count is not available this way.
Cost per token: it comes from a single 8,000-token control run measured at 0.01 CNY resolution, so it is good to about 15%. That is enough to tell 0.06 from 0.42, not enough to tell 13,600 billed tokens from 14,600.
Lag: the balance moved about 9–13 minutes after the requests ended. A check made one minute later would have reported no cost at all. Wait before reading it.
No usage and no stop signal in the stream: an aborted stream never delivers the final chunk that carries usage, and nothing else on the stream says "stopped at token N". The client cannot learn the billed token count from the stream it aborted. In this test the stop point could only be inferred from billing, and only roughly. Other providers may expose more, such as an explicit cancel event with a final status, but this experiment does not cover them.
Overshoot: if a provider finishes work already in flight before it notices the disconnect, the extra tokens would be a small number past the abort point. That is far below what a two-decimal balance can resolve, so this measurement can rule out running to max_tokens, but not a few tokens of overshoot.
Scope: one model, streaming only, thinking disabled, off-peak pricing, a single day. Thinking mode, non-streaming requests, and aborts before the first chunk were not measured separately. Non-streaming calls are harder to test this way, because there is no first chunk to anchor the abort on.

One more observation concerns the abort timing. With a fixed 5-second abort in the first run, 3 of 5 requests had received no content at all when the abort fired. The time to first token can exceed an aggressive abort window, so a cancellation test that starts its timer at the request start may measure the queue rather than generation. The final experiments start the timer at the first content chunk.

This is one provider’s behavior on one day, written down with its method so it can be repeated. It is not a general statement about LLM APIs.

To repeat it:

git clone https://github.com/aiengineering-book/llm-cancellation-experiment.git
cd llm-cancellation-experiment
pnpm install
cp .env.example .env        # set LLM_API_KEY
pnpm experiment:billing

ABORT_AFTER_MS sets the abort point, counted from the first content chunk. Run it once with 3000 and once with 15000. The run spends real money, up to a few cents, and takes 15-30 minutes because it waits for the balance to settle.

Accounting for a Cancelled Request

The measurement says what one provider did. It does not make the cost of a cancelled request known, so record it as a range instead of a number:

Lower bound: the tokens your client received before the abort.
Upper bound: max_tokens, plus the prompt, because you cannot see where the provider stopped.

The DeepSeek result puts those requests near the lower end. Another provider, or a self-hosted server, can sit anywhere in the range. Log the last usage frame you actually received, and reconcile against the provider's reported usage later, instead of estimating cost from the abort time.

Side Effects Do Not Cancel

An HTTP abort stops your wait on the stream. It does nothing for work that has already started somewhere else. If the model emitted a tool call and your code began executing it, the abort does not reach the work that tool kicked off downstream.

So the cancellation chain has one more boundary after the provider: the tools. A tool that sends an email or writes a record needs to check for cancellation itself, and be safe to run twice. The next article in this series covers that, along with what to do when the work should survive the connection entirely.

Why This Matters for Real Providers

With a real provider, replace the mock provider with the provider’s streaming API. The result above used DeepSeek, so the response ID here comes from the id field on each streaming chunk. Other providers expose it differently.

Record the provider request/response ID as early as possible.

For example, OpenAI’s Responses streaming API emits a response.created event containing a unique response ID, and its response object exposes status and usage fields.

Your instrumentation should therefore capture something like:

requestId = resp_...
startedAt = ...
firstTokenAt = ...
abortAt = ...
lastChunkAt = ...

Then, after the request has terminated, obtain the provider-side record or usage information through whatever mechanism that provider exposes.

The question you want to answer is not:

“Did my fetch reject?”

It is:

“For request resp_..., what was the final provider-side status and usage?"

Measure, Don’t Assume

A controlled test should look like this:

                 Control
                    │
                    ▼
             Long generation
                    │
                    ▼
              record usage

versus:

              Cancellation
                    │
                    ▼
             Long generation
                    │
              abort at T
                    │
                    ▼
              record usage

Then compare:

Control      Cancelled
-----------------------------------------
Request ID        resp_A       resp_B
Abort time        —            5.0s
Final status      completed    ?
Output tokens     8,421        ?
Billed tokens     8,421        ?

You need provider-side evidence for the final two columns.

Get Kun’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

If the provider does not expose enough information to establish this, the honest conclusion is:

We verified HTTP cancellation, but we could not independently verify provider-side inference termination or billing behavior.

That is a much stronger engineering conclusion than guessing.

Cancellation Is Not the Same as “Delete the Work”

There is another subtle point.

Even if a provider supports cancellation, cancellation is not necessarily instantaneous.

Imagine:

t = 0s       request starts
t = 5s       client disconnects
t = 5s       server aborts fetch
t = 5.1s     provider receives disconnect
t = 5.2s     inference scheduler stops generation

There is a propagation delay.

In a distributed system, you should expect this.

The correct model is:

abort requested
      │
      ▼
signal propagated
      │
      ▼
provider receives cancellation
      │
      ▼
provider stops work

Each transition takes time.

And Billing Has Its Own Semantics

Even if inference stops at 5.2 seconds, billing may not be defined as:

“Charge exactly the number of tokens generated before 5.2 seconds.”

A provider might have:

token accounting rules;
minimum billable units;
cached input accounting;
reasoning-token accounting;
provider-specific cancellation rules;
delayed usage aggregation.

Therefore:

inference stopped

and:

billing stopped

are still separate claims.

For example, OpenAI’s current pricing is token-based and distinguishes input, cached input, and output pricing across models.

That makes it especially important to use the provider’s actual usage record rather than estimating cost from the number of chunks your server happened to receive.

A Useful Production Instrumentation Pattern

Treat the LLM request as an observable operation.

type LlmRequestMetrics = {
  requestId?: string;

  startedAt: number;
  firstChunkAt?: number;
  lastChunkAt?: number;
  abortedAt?: number;
  completedAt?: number;  clientDisconnected: boolean;
  abortReason?: 'client_disconnect' | 'timeout' | 'application';  receivedChunks: number;
  receivedBytes: number;  providerStatus?: string;
  inputTokens?: number;
  outputTokens?: number;
  totalTokens?: number;
};

Then your logs can tell the whole story:

llm.request.started
  request_id=resp_123

llm.request.first_chunk
  request_id=resp_123http.client_disconnected
  request_id=resp_123llm.request.aborted
  request_id=resp_123
  reason=client_disconnectllm.provider.completed
  request_id=resp_123
  status=cancelled
  output_tokens=1234

Now you can actually investigate the behavior.

Without the request ID, timestamps, and usage data, you are mostly guessing.

This Also Changes How You Think About “Saving Money”

Suppose you have:

100,000 generations / day

and:

10% of users abandon generation early

It is tempting to say:

“Cancellation saves us 10%.”

That conclusion is unjustified.

You need to know:

abandonment rate
        ×
average unused generation
        ×
provider cancellation semantics
        ×
billing semantics

Only then can you estimate the actual savings.

A useful metric is:

wasted_output_tokens

rather than simply:

abandoned_requests

For example:

Request A
generated: 10,000
user saw:  500
potentially abandoned: 9,500

versus:

Request B
generated: 1,000
user saw:  500
potentially abandoned: 500

Both are one abandoned request.

Their cost profiles are completely different.

The Three Questions Every AI Backend Should Ask

When implementing cancellation, ask these separately.

1. Did my application cancel?

You can test this locally.

signal.aborted === true

and:

catch (error) {
  // AbortError / cancellation
}

2. Did the provider cancel?

Check:

provider request ID;
final response status;
provider logs;
provider API;
provider documentation.

3. Did billing reflect the cancellation?

Check:

usage records;
token counts;
billing records;
controlled experiments.

Do not collapse all three into:

AbortController.abort()

A Particularly Important Example: Realtime APIs

Some APIs expose an explicit application-level cancellation operation.

For example, OpenAI’s Realtime API has an explicit response.cancel client event, and the server reports the response as cancelled.

That’s conceptually different from merely closing an HTTP connection.

You have:

HTTP disconnect

versus:

application-level cancel command

If a provider offers the latter, it gives you a much stronger semantic contract.

This illustrates the larger principle:

The stronger the cancellation guarantee you need, the more explicit the provider-side cancellation API should be.

The Distributed Systems Lesson

This isn’t really about AbortController.

AbortController is just where the problem becomes visible in TypeScript.

The deeper issue is distributed cancellation.

Your application is a chain:

Browser
   │
   ▼
API server
   │
   ▼
LLM gateway
   │
   ▼
Provider
   │
   ▼
Inference scheduler
   │
   ▼
Usage / billing

Cancellation must cross boundaries.

At every boundary, there is a question:

Does this component understand cancellation, and what does cancellation guarantee here?

That is a distributed-systems question.

The Correct Mental Model

Don’t think:

abort()
   ↓
everything stops

Think:

abort()
   ↓
my operation stops
   ↓
cancellation signal crosses boundary
   ↓
remote system may honor it
   ↓
remote work may stop
   ↓
usage may change
   ↓
billing may change

Every arrow needs a contract.

Some arrows are controlled by you.

Some are controlled by the provider.

Final Takeaway

AbortSignal is still essential for a production LLM server.

If a user closes a streaming chat response, you should normally propagate that cancellation to the upstream request when the generation is connection-bound. Your server-side architecture should also distinguish these ephemeral responses from persisted work and long-running Agent tasks.

But don’t make a stronger claim than your evidence supports.

This:

fetch() aborted

proves:

My application stopped waiting for that HTTP operation.

It does not automatically prove:

The provider stopped inference.

And it certainly does not automatically prove:

The provider stopped billing.

Those are three different facts.

Instrument the request. Capture the provider request ID. Run a controlled cancellation experiment. Compare provider-side usage.

Then you know.

The Rule

Cancellation is a mechanism.
Inference termination is a provider behavior.
Billing is an accounting result.

Don’t assume one implies the others.

Measure the boundary.

References

Node.js Event Loop, Timers, and process.nextTick()
OpenAI Responses API — Streaming Events
OpenAI API Pricing
DeepSeek API — Create Chat Completion
DeepSeek API — Models & Pricing
aiengineering-book/llm-cancellation-experiment — code for the experiments in this article

Kun

You Aborted the LLM Request. Did You Actually Stop the AI?

The Cancellation Chain

Claim 1: The browser disconnected

Claim 2: Your server aborted the upstream request

Claim 3: The provider stopped inference

Claim 4: Billing stopped at the same point

AbortSignal Only Controls What Holds the Signal

A Better Diagram

Why “The Connection Closed” Is Not Enough

The Experiment

Experiment A: Does my server cancel the HTTP operation?

Experiment B: Did the provider stop generating?

A Local Provider That Makes the Boundary Visible

What You Should See

A Real Provider Result: DeepSeek

What was measured

What this does and does not establish

Accounting for a Cancelled Request

Side Effects Do Not Cancel

Why This Matters for Real Providers

Measure, Don’t Assume

Get Kun’s stories in your inbox

Cancellation Is Not the Same as “Delete the Work”

And Billing Has Its Own Semantics

A Useful Production Instrumentation Pattern

This Also Changes How You Think About “Saving Money”

The Three Questions Every AI Backend Should Ask

1. Did my application cancel?

2. Did the provider cancel?

3. Did billing reflect the cancellation?

A Particularly Important Example: Realtime APIs

The Distributed Systems Lesson

The Correct Mental Model

Final Takeaway

The Rule

Further Reading

References

Written by Kun

Responses (2)

More from Kun

Your LLM App Is Wasting Money: What Happens When Users Close the Tab?

You build an AI chat application.

Building a Multi-Agent System in TypeScript

Building a Multi-Agent System in TypeScript

Building an MCP Server with TypeScript from Scratch

The Model Context Protocol (MCP) has a fragmentation problem in its documentation: most examples are either too minimal to be useful, or…

How to Defend Against Prompt Injection in Production

Prompt injection is the AI equivalent of SQL injection — and most AI applications in production today have no defense against it.

Recommended from Medium

How To Run Claude Like a Tech Lead, Not a Senior Engineer

You already know how to brief one session. The leverage now is running four of them and still shipping code you’d put your own name on.

8 New AI Terms You Should Know to Be Ahead of 90% of People

A few more AI terms to know beyond tokens, prompts, and hallucinations.

What Should Programmers Do While the AI Writes Code?

Hint: not start another AI coding loop

Prompting Is Now the Slowest, Buggiest Way to Code with AI. Here’s the Fast, Clean One

Forget clever prompts. Here’s the ultimate hands-on guide to AI coding productivity, with insights you won’t find anywhere else

Building an Agentic Knowledge Graph for Billions of Events

Rebuilding Netflix-inspired observability with AI

10 AI Tools Everyone Will Be Using in Six Months, But Almost Nobody’s Found Yet

You’re going to see these in someone’s toolkit next year and wonder why you didn’t hear about them first. Here’s your head start.

Responses (2)

More from Kun

Your LLM App Is Wasting Money: What Happens When Users Close the Tab?

You build an AI chat application.

Building a Multi-Agent System in TypeScript

Building a Multi-Agent System in TypeScript

Building an MCP Server with TypeScript from Scratch

The Model Context Protocol (MCP) has a fragmentation problem in its documentation: most examples are either too minimal to be useful, or…

How to Defend Against Prompt Injection in Production

Prompt injection is the AI equivalent of SQL injection — and most AI applications in production today have no defense against it.

Recommended from Medium

How To Run Claude Like a Tech Lead, Not a Senior Engineer

You already know how to brief one session. The leverage now is running four of them and still shipping code you’d put your own name on.

8 New AI Terms You Should Know to Be Ahead of 90% of People

A few more AI terms to know beyond tokens, prompts, and hallucinations.

What Should Programmers Do While the AI Writes Code?

Hint: not start another AI coding loop

Prompting Is Now the Slowest, Buggiest Way to Code with AI. Here’s the Fast, Clean One

`AbortSignal` Only Controls What Holds the Signal