LLM Billing: What Are You Actually Paying For?

Without any optimization, input tokens grow linearly with each turn, and the total cost ends up roughly proportional to n squared.

In practice it is not that bad. API providers have caching, and tools do context compression. Let me walk through each piece.

The basic fact

LLMs are stateless. They do not remember what was said in the previous turn.

So every message you send must include the entire conversation history. Tools just stitch the history together and ship it to the API in one shot.

Running the numbers (assuming no optimization)

Assume per turn:

• The user sends 100 tokens
• The model replies with 200 tokens

The trend: input tokens per turn is roughly 300 × (n-1), growing linearly.

Cumulative cost:

1
2
3
Total cost ≈ unit price × (100 + 400 + 700 + 1000 + ...)
           = unit price × arithmetic series sum
           = unit price × (150n² - 50n)

For large n, roughly 150 × unit price × n².

Total cost is proportional to the square of the number of turns. Your gut feeling was right.

But three things help in practice

1. KV Cache

This is the most important mechanism.

When the API processes your input, it runs every token through the neural network. Two phases:

1. Prefill: computes attention relationships between all input tokens (the heavy part)
2. Decode: generates output tokens one by one

If the first 300 tokens of turn 2 are identical to turn 1 (the conversation history), the API provider caches the intermediate results from turn 1 and reuses them in turn 2.

This means:

• Turn 2 sends 400 tokens, but only the last 100 (the new message) need recomputation
• The first 300 tokens get the cache hit price

DeepSeek V4 Flash pricing: | Type | Price | |——|——-| | Cache miss | $0.14 / M tokens | | Cache hit | $0.028 / M tokens (80% off) |

Back to the example:

With caching, the incremental cost per turn is basically constant (only the new input is charged at full price).

One caveat: a DeepSeek API key shares its cache globally, but only if the input prefix is character-for-character identical. Change one word and the cache misses. So if a tool truncates or rewrites history, caching does not help.

2. Context Compression

Tools like OpenCode have built-in context compression. When the conversation gets too long, they:

1. Summarize most of the earlier history into a digest, replacing the raw conversation
2. Keep only the most recent turns in full detail
3. Drop irrelevant tool call results

Effect:

1
Raw history: 10000 tokens → compressed: 2000 tokens

Same logic as taking meeting notes. You do not need every word everyone said. You just need the conclusions and the key data.

3. Output costs money too

So far we only looked at input, but output is also billed:

• DeepSeek V4 Flash output price: $0.28 / M tokens (double the input price)
• Output tokens per turn is roughly the model’s reply length, it does not accumulate
• So output cost grows linearly with turns, not quadratically

What costs look like in practice

A typical OpenCode debugging session:

A full debugging session (10 turns): roughly $0.0005-0.001 total.

Even with heavy use at 50 turns per day, costs are around $0.01-0.05.

Visualization

1
2
3
4
5
6
7
8
9
10
11
Cost
  ↑                         ✦ No cache (quadratic)
  │                      ✦
  │                   ✦
  │                ✦
  │             ✦
  │          ✦
  │       ✦
  │    ✦                            --- With cache (linear)
  │ ✦                             .... Output cost (linear)
  └──────────────────────────────→ Turns

With caching:

• Input cost (linear): each turn adds roughly constant × cache miss price + history × cache hit price
• Output cost (linear): each turn adds roughly reply length × output price
• Total cost is roughly linear, not quadratic

Summary

Bottom line: do not worry about the n² problem. Caching and context compression keep actual costs close to linear. A heavy session typically costs a few cents.

The coding agent problem: Agent loops multiply interactions

Normal chat vs coding agent

What the agent loop looks like

Normal chat:

1
User → sends message → model replies → done

A coding agent fixing a bug:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
User → "Help me look at this bug"
  ↓
AI thinks → decides which file to read
  ↓ calls read tool
Tool returns file content (500 lines of code)
  ↓
AI thinks → decides what command to run to reproduce the bug
  ↓ calls bash tool
Bash returns error output
  ↓
AI thinks → decides which line to change
  ↓ calls edit tool
  ↓
... this may repeat 10-50 times
  ↓
Final result

Each step is a separate LLM call. A single task can easily take 10-50 calls.

Tool results are often large

Normal chat:

1
2
User sends 100 words → model replies with 200 words
Increment per turn: ~300 tokens

Coding agent:

1
2
3
User sends 50 words → AI calls tool to read a file
Tool returns 500 lines of code → maybe 5000 tokens
Increment this turn: 5000 tokens (10-20x normal chat)

Running grep -r "keyword" ./src might return tens of thousands of tokens. All of it feeds into the next turn’s input.

A real example

With DeepSeek V4 Flash, a 20-turn debugging task:

The problem: cumulative input keeps growing because everything from previous turns gets sent again.

Caching does not fully save coding agents

The “caching makes costs linear” story breaks down for coding agents:

1
2
3
4
Turn 1:  500 tokens  → full price
Turn 2:  3500 tokens → first 500 hit + last 3000 miss (full price)
Turn 3:  3700 tokens → most of first 3500 hit + last 200 miss
Turn 5:  9200 tokens → file content is huge, thousands of new miss tokens every time

Large chunks of new content like file contents are always cache misses. History tokens hit cache fine, but the new file content is so large that per-turn costs stay significant.

What OpenCode does about it

Real cost estimates

Heavy monthly use can run into tens or hundreds of dollars.

Tips to control costs

Summary: coding agents really are much more expensive

The cost gap between normal chat and coding agents is 10-100x, and coding agent costs are unpredictable. You never know how many turns a bug fix will take.

If you use it heavily, watch your bill, set spending limits, and cut your losses early instead of pushing through a dead end.

Appendix: Prefill and Decode explained

Why KV cache makes cache hits possible

The core of Transformer token processing is the attention mechanism. Each token computes relationships with every token before it.

This produces two vectors per token: K (Key) and V (Value). After computation, the server stores each token’s K and V, which is the KV cache.

Prefill phase: process all input

When you send text to the API, say 500 tokens:

1
2
3
4
5
6
500 tokens need pairwise attention computation
That is 500 × 500 = 250,000 calculations

After each token is computed, its K and V go into the cache

All 500 tokens are computed in one shot. This is Prefill.

The cost of Prefill is baked into the “input token” price. Your payment covers those 250,000 calculations.

Decode phase: generate output token by token

The model starts generating text:

1
2
3
4
5
6
7
8
9
10
11
Generating the 1st output token:
- Its K and V must be computed fresh
- It attends to the 500 cached K vectors from Prefill (reusing cache)
- Much cheaper than Prefill

Generating the 2nd output token:
- Now the cache has the 1st token's K and V too
- It attends to the 500 Prefill K vectors + the 1st token's K
- Each new token reuses all previously cached tokens

This token-by-token generation is Decode.

How caching affects your bill

KV Cache saves server compute. For your bill:

1
2
3
4
5
6
Your bill =
  Input tokens × input price (Prefill covers all input)
+ Output tokens × output price (Decode generates one by one)

KV Cache does not change the price tags you see.
It lowers DeepSeek's compute costs, so DeepSeek can offer lower prices.

DeepSeek’s cache hit price is 1/5 of the miss price ($0.028 vs $0.14). That is how they pass the compute savings on to you.

• Previous
  LLM Serving: GPU Memory, KV Cache, and Large-Scale Inference