Skip to contentQloudOpen models
Console login
← Back to Home

Explore / Agent inference

Explore the economics.

Every turn carries the last.

Grow a session. Switch caching. See what gets reused.

Reused prefixNew input
  1. 01Model output appended
  2. 02Tool result appended
  3. 03Model output appended

Add a turn, a tool result, or three parallel subagents.

Prefix cache

Turn 03 / 12

Reuse the prefix.

Earlier input is reused. Only the new tail adds input work in this model.

Input work across the session

Cache on
Cache off

Relative input units. Not latency or cost.

Memory tiers & assumptions

KV demand this turn:

HBMDRAMNVMe

A simplified session model, not measured performance. Assumes perfect prefix hits and equal work per input unit. Fan-out lasts one step; each branch is counted independently. Tier sizes are arbitrary. Decode, cache misses and transfer costs are excluded.

Turn 3: model output appended. Prefix served from cache.