Explore / Agent inference
Explore the economics.
Every turn carries the last.
Grow a session. Switch caching. See what gets reused.
- 01Model output appended
- 02Tool result appended
- 03Model output appended
Add a turn, a tool result, or three parallel subagents.
Turn 03 / 12
Reuse the prefix.
Earlier input is reused. Only the new tail adds input work in this model.
Input work across the session
Relative input units. Not latency or cost.
Memory tiers & assumptions
KV demand this turn:
A simplified session model, not measured performance. Assumes perfect prefix hits and equal work per input unit. Fan-out lasts one step; each branch is counted independently. Tier sizes are arbitrary. Decode, cache misses and transfer costs are excluded.
Turn 3: model output appended. Prefix served from cache.
What will a completed job cost?
Compare Realtime, Async and Batch for the same model and workload.
Same model and workload. Async and Batch discounts are relative to Realtime. Prefix reuse can reduce input cost further.
Total tokens across all inference calls in one job attempt.
Share of input billed at the cached-input rate. Batch savings do not require cache hits.
Completion & billed attempts
Estimated total billed spend divided by completed jobs. Assumes equal token use per billed attempt, including failed attempts and charged retries.
Estimated cost per completed job
$1.20/ job
Batch · 70% lower than Realtime token rates
Realtime: $4.00 / completed jobSame GLM-5.3 workload and completion rate
- Fresh input
- $0.2700
- Cached input
- $0.0000
- Output
- $0.9300
Batch rates / 1M tokens
- Fresh input
- $0.270
- Cached input
- $0.078
- Output
- $0.930
Charges remain metered by input, cached input and output tokens.
Estimates at GLM-5.3 list rates. Jobs are billed by the tokens they use, not at a fixed price.
How inference is optimized.
Inspect synthetic engineering hypotheses, the workload signals they address and the gates they must pass. These examples are not evidence of deployed performance.