What prompt caching is actually worth

Prompt caching is the rare optimization that costs almost nothing to adopt and pays continuously. If a large part of your prompt is identical on every call — and for most production applications it is — you are currently paying full input price to resend it thousands of times a day.

This is one lever from the full cost guide, pulled out because the details determine whether you get the discount at all.

How it is billed

The pattern is consistent across both vendors: a cache read is priced at roughly a tenth of a normal input token. corouter honors these cache multipliers at the same ratio against our outlet rates, so the discount compounds with the lower base price rather than being flattened out.

What qualifies

Caching works on a prefix. The cached portion has to be the beginning of your prompt, byte-identical to the previous call, and long enough to clear the vendor's minimum. Three consequences follow, and they are where most implementations lose the discount:

• Order matters. Stable content first — system prompt, tool definitions, few-shot examples, policy documents — then the variable user input. A timestamp at the top of your system prompt invalidates the entire cache on every call.
• Byte-identical means byte-identical. Reordered JSON keys, a changed whitespace, or an interpolated user name inside the prefix all count as a miss.
• Caches expire. They are designed for repeated traffic within a short window, not for storage. A route called twice an hour will not benefit the way one called twice a second does.

What it is worth

Take a workload of 50M input tokens a month where 80% of every prompt is a stable prefix. Without caching you pay the full input rate on all 50M. With caching you pay full rate on 10M and the cache-read rate on 40M. Since the cache read runs at roughly a tenth of the input rate, the input side of your bill drops to a little over a quarter of what it was.

That is a larger effect than most model-switching decisions, and unlike switching models it changes nothing about the outputs you get. Run your own split through the cost calculator — it takes cached input as a separate field for exactly this reason.

Structuring a prompt for caching

1. Put every fixed instruction, tool schema, and example in one contiguous block at the top and never interpolate into it.
2. Move anything dynamic — user message, retrieved documents, current time — strictly below that block.
3. Serialize deterministically. Fix your JSON key order so identical content produces identical bytes.
4. Verify with the usage figures the API returns. If cached token counts stay at zero, your prefix is not matching, and no amount of configuration will fix a prompt that changes at the top.

Then reprice the rest

Caching cuts what you pay for repeated input. The remaining levers cut everything else: choosing the right model tier, comparing two candidates directly, and measuring before optimizing. The current rates for every model we carry are on the pricing page.