AI Engineering · 8 MIN

Why don't you know where your AI bill goes, and how do you cut it?

Most teams cannot say where their OpenAI or Anthropic money goes. Here is how to reconcile your logs with the vendor bill, find the leaks, and cut spend without hurting quality.

By NactorePublished 20 Nov 2026All articles

Most companies do not know where their OpenAI or Anthropic money goes because their own cost tracking does not match how the vendor actually bills. Cached tokens, retries, long system prompts and reasoning tokens all get counted differently, so the dashboard says one thing and the invoice says another. You cut the bill by first making those two numbers agree, then removing calls you never needed, sending simple work to smaller models, trimming what you send, caching what repeats, and putting a budget on every feature. This post walks through each step and ends with a one-week audit your team can run.

Key takeaways
  • Internal LLM cost tracking is often wrong, so reconcile your call logs against the vendor bill before you optimize anything.
  • On one production support agent, checking 10,305 AI calls against the vendor bill showed the internal tracking was about three times off.
  • The cheapest AI call is the one you skip. Plain rules can settle many easy cases before a model is ever called.
  • Model routing, shorter prompts and caching cut cost further, but only after an eval confirms quality holds.
  • Per-feature budgets and alerts turn a surprise invoice into a number someone owns.
  • Nactore builds, ships and measures AI features as one partner, with cost per successful task tracked on everything we ship.

Why don't companies know where their AI money goes?

Companies lose track of AI spend because the number they log is not the number they pay. Most teams estimate cost from their own code by multiplying tokens by a price list. The vendor bills on its own count, which includes discounts, retries, hidden reasoning tokens and context you forgot you were sending. The gap grows quietly every month.

On a production support agent we know closely, the team checked 10,305 AI calls one by one against the vendor's bill. Their internal cost tracking turned out to be about three times off. Every decision made from that dashboard, from pricing to which feature to optimize first, was built on the wrong number.

How do you reconcile your logs with the vendor bill?

Reconciling means matching every AI call your system logged against what the vendor charged for the same period, and explaining every gap. Export usage from the vendor console, pull your own call logs for the same dates, and compare totals per model and per day. Where they disagree, the reason is almost always one of a few known causes.

  • Caching discounts. Both major vendors bill cached input at a different rate from normal input, and Anthropic also charges a premium to write the cache. If your code prices every input token the same, your numbers drift.
  • Retries. A timeout or a malformed response that your code quietly retries is a second paid call. Many logs only record the final success.
  • Long system prompts. The system prompt, tool definitions and examples are sent and billed on every call, not once per conversation.
  • Thinking and reasoning tokens. Reasoning models bill their internal thinking as output tokens, even when you never see that text. Output tokens are the expensive kind.

Log the raw usage fields the API returns on every response (input, output, cached and reasoning tokens) rather than counting tokens yourself. That single change makes reconciliation possible. Our guide to what to log for LLM observability covers the rest.

Where does LLM spend usually leak?

LeakHow to spot itTypical fix
Silent retriesVendor call count is higher than your logged call countLog every attempt, cap retries, fix the cause of failures
Reasoning tokensOutput tokens on the bill far exceed the text you receivedLower the reasoning effort setting, or use a non-reasoning model for simple tasks
Bloated system promptInput tokens per call stay high even for short user messagesCut unused instructions, examples and tool definitions
Too much retrieved contextInput cost rises with every document you add to the indexSend the few best passages, not every near match
Cache missesCached token fields stay near zeroPut stable content first, move timestamps and IDs out of the prefix
Easy cases sent to the modelMany requests get the same short answerAnswer them with plain rules before any model call
One big model for everythingA single model name covers every feature in the billRoute simple work to a smaller model after an eval

How do you cut the bill without hurting quality?

You cut the bill without hurting quality by removing waste in order of safety: first calls you never needed, then tokens you never needed, then price per token. Every change runs against a fixed eval set before it ships, and the number you track is cost per successful task, not price per token.

Handle the easy cases with plain rules first

Many requests do not need a model at all. A refund status or a known error code can be answered by a lookup and a template. In one production system, a deterministic rule layer closed about 60% of support tickets with no AI call at all, and AI spend on that agent fell by about 53% after the fixes. Rules are also faster and easier to debug.

Route simple work to smaller models

Classification, extraction and short rewrites rarely need the largest model. Send them to a smaller one and escalate only the hard cases. Prove it with your eval set first, because a cheaper model that fails and retries can cost more per successful task. Our guide to choosing an LLM for production covers the selection method.

Trim prompts and context

Read your system prompt line by line and delete anything the model no longer needs. Cut examples that do not change behavior. Send three good retrieved passages instead of twenty. Cap output length and ask for short, structured answers.

Cache what repeats

Prompt caching bills a repeated prompt prefix at a lower rate. It only works when the prefix is identical, so put stable content first and variable content last. For identical questions, cache the final answer in your own store and skip the call. The details are in our post on prompt caching for LLM cost control.

Set per-feature budgets and alerts

Tag every AI call with the feature that made it. Give each feature a monthly budget, an owner and an alert when daily spend crosses a threshold. Then a runaway loop or a prompt change shows up the same day, not on next month's invoice.

Pro tip

Track cost per successful task, not cost per call. A feature whose calls got cheaper while its success rate dropped did not get cheaper.

What does a 1-week AI bill audit look like?

A one-week AI bill audit gives a team a reconciled cost number per feature and a ranked list of fixes. It needs read access to the vendor console, your application logs and someone who knows the product.

  1. Day 1: pull the data. Export vendor usage for the last full month and your own call logs for the same dates.
  2. Day 2: reconcile. Match totals per model and per day. Explain every gap using retries, caching, system prompts and reasoning tokens.
  3. Day 3: attribute. Tag spend by feature. Find the few features that drive most of the bill.
  4. Day 4: sample. Read a few hundred real requests from the top features. Mark which could be answered by rules, a smaller model or a cache.
  5. Day 5: estimate safely. For each candidate fix, build a small eval set from real requests and test quality before you trust any saving.
  6. Day 6: rank. Order fixes by saving, risk and effort. Easy rule-based wins usually come first.
  7. Day 7: guardrails. Add feature tags, budgets and alerts so the bill stays explained after the audit ends.
You cannot cut what you cannot see, so make the bill agree with your logs first.

Frequently asked questions

Why does our dashboard not match the OpenAI or Anthropic invoice?

Usually because your code estimates cost from its own token counts while the vendor bills on its count. Retries, caching rates, system prompts sent on every call and hidden reasoning tokens are the common causes. Log the usage fields from each API response to close the gap.

Will cutting costs make our AI worse?

Not if every change is tested against a fixed eval set built from real requests. Ship a change only when quality holds, and measure cost per successful task.

Should we switch vendors to save money?

Switching rarely fixes waste. Retries, bloated prompts and unneeded calls follow you to any vendor. Reconcile and remove waste first, then compare vendors on your own eval set.

Who should own the AI bill?

Each feature should have an owner and a budget. Finance sees the total, but the person who can change the prompt should get the alert.

The takeaway

An AI bill only stays under control when the number you log matches the number you pay and every feature has an owner. That is engineering, measurement and growth work done together, which is how Nactore works as one partner that builds, ships, ranks and grows, measured on your results. We run a fixed-scope AI bill audit that finds where your money goes and the top fixes to cut it. Write to hello@nactore.com to start.

Want to apply this to your business?

Book a free 30-minute call. We will tell you what we would do first.