Stop Burying Token Costs in COGS

The token line is where your unit economics go to die

If your LLM spend lands as one undifferentiated number in COGS, you have no idea which AI features make money and which ones burn it. Attribute token costs to the feature, team, or customer segment driving them, or you will over-invest in cheap low-value AI and systematically under-price the expensive high-value usage that actually moves the business.

CJ Gustafson at Mostly Metrics framed the question CFOs now face: what do you do when a variable cost that used to be a rounding error becomes a material line that scales with product decisions you don’t control? The answer most finance teams reach for is the wrong one. They watch the aggregate number, wince when it grows, and ask engineering to “optimize.” That is cost containment, not accountability, and it tells you nothing about whether the spend is producing margin.

The default is a blob. A single monthly invoice from OpenAI or Anthropic or Bedrock, booked to cost of goods sold, spread across every AI feature the company ships. The P&L owners who actually drive that spend never see the number. The product manager who added an agentic retry loop, the team that widened the context window, the segment of power users hammering the summarization endpoint. They can’t see it. It doesn’t exist at their level of resolution. What nobody can see, nobody manages.

Token spend scales with product decisions finance never made

Cloud cost sprawl is a solved problem in principle. You tag EC2 instances, you build a showback report, you nudge the team that spun up an oversized cluster. Token costs are worse, and the reason is structural.

Compute cost for a traditional service scales roughly with traffic. Double the users, roughly double the bill. You can model it. Token cost scales with choices made deep inside the product, and those choices multiply.

An agentic feature that loops five times per request costs five times a single-shot call, and the loop count is a product decision, not a traffic number. A context window stuffed with retrieved documents costs more per call than a lean prompt, and the retrieval strategy lives in an engineering repo. Retries on timeout, fallback to a larger model, chain-of-thought prompting that triples output tokens. Every one of these is a lever pulled by product or engineering, and every one changes the cost per request by a multiple, not a margin.

Finance owns the P&L consequence of decisions it has no visibility into and no vote on. That is the trap. The people who can move the cost curve sit three orgs away from the invoice, and the invoice arrives in a shape that hides which of their decisions did the damage.

Attribution means tagging the call, not reading the invoice

Real attribution starts at the API call. Every request to a model provider carries metadata: which feature invoked it, which team owns that feature, which customer or segment triggered it, which model tier served it. That tag rides with the usage record into a cost store, and the cost store maps token counts to dollars at the model’s current rate.

This is not exotic tooling. It is the same discipline that made cloud FinOps work, applied one layer up. A middleware wrapper around every model call stamps the request with a feature ID and a team ID. A pipeline joins those tagged records against provider pricing. A dashboard shows cost per feature, cost per customer, and cost per thousand requests, broken out by the dimensions that matter to the people who can act on them.

Then you decide between showback and chargeback. Showback puts the number in front of the team that generated it and lets accountability work through visibility. Chargeback moves the dollars onto that team’s budget and makes the tradeoff real. Showback is the crawl. Chargeback is where the incentive actually bites, because a cost you can see but never pay for is a cost you eventually ignore.

The point of both is the same. A product manager should be able to answer, before shipping, what a new agentic flow will cost per request and what that does to the margin on the feature it sits inside. Today most can’t, because the number never comes back to them in a form they can read.

Engineering will call this overhead, and finance has to force it

Granular tagging is friction at the exact moment engineering least wants friction. Shipping an AI feature is already hard. Adding a mandatory step where every model call must carry a feature ID and a team ID reads, to the engineer under a deadline, as bureaucracy the finance team invented to feed dashboards.

The resistance is predictable and it is not irrational. Tagging discipline decays the instant it becomes optional. One untagged endpoint, one service that forgot to pass the metadata, and the attribution report grows a bucket of “unallocated” that everyone learns to discount. The whole system is worth exactly as much as its coverage, and coverage erodes from any gap.

So finance can’t win this with a request. It wins with tooling and incentives. Tooling means the tagging is built into the shared model-calling library, so the default path stamps the metadata automatically and the untagged call is the one that takes extra effort. Make the right thing the easy thing. Incentives mean the unallocated bucket is somebody’s problem by name, and margin on AI features shows up in the goals of the product leaders who own them. When a team’s budget carries its own token spend, the argument about overhead ends. The team that pays the bill stops calling the meter bureaucracy.

This is the organizational fight, and finance has to be willing to have it. The alternative is a beautiful attribution architecture with 40 percent coverage, which is worse than none because it looks authoritative and lies.

Attribution is the only thing that lets you price and prune

Here is what the tagged number buys you, and none of it is available without it.

You can price AI features against their real cost. A feature that costs $0.40 per request in tokens cannot be bundled into a $9 plan that assumes marginal cost near zero, and attribution is how you learn that before the plan ships instead of after the cohort scales. You can find the customer segment whose usage pattern costs 10 times the median and either price them into profitability or design the feature to bound the cost. You can kill the AI feature that everyone loved in the demo and nobody will pay for once you see it carries negative margin at volume.

You can also hand product teams something they’ve never really had for AI: accountability for margin, not just for shipping. A product manager who owns the cost line of their feature makes different decisions about retry logic and context size and model tier. Not because finance nagged them, but because the number is theirs now. This is the same margin discipline the gap between AI-native and classic SaaS gross margins makes unavoidable, pushed down to the feature level where the decisions actually get made.

The company flying blind will keep pouring budget into cheap AI that demos well and generates little, while quietly subsidizing the expensive usage that drives real value and could bear a real price. Both errors come from the same missing capability. You cannot manage the unit economics of a thing you have chosen not to measure, and right now most companies are choosing not to measure the fastest-growing variable cost on their P&L.