Tracking AI Model Cost and Token Use per Feature With RollupDB

An app calls two language models, a large one and a small one, from three features: a support chat, document summaries and search. Every call has a token count and a price. Finance wants the monthly bill split by feature. Engineering wants to know which feature sends the longest prompts, and whether moving support chat to the small model last quarter actually saved money.

The provider’s dashboard shows the bill, not your features. Logging every call answers everything, and in a year it’s millions of rows, most holding prompt text you’d rather not keep at all. What the questions need is sums and counts per model and feature over time, plus the shape of the token counts. RollupDB keeps that and lets the calls go.

Two Streams

A reading in RollupDB is one number, so cost and tokens are two streams with the same dimensions, the model and the feature. Cost needs exact sums and no percentiles. Tokens get a sketch, because the long prompts are the expensive ones.

{
  "name": "llm_tokens",
  "version": 1,
  "effective_at": "2026-01-01T00:00:00Z",
  "dimensions": ["model", "feature"],
  "statistics": ["count", "sum", "min", "max", "avg"],
  "max_groups": 200,
  "tiers": [
    {"width_seconds": 1,     "retain_seconds": 3600,     "grace_seconds": 120},
    {"width_seconds": 60,    "retain_seconds": 604800,   "grace_seconds": 120},
    {"width_seconds": 3600,  "retain_seconds": 31536000, "grace_seconds": 120},
    {"width_seconds": 86400, "retain_seconds": 0,        "grace_seconds": 120}
  ],
  "sketch": {"algorithm": "kll_float64", "encoding_version": 1, "k": 200, "m": 8}
}

llm_cost is the same without the sketch line. The app sends one reading to each stream per call, with the cost in dollars worked out from its own price table.

Where the Money Goes

We ran 3,000 simulated calls through the released 0.4.1 binary. The prices are made up, a large model at about twenty times the small one’s rate, so read the shape, not the dollars.

rollupdb sql ai.rollupdb "SELECT model, feature, sum(count) AS requests,
  round(sum(sum), 2) AS dollars, round(sum(sum) / sum(count), 4) AS per_request
  FROM llm_cost GROUP BY model, feature ORDER BY dollars DESC"

Support chat on the large model is 22% of the calls and 86% of the spend. The sums are exact, so this is the bill, not an estimate of it. The 293 support chats that went to the small model cost under a quarter of a cent each, against nearly five cents on the large one. That’s the comparison to run before and after a model switch.

Tokens, from the other stream:

rollupdb sql ai.rollupdb "SELECT model, feature, sum(sum) AS tokens,
  round(rollup_quantile(sketch, 0.95)) AS p95_tokens, max(max) AS biggest
  FROM llm_tokens GROUP BY model, feature ORDER BY tokens DESC"

Summaries and search used about 1.26 and 1.25 million tokens, support chat on the large model 1.09 million. The large model’s support chats have the longest calls, 3,162 tokens at the 95th percentile and 5,065 at most. Totals and the maximum are exact. The p95 is approximate.

Changing Your Mind Later

Say you start without the token sketch. A new stream version can add it later; buckets from then on carry percentiles, and queries over older time say plainly that percentiles weren’t kept there. Retention can change the same way. The dimensions can’t. A per-customer breakdown has to start as its own stream on the day you decide you need it, and customer IDs make for many series, so set max_groups with that in mind.

What Stays Out

No prompts, no responses, no user IDs. The file holds numbers per model and feature per time bucket, which is easier to keep for years and easier to hand to finance. If you need to read actual conversations, that’s a different store with a much shorter life.