> ## Content Index
> Fetch the complete content index at: https://www.oliphonic.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Stop counting tokens and start counting what actually matters
- URL: https://www.oliphonic.com/stop-counting-tokens-and-start-counting-what-actually-matters/
- Published: 2026-09-27T15:48:47.000Z
- Updated: 2026-09-27T15:48:47.000Z
- Author: Oliphonic

*AI pricing is obsessed with tokens. Businesses should be obsessed with something (unfortunately) much harder to figure out: what it actually costs to get a useful task completed properly.*

AI cost conversations tend to feel like comparing a unit of measure that no-one actually really has a proper handle on - **tokens.**

*"What's a token?" Well duh Oli, a token is the* **atomic discrete unit of information resulting from the decomposition of continuous sequential data**, **mapping raw input strings into compressed, high-dimensional vector representations.**

...got it. Cool. So... Opus 5.5 now costs **$4 / million tokens input**, **$20/M output** and **$0.2/M** cache. Great, let me go back to my boss and accurately describe how much it's going to cost to bring AHT down by 20% in the contact centre using AI assistants. 

**...yeah right.**

Token prices are useful, obviously. If you are running millions of requests, a cheaper model can save a lot of money. But the way AI is often discussed makes the token feel like the unit of economic value. It isn’t. A token is just a tiny unit of model input or output. Nobody in a business wakes up desperately hoping to purchase 17 million attractively priced subwords before lunch.

They want something done. A customer email classified. An invoice extracted. A sales call summarised. A support ticket resolved. A piece of code fixed. A proposal drafted. An application checked. A fraud case investigated.

The useful unit is not **cost per token**. It is **cost per successfully completed task at the quality you actually need**.

That sounds like a small distinction. I think it is a very large one.

## The three-pence model that costs £47

Imagine I can use a wonderfully cheap model to process an invoice for 3 pence. Fantastic. Unfortunately it gets the tax treatment wrong occasionally, formats dates inconsistently, needs three retries on unusual documents and produces a confidence score that appears to have been generated by a random number generator.

So we add another prompt. Then a retry. Then a validation step. Then a second model to check the first model. Then a human reviews the edge cases. Someone builds a workflow around all of this. Someone else monitors it. A third person spends Thursday afternoon working out why invoices from Belgium have suddenly been translated to Chinese.

The 3 pence inference is still 3 pence. The task is not.

This is why cost models for enterprise AI should include the boring things: retries, latency, orchestration, human review, failures, escalation and the cost of getting something wrong. A model that costs five times more per request but completes 98 per cent of tasks correctly on the first attempt may be dramatically cheaper than the bargain model that turns each request into a small administrative saga.

This is not unique to AI. A £12 drill that breaks halfway through your kitchen renovation may not have actually saved you money after you avoided the £50 drill.

## The cleverest model should not do everything

There is another odd assumption creeping into AI architecture: that the best model should be used for every task.

**Why?**

We don't ask senior barristers to alphabetise receipts. We don't expect neurosurgeons to fix a splinter we may have. Yet organisations routinely talk about selecting “the model” for an AI solution, as though one model must become **the permanently employed brain of the entire thing.**

![](https://storage.ghost.io/c/9c/fa/9cfabe9c-bcc0-471f-a6fa-271bee166fa4/content/images/2026/09/ChatGPT-Image-Sep-27--2026--04_36_09-PM.png)

Best model means best results! .... right??

A more sensible future probably looks like **routing**. Easy, repetitive, well-understood tasks go to smaller and cheaper models. Difficult, ambiguous or high-risk cases go to a frontier model. Extremely high-risk cases go to a human. The system continuously evaluates whether the cheap path is actually working and escalates when it is not.

The economic goal is not to minimise model price. It is to minimise the cost of reliably producing the required outcome. That changes the architecture quite a lot.

## Teach the clever model once

This is where model distillation becomes particularly interesting.

The basic idea is pretty intuitive. Start with a large, capable model that can perform a specialised task well. Use its successful outputs to help train a smaller model to imitate the useful behaviour on that narrower task. The expensive model becomes something like a teacher; the smaller model becomes the graduate who has learnt one particular job extremely well.

This is not a theoretical oddity. OpenAI explicitly documents the pattern: get strong performance from a larger model, capture high-quality outputs, then use them as training data for a smaller model so that the cheaper model can perform similarly on that particular task. [OpenAI](https://openai.com/index/api-model-distillation/?utm%5Fsource=chatgpt.com)

A frontier lab may spend extraordinary amounts of money discovering general capabilities: training huge models, building infrastructure, creating datasets, running experiments and paying clever people to stare at loss curves. But once a useful behaviour exists, reproducing a narrow slice of it may be much cheaper than discovering it in the first place.

Intelligence could therefore have a strange cost curve: **extremely expensive to discover, comparatively cheap to reproduce for specific jobs**.

That should make both enterprise architects and investors sit up slightly straighter. Heck, the conspiracy theorists amongst us may even presume that this very fear led to the call from all the Frontier labs to "slow down AI" for fear it may end the human race (**read:** better regulate the sector to cement their frontier models as **the only** models and prevent distillation - see my earlier blog article on Jacob Coxon...).

## The awkward bit: who paid to discover the intelligence?

Distillation is not inherently controversial. It is a standard technical approach and model providers themselves support legitimate versions of it. Keyword "legitimate".

The controversy begins when one company uses another company’s model outputs to train a competing system in ways that may violate the original provider’s terms or protections. The DeepSeek debate pushed this into public view: OpenAI alleged that actors associated with DeepSeek used outputs from US models for distillation, while reporting at the time stressed both how widespread distillation techniques are and how difficult alleged unauthorised use can be to prove or police. [Reuters](https://www.reuters.com/technology/artificial-intelligence/why-blocking-chinas-deepseek-using-us-ai-may-be-difficult-2025-01-29/?utm%5Fsource=chatgpt.com)

The distinction matters. There is a perfectly ordinary enterprise pattern where I use a permitted frontier model to generate examples for my own narrow specialist model. There is also a very different competitive scenario where an organisation attempts to reproduce broad capabilities from a rival without bearing the original discovery cost.

Those are not the same thing.

But economically they point towards the same uncomfortable question: if the first copy of a capability costs billions and the ten-thousandth specialised copy costs very little, where does the actual value sit?

In the model weights? The infrastructure? Proprietary data? Distribution? Brand? Safety systems? Integration into business workflows? Or simply being the organisation that discovers the next capability before everyone else learns to commoditise the last one?

## Cheap intelligence can still produce expensive work

There is also a slightly comic danger here. The model industry becomes obsessed with squeezing another 30 per cent out of inference cost while enterprises continue wrapping the model inside processes containing six approval gates, three committees and a human whose job is to copy the result into SAP.

Congratulations. We saved £0.004 on the model call, but the process still costs £27.

This is why I increasingly like **cost per completed task** as the unit that forces everyone to behave like adults. It connects architecture to the business outcome. It makes you ask whether the smaller model really works, whether the frontier model is needed, how often escalation happens, what human intervention costs and whether automation is actually removing work rather than merely relocating it.

It also makes evaluation unavoidable. You cannot optimise cost per completed task unless you can define “completed” and “good enough”. That means test cases, quality thresholds, failure categories and an honest understanding of what happens outside the demo.

Less exciting than a benchmark leaderboard, admittedly, while potentially much more valuable.

Interestingly, Anthropic’s 2026 Economic Index found that higher-compute Claude sessions tended to be associated with higher-value work outputs, albeit noisily. That is another reason simplistic “fewer tokens = better economics” thinking can be misleading: spending more compute may be entirely rational if the extra capability produces a more valuable or more reliable result. [Anthropic](https://www.anthropic.com/research/economic-index-june-2026-report?utm%5Fsource=chatgpt.com)

![](https://storage.ghost.io/c/9c/fa/9cfabe9c-bcc0-471f-a6fa-271bee166fa4/content/images/2026/09/ChatGPT-Image-Sep-27--2026--04_44_36-PM.png)

This image took way more prompting than expected to get right

## The cheapest model might be the expensive one

As mentioned, the cheapest model may in fact be the most expensive model in the workflow. The expensive frontier model may be cheap if it gets the task right immediately. A tiny distilled model may be astonishingly economical if it handles 95 per cent of predictable work and escalates the strange 5 per cent. And a supposedly premium architecture may become absurdly costly if every output still requires somebody to inspect it manually.

Thinking that is more like:

**For every thousand real jobs we give this system, how many does it complete correctly, how long does it take, how often does a human intervene, and what is the total cost?**

And if the future really does involve armies of models — frontier models discovering capabilities, smaller models learning them, routers deciding who does what and humans appearing only when something genuinely odd happens — then the winning AI architecture may not contain the smartest possible model everywhere.

It may simply know when **not** to use it.