AI Tokens Explained: 4 Costly Mistakes to Avoid

7 min read

TL;DR Tap to expand the short version

AI tokens explained simply: they’re the unit that decides your AI bill.
English text runs at roughly 1.3 tokens per word, so what you type is always billed as more units than it looks like.
Uber burned through its entire 2026 AI budget in four months after giving 5,000 engineers access to Claude Code.
Output tokens (what the AI generates) typically cost three to five times more than input tokens (what you send), because generating text takes more computation than reading it.
Fixes that actually help: start fresh chats for new topics, use smaller models for routine tasks, and keep prompts direct rather than padded.

AI Token Basics

Every message sent to an AI tool is broken down into tokens before the model can process it, and a token is not the same thing as a word. English text runs at roughly 1.3 tokens per word, or about three tokens for every four words, a ratio confirmed across independent tokenizer analyses including howmanytokens.org and Token breakdown. That gap between what people picture, words, and what they’re actually billed for, tokens, is where confusion and cost both start.

Uber found this out the hard way. After giving roughly 5,000 engineers access to Anthropic’s Claude Code in December 2025, the company burned through its entire 2026 AI tooling budget by April, just four months into the financial year. Uber’s chief technology officer, Praveen Neppalli Naga, confirmed the overrun to The Information, saying the company was “back to the drawing board” on its spending assumptions.

The adoption curve moved fast. Forbes reported that the share of engineers classified as agentic coding users rose from 32% in February to 84% by March, and monthly per-engineer costs ranged from $500 to $2,000. Uber had also run an internal leaderboard ranking teams by AI usage volume, a system that rewarded higher token consumption even though the teams driving adoption weren’t the ones accountable for the budget. Uber’s president and chief operating officer, Andrew Macdonald, later said the company was finding it harder to justify rising token spend without clear evidence it was producing measurable improvements for customers.

Run that per-engineer range against a smaller team and the scale of the problem changes shape. A 50-person engineering group at the low end of Uber’s range, $500 per engineer a month, lands at $25,000 monthly, a figure most finance teams would fold into general software spend without a second look. At the high end of $2,000 per engineer, the same team hits $100,000 a month, and that is before accounting for the rest of the AI tooling stack sitting alongside it. The distance between those two numbers is not a rounding error. It is the difference between a line item nobody tracks and one that needs its own approval chain, and most companies do not know which side of that line they are on until the invoice arrives.

The same pattern is showing up well beyond engineering teams. A UBS survey found roughly 60% of the enterprises it polled are now throttling AI spending, as of June 2026. Separately, the FinOps Foundation’s 2026 State of FinOps report found that 73 percent of enterprises said their AI costs exceeded original projections. Gartner has warned that organisations that neglect data structure and context quality end up with AI agents that are inaccurate, inefficient, and expensive to run, and it forecasts that businesses prioritising well-structured context in their AI systems could lift agentic AI accuracy by up to 80 percent and cut costs by up to 60 percent by 2027.

What Drives the Cost

Model

Two things determine token costs, the model handling the request and how much text moves in and out. Providers typically charge more for output tokens than input tokens, often three to five times more, because generating a response takes more computation than reading one.

Context windows

Context windows add a second layer. This is the maximum number of tokens a model can process in a single exchange, and it behaves differently depending on the tool. Anthropic’s own documentation states that in Claude models from Sonnet 3.7 onward, exceeding the context window returns an error rather than silently dropping earlier content, and the claude.ai interface shows a warning as a conversation approaches its limit, sometimes summarising earlier turns to stay within it. That warning is the version of this problem most teams meet first, and the recovery steps are covered in our guide to Claude usage limits. Capacity also varies by model tier. Anthropic’s current Opus 4.8 and Sonnet 5 models run a 1 million token context window, while Haiku 4.5 runs 200,000, per Anthropic’s platform documentation.

Reducing Token Waste

None of this requires becoming a developer. The practical fixes that surface repeatedly in enterprise cost reporting are as follows:

Start a fresh conversation for a new topic rather than letting one long chat history keep accumulating.
Match routine tasks to smaller and cheaper models rather than defaulting to the most powerful option available
Keep prompts direct rather than padded with unnecessary context.

What’s changing is the expectation that this now needs tracking at all. As Uber’s experience shows, token consumption can outrun a full year’s budget in a matter of months when nobody is watching the meter. Businesses that treat AI usage the way they’d treat any other variable cost, monitored, benchmarked, and reviewed, are the ones least likely to be caught by surprise when the bill arrives.

FAQ

What exactly are AI tokens and how do they work?

Tokens are the building blocks an AI uses to read and write, often representing parts of words or punctuation. These units are how models measure their processing work and how providers track your usage.

Why is there a price difference between input and output tokens?

Generating text requires significantly more computational power than simply reading it, which is why output tokens usually cost three to five times more. Budgeting with that ratio in mind keeps forecasts much closer to the real bill.

What happens if my conversation gets too long for the context window?

The context window acts as the AI’s short-term memory, and once you hit the limit, the model may stop processing or trigger an error. Some platforms help by providing warnings or summarizing older parts of the chat to keep you within the limit.

Is it true that different words count as different amounts of tokens?

Yes, a simple word like ‘the’ is usually just one token, while longer or more complex words might be broken down into three or more. That is why two prompts of similar length can consume noticeably different amounts of your budget.

What are some practical ways to save money on AI token usage?

You can lower costs by starting fresh conversations for new topics to keep history short and by using smaller, more efficient models for routine tasks. Being direct and avoiding unnecessary padding in your prompts also helps keep your token count down.

Why Uber’s Engineers Burned a Year’s AI Budget in Four Months

Table of Contents

AI Token Basics

What Drives the Cost

Model

Context windows

Reducing Token Waste

FAQ

What exactly are AI tokens and how do they work?

Why is there a price difference between input and output tokens?

What happens if my conversation gets too long for the context window?

Is it true that different words count as different amounts of tokens?

What are some practical ways to save money on AI token usage?

Like this:

Why Uber’s Engineers Burned a Year’s AI Budget in Four Months

Table of Contents

AI Token Basics

What Drives the Cost

Model

Context windows

Reducing Token Waste

Related reading

FAQ

What exactly are AI tokens and how do they work?

Why is there a price difference between input and output tokens?

What happens if my conversation gets too long for the context window?

Is it true that different words count as different amounts of tokens?

What are some practical ways to save money on AI token usage?

Share this:

Like this:

Discover more from Industry Contents