Managing the Cost of AI: Tokens, Credits and Real Return
How to manage AI cost: token consumption, per-feature breakdown, budget limits, cost-benefit comparison and savings.
AI features differ from classic software modules in one respect: you pay as you use them. A feature that looks cheap at launch shows up on the invoice three months later — and most teams only notice once the bill has grown.
What tokens are and why they matter
Language models split text into “tokens”. Roughly, one token is about four characters. Cost is proportional to the tokens processed, and there are two sides to it:
- Input tokens: everything you send — the question, the system prompt, chunks retrieved from the knowledge base, the conversation history.
- Output tokens: what the model produces. Usually more expensive than input.
The most common surprise: the cost usually comes from the context accompanying the question rather than the question itself. A long system prompt plus 20 messages of history sent with every request dwarfs the query.
What each feature consumes
Conversation summaries
Large input (the whole thread), small output. Expensive on long threads; set a threshold.
Channel AI assistant
Runs on every message and retrieves from the knowledge base. Grows linearly with volume — the item to watch most.
Intent recognition
Short input, very short output. Low unit cost, but runs very often.
AI reporting
Medium input, potentially large output. Used infrequently, so the total impact is usually limited.
Knowledge base preparation
Processing documents is a one-off cost, repeated as often as you update them.
HR candidate evaluation
Fixed per candidate; multiplied by headcount in bulk runs.
Five ways to reduce cost
- Trim the context. Sending the last 8 messages plus a summary instead of 30 usually cuts cost noticeably without hurting quality.
- Shorten the system prompt. A three-page rulebook is paid for on every request and diluted by the model anyway. Short, clear rules are cheaper and more effective.
- Set thresholds. Do not summarise short conversations; use flows rather than the assistant for routine questions.
- Fix what repeats. A report produced monthly in the same format should become a fixed report — AI is for exploration, not repetition.
- Match the model to the job. Simple work like intent recognition does not need the most capable model; choosing per task makes a real difference.
Budget protection
Before any technical optimisation, what you need is a limit. At minimum:
- A balance or monthly ceiling must be defined.
- A low-balance notification must go out — a feature stopping silently is the worst outcome.
- Consumption should be visible per feature; a total does not tell you where it went.
- Abnormal growth must be noticeable. An assistant stuck in a loop through an integration fault can burn a monthly budget overnight.
Measuring the return
Cost alone means nothing; read it against what it buys. Usable comparisons:
- Cost per conversation versus the cost of the agent minutes that conversation would take.
- Share of conversations resolved by the bot — handovers do not count as savings.
- Reduction in average handle time and what that means in staff capacity.
- Satisfaction difference: a cheap but unsatisfying resolution costs more over time.
Frequently asked questions
Is a flat-price plan better?
It gives predictability but costs more at low usage. Moving to a fixed package before measuring three months of real consumption usually means overpaying.
Costs went up — should I switch the feature off?
Look at where it came from first. Most increases trace to a single item — usually an assistant running on every message, or untrimmed context.
Does self-hosting a model make it cheaper?
It can at high, sustained volume; at low volume hardware and maintenance exceed pay-per-use. Decide by looking at your own volume curve.
Next step: to see per-feature consumption and cost screens, request a demo.