Why AI shakes up FinOps
According to State of FinOps 2026, 98% of FinOps teams now manage AI spend, up from 31% two years earlier. And Flexera links the first rise in cloud waste in five years to AI usage. Three reasons:
- New billing: per token, input and output, with very different prices from one model to another.
- Unpredictable consumption: a new use case, an agent loop or an overly long prompt can multiply the bill within days.
- Value that is hard to measure: without a cost per use, you can't tell whether a use case pays off.
Payment modes
| Mode | Principle | When to choose it |
|---|---|---|
| Pay-as-you-go (per token) | Price per million input and output tokens | Getting started, variable load |
| Provisioned capacity | Throughput reserved by the hour or month (PTUs on Azure OpenAI, provisioned throughput on Bedrock) | Heavy, steady load, guaranteed latency needed |
| Batch | Asynchronous processing, about 50% cheaper than real time on eligible models | Anything that doesn't need an immediate answer: classification, summaries, extraction |
Optimization levers
- The right model for each task. A small model is often enough to classify, extract or route; keep large models for the tasks that need them.
- Bounded answers. Limit answer length and clean up prompts: every token counts, input and output.
- Prompt caching. When a long context comes back with every call, caching it sharply cuts the cost of input tokens.
- Batch for non-urgent bulk processing.
- Provisioned capacity only when the load is steady and measured, like a reservation.
- GPUs: automatic shutdown of training and test machines, Spot for interruptible jobs, scale to zero for lightly used endpoints.
Guardrails to set from day one
- One tag or deployment per use case, to know who consumes what.
- Quotas and rate limits per application.
- Daily budget alerts on AI resources: a monthly alert arrives too late.
- A cost per use tracked in the dashboard: cost per conversation, per document, per active user.
It is the same logic as for the rest of the cloud: understand, optimize, operate, stay in control. At a much faster pace.