Token and AI API budgets: the FinOps guide for SMEs in 2026

Introduction: controlling variable spend without suppressing use

An AI application often combines subscriptions, usage-based calls, storage and human work, so a reliable budget starts from the units actually consumed and the value obtained rather than a simple licence count.

• Demand: requests, documents, conversations and agent steps determine the volume processed.
• Consumption: input tokens, output tokens, caching, tools and any reasoning tokens make up the variable bill.
• Service: latency, availability and quality determine the acceptable model tier for each use case.
• Value: cash savings, incremental margin, released capacity and avoided risk must be tracked separately.

Financial risk comes less from the token itself than from missing ownership, executable limits and quality measurement, because a team can optimise the unit price while multiplying unnecessary calls or human rework.

• A shared key prevents spend from being attributed to a team, application and use case.
• A dashboard reveals drift after the event; a policy on the request path can slow, reroute or reject it.
• A cheaper model is economical only when its error rate, escalations and control costs remain acceptable.
• An agent with no step limit can turn an ordinary error into a consumption loop.

This guide therefore sets out a complete control chain from inventory and gateway to caching, circuit breakers and ROI calculation, using bounded examples that each SME should adapt to its own evidence.

• Prices are a snapshot as at 6 September 2026 and must be checked against provider catalogues before a decision.
• Quota, similarity and escalation thresholds are design examples rather than universal standards.
• Benefits enter ROI only when they are attributable, documented and counted once.

Governance: connecting use, risk and cost centre

Useful governance creates a traceable line between every application, its owner, providers, data, risks and budget, making it possible to arbitrate or stop a use case that has become disproportionate.

• Inventory: application, model, version, provider, purpose, processed data and dependencies.
• Accountability: business owner, technical owner, cost centre and person authorised to raise a cap.
• Assessment: expected quality, risks, affected population, human oversight and fallback scenario.
• Measurement: cost per useful result, errors, rework, latency and incidents over a stated period.

ISO/IEC 42001: a management system, not an automatic assurance

ISO/IEC 42001:2023 provides an AI management-system framework that an organisation can use to structure policies, responsibilities, risks, objectives and continual improvement, without alignment or certification alone guaranteeing that a particular system complies with the law [1].

• The standard helps organise decisions and evidence across the AI lifecycle.
• Certification is separate from using the framework and applies to a defined audit scope.
• Legal duties still depend on the organisation’s role, the system and its use case.
• Budget control benefits from the same documentation, but must still be implemented in technical and financial systems.

AI Act: applying the timetable to the right actor and system

As at 6 September 2026, the European timetable must be read by obligation, system category and organisational role, because an SME may be a provider, deployer, importer or distributor and does not carry the same duties in every situation [2][3].

• Article 50 has applied since 2 August 2026 to providers and deployers within scope, according to the transparency obligation concerned.
• The 2 December 2026 grace period applies only to providers of systems placed on the market or put into service before 2 August 2026, for the Article 50(2) marking and detection duty; it does not defer Article 50 as a whole [3].
• Sections 1 to 3 of Chapter III, except Article 6(5), apply on 2 December 2027 to high-risk systems under Article 6(2) and Annex III, then on 2 August 2028 to those under Article 6(1) and Annex I [2].
• For high-risk systems already placed on the market or put into service before the applicable date, the transition depends in part on significant design changes; a specific regime covers systems intended for public authorities [2].
• Article 4 now requires providers and deployers to take measures supporting the development of AI literacy among the people concerned, taking account of their experience and the context of use [2].

Compliance and resource efficiency may use some of the same management data, but they address different objectives that management should document without treating lower token use as legal or environmental proof.

• AFNOR Spec 2314 offers a method for measuring and reducing AI’s environmental impacts [4].
• Consumption logs can support financial monitoring and resource indicators when their limitations are explicit.
• Context reduction is beneficial only if quality, security and evidence retention remain at the required level.

The AI gateway: where policies are enforced

An AI gateway between applications and providers centralises identity, routing and consumption rules, provided applications cannot bypass it through direct keys.

• Identity: associate every call with a user, team, environment, application and cost centre.
• Policy: check the permitted model, cap, rate, region, data classification and right to use a tool.
• Resilience: manage timeouts, bounded retries, fallback providers and degraded modes.
• Metering: reconcile the pre-call estimate, usage returned by the provider and the actual invoice.

Observability informs a decision while a control component executes it, a crucial distinction because tools such as Langfuse receive traces outside the model request path and cannot block spend on their own [5].

• Gateway or policy engine: sits on the synchronous path and permits, reroutes, throttles or rejects a request.
• Telemetry collector: receives spans and measurements, often asynchronously, and makes them available for analysis.
• Observability platform: analyses cost, latency, quality and traces; an alert is not a cut-off.
• Governance loop: turns observations into revised policies that are tested and then deployed to the gateway.

Tool selection should focus on capabilities verified in the chosen version rather than a general league table, because features and performance change with hosting mode, plugins and load.

• The control path determines whether a policy can be enforced.
• The telemetry path determines the quality of measurement and diagnosis.
• Both should be tested together under load and during a provider outage.

Component	Main function	Can it block a request?	Checks before selection
API/LLM gateway or proxy	Authentication, quotas, routing and rate limiting	Yes, if the policy runs on the call path	Covered models, budget granularity, high availability and prevention of bypass
Model router	Select a model by task, cost and quality	Yes, if integrated into the control path	Evaluation sets, fallback rules, added latency and explainability
LLM observability	Traces, estimated costs, evaluations and debugging	No, unless paired with a separate control component	Usage capture, data protection, sampling and invoice reconciliation
Exact or semantic cache	Controlled reuse of a result	Yes, by serving a response without a new call	Freshness, tenant isolation, invalidation and match quality

Quotas: budgeting for a result rather than an abstract volume

An actionable quota combines a monetary envelope, period, owner and overrun behaviour, then translates these into technical metrics that the gateway can evaluate before and after each call.

• The monthly budget defines the maximum acceptable spend for the use-case portfolio.
• An explicit reserve covers legitimate peaks and should not conceal an inadequate forecast.
• Sub-quotas by application or cost centre prevent one flow from exhausting the shared allowance.
• Cost per useful result complements token counts: case handled, conversation resolved, test fixed or report approved.

Consumption profiles differ primarily through the shape of the work, making a model based on local measurement more robust than generic token ranges by department.

• First measure flow-specific drivers: turns, pages, tools, context and outputs.
• Then connect those drivers to a verifiable business result.
• Finally define the cost policy that protects the result without degrading its quality.

Use case	Main cost driver	Outcome metric	Suitable policy
Customer support	Number of turns, customer context and generated answers	Correctly resolved conversation	Economy model by default, turn cap and escalation on defined grounds
Finance documents	Pages, OCR, extraction and validation	Correctly structured document	Deterministic pre-processing, structured output and exception review
Software engineering	Repository size, tool calls and test-fix cycles	Mergeable task or corrected defect	Per-task budget, step limits and stronger model for evidenced complexity
Marketing	Output volume, variants and revisions	Approved content or campaign margin	Variant limit, output template and cost attributed to the campaign

A graduated policy preserves service where possible while preventing overspend, using thresholds selected for criticality and tested in a pre-production environment.

• Alert: at an initial threshold, notify the owner and explain the consumption rate.
• Conservative mode: reduce verbosity, disable expensive options or route to a validated cheaper model.
• Exceptional approval: reserve cap increases for an identified role, recorded duration and justification.
• Cut-off: at the hard limit, reject the call or serve a degraded mode without incurring new unauthorised spend.

Dynamic routing: buying the required level of quality

Routing reduces spend only when the smaller model meets a measured quality threshold and the cost of classification, failures, retries and escalations is included in the comparison.

• Define a representative evaluation set by intent, language, risk and context length.
• Assign a default model to each task class, including data and regional constraints.
• Escalate on an observable signal: schema validation, evaluation score, failed tool or business rule.
• Recalculate cost per correct result periodically because prices and performance change independently.

The router follows a short, auditable sequence that separates the initial decision from output control and limits expensive cascades.

• Classify intent, risk and complexity with rules or a small model.
• Check access, data, budget and availability policies.
• Call the cheapest admissible model that reaches the measured quality level.
• Validate the output, then permit no more than the defined number of escalations for that flow.

A pricing snapshot rather than a permanent ranking

Public prices as at 6 September 2026 show why the router catalogue should be dated, versioned and reconciled with invoices rather than copied once into a spreadsheet [6][7][8].

• The OpenAI rows provide comparable public input and output prices.
• The DeepSeek and Anthropic rows identify pricing dimensions that prevent a simple two-column comparison.
• Cache, batch and volume discounts and tool fees still need to be calculated for each scenario.

API offer	Input / 1M tokens	Output / 1M tokens	Point to check
OpenAI GPT-5.6 Luna	$0.20	$1.20	Announced from 30 July 2026; check caching, batch and tools separately [6].
OpenAI GPT-5.6 Terra	$2.00	$12.00	Same reference date; compare cost per correct result rather than token price alone [6].
OpenAI GPT-5.6 Sol	$4.00	$20.00	Reserve the premium tier for tasks where evaluation justifies the difference [7].
DeepSeek V4 line	Variable	Variable	Since 16 August 2026, pricing distinguishes peak and off-peak periods, with off-peak announced at 50% less; consult the live tariff [8].
Anthropic Claude	Model-dependent	Model-dependent	The catalogue distinguishes input, five-minute or one-hour cache writes, cache reads and output [9].

A reasoning-model budget should use the billable counters returned by the provider, because visibility, naming and pricing treatment for internal tokens and tool calls vary by API.

• Set the maximum reasoning effort where the API allows it.
• Cap output and tool calls independently of the visible answer length.
• Measure the actual distribution by task type, including its upper tail, instead of applying a universal multiplier.
• Include search, code execution and other paid tools in the request’s fully loaded cost.

Context engineering: separating compute, latency, memory and billing

Longer context increases several resources under different scaling laws, so the claim that “twice the context costs four times as much” holds only in a narrow technical case and describes neither all inference nor API billing [10].

• Prefill: with standard dense attention and the same implementation, theoretical attention work grows quadratically with length; doubling context may therefore approach four times that work if this phase dominates.
• Decoding: the KV cache avoids recomputing the entire prefill for every generated token; latency then depends partly on cached length, memory and hardware.
• KV-cache memory: it grows approximately with sequence length, layer count and the chosen representation.
• API billing: it follows provider units and options — input, cache, output and tools — rather than the GPU’s exact internal operations.
• Observed latency: optimised kernels, batching, parallelism, network, load and hardware can move measurements away from the simple complexity order.

Good context engineering removes noise before the call, retains necessary evidence and checks that compression does not harm responses on the business evaluation set.

• Filter documents by access rights, date, source and relevance before injection.
• Summarise history with references to original evidence instead of copying every turn.
• Use structured outputs and length limits suited to the expected result.
• Test compressors such as LLMLingua on local tasks; published research results remain dependent on dataset and model [11].

Caching: saving a call without serving stale answers

Exact and semantic caching reduce calls and latency when requests repeat, but their net benefit depends on hit rate, indexing cost, false matches and required freshness.

• Exact caching normalises and hashes a request, then reuses only an identical match.
• Semantic caching computes an embedding and reuses an answer when similarity crosses a validated threshold.
• Provider caching reuses a prefix under the relevant API’s rules, durations and prices.
• A model call remains necessary when match or freshness controls fail.

A defensive cache architecture checks tenant identity, source version, match and lifetime policy before reusing a result.

• Local layers decide whether an existing answer is admissible for the request.
• Provider caching reduces some prefix processing under its own pricing contract.
• Model fallback produces a new answer when any check fails.

Layer	Intended benefit	Main risk	Required control
Exact match	Avoid the call and embedding	Cache key is too broad	Include tenant, language, prompt version and rights in the key
Semantic similarity	Recognise rephrasings	Semantic false positive	Evaluate threshold by intent and exclude sensitive uses
Provider cache	Reduce the cost of a repeated prefix	Mis-modelled rules or charges	Read cache counters, duration and write/storage charges
Model fallback	Produce a new answer	Cost and latency	Bound budget, timeout and retry count

Lifetime should reflect volatility and business risk, with event-driven invalidation when the source changes and no generic threshold presented as safe for every type of content.

• Prices, stock, rights and urgent instructions require short expiry or direct verification.
• Versioned documentation and stable FAQs can last longer if their version remains in the key.
• Personal data and customer-specific answers must be isolated and governed by the retention policy.
• Hit rate, freshness errors and net savings should be measured together.

Agents: containing loops and powers

An agent can consume at an accelerating rate as its history grows or it repeats tools and attempts, without that growth being universally quadratic or the loop necessarily being visible to the user.

• Each step can add context, tool results and another billed output.
• An automatic retry can repeat a permanent error and multiply cost without increasing the chance of success.
• Delegation to other agents expands the call tree and should share the same parent budget.
• Stopping the application does not necessarily cancel operations already launched; control must cover the data plane and tools.

A containment architecture combines budget, time, identity and tool scope so that no single failure gives the agent indefinite spend or access.

• Budget and steps: cap each task, session, child agent and tool, checked before every new action.
• Time: set an overall deadline, per-call timeout and retry count with backoff.
• Identity: use a distinct, revocable identity, short-lived secrets and minimum permissions.
• Data and tools: authorise named resources, require approval for sensitive writes and retain an audit trail.
• Human recovery: save state, show a clear stop reason and provide a bounded override procedure.

ROI: separating cash, margin, capacity and risk

Financial ROI compares attributable benefits realised during a period with total cost over that same period, while quality and capacity indicators explain performance without automatically becoming cash flows.

ROI (%) = [(attributable benefits − TCO) / TCO] × 100

• TCO includes APIs and tools, data, integration, infrastructure, evaluation, oversight, change, security and governance.
• Cash savings are expenses that were actually reduced or removed.
• Incremental margin applies the contribution rate to attributable incremental revenue; gross revenue is not a benefit.
• Released capacity is measured in hours and theoretical value, entering ROI only when a distinct economic use is evidenced.
• Avoided risk uses a documented change in expected loss, with explicit assumptions and uncertainty.

An illustrative annual example shows why adding saved hours to the value of their redeployment would count the same effect twice, even when each line seems defensible in isolation.

• The scenario assumes annual TCO of €80,000.
• It includes only financial benefits whose realisation is specified.
• Capacity value remains a separate operational indicator in this calculation.

Illustrative item	Amount included	Treatment
Project TCO	€80,000	Denominator and subtracted cost
Contract and service savings	€25,000	Documented cash benefit
€120,000 incremental revenue at 40% contribution margin	€48,000	Margin rather than gross revenue
Reduction in expected loss	€15,000	Benefit subject to documented assumptions
1,200 released hours at €45/hour	€54,000 capacity	Tracked outside ROI because realised effects already appear above

In this bounded example, attributable benefits are €88,000, net benefit is €8,000 and annual ROI is 10%, a result that changes as soon as an attribution, margin or risk assumption changes.

• Calculation: (25,000 + 48,000 + 15,000 − 80,000) / 80,000 = 10%.
• The €54,000 capacity value is not added without evidence of a separate saving, margin or avoided cost.
• Sensitivity analysis should vary adoption, volume, quality, unit cost and contribution margin.
• Payback complements ROI by showing when cumulative cash flows cover TCO.

The management dashboard then connects financial outcomes to operational evidence that explains them and makes them auditable.

• Each financial outcome has an earlier or concurrent operational indicator.
• Each row specifies the control needed to isolate the effect of AI.
• Unobservable assumptions remain documented in the sensitivity analysis.

Area	Possible financial outcome	Operational evidence	Attribution safeguard
Finance and operations	Lower cash cost or redeployed capacity	Cycle time, rework and processed volume	Before/after comparison with volume changes controlled
Support	Avoided cost or retained margin	Correct resolution, escalations and satisfaction	Comparable quality and demand, residual human cost included
Security and compliance	Lower expected loss	Incidents, severity and detection time	Documented probability and impact, explicit baseline scenario
Sales and marketing	Incremental margin	Conversion, basket size and acquisition cost	Comparison group or period, cannibalisation deducted

A workable FinOps cadence for SMEs

An SME can start with one use case and a short monthly loop, provided it defines the expected outcome, acceptable loss cap and post-measurement decision rule before the pilot begins.

• Before: establish baseline, forecast TCO, quality criteria, legal constraints and budget limit.
• During: measure cost per correct result, cost distribution, cache, escalations, human rework and incidents.
• After: reconcile invoice and telemetry, calculate attributable benefits, document variances and decide whether to stop, correct or expand.
• At every change: reassess model, price, contract, data, prompts and cache rules before shifting traffic.

Conclusion: a verifiable economic architecture

Controlling AI API budgets requires a coherent chain in which governance assigns, the gateway enforces, observability measures, routing arbitrates, caching avoids and finance values only effects that have actually been realised.

• Spend without an owner or measured result remains hard to defend, even with cheap tokens.
• An alert without an enforcement mechanism informs but does not protect the budget.
• Longer context affects compute, memory, latency and price in different ways that must be measured separately.
• Time saved first describes capacity; its financial effect depends on what the organisation does with it.
• The Junyr Method™ connects these decisions so AI cost becomes a governed variable rather than a month-end surprise.

Going further

The free Junyr AI maturity audit provides an initial thirty-minute conversation to position the organisation, identify its main obstacle and select a sensible first project, followed by a one-page deliverable.

• Bring one use case, the available consumption data and the main cost uncertainty.
• Use the discussion to identify the priority investigation.
• Scope any detailed architecture review and quantified recommendations as a separate engagement.

For the wider method, read AI Maturity of French SMEs.

Sources: verified or consulted on 6 September 2026

[1] ISO, ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management system. URL: https://www.iso.org/standard/42001

[2] European Union, Regulation (EU) 2026/1744. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202601744

[3] European Commission, Transparency obligations under Article 50 of the AI Act. URL: https://digital-strategy.ec.europa.eu/en/faqs/transparency-obligations-under-article-50-ai-act

[4] AFNOR, General-purpose framework for frugal AI — AFNOR Spec 2314. URL: https://www.afnor.org/en/news/artificial-intelligence/reference-framework-reduce-environmental-impact-ai/

[5] Langfuse, Route OpenTelemetry to Langfuse through Azure API Management. URL: https://langfuse.com/integrations/gateways/azure-api-management

[6] OpenAI, Advancing the price-performance frontier with GPT-5.6, 30 July 2026. URL: https://openai.com/index/advancing-the-price-performance-frontier-with-gpt-5-6/

[7] OpenAI, Compare models — OpenAI API. URL: https://developers.openai.com/api/docs/models/compare

[8] DeepSeek, DeepSeek-V4-Pro GA Release — API pricing update, 13 August 2026. URL: https://api-docs.deepseek.com/news/news260813/

[9] Anthropic, Claude API pricing. URL: https://docs.anthropic.com/en/docs/about-claude/pricing

[10] Huang et al., APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs, ACL 2025. URL: https://aclanthology.org/2025.acl-long.525/

[11] Microsoft Research, LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. URL: https://arxiv.org/abs/2310.05736

Article written by Paul-Antoine TUAL, AI Transformation Leader, creator of the Junyr Method™.