Category: AI Strategy

AI system of record, defined: An AI system of record is the authoritative, vendor-neutral record of everything an organization’s AI does. It captures every AI interaction and outcome, from coding agents to assistants to autonomous agents, down to the token and its cost, and structures it into one data model by user, team, department, project, agent, model, and vendor, kept over time.

Every system of record a large company runs today was built on the same assumption: a person types something in. A recruiter enters a new hire into the HR system, a sales rep logs a call in the CRM, and an accountant posts a journal entry to the general ledger. Even Git, the system of record for source code, only changes when a developer commits. These records are trustworthy because people are required to feed them, and their volume is bounded by how fast people work.

AI breaks that assumption. A single engineer running a coding agent can generate thousands of model calls in an afternoon, an autonomous agent can work through a queue of support tickets overnight with nobody watching, and employees paste documents into chat assistants the company never approved. None of that activity gets typed into anything. It is scattered across vendor consoles, cloud bills, and browser tabs, and each of those sources sees only its own slice. That is why most enterprises can tell you exactly how many laptops they own, but not how many AI tokens they consumed last month, who consumed them, or what came back. An AI system of record closes that gap, and it is the first system of record that nobody types into.

What Is a System of Record?

The term is older than the cloud. In enterprise IT, a system of record is the authoritative source for a given kind of data: the one place everyone agrees is correct when two reports disagree. Geoffrey Moore popularized the contrast between systems of record, the transactional backbones such as finance, HR, and CRM that keep a business accurate, and systems of engagement, the collaborative tools people use every day; Josh Bersin’s 2012 piece in Forbes is a good primer on the distinction. Systems of record are not supposed to be exciting. Their job is to be complete, consistent, and durable, so that every report, audit, and decision downstream starts from the same numbers.

Most enterprises already run one for each thing that matters to them, and each one answers a question nobody else in the company is allowed to answer differently.

Domain	System of record	The question it answers
People	HRIS, such as Workday	Who works here, in what role, and at what cost?
Customers	CRM, such as Salesforce	Who are our customers, and what have we sold them?
Money	General ledger or ERP	What did we spend and earn, and where?
Code	Git	What changed, when, and who changed it?
AI	AI system of record	What did our AI do, what did it cost, and what did it produce?

The last row differs from the others in three ways. Its data is captured automatically rather than entered by people, so completeness depends on coverage rather than on discipline. It spans vendors by design, because no company runs all of its AI through one provider. And its basic unit is the individual interaction and the tokens behind it, which means the volume is measured in millions of events rather than thousands of transactions.

Why AI Spend Tracking and Governance Now Need a System of Record

Enterprises got through the first years of generative AI without one, mostly because the spend was small and the usage was experimental. Three shifts have ended that.

AI usage is fragmented across vendors

Most organizations believe they know what AI they run, and most are wrong. In a Cloud Security Alliance survey published in April 2026, 68% of respondents reported high confidence in their visibility into AI agents, yet 82% had discovered previously unknown agents in their environment in the past year. Part of the reason is structural, since each AI vendor reports on itself and only itself. The OpenAI admin console knows about OpenAI, the coding tool’s dashboard knows about the coding tool, and none of them can see the AI apps employees adopted on their own, which is where shadow AI lives. Adding up the vendor totals by hand gives you a number, but not one anyone can trace back to a team or a decision.

Cost moved from seats to tokens

Even GitHub Copilot, long the archetypal per-seat AI tool, has changed models: GitHub announced in April 2026 that all Copilot plans would move to usage-based billing on June 1, with credits consumed based on token usage, including input, output, and cached tokens. A seat license is predictable, and procurement can manage it with a spreadsheet. Usage-based pricing ties spend to behavior, so a handful of power users or one runaway agent can move the monthly bill more than the rest of the company combined. Falling unit prices do not fix this, because usage grows faster than prices fall, a pattern we traced in Your AI Got Cheaper. Your Bill Didn’t. Once spend is variable, it has to be recorded at the level where it varies, which is the interaction and the token.

Agents act without a human in the loop

The same Cloud Security Alliance survey found that nearly two in three organizations (65%) had experienced AI agent-related incidents in the previous 12 months. When a person uses an assistant, there is at least someone who can explain what happened. When an agent opens a pull request, resolves a ticket, or calls an external API on its own, the only account of what it did is whatever was recorded at the time. Governance, audit, and incident response all depend on that account existing, being complete, and being kept somewhere the agent’s own vendor does not control.

There is also a strategic reason underneath the operational ones. Models and compute are available to every competitor on the same terms, but the record of how your own organization uses AI, and what it gets back, has no market and cannot be bought, an argument Paul Brzozowski develops in ClaudeForce, and the One AI Input You Cannot Buy.

The Five Properties of an AI System of Record

Plenty of tools hold some AI data. What makes a record a system of record is a specific set of properties, and each one exists because a particular question becomes unanswerable without it.

1. Complete capture

Without complete capture, every number is a partial number presented as a total. The record has to cover coding agents, workforce assistants, and autonomous agents, sanctioned tools and unsanctioned ones, because the questions leaders ask are about all of their AI rather than the part that happened to be convenient to instrument. A record that sees only approved tools produces confident answers about the wrong total, which is worse than having no answer at all.

2. Cost down to the token, reconciled to the bill

Without token-level cost, AI ROI has no denominator. Multiplying token counts by list prices gives a useful estimate, but discounts, caching, and committed-use agreements all change what you actually pay. The finance-grade version reconciles usage to the provider’s real invoice, as we described for Google Vertex AI cost reconciliation, and a good record labels which figures are reconciled bills and which are estimates, so nobody mistakes one for the other.

3. One data model with attribution

Without attribution, you know what was spent but not by whom or for what. Every interaction needs to resolve to a user, a team, a department, a project, an agent, a model, and a vendor, in one consistent schema rather than one per tool. Those dimensions overlap by design: the same dollar legitimately belongs to a developer, their department, their project, and the provider all at once, which is what lets a CFO and an engineering lead look at the same record from different angles and still reconcile. That structure is what makes AI spend governance and cost attribution possible.

4. Outcomes linked to activity

Without outcomes, you are measuring activity and calling it value. Tokens are an input, and ranking teams by how many they burn rewards consumption rather than results, which is the trap we described in Tokenmaxxing Is the New Lines of Code. A system of record ties interactions to the outcomes the business cares about, such as merged pull requests, resolved tickets, or hours returned to a team, through custom KPIs the business defines itself. That link is what turns a usage log into AI ROI measurement.

5. Vendor-neutral and durable

Without neutrality and history, you cannot compare vendors or look back. A provider’s own console is structurally one-sided, since it has no reason to show you where a competitor does the same work for less. The record also has to persist: trends, month-end forecasts, and year-over-year comparisons all depend on history, and so does the audit trail that AI governance requires, particularly now that the EU AI Act is enforceable.

What an AI System of Record Is Not

The phrase is used loosely, and several adjacent categories get mistaken for it. The distinctions matter when you are deciding what to buy or build.

Often confused with	What it does	Why it is not the record
An AI dashboard	Displays charts built on some underlying data	A dashboard is a view. The record is the data itself, kept over time, that any view can be built from.
A vendor admin console	Reports usage and billing for one provider’s product	It sees one slice of your AI and has no reason to show you the rest.
LLM observability and tracing	Traces prompts, latency, and errors inside applications an engineering team instruments	It covers the apps a team chose to instrument, for engineers, with no view of workforce tools, shadow AI, or cost against outcomes.
An “agent system of record” workspace	Gives agents and people a shared place to do work on common data	That is where AI works. An AI system of record is the account of what AI did, wherever it worked.
AI on top of existing systems of record	Uses models to read and act on CRM, ERP, or HR data	That puts AI on a record. It does not keep a record of the AI.

The last row deserves a second look, because the two ideas reinforce each other. The more AI reads and writes to your CRM, your ERP, and your codebase, the more consequential its actions become, and the more you need an independent record of what it did there. The relationship with AI analytics is simpler: AI analytics is the practice of measuring AI usage, cost, and impact, and the AI system of record is the data that practice runs on.

The Hard Parts of Building One

Capturing AI activity is the tractable part. The difficulty is making the record accurate enough that finance, security, and engineering all accept it, and anyone evaluating an approach, including ours, should ask how it handles the following.

Attribution when the provider does not know the person

Provider data often identifies an API key rather than a human. OpenAI, for example, reports usage per key, so a key shared by a team or a service collapses many people into one line. Resolving identity means joining several signals, such as single sign-on identity, commit emails, and device mapping, and a trustworthy record shows the portion it could not attribute instead of quietly spreading it across everyone.

Estimates versus bills

Token counts arrive in near real time, while reconciled costs can lag by a day or more; Google’s billing export, for instance, typically trails usage by 24 to 48 hours. A record needs both: the estimate for timely alerts and the reconciled figure for the numbers that go to finance, each clearly labeled.

Seats and tokens do not add up cleanly

Workforce assistants are still mostly licensed per seat, while coding agents and model APIs increasingly bill per token, and some tools, like GitHub Copilot in 2026, switch from one to the other. Forcing both into one cost-per-interaction figure produces a number that looks precise and is not. Seat costs have to be allocated, for example pro-rated across the projects a licensed user works on, and kept distinct from metered usage.

What gets captured, and where it lives

This is the first question a CISO asks, and it should be. Detecting sensitive data in prompts requires seeing prompt content, while measuring adoption of a long tail of AI apps only needs metadata such as the app, the user, and the time. A well-designed record captures content only where it is needed, supports redaction of sensitive fields before data leaves the application, and can be deployed inside the customer’s own infrastructure when regulation or contracts require it. The record is itself sensitive data, and it has to be governed like any other system of record, a standard we hold ourselves to on our Trust & Security page.

Linking activity to outcomes

Counting tokens is mechanical, but deciding what a pull request or a resolved ticket was worth requires business definitions and assumptions, such as how much time a task used to take and what that time costs. Those assumptions should be visible and adjustable rather than buried, and ambiguous cases should be under-counted rather than over-claimed, which is how we approach detecting AI-assisted code.

What an AI System of Record Answers, and Who Owns It

One record serves several leaders, each asking different questions of the same data. These four uses are where most organizations start.

Use	Primary reader	Questions the record answers
AI ROI	Head of AI, CFO	Which AI programs return more than they cost? Which should scale, and which should stop?
AI FinOps	CFO, finance	What will AI cost by month-end, and which team, project, or model is driving the change?
AI governance	CISO, compliance	Where is sensitive data going into AI tools, including unapproved ones? Can we show an auditor what our AI did?
AI engineering productivity	VP of Engineering	Are AI coding tools shortening cycle time, and which developers have licenses they are not using?

Ownership follows the general ledger model. Finance does not own every expense, but it owns the ledger, and every department accepts the ledger’s numbers. An AI system of record works best the same way: one accountable owner, usually the CIO or Head of AI, with finance, security, and engineering as primary consumers of a shared record rather than keepers of competing ones. In practice, the right owner is whoever answers to the board for the AI program as a whole.

To check whether your organization already has one, try answering these five questions from what you have today, without starting a project:

1. What was our total AI spend last month across every vendor, and can we produce it within a day?
2. Which team, project, and model drove the largest change in that spend?
3. How many AI tools did employees use that IT never approved?
4. Which AI program produced a measurable business outcome, and what did each unit of that outcome cost?
5. If an auditor asked what a specific agent did last quarter, could we show them?

If more than one of those answers requires a spreadsheet and a week of work, the organization has AI data, but not yet an AI system of record.

Olakai: The System of Record for Enterprise AI

Workday knows your people. Salesforce knows your customers. Olakai is the system of record for your AI. We built it around three steps: capture, structure, and act.

Capture. Olakai captures every AI interaction and outcome across the organization, from coding agents to assistants to autonomous agents, down to the token and its cost. It collects from a browser extension for workforce chat assistants and shadow AI, from provider admin APIs for Anthropic, OpenAI, Cursor, GitHub Copilot, and Google Vertex AI, from coding agent hooks, from pull request analysis that needs nothing installed on developer machines, and from an SDK for the agents you build yourself. The full list is on our integrations page.

Structure. Everything is structured into one data model, organized by user, team, department, project, agent, model, and vendor, and viewed through two lenses: Olakai Agentic for AI coding tools and autonomous agents, and Olakai Assistive for the chatbots, copilots, and AI apps your employees use.

Act. Budgets, month-end forecasts with a confidence range, and policy alerts turn the record into decisions before the invoice arrives, and Kai lets anyone ask the record a question in plain language. Forecasts are projections rather than guarantees, and we label them that way. Olakai runs as SaaS or fully inside your own infrastructure, and you can see how the pieces fit together on the platform overview.

Frequently Asked Questions

What is an AI system of record?

An AI system of record is the authoritative, vendor-neutral record of everything an organization’s AI does. It captures every AI interaction and outcome, including coding agents, assistants, and autonomous agents, down to the token and its cost, and structures it into one data model by user, team, project, agent, model, and vendor.

How is an AI system of record different from an AI dashboard?

A dashboard is a view of some data. An AI system of record is the underlying data itself, captured completely, attributed consistently, and kept over time, so any dashboard, report, forecast, or audit can be built from it and every team works from the same numbers.

What is the difference between an AI system of record and AI observability?

AI observability tools trace prompts, latency, and errors inside the applications an engineering team instruments. An AI system of record covers all AI across the organization, including workforce tools and shadow AI, and connects usage to cost and business outcomes for finance, security, and engineering leaders.

Why can’t AI vendors provide their own system of record?

Each vendor sees only its own product, and most enterprises use several AI vendors at once plus tools employees adopt on their own. A vendor console also has no reason to show where a competitor does the same work for less. A system of record has to be vendor-neutral to be complete and credible.

Does an AI system of record store our prompts?

It depends on the use. Detecting sensitive data requires seeing prompt content, while measuring adoption needs only metadata such as the app, user, and time. A well-designed record captures content only where needed, supports redaction before data leaves the application, and can run inside your own infrastructure.

Key Takeaways

• It is the first system of record that nobody types into, because AI activity is captured automatically across many vendors.
• Its five properties are complete capture, token-level cost reconciled to the bill, one data model with attribution, outcomes linked to activity, and vendor neutrality with durable history.
• It is not a dashboard, a vendor console, or an observability tool, and it is the data that AI analytics, AI FinOps, and AI governance run on.

Every company will eventually keep a record of its AI, the same way it keeps one for its people, its customers, and its money. If you want to see what yours looks like, schedule a demo, and we will connect to the tools you already run and show you your own AI system of record.

From the AI ROI Series, recorded 25 August 2026. Anthropic is going public, and it is shaping up to be the largest listing in history, bigger than SpaceX, which raised $75 billion back in June. The Financial Times reports that investors are targeting $2 trillion for the company. Almost all of the coverage stops at that number, and the arithmetic underneath it points straight at your own budget, so that is where I want to go.

A headline valuation is a function of how much stock changes hands

When a company lists, it does not sell itself. It sells a small piece of itself, and that piece sets the price of everything else. SpaceX sold 4.2% to raise its $75 billion. Anthropic wants to beat that raise and clear $2 trillion. So put $100 billion into $2 trillion and you get about 5% changing hands, which means 95% of the company is being priced by the 5% that sells.

That is the Wall Street half, and I am done with it, because we are buyers here rather than investors. What I care about is what has to be true for $2 trillion to hold, since the answer to that question arrives on your invoice. Two things have to hold, and they pull against each other.

First, the revenue

Bankers expect Anthropic past $100 billion annualised by year end. They started the year at around $9 billion. That is roughly an elevenfold move inside twelve months, and the thing to be clear about is where it comes from. That revenue is enterprises buying tokens. Their revenue forecast is your AI budget, and the growth has to come out of somebody’s line item, which means the plan you are writing for 2027 is on the other side of the same equation.

It is worth being precise about what that implies, because it is easy to read as rhetoric. A vendor growing from $9 billion to $100 billion in a year is not doing it on new logos alone at that scale. A large share has to come from existing customers spending more, which is the same creeping invoice I described in the agent portfolio piece: adoption spreads, usage climbs, and the bill climbs with it, always with a good reason attached. From the vendor’s side that curve is the growth story underwriting the listing. From your side it is next year’s budget variance.

Second, the margin, and this is the fragile one

Gross margins run around 44%. The valuation is priced on 40% to 50%, sustained for years. Anything under 35% and the analysis says most of the valuation goes with it. So the whole structure rests on a band of about ten points, in a business whose input costs are being set by the compute market I have been complaining about all year.

Gross margin	What it implies
40% to 50%, sustained	The band the $2 trillion valuation is priced on
~44%	Where margins actually run today
Below 35%	Most of the valuation goes with it

Now hold that next to the capability chart

Ten frontier models, 16 points of capability between them, priced 12 times apart. The top two, the most capable and the most expensive, are Anthropic’s. Sixteen points of capability across a twelve-fold price range means the premium buys real capability, and buys it at a rate that gets harder to defend the further you get from the tasks that genuinely need it. So the rational move for any buyer is to push work down the curve wherever the job allows it, and I am seeing a great deal more of that, seriously, among our own customers.

Here is the part I think is genuinely underappreciated. Every enterprise that pushes work down the curve takes a point of somebody’s margin. The buyer behaviour that is rational for you individually is the same behaviour that presses on the one variable the valuation cannot afford to lose. That is my thesis, and it is why enterprise buyers are the variable in this arithmetic rather than the audience for it. Routing work to the cheapest model that finishes the job stopped being purely a cost tactic somewhere in the last year.

You can only make that call if you can see it

Pushing work down the curve sounds like a procurement decision and is actually a measurement one, because the phrase “wherever the job allows it” is doing all the work in that sentence. Deciding which jobs allow it means knowing, per task type, what the cheaper model finishes and what it does not, which is a question about your own workload rather than about any leaderboard. Get that wrong in the cautious direction and you pay the premium on everything forever. Get it wrong in the aggressive direction and you cut the bill while quietly degrading the output, which I have written about at some length after getting the unit wrong myself.

None of the above is actionable without visibility across all three of the places AI now runs: your coding AI, your assistive AI, and the autonomous agents most organisations are piloting for next year. One record, rather than three vendor consoles and a cloud bill you reconcile by hand in January. Capture everything, attribute it to a team, an agent, and a model, and then it can answer a question you had not thought of when you started collecting.

Not a dashboard, though. A dashboard is just a view, and a view only shows what somebody already collected, which in practice means vanilla metrics chosen before anyone knew what would matter. That distinction is the same one behind a falling rate card and a rising bill, and behind an agent portfolio nobody had split by agent. In both cases the number that mattered existed only after somebody kept the record that could produce it.

2027 budgets are being written right now

That is the window, and it closes. As always, check my math and tell me if I have this wrong, because I welcome that all day long. But the questions I would want answered before you sign next year’s number are these. Do you have one source of truth, or several? Are you capturing every AI interaction across coding, assistive, and agentic use, or only the ones a vendor happens to report to you? Can you make sense of it at scale without a project to do so? And are you collecting continuously, so that the AI ROI question can be answered with evidence rather than reconstructed under deadline?

I’m Paul, co-founder of Olakai. Olakai is the system of record for enterprise AI: one record across every tool, every agent, and every token, in your own environment. Your AI is an investment, so let’s measure it like one.

From the AI ROI Series, recorded 11 August 2026. Two headlines that week are worth translating into tokenomics. Intel asked Wall Street for $15 billion and Wall Street handed over $20 billion, its first share sale since 1971, with the reason given in the filing being general corporate purposes. Then Mark Zuckerberg, in a manifesto about abundance, called compute finite and therefore carrying an opportunity cost, which is the line most people skipped.

It is the same message from both. Compute gets more expensive from here, even while the unit price per token keeps falling on paper. Both of those are true, which is why the rate card is the wrong thing to watch.

Waste rarely looks like failure

We measure the value of AI, which means most of our time goes on measuring the waste, and the awkward thing about waste in an AI programme is that it usually looks like growth. Adoption climbs, usage climbs, and spend climbs with them, and everybody agrees it is going well. A rising bill on a good rollout and a rising bill on a wasteful one look identical from the outside. You cannot tell them apart without knowing what it costs to finish one piece of work, which is the number almost nobody has.

Eighteen months, and the number nobody in the building had seen

One of ours, no names. Eighteen months into an AI programme, adoption spreading, engineering shipping agents, coding proficiency already high, everybody pleased. Then around April the invoices started creeping as usage-based pricing kicked in. Not a spike, which somebody would have noticed. A bit more every month, always with a good reason attached, because more teams were using more of it. Somewhere upstairs, somebody finally asked what they were actually getting for it, which is the question we now get asked more than any other.

They proved it themselves, as it happens, since they are proficient users and we only assisted with the initial lift. The thesis was what it cost them to finish one piece of work. Over those eighteen months, while spend tripled and adoption climbed, the cost of finishing one piece of work went up about 40%. Remember what the rate card was doing over the same period. Every model they used got cheaper per token, and the cost to finish a job still rose 40%.

Then we split it by agent

Which nobody had done either. They had dozens running. Four of them were carrying about 70% of the measurable return. The bottom third was consuming roughly 30% of the spend and returning almost nothing. Every one of those started life as an experiment, as it should have, and that is the pattern I think is defining enterprise AI right now. Teams were told to find AI use cases, engineering did what good engineering does and built agents, tried things, shipped them. Some worked brilliantly and some did not, and not one of them was ever switched off. They did not have an AI programme so much as dozens of experiments with a budget and no way to tell which was which.

We ended the bottom tier, moved that money into the four that were working, and they got next year’s growth out of this year’s envelope. The invoices stopped creeping.

Which is the part worth keeping, because we are not here to save costs on enterprise AI. We are here to help organisations make the most out of it, and those are different jobs with different answers. This is directional and it is one company, so check it against your own estate rather than mine.

Why this is a forecasting problem

Those invoices began as a visibility problem, became a cost problem, and ended as a board question, in that order. Everyone I talk to has more agents planned for next year and nobody has fewer, and most are watching the same two numbers that company was, which will look excellent right up until somebody asks what they bought. That is survivable while compute is cheap. Intel just raised $20 billion because it will not be, and when that price moves it moves under all of it, including the agents returning nothing. Cheap tokens fund waste rather than fixing it, and at the moment they are funding the fabs too.

So the question is not whether you will run more agents next year, because you will. It is whether you will be able to say which of them earned it, which is a question about what your agents actually produced rather than about how many you shipped. The same discipline that makes a build-or-buy decision answerable makes this one answerable, and it starts from the same place: a baseline you measured rather than assumed.

Four questions to take into your next review

1. Do you know what is running in your enterprise?
2. Are you ready for what is next?
3. Are you ahead of the competition?
4. Are you making the most out of your tokens?

Those are table stakes for every AI leader through to the CFO, and they need answering with data behind them rather than with confidence. If your answer to the first one is a list of tools rather than a list of outcomes, that gap is the same one behind a falling rate card and a rising bill, and it is why measured AI ROI and the metrics underneath it are worth building before the next budget cycle rather than during it.

I’m Paul, co-founder of Olakai. Measuring what AI actually costs and what it actually returns, on your own workload, is the work I spend my days on, and I am generally happy to be told I am wrong. Schedule your AI evaluation, and we will show you your own record, in your own environment.

From the AI ROI Series, recorded 28 July 2026. Two things happened that week, and they are more connected than they look. Visa cut 2,600 jobs, about 7% of the company, and the cuts fell mostly on technology and product teams. The CEO’s memo said AI is accelerating the evolution of how work gets done, although Visa’s own people were careful to say AI was not the only reason, and I am not going to overstate it.

Hold on to which teams got cut, though, because technology and product is the exact group that burns almost all of an enterprise AI budget. The second thing is that Anthropic published a position on open-weight models after taking a beating for not signing the open letter. Somewhere between those two stories, half of LinkedIn decided the answer is to stop paying vendors and run the models yourself.

So let us do what we do here, and price it.

First, what Dario actually wrote

The version going around is not the version he wrote, so this part is worth getting right before any arithmetic. He did not call for a ban. He said it plainly: “Anthropic has never advocated for banning open-weight models.” He called open models without the dangerous capabilities a public good, and the safety testing he asked for would apply to Anthropic’s own models too. He asked for three things: keep advanced chips away from authoritarian governments, stop industrial-scale distillation, which is copying frontier models through the API, and test any sufficiently capable model before release, open or closed.

Is he neutral? Of course not, he sells closed models, and you should read him like an S-1. But if this were straight protectionism he would have backed the ban, because banning Chinese open models inside US companies is the single policy that most helps his revenue, and he turned it down. The sentence everyone quoted instead was the one about open models costing nothing besides the compute needed to run them. So we priced the compute.

What actually changed in the pricing

A developer on a flat subscription costs about $200 a month. No meter, no visibility, burn as much as you like. That same developer, doing identical work, billed by the token, costs $1,200 a month or more. Six times, for the same output. Nobody’s usage exploded, the visibility did, and that invoice is what sent everyone hunting for a cheaper answer in the first place.

The arithmetic, on a composite company

The company I carry through these episodes is 250 people, 40 of them developers, running about 115 billion tokens a year. The open-weight example is Kimi K3, the 2.8 trillion parameter model everybody points at. Three scenarios, with every assumption bent in favour of building.

Scenario	Annual cost	vs buying
Buy it from a vendor	$620,000	baseline
Build it, engineers already on payroll and reassigned, no new salaries	$748,000	~1.2x
Build it, with three specialists who can run it in production	$1.47M	~2.4x

On the metal alone it is nearly competitive at 1.2 times, which honestly surprised me. Then you put the people back and the arithmetic stops working, because you will need those people whether or not you have budgeted for them. The third scenario also carries $1.58 million of hardware on day one, locked to one model, in a market where something changes every month.

Where the $620,000 actually sits

Before anyone asks whether that figure is just the developers, no, it is all 250 people. The 40 developers burn $576,000 of it. Everybody else accounts for $44,000. So developers are 93% of the spend, which brings us back to Visa, because the teams getting cut are the teams generating almost the entire AI bill.

Where the money goes when you build	Share
People	about half
Metal	about a third
Power	3%

Power at 3% is the number I got most wrong going in, and I expected it to be much bigger personally. I do not know about you, but if anyone is selling you self-hosting on an energy argument, they have not built one, and that holds even against an 18% year-on-year increase in US electricity prices.

The one line to take to your board

Your entire annual AI bill, all 250 people, is $620,000. The three engineers you need to run the thing yourself cost $722,000. Maybe you already have them and maybe you do not, but three engineers cost more than the whole company’s AI bill, before a single server, before power, hosting, or support. All of that assumes $1,200 a month per developer, which is the assumption most likely to be wrong for your team, so run it with your own number. Push the developer usage rate as hard as you like and the metal gets you down to about 1.1 times in my analysis. The staff number never gets there.

It is worth saying that Kimi K3 is not cheap to buy either. It prices the same as Sonnet 5, and it is very capable, but cheap is the wrong word for it at the end of the day. The Wall Street Journal ran a piece the same week arguing AI pricing had peaked and that self-hosting was the era we were entering, which I found quite surprising from that masthead.

Four reasons to build, and cost is not one of them

There are four cases where building is the right call: an air gap, sovereignty, the model being your actual product, or idle hardware of this class that you already own. Those are real, and they are edge cases. Everything else is a measurement problem wearing a procurement costume. Token prices genuinely are falling, that part is true, but consumption per task is climbing faster, and the subsidised era is over. Usage-based pricing is a cash business rather than philanthropy.

What has been bothering me for weeks is the speed of the round trip: a non-expert explains enterprise AI to other non-experts, and within seventy-two hours it is an urban legend that you should just deploy Kimi K3 yourself because it is free. That is a story your uncle tells you after a few beers at the family barbecue.

What to check before you price a build

If you are chasing open weights to save money without knowing your own baseline, you have nothing to compare against, and you will not save anything because you never knew what you were spending. So the check is the boring one. Do you know your current cost per developer per month, measured rather than assumed? Do you know what share of your total AI spend sits with the 40 or so people who generate most of it, which is a question about your own engineering usage data rather than a vendor’s console? And do you know your cost per unit of output well enough that you could tell whether a build actually beat it a year from now?

Those who do not measure end up exposed, which is the same pattern behind the AI bill crowding out other budget lines and behind most of the tool sprawl I see in engineering organisations. A baseline is what makes the build-or-buy question answerable at all, and keeping that baseline across every tool and every token is the part I would put in place before a procurement exercise rather than after one. It is the same discipline that makes routing and AI ROI measurable instead of anecdotal, and it is why the metrics layer comes first.

One question, answerable from your last quarter without looking anything up: if you self-hosted tomorrow, what number would you compare the result against?

I’m Paul, co-founder of Olakai. Measuring what AI actually costs and what it actually returns, on your own workload, is the work I spend my days on. Maybe I am completely wrong here and it works for you, in which case I would genuinely like to hear about it. Your AI is an investment, so let’s measure it like one.

“Custom KPIs” sounds like a settings screen — pick a formula, name a metric, done. What’s actually interesting about Olakai’s KPI system for AI agents is the four-layer architecture underneath that screen, and specifically the parts of it you’re not allowed to customize. That restriction is the feature, not a limitation, and it’s worth understanding why before assuming more configurability would automatically be better.

Four layers, decreasing rigidity

At the base sit Raw Metrics — pure aggregations straight from event data, always visible on every agent, zero configuration required. Interaction Volume counts total prompt requests; Token Consumption sums tokens across all of them. Neither can be overridden, because there’s nothing to argue about: they’re direct counts, not judgment calls.

Above that sit Metric Slots — standardized measurement points that every new agent gets automatically provisioned with, no setup required. Each slot has an enforced output contract: a fixed unit that can never change, paired with a formula that can. Execution Cost always reports in USD, by default calculated from total tokens times cost per million tokens, with market-rate pricing applied automatically when a recognized model like Claude Sonnet or GPT-4o is detected instead of a flat default rate. Time Saved always reports in minutes, by default estimated through an AI classifier that reads the conversation and buckets it into one of five tiers, from zero minutes for a trivial exchange up to sixty for something that would have taken an hour manually — and coding-agent sessions get a purpose-built variant of that classifier with an 480-minute ceiling that reads structural signals like tool calls and files edited, because most of the real work in a coding-agent session lives in tool calls and file edits, not in the visible transcript text. Value Created always reports in USD, calculated from time saved times an hourly rate. Governance Compliance always reports as a percentage, measuring the share of interactions under a configurable risk threshold. You can change how each slot calculates its number. You can never change what unit it reports in.

Composites sit above the slots, computed automatically and not directly editable at all — their values come entirely from the slots feeding them. The flagship composite is ROI: Value Created divided by Execution Cost, expressed as a multiplier. Below 1x means the agent costs more than it saves. 1x to 5x is good, worth continued investment. Above 5x is excellent, worth expanding to new use cases. You can’t tune ROI directly — the only way to improve it is by refining the Execution Cost and Value Created slots feeding it, adjusting the underlying cost formula or hourly rate assumption rather than nudging the output number itself.

Custom KPIs sit at the top, fully open: your own formula, classifier, or LLM-based extraction, any name, any unit, any aggregation. No output contract, no restriction.

Why the restriction is the point

Raw Metrics and Metric Slots being non-fully-configurable is exactly what makes cross-agent benchmarking and portfolio-level ROI mean anything at all. Every agent’s Execution Cost reports in USD no matter how it’s calculated internally, so a Head of AI comparing thirty agents across different teams is comparing genuinely comparable numbers, not thirty differently-defined “cost” figures that happen to share a column header. Full flexibility everywhere would look more powerful in a demo and break the one thing that makes the ROI composite trustworthy at scale — the constraint is a deliberate design choice, not a missing feature waiting to be built.

Assistive IQ measures the same question a different way

This four-layer system is specifically how Agent IQ measures autonomous agents. Assistive IQ — chatbots, copilots, browser-monitored tools — answers the same executive question, “is AI creating more value than it costs,” through a genuinely different measurement pipeline, and Olakai says so directly rather than pretending it’s one unified system end to end. Assistive’s value signal comes from Advanced Analytics estimating time saved per interaction, not from a formula-slot architecture; its cost signal is app-level subscription and licensing economics, since most assistive tools are billed per seat rather than per token. Run the same shape of calculation through that pipeline — 10,000 monthly interactions, 5 minutes saved each, an $55 hourly rate, against a $2,000 monthly subscription — and you get 833 hours saved, $45,815 in value created, and a 22.9x ROI. Same ROI shape, value divided by cost, applied to a different cost basis because the underlying billing reality is different.

Assistive’s answer to “one number for executive reporting” isn’t a Productivity Score — it’s the OLA Index, a 0-100 adoption score built from user penetration, engagement depth, use-case breadth, and consistency of usage. Different math, same instinct: give a non-technical executive one trustworthy number instead of a dashboard full of raw counts.

Why the honesty is worth more than a unified story

It would be a cleaner marketing story to claim one KPI engine spans every AI use case on the platform. It would also be false, and false in a way that would eventually get caught the moment someone tried to compare an Agent IQ ROI figure against an Assistive IQ one and found the cost basis didn’t reconcile. Cost basis is the part that’s genuinely cross-cutting here — per-token billing and subscription billing both show up inside Agentic traffic and Assistive traffic alike, not neatly split one basis per product — which is exactly the kind of nuance that only survives if the documentation, and the content built on top of it, admits the system isn’t unified yet rather than smoothing over the seam.

Out-of-the-box defaults that work immediately, full customization available exactly where precision matters, and honesty about where two products still measure differently — that combination is what a genuinely business-friendly interface looks like in practice, not a slogan on a features page.

Want to see how Agent IQ’s KPI slots and Assistive IQ’s OLA Index would read against your own AI usage? Talk to an Expert.

From the AI ROI Series, recorded 21 July 2026. Apple raised prices on Macs and iPads and pointed straight at AI, saying the data centre buildout has driven memory chip costs up faster than they have ever seen and that they can no longer shield customers from it. Microsoft hiked the Xbox. Analysts expect smartphones to rise around 20% this year. Whatever else those are, they are not normal inflation.

So the AI tax has stopped being an enterprise problem and turned up in the phone in your pocket. Which leaves the only question I actually care about here: if the cost of AI is leaking all the way into a MacBook, what is it doing to your token bill?

The arithmetic nobody is connecting

Your vendor will show you a chart of the per-token sticker price trending down, and they are all doing it, and they will call that savings. The sticker did fall, and that part is true. But the moment your enterprise plan flips to pay as you go, the same work runs five to ten times more, because you are now being metered on consumption rather than on seats. Both facts are true at once, which is why the chart and the invoice disagree so violently.

The biggest labs are monetising hard on the back of it. Anthropic just passed OpenAI in business spend, almost entirely because of Claude Code, since coding is the battlefield. And the uncomfortable part, which I do not think gets said plainly enough, is that they make more money when you burn more tokens. We have all become dependent, and we have not yet seen autonomous agents running at full scale. That is the 2027 bill, and it is still coming.

Meanwhile almost nobody can answer the basic question. You have seen your invoices, so you roughly know what AI costs you, but is it working? Only about 15% of companies forecast their AI spend within 10% of reality, and most miss by more than a quarter, which means most teams are being driven by need rather than by science. That is exactly why boards are now turning to their executives and telling them to get a handle on it.

Even Google is playing the same game

Before anyone calls Google the cheap exception, look a little closer, because Google is running the smartest version of the same play. A free tier good enough to live on, Gemini bundled straight into your Workspace seats, and an API priced below cost, all to get you embedded before the meter starts to matter. The meter is still there, sitting behind the bundle for now. And when even Google has to cap Meta’s compute and tell them to use fewer tokens, the capacity ceiling and the pricing that follows it are quite real. Nobody is exempt from that, including the people selling you the exemption.

Tokenmaxxing got us here. Tokenwising is how we make it pay.

Tokenmaxxing was the 2025 story: burn everything, more is better. It was a reasonable place to end up, honestly, because it proved the value at a point when the value was still in question. 2026 is about tokenwising, which is spending like it is your own money. Here is what that looks like on a real floor, without naming names.

The pilot

A client of ours, an engineering org. Over the last 60 days we ran a pilot with about 30 developers, roughly a third of the department. The goal was never to brag about lines of code, since shipping thousands of lines of AI-written code is a vanity metric, and I have said so about token leaderboards often enough. The goal was quality: code that passes testing, ships, and moves the business.

Today about half their code is AI-written, and they want to push past 80% in the next six months. That is aggressive, it is expensive, and it is exactly the sort of thing you should not do blind. So we measured three things: who is getting real lift and who is simply burning tokens, which model each task actually needs rather than defaulting to the most expensive one available, and where the waste hides, which is in the re-prompts, the abandoned runs, and the agents quietly looping. None of that was about slowing them down. It was about making the 80% push a science project rather than a guess.

One concrete piece of it is model routing. Match the model to the task and the subtask, and stop burning your most capable and most expensive model on mundane coding work where it produces no better result. Done well, that honestly takes 40% to 50% off the bill on those tasks, which is the difference between an 80% target that pays for itself and one that quietly bleeds. Route on the wrong unit, though, and you will cut the bill while destroying the value underneath it, which is a trap I walked into publicly and corrected later.

The part I am most excited about, which is real guardrails

Here is what is actually new. Your vendors will let you set one big account-level spend limit, and at enterprise scale that is close to useless, because it does nothing to stop a single agent going rogue and eating half the quarter’s budget over a weekend. Budgets, forecasting, and visibility with alerts baked into the workflow are largely solved at this point. The hard part is enforceable limits, real ones, at the project, team, department, and even the individual-developer level. Not a warning after the money is gone, but an actual ceiling, which is the thing that lets a chief data officer sleep. It is early, and I suspect a lot of you are quietly wrestling with the same problem, so I will go deeper on it in coming episodes. Some of the mechanics are already written up in how the budgets and alerts work.

The move

Tokenmaxxing got us here, and tokenwising is how the AI transformation actually pays. The check I would run this month is narrow enough to finish in an afternoon. Do you know your cost per unit of output, rather than your cost per seat or your total invoice? Do you know which of your developers are getting genuine lift, measured against something, and which are producing volume? And if one agent ran unattended over a weekend, is there anything in your stack that would stop it, or only something that would tell you about it on Monday?

Most organisations can answer none of the three, which is the whole reason a measured view of AI ROI and a real record of what your coding tools are doing matter more this year than they did last year. The falling rate card is going to keep making the case that things are getting cheaper, and your invoice is going to keep disagreeing, and only one of those two has your name on it.

One question to take into your next standup: how much of your code is AI-written right now, and do you know whether it is making you money or simply making more code?

I’m Paul, co-founder of Olakai. Measuring what AI actually costs and what it actually returns, on your own workload, is the work I spend my days on. Schedule your AI evaluation, and we will connect to what you already run and show you your own record, in your own environment.

Gartner surveyed 782 infrastructure and operations leaders. Only 28% said their AI projects were fully meeting return-on-investment expectations. One in five — 20% — reported their AI initiatives had failed outright. The remaining majority sat somewhere in between: technically live, technically “in production,” and still unable to show the business a return anyone would call a win.

That finding, published April 7, 2026, is one of the more sobering data points to come out of enterprise AI research this year — not because it’s shocking, but because it’s precise. Gartner’s research isn’t describing a handful of failed pilots. It’s describing the median enterprise AI program: deployed, adopted, budgeted for — and still unmeasured against the outcomes it was funded to deliver.

A second Gartner study just took away the easiest excuse

A month later, on May 5, 2026, Gartner published a companion release built on a separate survey — 350 executives at companies with more than $1 billion in revenue. It found that roughly 80% of organizations piloting or deploying autonomous AI report some form of workforce reduction. On its own, that stat fuels the standard board-level story: AI is cutting costs, headcount is coming down, the investment is paying for itself. Gartner’s data says otherwise. The rate of workforce reduction was nearly identical between companies reporting high AI ROI and companies reporting flat or negative ROI. Layoffs happened either way. They just weren’t correlated with whether the AI actually worked.

That single finding dismantles a narrative a lot of executive teams have been quietly leaning on. Cutting headcount around an AI rollout isn’t evidence of AI value — it’s a budget action that companies take whether or not the underlying technology is delivering. If your board is citing reduced headcount as proof your AI investment is working, Gartner’s own data says that proof doesn’t hold. The 28% of companies fully realizing ROI aren’t the ones who cut the most people. They’re the ones who can actually show what changed.

The pattern isn’t unique to Gartner’s sample

PwC’s 29th Global CEO Survey, published in January 2026, surveyed 4,454 CEOs across 95 countries and landed on a strikingly similar shape of problem. Fifty-six percent of CEOs report zero revenue or cost benefit from their AI investments to date. Only 12% report benefiting on both fronts — revenue and cost — at once. Two different research firms, two different survey populations, and the same structural story: a small minority of enterprises can point to AI value with confidence, and a majority cannot, despite comparable or larger spend. We’ve written before about a related but distinct data point — the enterprise AI revenue gap that Deloitte’s own research surfaced — and this Gartner/PwC pairing confirms it’s not an anomaly specific to one vendor’s survey methodology. It’s the default outcome when AI adoption outpaces AI measurement.

What separates the 28% from everyone else is not a better model, a bigger budget, or a more ambitious use case. Gartner’s research points to something less exciting and far more fixable. Among the I&O leaders who reported failure, the dominant root cause was misaligned expectations — leadership assumed AI would immediately automate complex tasks or produce cost reductions on a timeline the technology was never going to meet. Among those who reported success, the top two factors were integrating AI into existing workflows rather than bolting it on as a parallel process, and securing full executive support before and during the rollout, not just at launch. Neither of those factors requires a different AI vendor. Both require a measurement layer that tells leadership, in real time, whether expectations and reality are converging or diverging.

Why “run another pilot” isn’t the fix

The instinctive response to a disappointing AI rollout is usually to relaunch it — a new pilot, a new vendor, a new proof of concept scoped more carefully this time. That instinct is understandable and, per Gartner’s own root-cause data, largely misdirected. The 72% of organizations not seeing full ROI don’t have a pilot problem; they have a visibility problem. They can’t see, in any unified way, which teams are using AI productively, which usage is idle license spend, where the workflow integration succeeded, and where it quietly reverted to the old process the day nobody was watching. Our own research on structured, time-boxed pilots — see the 30-day AI pilot framework — makes a version of this same point: the pilot itself isn’t usually the failure point. The failure point is what happens after the pilot, when nobody is instrumenting the rollout against a defined success bar.

This is precisely the gap Olakai was built to close. Olakai is a vendor-neutral Enterprise AI Intelligence Platform — a measurement layer that sits above whatever AI tools, agents, and copilots an organization already has, rather than replacing any of them. It doesn’t require betting on a different model or ripping out an existing rollout. It requires instrumenting the AI that’s already live: which teams are using it, what outcomes it’s producing against the KPIs that actually matter to the business, and where the gap between expectation and reality is widening instead of closing. That’s the system of record enterprises are missing — and it’s exactly the layer that turns Gartner’s root-cause findings into an operating discipline instead of a postmortem.

What this means for the CFO’s office

For a CFO, these two Gartner releases together are close to a mandate. The first says most AI spend under your purview is not clearing the ROI bar the business case promised. The second closes off the one metric finance teams have been quietly using as a proxy for success — headcount reduction — because Gartner’s data shows that number moves the same way whether or not the AI is actually working. That leaves finance with a harder but more honest question: not “did we cut costs somewhere near the AI rollout,” but “can we show, tool by tool and team by team, what this specific AI investment returned.” Answering that question requires the same instrumentation an engineering leader would want for infrastructure spend — usage data, adoption data, outcome data, tied to the specific KPIs the board actually cares about, not vanity metrics like prompt volume or seat counts. We built Olakai’s CFO use case around exactly that requirement, because “we reduced headcount” is not a board-defensible ROI answer anymore, and after May 5, 2026, most CFOs know it.

None of this is really an argument against AI investment. Gartner’s own root-cause data says the fix is inexpensive relative to the AI spend itself: align expectations up front, integrate into existing workflows instead of running parallel processes, and keep executive sponsorship active past the launch date. The organizations getting this right aren’t spending dramatically more than the ones getting it wrong — they’re measuring more precisely. We saw a similar pattern in our review of 100+ AI agent deployments: the deployments that scaled were rarely the ones with the most sophisticated technology. They were the ones with a clear, agreed-upon definition of what success looked like before the rollout started, and a way to check that definition against reality every month, not just at the annual budget review.

The choice in front of most enterprises right now

Gartner’s numbers describe where most enterprises already are: 72% short of full ROI, 20% at outright failure, and a workforce-reduction number that no longer means what boards have been telling themselves it means. None of that is a verdict on AI technology. It’s a verdict on the absence of a measurement layer sitting between the AI tools an enterprise buys and the outcomes it’s actually able to prove. Our own AI ROI page lays out what that measurement discipline looks like in practice — the KPIs, the adoption tracking, the governance tie-in — because the fix Gartner’s research points to isn’t a new pilot or a headcount announcement. It’s visibility into the AI you already have.

Enterprises now have a clear choice, and Gartner just made it a quantified one. Build the measurement layer now, while the gap between the 28% and everyone else is still closeable with better instrumentation rather than a strategy reversal — or keep operating on faith, keep citing headcount numbers the board can no longer treat as proof, and end up counted among the 72% a year from now when the next survey runs. Build this with Olakai, or explain to your board next year why your AI program is still one of the 72%.

Cursor built a CFO council this week, and quietly proved a thesis Olakai has been on for months.

The quick version of the week first: Microsoft cut another wave of jobs largely to fund its AI bet, continuing the pattern we broke down in why the accountability bar for AI spend just went up. The big labs kept softening their 2025 predictions about half of knowledge-worker jobs vanishing. xAI shipped Grok 4.5, OpenAI pushed a new ChatGPT release after a federal review, and — the one that drew a smile around here — Elon Musk publicly called Anthropic “obviously the leader in AI right now,” adding “I was clearly wrong,” a rival conceding the lead in writing days after launching his own model.

But none of that is the story worth unpacking today.

Cursor built a CFO council

This week Cursor, the AI coding tool, launched a CFO Council, a working group for chief financial officers, and published a stack of data alongside it. Sit with that for a second: a coding tool built a forum for finance leaders.

Here is why it matters. Cursor, more than almost any other AI coding vendor, turned coding into a metered cost. It was early and aggressive on usage-based pricing — its top consumer tier taps out around forty dollars a month, and past that, users are into custom, usage-based territory fast, even as individuals. Cursor made tokens a line item, and now it is walking straight into the CFO’s office to help make sense of the bill it helped create.

That is the point we have been making for months: the responsibility for winning at AI, the tokens, the consumption, the return, has moved into the CFO’s office. Coding used to be the CTO’s world, engineering’s world, and finance stayed out of it. Not anymore. Cursor just pulled the CFO directly into the coding room, in public, because somebody has to answer for the spend — a shift we mapped out in detail for what CFOs need from AI ROI reporting.

The data is fascinating, and a warning

Cursor’s headline number, published on its CFO Council blog: companies in the top quintile of token usage saw 16.5% year-over-year revenue growth, versus 5.1% for the bottom quintile. Use more tokens, grow faster — and AI genuinely does create real value. But that is a correlation, seen from thirty thousand feet. High token usage lining up with high revenue growth does not mean the tokens caused the growth. A thousand other things drive a company’s top line. What is missing in between is measurement: the step-by-step evidence that connects a token to an outcome.

Cursor’s own Developer Habits Report makes the case just as clearly from the other direction. The top 1% of users generate 46 times more AI-assisted code per day than the median developer. Value is wildly concentrated, so “we used a lot of tokens” tells a finance team almost nothing on its own. The real questions are who, what, and did it ship. Cursor’s data also shows cost per agent request swinging nearly nine times across model families — from roughly $1.57 on the most expensive model down to $0.18 on the cheapest — so the identical request is priced completely differently depending on where it gets routed, the exact dynamic we unpacked in why acceptance rate is the wrong metric for coding-tool ROI.

Put it together, and Cursor has accidentally proven the thesis it did not set out to prove. The correlation is interesting. It is not proof. Proof comes from measurement — who, what, and why, token by token, tied to what actually shipped. That is the game for 2026 and 2027.

What this means beyond coding tools

Cursor is a coding-specific example, but the same trap applies to every AI vendor a company runs, from customer-support copilots to autonomous agents handling multi-step workflows. A vendor’s dashboard will always show the metrics that make the vendor look good. A quintile chart on revenue growth is a marketing asset for Cursor, not an ROI audit for the buyer. The only way to get an honest answer is a measurement layer that sits above any single vendor, pulling cost-per-outcome data across every tool a company runs, which is precisely the gap custom KPI tracking is built to close — visibility a vendor’s own blog post will never hand over voluntarily.

The read

The headline this week is simple: the CFO just walked into the coding room, and Cursor held the door open. The number that should stick, though, is not the 16.5% versus 5.1%. It is the fact that Cursor felt it needed to build a CFO council at all. When the company selling the tokens starts speaking finance’s language unprompted, that is the clearest signal yet that unmeasured AI spend has become too large a line item to leave unmanaged, and vendor-neutral proof, not vendor-supplied correlation, is what the CFO’s office actually needs.

One question worth asking internally: is the CFO already in the AI tokens conversation at your company, or still on the outside looking in?

Talk to an Expert →

In May 2026, Microsoft’s Experiences + Devices division quietly pulled Claude Code licenses from its engineers. The reason wasn’t performance. Token billing had reportedly climbed to roughly $2,000 per engineer per month, and nobody inside the division had seen it coming until the invoice did. Weeks earlier, Uber had burned through its entire 2026 AI coding budget in four months flat, after adoption across its 5,000-engineer org surged from 32% to 84% almost overnight, with its heaviest users individually costing the company $2,000 a month. Both stories made headlines for the same reason: the bill was a surprise. Neither company lacked data from its AI coding vendors. Each had a perfectly good dashboard — for one tool.

That’s the part that should worry every VP of Engineering reading this. Uber and Microsoft aren’t outliers because they use AI coding tools aggressively. They’re outliers because their overruns became public. Jellyfish’s 2026 AI Engineering Trends report, which analyzed more than 20 million pull requests across 700+ companies and 200,000+ engineers, found that Claude Code, Gemini Code Assist, and GitHub Copilot now cluster within nine points of each other at the top of enterprise adoption, with twelve more tools trailing close behind. A year earlier, Copilot alone held a commanding 42% share. That era is over. The modern engineering org doesn’t pick a coding assistant. It accumulates several, one team at a time, until nobody in leadership can name all the tools running against the company’s codebase — let alone say what each one costs, who’s actually using it, or whether it’s paying for itself.

Sprawl is the default, not the exception

It’s tempting to treat “which AI coding tool should we standardize on” as the strategic question. It isn’t, anymore. Claude Code lands with one team because a senior engineer swears by its planning mode. Cursor spreads through another because it’s the fastest way to onboard a new hire onto an unfamiliar repo. GitHub Copilot ships by default because it’s bundled into the existing GitHub Enterprise contract. Codex creeps in through a few engineers experimenting on side projects. Gemini Code Assist arrives bundled with a Google Workspace renewal nobody scrutinized closely. None of these adoptions individually looks like a decision worth escalating. Collectively, they add up to an organization running four or five AI coding vendors with zero shared measurement layer between them — which is precisely the fragmentation problem we’ve written about at the platform level in what agentic AI actually means for the enterprise, and precisely the gap Olakai was built to close.

We’ve written before about the harder problem of proving that any single AI coding tool is generating value rather than just generating code, and about why acceptance rate is the wrong metric to chase when you’re evaluating one vendor in isolation — see Your AI Coding Tools Are Generating Code. Are They Generating Value? and AI Coding Tool ROI: Why Acceptance Rate Is the Wrong Metric. Those posts assumed a single tool as the unit of analysis. This one doesn’t. The question enterprises are actually facing in mid-2026 isn’t “is Copilot worth it” — it’s “we run five of these, and I have five different answers to that question, none of which use the same metric, currency, or time window.” That’s a portfolio problem, and native vendor dashboards were never built to solve it. Copilot’s dashboard sees Copilot. Cursor’s admin panel sees Cursor. Neither will ever tell you which tool your best engineers are quietly switching away from, or which team is paying triple the per-seat cost of another team doing comparable work.

Nobody is actually measuring this — even one tool at a time

Before an organization can worry about comparing five AI coding tools, it has to be tracking metrics on any of them, and most aren’t. Jellyfish’s same report found that only 46% of organizations are actively tracking AI-specific metrics at all — adoption, acceptance rate, model usage, anything. That’s not 46% tracking consistently across every vendor in use. That’s 46% tracking anything, from any vendor, in any form. The other 54% are running a multi-vendor AI coding program on instinct: a sense that “the team seems to like Cursor” or “we haven’t heard complaints about Copilot,” with no underlying data to confirm or contradict it.

Layer cost onto that visibility gap and the picture gets worse. Research from DX covering more than 400 organizations found blended per-developer spend across tiers and tools now running $200 to $600 a month, with agentic token consumption alone sometimes reaching $200 to $2,000-plus per engineer per month depending on usage intensity — the same range that blindsided Microsoft’s E+D division. And forecasting that spend is failing broadly, not just at the companies that make the news: a Mavvrik/Benchmarkit survey of 372 enterprises found only 15% forecast AI costs within 10% of actual, while nearly one in four miss by more than 50%. Multiply that forecasting failure across four or five tools running in parallel, each billed differently, each reported through a different console, and “surprise” stops being a risk and starts being the expected outcome.

What a unified view actually requires

Solving this isn’t a matter of asking engineering managers to check five dashboards instead of one and mentally reconcile the numbers. It requires a measurement layer that sits above every vendor and normalizes what each one reports into a single, comparable view. That’s the specific gap Olakai Agentic is built to close: a vendor-neutral analytics and governance layer across Claude Code, Cursor, GitHub Copilot, Codex, Gemini Code Assist, and whatever the team adopts next, without requiring the organization to standardize on one vendor first.

Concretely, that means three things a single-vendor dashboard structurally cannot give you. First, cost-per-PR comparison across tools on common ground — not Cursor’s definition of a productive session next to Copilot’s definition of an accepted suggestion, but one measurement standard applied consistently, so a VP of Engineering can see that Team A’s tool costs three times what Team B’s does for comparable throughput and ask why. Second, adoption cohorts that span vendors, showing who’s actually using what — the power users worth studying, the licenses sitting idle regardless of which tool issued them, and the teams quietly switching tools without anyone approving the shift. Third, budget forecasting that aggregates spend across every provider into one number the CFO can trust, with alerts before a team’s token usage on any single tool turns into the kind of invoice that ends a pilot. This is the same measurement-layer thinking behind our Analytics & Custom KPIs capability, applied specifically to the vendor sprawl that now defines every engineering org’s AI coding stack.

For a VP of Engineering, the practical shift is to stop evaluating AI coding tools one procurement cycle at a time and start treating the portfolio itself as the thing to manage. That means asking which teams are using which tools before the next contract renewal, not after a token bill forces the conversation; it means comparing cost-per-outcome across vendors on the same axis instead of trusting each tool’s self-reported acceptance rate; and it means building budget alerts before adoption surges the way Uber’s did, not after. Olakai Agentic is purpose-built for exactly that workflow, and it’s the specific reason we built a dedicated page for engineering leadership — see Olakai for VPs of Engineering for how the cross-tool view maps to the decisions this role actually has to make.

None of this requires an organization to consolidate down to one AI coding tool, and for most engineering teams that wouldn’t even be the right call — different tools genuinely suit different workflows, and forcing a single vendor sacrifices real productivity gains for the sake of simpler reporting. The fix isn’t fewer tools. It’s a unified measurement layer across every coding tool the organization already runs, so the next $2,000-a-month surprise shows up on a dashboard weeks before it shows up on an invoice.

If your engineering org is running three, four, or five AI coding tools right now with no shared view across them, that’s not a future governance project — it’s the state of your AI spend today, and it’s already accumulating risk you can’t see. Talk to an Expert to see how Olakai Agentic brings every AI coding vendor into one measurement and governance layer.

Two weeks of writing about AI token economics kept leading to the same corner: you cannot control what the vendors charge, only how wisely you spend it. That is the entire case for model routing, and this week it got a perfect teaching example.

On Tuesday, Anthropic launched Claude Sonnet 5, and the whole pitch fit in one sentence: near-Opus performance, at a fraction of the price. Sit with that for a second, because that sentence is the entire case for routing, stated by a frontier lab about its own model lineup.

How the pricing actually works

You pay per token, split into input (what you send) and output (what the model writes back), and output is the expensive side, usually about five times the input rate. Here is the current Claude ladder, per million tokens.

Model	Input	Output	Notes
Haiku 4.5	$1	$5	Fastest, cheapest current tier
Sonnet 5 (intro)	$2	$10	Through Aug 31, 2026
Sonnet 5 (standard)	$3	$15	After Aug 31
Opus 4.8	$5	$25	Premium, the common default
Fable 5	$10	$50	Top tier, twice Opus

Two more levers sit on top of that ladder: batch processing takes 50% off, and prompt caching takes up to 90% off the input you reuse. Keep both in your back pocket.

Now, Sonnet 5 specifically. Anthropic’s own benchmark numbers put it close to Opus 4.8, roughly 63 versus 69 on agentic coding and basically tied on knowledge work, at about 40% of the price. That is the headline, and it is real. Here are the two things worth checking before anyone lets a vendor’s pricing banner do the talking. Sonnet 5’s new tokenizer turns the same input into up to 35% more tokens, so a slice of that discount comes right back. And the two-dollar rate is an introductory price that reverts to three and fifteen at the end of August. Real savings are real, you just calculate them on tokens, at the price you will actually pay in September, not the launch banner.

The move: match the model to the task

Here is the whole idea, and it is almost embarrassingly simple. Most AI calls never needed the top model in the first place. Across the deployments we see at Olakai, somewhere between 60 and 80% of the work, the summarizing, the extracting, the routine code, gets handled just as well by a model that costs five or ten times less. This is not a hunch. Researchers at UC Berkeley, Anyscale, and Canva published peer-reviewed routing work (RouteLLM, presented at ICLR 2025) showing roughly 85% cost savings while holding 95% of frontier-model quality, and in practice a well-chosen model pair lands around half the cost at about 98% of the quality. The reason it works is simple: most teams were overpaying on the easy stuff the whole time.

Being a smarter token manager is just this: send each task to the cheapest model that can actually do it, and save the expensive model for the work that truly needs it.

How it plays out with AI coding agents

Coding is where the token bill actually lives for most engineering orgs, so it is worth making this concrete with three scenarios that show up constantly in AI coding tool deployments.

The planner and the executors. A coding agent is not one thing. It is a planner that decides the approach, and a swarm of executors that do the grunt work: writing boilerplate, generating tests, fixing lint, editing files. The judgment lives in the planner, so give it the best model available. The executors are mostly routine, and they run over and over across a long session, which is exactly where tokens pile up. Point the executors at Haiku or Sonnet 5 instead of Opus, and the build gets dramatically cheaper with no drop in the quality that matters. One measured example: a 14-million-token build came in 57% cheaper with the executor on Haiku 4.5 instead of Opus 4.8, and the planner never changed.

Route by difficulty. Not every ticket is hard. Renaming variables, scaffolding a test, a simple endpoint, a formatting pass, that is easy work, and it should go to Haiku. A feature or a mid-size refactor is Sonnet 5 territory. The gnarly stuff, tricky architecture, a subtle concurrency bug, security-sensitive code, is where it makes sense to spend on Opus. On most engineering teams the easy and medium buckets make up the vast majority of tickets, which means most traffic should never touch the frontier model at all.

Watch the loops. Agentic coding burns tokens in a way chat never did, because agents retry, re-prompt, and loop, and every wasted token in turn one gets paid for again on every turn after it. A single long, unoptimized Opus session can run twenty dollars or more; the same session, routed and cleaned up, can be two or three. Multiply that across a twenty-developer team running dozens of sessions a day, and the difference is a five-figure monthly bill that is mostly avoidable. Route the routine sub-steps down, and cap the loops so one stuck agent cannot run up the tab.

The same logic holds outside of code. Summarizing a long document costs about eleven cents on Haiku, fifty-five cents on Opus, and a dollar-ten on Fable, for the same summary. Run a million of those a month and that is a hundred and ten thousand dollars against well over a million. Burning the most capable model on a routine document summary is paying ten times over for an answer nobody can tell apart from the cheaper one.

The catch, and it is the important one

Routing is not free money, and it is not fire-and-forget. The classic way it bites: a team builds a router, cuts the bill 40%, finance is thrilled. Then the provider quietly tweaks the cheap model, a quality check starts failing, and the router silently sends everything back to the most expensive model. The next bill triples. Nothing errored. Nothing alerted. Savings only ever count net of quality, and a weaker answer that triggers retries and manual cleanup can quietly eat the very savings it created.

Routing without measurement saves money right up until it costs more than it saved. Doing it properly takes three things running underneath the router: cost per outcome for each model, not just the total bill, so a route can be proven to actually pay off; a live watch for silent escalation and quality drift; and enforceable limits around the whole system, so a misrouted or runaway job cannot eat the quarter’s budget before anyone notices. This is precisely the layer we built Agent IQ to sit on top of, and it is why cost-per-outcome tracking, not just total spend, is the metric that matters. Visibility tells a team it happened. A limit stops it before it does.

What this means for the CFO conversation

For a finance leader watching AI coding spend climb, routing is the single highest-leverage lever available before the next budget review, but only if someone can show the receipt. “We switched to a cheaper model” is not a number. “We cut cost per accepted line by 40% while holding acceptance rate flat” is. That distinction is the difference between a CFO who trusts the next AI budget request and one who starts asking for a moratorium, a pattern we have written about in why acceptance rate is the wrong metric on its own for judging coding-tool ROI.

The playbook

This is what leading an AI transformation actually looks like in 2026. Not chasing the biggest model. Matching the model to the task, measuring that the swap held quality, and proving the savings with a number a CFO can defend in a board meeting. That is the vendor-neutral measurement layer Olakai exists to provide: one place to see cost per outcome across every model and every vendor, not a router’s word for it. That is how a team ships more, spends like it is its own money, and walks into the next budget review with the receipt instead of an excuse.

One question worth taking into the next architecture review: what share of your AI calls hit your most expensive model by default, and do you actually know whether they needed to? If the honest answer is “we’re not sure,” that is the gap talking to an Olakai expert is built to close.

Talk to an Expert →