“Custom KPIs” sounds like a settings screen — pick a formula, name a metric, done. What’s actually interesting about Olakai’s KPI system for AI agents is the four-layer architecture underneath that screen, and specifically the parts of it you’re not allowed to customize. That restriction is the feature, not a limitation, and it’s worth understanding why before assuming more configurability would automatically be better.
Four layers, decreasing rigidity
At the base sit Raw Metrics — pure aggregations straight from event data, always visible on every agent, zero configuration required. Interaction Volume counts total prompt requests; Token Consumption sums tokens across all of them. Neither can be overridden, because there’s nothing to argue about: they’re direct counts, not judgment calls.
Above that sit Metric Slots — standardized measurement points that every new agent gets automatically provisioned with, no setup required. Each slot has an enforced output contract: a fixed unit that can never change, paired with a formula that can. Execution Cost always reports in USD, by default calculated from total tokens times cost per million tokens, with market-rate pricing applied automatically when a recognized model like Claude Sonnet or GPT-4o is detected instead of a flat default rate. Time Saved always reports in minutes, by default estimated through an AI classifier that reads the conversation and buckets it into one of five tiers, from zero minutes for a trivial exchange up to sixty for something that would have taken an hour manually — and coding-agent sessions get a purpose-built variant of that classifier with an 480-minute ceiling that reads structural signals like tool calls and files edited, because most of the real work in a coding-agent session lives in tool calls and file edits, not in the visible transcript text. Value Created always reports in USD, calculated from time saved times an hourly rate. Governance Compliance always reports as a percentage, measuring the share of interactions under a configurable risk threshold. You can change how each slot calculates its number. You can never change what unit it reports in.
Composites sit above the slots, computed automatically and not directly editable at all — their values come entirely from the slots feeding them. The flagship composite is ROI: Value Created divided by Execution Cost, expressed as a multiplier. Below 1x means the agent costs more than it saves. 1x to 5x is good, worth continued investment. Above 5x is excellent, worth expanding to new use cases. You can’t tune ROI directly — the only way to improve it is by refining the Execution Cost and Value Created slots feeding it, adjusting the underlying cost formula or hourly rate assumption rather than nudging the output number itself.
Custom KPIs sit at the top, fully open: your own formula, classifier, or LLM-based extraction, any name, any unit, any aggregation. No output contract, no restriction.
Why the restriction is the point
Raw Metrics and Metric Slots being non-fully-configurable is exactly what makes cross-agent benchmarking and portfolio-level ROI mean anything at all. Every agent’s Execution Cost reports in USD no matter how it’s calculated internally, so a Head of AI comparing thirty agents across different teams is comparing genuinely comparable numbers, not thirty differently-defined “cost” figures that happen to share a column header. Full flexibility everywhere would look more powerful in a demo and break the one thing that makes the ROI composite trustworthy at scale — the constraint is a deliberate design choice, not a missing feature waiting to be built.
Assistive IQ measures the same question a different way
This four-layer system is specifically how Agent IQ measures autonomous agents. Assistive IQ — chatbots, copilots, browser-monitored tools — answers the same executive question, “is AI creating more value than it costs,” through a genuinely different measurement pipeline, and Olakai says so directly rather than pretending it’s one unified system end to end. Assistive’s value signal comes from Advanced Analytics estimating time saved per interaction, not from a formula-slot architecture; its cost signal is app-level subscription and licensing economics, since most assistive tools are billed per seat rather than per token. Run the same shape of calculation through that pipeline — 10,000 monthly interactions, 5 minutes saved each, an $55 hourly rate, against a $2,000 monthly subscription — and you get 833 hours saved, $45,815 in value created, and a 22.9x ROI. Same ROI shape, value divided by cost, applied to a different cost basis because the underlying billing reality is different.
Assistive’s answer to “one number for executive reporting” isn’t a Productivity Score — it’s the OLA Index, a 0-100 adoption score built from user penetration, engagement depth, use-case breadth, and consistency of usage. Different math, same instinct: give a non-technical executive one trustworthy number instead of a dashboard full of raw counts.
Why the honesty is worth more than a unified story
It would be a cleaner marketing story to claim one KPI engine spans every AI use case on the platform. It would also be false, and false in a way that would eventually get caught the moment someone tried to compare an Agent IQ ROI figure against an Assistive IQ one and found the cost basis didn’t reconcile. Cost basis is the part that’s genuinely cross-cutting here — per-token billing and subscription billing both show up inside Agentic traffic and Assistive traffic alike, not neatly split one basis per product — which is exactly the kind of nuance that only survives if the documentation, and the content built on top of it, admits the system isn’t unified yet rather than smoothing over the seam.
Out-of-the-box defaults that work immediately, full customization available exactly where precision matters, and honesty about where two products still measure differently — that combination is what a genuinely business-friendly interface looks like in practice, not a slogan on a features page.
Want to see how Agent IQ’s KPI slots and Assistive IQ’s OLA Index would read against your own AI usage? Talk to an Expert.