Is it cheaper to self-host an open-source model or keep paying per token for an API?
For most enterprises, paying per token for a vendor API is cheaper than self-hosting an open-weight model once specialized engineering staff and infrastructure are accounted for. In a composite 250-person company, buying from a vendor costs $620,000 annually, while self-hosting costs $748,000 if existing engineers are reassigned with no new salaries, and $1.47 million if three production specialists are hired.
The Arithmetic: Build vs. Buy Scenarios
When comparing vendor API costs against self-hosting for a 250-person company with 40 developers running 115 billion tokens annually, buying from a vendor costs $620,000. Building with existing engineers reassigned and no new salaries costs $748,000 (about 1.2 times buying). Building with three specialists to run the model in production costs $1.47 million (about 2.4 times buying).
Source: Build or Buy: The Arithmetic on Running Your Own Model
Specialized Staffing and Hardware Costs
The personnel required to operate a model in production drive the cost disparity. Three dedicated engineers cost $722,000, exceeding the company's entire $620,000 annual AI API bill before accounting for servers, power, hosting, or support.
Source: Build or Buy: The Arithmetic on Running Your Own Model
Where Self-Hosting Budget Goes
When an organization builds and runs its own model, people account for about half of the total spend, hardware accounts for about a third, and power accounts for 3%.
Source: Build or Buy: The Arithmetic on Running Your Own Model
Valid Non-Cost Reasons to Build
Building and hosting a model is justified in four specific edge cases: strict air-gap requirements, data sovereignty mandates, scenarios where the model itself is the core commercial product, or situations where an enterprise already owns idle hardware of that class.
Source: Build or Buy: The Arithmetic on Running Your Own Model
Also asked as
- Is running an open-weight model cheaper than paying vendor API token costs?
- What does it cost to self-host an open-source LLM versus buying tokens from an API?
- When does it make financial sense to build and host your own AI model instead of buying API tokens?
Related questions
- What key token metrics should CFOs monitor to manage AI expenditure?
- How does Olakai detect unused and idle AI SaaS seat licenses?
- How do AI spend budgets and alerts work in Olakai?
- How do you calculate ROI on enterprise AI investments?
- How do you reconcile AI spend across OpenAI, Anthropic and Google Vertex in one view?