As enterprises continue deploying AI to increase efficiency, something strange is happening. A May 2025 Gartner survey of 506 CIOs found that 72% are only breaking even or losing money on AI investments. Some companies have even seen AI costs jump 10x from prototype to staging.
On paper, most models look cheap. Token costs are low, or at least predictable. So where is the extra cost coming from?
This is the Iceberg Effect. Token cost is just the part of AI spend that’s visible. But the unattributed costs lurking below the surface are far larger than most teams realize.
Most people picture an AI agent working like this:
Prompt comes in → Agent makes a plan → Agent calls a few tools → Agent answers back
The reality is closer to a small team iterating on a project — planning, re-planning, and calling tools speculatively. Because AI is non-deterministic, running the same prompt 10 times can produce 10 different answers reached by 10 different paths. One prompt may call three tools. The next may call 30, and each of those tool calls incurs costs. An agent may hallucinate, retry, spawn sub-agents, or fail and require a human in the loop (and repeat these steps unpredictably until the problem is solved).
In the span of a single request, costs compound beneath the surface where teams can’t see what’s happening. Here's a deep dive into the flow of an AI call, and how costs add up at each stage.
The Real Shape of an AI Call
[Infographic: Iceberg diagram with looped arrow-flow. Above the waterline: prompt hits the model ($0.02), response gets delivered ($0.01). Below the waterline: tool calls, retries, sub-agents, logging, storage — the bulk of the cost. Arrows loop back through the underwater stages to show steps can repeat. End total: $1.43 vs. $0.03 visible on the invoice — a 72x gap.]
Want the full picture of how AI economics break down at scale? Read the Whitepaper: The Financial Blind Spots in Autonomous AI.
Step 1: The Prompt Hits the Model
The prompt arrives. The model reads it, interprets intent, and decides what to do next. In practice, this could look like:
- A customer support ticket comes in, and the model drafts a response plan
- A user submits an application form, and the model decides which checks to run
- A researcher asks a broad question, and the model breaks it into search, enrichment, and scoring sub-tasks
A single model call is cheap (e.g., $5.00 per 1 million input tokens). But agents rarely make one model call per request. According to the Tokenomics Foundation, reasoning and agentic workloads use anywhere from 5x to 30x more tokens per task than a comparable chat interaction. Every tool call, retry, and re-plan sends the model another prompt to interpret what happened and decide what's next.
Cost per call: $0.001–$0.05. Multiply by how many times the model gets called in a single user request for the real number.
Step 2: The Agent Reaches Out
What makes Agentic AI so valuable is its ability to call tools to complete a task. That includes actions like:
- Pulling records and reporting data via third-party APIs
- Querying databases
- Fetching context from various sources
A single user request might trigger three or four tool calls before the agent has enough context to act. And each one has a two-sided cost:
- Token cost: When a request comes through, the model reasons before the call to decide what to ask for and which tools to use. After, it reasons again to interpret what came back and decide what's next. Those charges show up on the LLM bill as tokens.
- API call cost: Every time an agent calls an API or queries a premium database, each platform charges for the call. That lives on a completely different invoice.
The second cost is often hidden because most software your AI calls via API doesn't distinguish AI traffic from human or software traffic. No line item says, "These queries came from an AI agent." Your CRM just sees a stack of API calls and bills accordingly. Without that attribution, the true cost of an AI call becomes a black box.
Cost per call: $0.05–$5.00+. Most software lookups run pennies. Premium data APIs (credit, identity, enrichment) can cost several dollars each.
Step 3: Retries, Loops, and Error Handling
Most of the time, agents don't get it right on the first try. According to Carnegie Mellon's TheAgentCompany benchmark, the most capable AI agents could only complete real-world office tasks on their own 30% of the time. The other 70% failed or required human intervention.
When an API returns a misconfigured response or the context isn't clear enough, the agent tries again. Sometimes it tries a different tool. It might:
- Re-run the call and re-reason over the new result
- Retry hundreds of times before hitting a rate limit
- Pass a task back and forth with another agent in a feedback loop until something works
Most of the time, these retries are invisible. But this is where costs grow exponentially. A clean retry adds pennies. But it's hard to predict when a failure will turn into a loop.
One team documented a $47,000 LLM API bill caused by a loop between four LangChain-built agents that ran for 11 consecutive days before anyone noticed. The agent wasn't broken. It was working exactly as designed: retrying, reasoning, calling tools until it solved the problem.
Cost per call: $0.05–$47,000+. The range is potentially infinite without guardrails.
Step 4: The Response Gets Packaged
Once the agent has an answer, the interaction gets written to a vector store so the next query has context. An audit log captures every decision the agent made and every tool it called.
The final response gets synthesized, formatted, and delivered (through email, chat, Slack, a document, etc.).
None of this is expensive in isolation. Storage and logging are cheap. Sending an email is effectively free. But every call and retry generates its own log entries and storage writes. A vector store that grows by 50,000 entries a day is a small cost in month one, but a meaningful line item by month 12.
Cost per call: $0.005–$0.10, depending on what gets logged and stored.
AI Call Example Cost Breakdown
Let's break down how these steps might add up based on publicly available AI pricing.
Say a fintech team builds an AI-powered loan pre-approval tool. A user submits an inquiry, and the agent returns a pre-approval summary in two seconds. The cost the team could see (the token cost) was ~$0.02 per use.
The actual cost was closer to $1.67 — an 83x gap between what the invoice showed and what the workflow actually consumed.
At 10,000 monthly inquiries, the team's LLM bill would come in at around $700. But the actual AI infrastructure cost is closer to $15,000, with most of it unattributed, scattered across vendor systems. Plus, the cost of retries (here modeled conservatively) amplifies the total cost significantly.
This is just one example, and it’s not unique to fintech. Any agentic workflow with meaningful tool use can produce this kind of cost gap.
What Are Your AI Agents Really Doing?
By all accounts, AI workflows should be saving money. Automating work that used to require human time and effort should be decimating overhead. So why isn't it?
The real cost of an AI call is the sum of every step, and most of those steps don't show up on the model bill. They're buried in tool costs, vendor invoices, and storage line items no one has connected back to the workflows that triggered them.
You can't optimize what you can't see. And you can't scale AI when the economics live in seven different systems that don't talk to each other.
An AI Economic Control System closes that gap, capturing every move your agents make and tying it back to the customer, feature, or workflow that triggered it.
Want to know why your AI spend isn't as profitable as it could be? See what your agents are actually spending (for free) with Revenium.
