Agent memory economics: 100M tool calls

OriginChainDB Team · May 6, 2026

cost agent-memory scaling ttl tutorial

TL;DR - An autonomous AI agent with 100M tool calls per month produces ~50 GB of trace data and ~75 GB of embeddings. Storage is the small line item; what dominates is recompute cost (re-embedding stale state) and retention (keeping calls around long enough to be useful but not so long they become legal liability).

The shape of the cost

For an agent doing 100M tool calls per month:

Line item	Per call	Monthly total	Cost (entry configuration)
Tool-call trace (JSON)	~500 bytes	50 GB	~$5
Embedding of trace	768-dim f32 = 3 KB	300 GB	~$30
Re-embedding (stale window)	varies	varies	~$50-200 (LLM API)
Retention sweep	-	-	included

Total OriginChainDB footprint: ~$35/month at this scale. Total LLM/embedding spend: $50-200/month depending on the re-embedding cadence.

For comparison, the LLM tool-call costs themselves (the actual inference for each call) at 100M calls × ~$0.0002 average per call = $20,000/month.

So storage and embedding are 0.2% of the cost; the LLM is 99.8%.

The lever that matters: retention

The single largest cost lever isn’t storage rate - it’s how long you keep the data.

A 100M-call/month agent over 12 months at full retention: 600 GB of traces + 3.6 TB of embeddings = ~$420/month in storage by month 12.

The same agent with 90-day retention: 150 GB + 900 GB = ~$105/month, plateauing.

The question is: do you actually need every tool call from 11 months ago? Usually not - most retrieval queries hit the recent window (last 7-30 days). After 90 days the value-per-byte drops sharply.

Per-key TTL makes this trivial: tag every trace with a 90-day TTL on write; the substrate forgets it on its own. No cron job, no “DELETE WHERE created_at <…” that locks the table.

The lever that doesn’t matter: storage rate

People obsess over per-GB cost. For agent memory it’s a rounding error.

Even at 10x our cost (some specialty databases charge $1-2/GB-month for managed storage), a 100M-call/month agent’s storage bill is ~$300/month. Compared to the $20K LLM spend, optimising the storage rate from $0.10 to $0.03/GB-month saves you ~$200/month - a 1% savings on the total bill.

Spend the optimisation budget on the LLM line.

The lever that’s invisible: re-embedding

When you ship a new embedding model (or fine-tune the existing one), the old embeddings in storage no longer match query-time embeddings. You have two options:

Re-embed everything. 300M existing vectors × $0.00002 per embedding = $6,000 one-time.
Dual-version with gradual drift. Tag each embedding with model version; query-time embeddings use the same version; old vectors expire naturally via TTL; the new version takes over the corpus over your retention window.

Option 2 is what we recommend. Combined with sensible retention, model upgrades become a no-op - you just start writing the new version and let TTL handle the old.

The dual-version pattern needs the substrate to handle multi-version embeddings cleanly. OriginChainDB’s shape versioning does this - same shape, different version, queries scope themselves automatically.

The lever that’s structural: what you store

The biggest cost reduction comes from being honest about what you’re storing.

Common bloat we see:

Storing the full LLM response, not just the tool-call decision. The reasoning trace is 10x the size of the tool decision and rarely retrievable enough to justify keeping.
Storing every intermediate state of the agent’s planning. Useful for debugging, expensive at scale. Move it to a debug-only sink with short TTL.
Storing duplicate embeddings. If the same input embeds to (approximately) the same vector, dedupe at write time via a hash key.

The cleanest production agents store:

The tool decision (what was called, with what args, getting what result).
One embedding per intent, not per call.
A foreign key to the user/session/parent-trace.

Everything else either lives in shorter-TTL debug storage or doesn’t get stored at all.

A worked example

Production agent doing 1M tool calls/day (30M/month):

Naive storage:

Full reasoning + decision + result: 5 KB/call → 150 GB/month → $15
Embedding of every call: 3 KB/call → 90 GB/month → $9
Cumulative at 12 months retention: 1.8 TB + 1.1 TB = ~$290/month plateau

Disciplined storage:

Decision only: 500 bytes/call → 15 GB/month → $1.50
Embedding of distinct intents (~10% of calls): 0.3 KB/call effective → 9 GB/month → $0.90
Cumulative at 90 days retention: 45 GB + 27 GB = ~$7/month plateau

The same agent costs 40x less per month with disciplined storage. The LLM bill is unchanged either way.

What changes at 1B calls/month

Storage line scales linearly. LLM line scales linearly. Network egress (if you stream traces somewhere) becomes visible. Retention sweep becomes a real workload - at this volume, daily compaction matters; OriginChainDB handles it asynchronously and out of the foreground request path.

The lever that becomes more important at this scale: embedding model choice. A 384-dim model uses half the storage of a 768-dim model and is often “good enough” for tool-call retrieval (which doesn’t need the same fidelity as document search). Switching halves the embedding storage line and roughly halves embedding-generation cost.

FAQ

What’s the per-call storage cost for agent memory?

At our pricing, ~$0.0000005 per call for traces + ~$0.000001 per call for embeddings. Both rounding-error compared to the LLM cost.

Should I re-embed when I switch models?

Usually no - let TTL handle it. Dual-version embeddings during the transition window; old vectors expire naturally. One-shot re-embedding is only worth it if your retention window is very long and you can’t tolerate query-time inconsistency.

How do I know when to expire traces?

Look at your retrieval queries. If 90% of them only reach the last 30 days, set TTL to 90 days. If your debugging workflows need 6 months, set it to 6 months. Don’t pick a number from a blog post - measure your own access pattern.

What’s the right embedding dimension for tool-call memory?

Smaller than you think. 384 is fine for most cases. 768 is the default we recommend; 1536 (OpenAI’s text-embedding-3-large) is overkill for short tool calls.

Can I get a cost breakdown for my own workload?

We’re happy to model it - share your call rate, average payload size, and retention window and we’ll do the math.