AI feature cost at 100K to 10M users
TL;DR - An AI feature’s bill is roughly 95% LLM tokens, 3% embedding generation, and 2% database. A personalized feed lands near $0.015 per user per month at 100K, 1M and 10M MAU alike, because the dominant line scales linearly with users. Pick the smallest model that works before you optimize anything else.
Why this matters
AI features look free in the prototype phase. You’re paying $20/month for an OpenAI key, $50/month for a small managed Postgres, and your “vector store” is a pickle file on disk. The unit economics work because there are no users.
Then you ship. By the time you have 100K MAU, the math has shifted: the OpenAI bill is $5K, the database is $300, and the vector sidecar is another $200. By 1M MAU you’re at $50K/month if you didn’t think about caching. By 10M MAU it’s a real budget line.
Most of that pain is avoidable with engineering decisions you make before you scale, not after.
The model: a personalized content feature
Each user gets a personalized “for you” feed:
- 50 user actions per day generate signal (views, clicks, dismisses)
- 1 personalization read per session, ~3 sessions per user per day
- ANN search returns top 20 candidates from a 1M-item corpus
- LLM re-ranks the top 20 to produce the final 10
- User profile embedding is refreshed once per day if signal accumulated
That’s the per-user shape. Multiply by MAU.
Per-user daily numbers
| Operation | Count/day | Notes |
|---|---|---|
| Signal writes | 50 | small JSON, 100 bytes |
| Profile reads | 3 | 1KB per read |
| Profile embeddings refresh | 1 | 768-dim f32 = 3KB |
| ANN searches | 3 | top-20 from 1M corpus |
| LLM re-rank calls | 3 | ~2K input tokens, ~200 output |
| Hydrate top-10 reads | 30 | 30 batched gets |
Storage growth:
- 50 signal records/day × 100 bytes = 5KB/day per user
- 1 embedding/day × 3KB = 3KB/day per user
- ~8KB/day per user total
Costs at 100K MAU
| Line item | Volume | Unit cost | Monthly |
|---|---|---|---|
| LLM re-rank (Sonnet) | 9M calls | ~$0.0006 each | $5,400 |
| LLM re-rank (Haiku) | 9M calls | ~$0.00015 each | $1,350 |
| Embedding generation | 3M new vectors | ~$0.00002 each | $60 |
| Database I/O (OriginChainDB, entry configuration) | All operations | flat monthly | ~$50 |
| Database storage (24GB cumulative) | OriginChainDB, entry configuration | flat monthly | (included) |
Totals:
- With Sonnet: ~$5,500/month → $0.055 per user/month
- With Haiku: ~$1,460/month → $0.015 per user/month
Notice: the LLM is 95% of the bill. Storage is rounding error. Picking the smallest sufficient model is the highest-leverage decision you’ll make.
Costs at 1M MAU
Roughly 10x the volume. The interesting wrinkle is which line items scale linearly and which don’t.
| Line item | Volume | Monthly |
|---|---|---|
| LLM re-rank (Haiku) | 90M calls | ~$13,500 |
| Embedding generation | 30M new vectors | ~$600 |
| Database I/O (OriginChainDB) | Higher tier needed | ~$400 |
| Database storage (240GB cumulative) | Higher tier | ~$200 |
Total: ~$14,700/month → $0.015 per user/month
The per-user cost doesn’t change much from 100K to 1M because the LLM dominates and scales linearly. This is good: you can model what 10M will cost without a calculator.
The database lines do tier up - at 1M MAU you’re past the smallest plan. But the database is still <5% of the bill.
Costs at 10M MAU
| Line item | Volume | Monthly |
|---|---|---|
| LLM re-rank (Haiku) | 900M calls | ~$135,000 |
| Embedding generation | 300M new vectors | ~$6,000 |
| Database I/O | Enterprise tier | ~$2,500 |
| Database storage (2.4TB) | Enterprise tier | ~$1,500 |
Total: ~$145,000/month → $0.0145 per user/month
At this scale, the things to optimize:
- Cache LLM responses for repeated input shapes. Often 30-50% of re-rank calls have near-duplicate input.
- Embed less often. If a user’s signal hasn’t shifted meaningfully, skip the daily embedding refresh.
- Move re-rank to a cheaper model. Haiku → smaller open-weight model on owned infrastructure.
Each optimization is a 10-30% cost win. Stack them.
What dominates
Across all three scales:
- LLM tokens (~95%) - the bottleneck.
- Embedding generation (~3%) - small but real.
- Database (~2%) - almost free.
Three implications:
- Don’t optimize the database first. It’s not your bottleneck. A correct database with a wrong LLM is expensive; the reverse is fine.
- Pick the smallest model that works. Haiku vs Sonnet is a 4x cost delta on the dominant line item. Test it.
- Cache aggressively at the LLM boundary. Even simple input-hash caches save 20-30%.
What changes the model
The numbers above assume one application pattern. Reality varies:
- Heavier reasoning (multi-step agent loops): LLM cost goes up 5-10x. Database stays the same. The ratio gets worse.
- Lighter generation (single classification, no re-rank): LLM cost drops 90%. Database becomes 20-30% of the bill. Database choices matter more.
- Higher write volume (writing every tool call): Database I/O scales meaningfully. OriginChainDB’s commit-window throughput shows up here.
Always model your own application before generalizing.
The OriginChainDB angle
This post isn’t a sales pitch - the database is 2% of the bill. But the atomic multi-shape write in OriginChainDB does eliminate one cost line that other architectures pay: the dual-write infrastructure (streams, sync workers, reconciliation jobs) between a primary DB and a vector DB. That’s typically a $200-2000/month line item depending on volume - modest, but it’s real engineering time too.
For most teams, the right reason to pick OriginChainDB isn’t cost - it’s the architectural simplicity of one substrate.
FAQ
What’s the per-user cost of a typical AI feature?
$0.01 to $0.05 per MAU per month for a moderate-complexity feature. Higher for heavier reasoning, lower for light classification.
Why is the LLM 95% of the cost?
Because tokens are individually cheap but you generate billions of them. Even at $0.0001 per call, 1B calls/month is $100K. Storage and database I/O are sub-pennies that don’t add up to the same total.
Should I optimize my database to reduce AI costs?
Probably not. The database is rarely the bottleneck. Optimize the LLM line first.
What about embedding model costs?
Embeddings are <5% of the total bill at moderate scale. Worth caching, not worth obsessing over.
How does OriginChainDB pricing scale at large volumes?
We tier by compute + storage plan. Talk to us - we’ll model your specific workload.
What to read next
- RAG at production scale: latency budget - the runtime side of the same equation.
- Why we don’t need a separate vector database - the architectural simplification that saves the dual-write line item.
- Per-key TTL for ephemeral AI data - TTL’d caches as a cost-reduction primitive.