Idempotent tool calls for agent loops

OriginChainDB Team · May 5, 2026

agents tool-calls idempotency reliability tutorial

TL;DR - Production AI agents make tool calls that have side effects: sending emails, charging cards, posting webhooks. Network retries and LLM-generated duplicates mean you’ll get the same call twice, and the second one shouldn’t fire. The fix is an idempotency key derived from the intent, an atomic claim only one worker can win, and a stored result the losers read instead of re-firing.

The problem

Your agent decides to send an email. The HTTP call to your email service times out. The agent doesn’t know if the email was sent. It retries. Now the user gets two emails.

Or: the LLM, in some non-determinism, hallucinates that it should call send_email twice in the same response. Your tool-handling code dutifully sends both.

Or: a downstream queue retried the agent’s request after a 503. Same call, second invocation. Same problem.

Every agent that touches the real world hits this. The fix is idempotency keys - every side-effecting call carries a unique key, and the system that processes it tracks which keys it’s already seen.

The shape of an idempotent tool

Three pieces:

The key. A stable identifier for the intent, not the call. If the agent decides “send email about order #12345 to user U”, the key should be a function of (intent type, order, user) - send_email:order-12345:user-U. Same intent, same key, regardless of how many times the call fires.
The store. A K/V store that records “I have seen this key, and the result was R.” Reads have to be fast (every call hits this); writes have to be atomic (no double-fires inside a race).
The compare-and-swap. When a call comes in, atomically: (a) check the store for the key, (b) if seen, return the stored result; (c) if not seen, claim the key, perform the side effect, store the result.

Step (c) needs to be atomic. Otherwise two concurrent calls both see “not seen,” both claim the key, both fire the side effect. That’s the bug we’re trying to prevent.

Building this on OriginChainDB

Two OriginChainDB features make this clean: per-key TTL and atomic compare-and-swap.

async function handleToolCall(intent: ToolIntent): Promise<ToolResult> {
 const key = idempotencyKey(intent); // e.g. "send_email:order-12345:user-U"

 // Check if we've already handled this intent
 const existing = await oc.get("tool-call", key);
 if (existing) {
 return existing.result; // already fired, return cached result
 }

 // Atomically claim the key with a short pending TTL
 // (so a crash before completion auto-clears the claim)
 try {
 await oc.put({
 shape: "tool-call",
 id: key,
 set: { intent, status: "pending", started_at: Date.now },
 ttl: 300, // 5 min pending window
 if_match: { exists: false },
 });
 } catch (e) {
 if (e.status === 412) {
 // Another worker claimed it first; wait for it to complete
 return await waitForResult(key);
 }
 throw e;
 }

 // We have the claim. Fire the side effect.
 let result: ToolResult;
 try {
 result = await fireSideEffect(intent);
 } catch (e) {
 // Mark failed; let the TTL clean up so retries can try again later
 await oc.put({
 shape: "tool-call",
 id: key,
 set: { intent, status: "failed", error: String(e) },
 ttl: 300,
 });
 throw e;
 }

 // Persist the result. Long TTL so re-issued intents in the next
 // hour return the cached result instead of re-firing.
 await oc.put({
 shape: "tool-call",
 id: key,
 set: { intent, status: "completed", result, completed_at: Date.now },
 ttl: 3600, // 1 hour
 });

 return result;
}

The if_match: { exists: false } is the atomic claim. Because the substrate serializes the compare-and-swap, only one worker will succeed at the put; the rest get 412 and fall into the wait-for-result path.

The TTLs do double duty:

The 5-minute pending TTL means if a worker crashes mid-call, the claim auto-clears and retries can try again.
The 1-hour completed TTL means re-issued intents within a reasonable window get the cached result; after that, the system has “forgotten” and will re-fire (which is usually fine - by an hour later the agent’s situation has changed).

How to compute the key

The key is a function of intent. Two design choices:

Option A - derive deterministically from intent. sha256(JSON.stringify(intent)). Same intent always produces the same key. Good for “the LLM emitted the same call twice.”

Option B - pass the key in from the caller. The agent’s loop knows it’s retrying and reuses the key. Good for “the upstream queue retried our handler.”

In practice you want both. Compute a deterministic key from the intent, but let the caller override it. That way:

LLM duplicates (same intent, no caller key) deduplicate via the deterministic key.
Network retries (caller knows) reuse the explicit key.
Genuinely-new intents that look textually similar (rare) can still differ by the explicit key.

The wait-for-result path

When a second worker hits “I’d claim this but someone else already claimed it,” what should it do?

async function waitForResult(key: string, timeoutMs = 30_000): Promise<ToolResult> {
 const start = Date.now;
 while (Date.now - start < timeoutMs) {
 const r = await oc.get("tool-call", key);
 if (r?.status === "completed") return r.result;
 if (r?.status === "failed") throw new Error(r.error);
 await sleep(100);
 }
 throw new Error("timed out waiting for in-flight tool call");
}

Polling is fine here - the second worker’s job is to wait, not to do work. 100ms intervals are cheap reads on a fast K/V store. For longer-running tools (LLM generation, etc.), bump the interval.

For tools you know are slow (>5s), consider a real notification primitive - but for the typical email-send / webhook-post / db-update case, polling is correct and simple.

What this gets you

The agent’s tool layer becomes safe under:

LLM-driven duplicate calls (same intent, same key, dedupe via store).
Network retries (agent retries; reuses key; gets cached result).
Worker crashes mid-call (pending TTL clears; retry works).
Multiple agents converging on the same action (atomic claim ensures only one fires).

What it doesn’t get you: protection against side effects that are themselves non-idempotent on the receiving end. If your email service charges per request regardless of dedup, you’re still paying. The discipline is “idempotency at every layer that has state” - the tool layer is one of them.

FAQ

What’s an idempotency key?

A unique identifier for an intended operation, used to detect duplicate invocations. Same key = same intent = the system should fire the side effect once and return the same result on subsequent calls.

Should I derive the key or pass it in?

Both. Derive deterministically by default (handles LLM duplicates), but accept a caller override (handles retry scenarios where the caller knows it’s retrying).

What’s the right TTL for completed tool calls?

Long enough to cover all reasonable retry windows from the upstream caller, short enough that storage doesn’t grow forever. 1 hour is a reasonable default; tune based on your retry policy.

Does this work for streaming tool calls?

Streaming makes idempotency harder - partial results don’t fit the “completed once” model cleanly. The pragmatic answer: serve the stream from the original handler, and on retry, return the final result (cached) rather than re-streaming.

Does OriginChainDB’s `if_match` add latency?

~20-50µs over a non-CAS write. Negligible compared to the side effect itself.