Handle request and capacity limits

When requests arrive faster than an instance can handle them, reduce the number running at once and read the retry guidance. This guide separates retryable reads from writes whose outcome needs checking.

Different limits protect different resourcesCopy Anchor Link

Requests, bytes, concurrent queries, natural-language calls, memory and result size have separate limits. Scope and values depend on the API, configuration and deployed engine. Use the response and current console configuration rather than hard-coding an old pricing-tier name or quota.

Read the actual responseCopy Anchor Link

Throttling can return 429. Temporary capacity pressure can also return 503 with Retry-After. Honor that header when supplied and retain the request ID. Do not assume every successful response contains X-RateLimit-Limit, X-RateLimit-Remaining or reset headers.

A 413 requires smaller work. A 501 requires a supported feature or topology. Neither is fixed by repeating the same request rapidly.

Bound retries for readsCopy Anchor Link

Check your installed client's retry settings before adding an application retry loop. Set a total attempt limit so the two layers cannot multiply retries. This complete Python example retries only a schema-list read, with four attempts and a bounded wait. Set OC_BASE_URL, OC_TENANT and OC_BEARER, and add identity headers when required.

import json
import os
import time
from urllib.error import HTTPError
from urllib.request import Request, urlopen

# Read-only request. Do not reuse this loop for an uncertain write.
url = f"{os.environ['OC_BASE_URL'].rstrip('/')}/v1/tenants/{os.environ['OC_TENANT']}/schemas"
headers = {"Authorization": f"Bearer {os.environ['OC_BEARER']}"}
for attempt in range(4):
    try:
        with urlopen(Request(url, headers=headers), timeout=15) as response:
            print(json.load(response))
        break
    except HTTPError as error:
        if error.code != 429 or attempt == 3:
            raise
        value = error.headers.get("Retry-After")
        error.close()
        try:
            delay = float(value) if value else 2 ** attempt
        except ValueError:
            # This small example expects delay-seconds, not an HTTP date.
            # Hand the response to your scheduler instead of retrying early.
            raise RuntimeError("Handle the returned Retry-After date")
        if not 0 <= delay <= 30:
            raise RuntimeError("Retry belongs in a longer-lived scheduler")
        time.sleep(delay)

In a worker fleet, add jitter to an exponential-backoff policy and enforce a total deadline. Treat the server’s retry delay as a minimum. Close response bodies before retrying. Keep cancellation and shutdown responsive.

Handle writes separatelyCopy Anchor Link

A response timeout or gateway failure does not prove a write was rejected. Reconcile its outcome, or use a stable key on an endpoint that explicitly supports deduplication. An idempotency key prevents duplicate work only on endpoints that document that behavior; it does not make arbitrary SQL safe to replay. See idempotency and transaction failures.

Reduce pressure at the sourceCopy Anchor Link

• Use a bounded worker pool and a queue instead of launching one request per row at once.
• Use supported batch APIs and checkpoint completed chunks.
• Select only needed columns and bound result sets.
• Investigate repeated failures using status, request ID and instance usage before changing capacity.