Skip to content

Rate Limits And Limits

Runtime and management APIs may enforce rate limits by key, route, or environment configuration. Treat 429 as retryable after backoff.

Runtime Usage Keys

Customer-key usage routes are intended for high-volume runtime traffic but should still use bounded queues, retries, and backoff in SDK integrations.

User-key partial logging has its own rate-limit scope because it can create unresolved resources. Use stable external IDs and idempotency keys when retrying partial logs.

Payload Size

Keep usage metadata compact. Avoid logging prompts, provider credentials, raw request bodies, or sensitive customer data. Webhook delivery stores bounded response snippets for diagnostics.

Retry Guidance

  • Retry network errors, timeouts, 429, and transient 5xx responses.
  • Use idempotency keys for retried writes.
  • Do not retry malformed requests until the payload is corrected.
  • For webhooks subscribers, return 2xx only after durable processing.