Skip to main content

Configure client behavior

Retry with exponential backoff

Async fan-out requests

Runtime guidance

Keep max_tokens small, use deterministic prompts, and cap retries to avoid tail latency.
Pair this guide with /errors to standardize retryable vs non-retryable failures.
Last modified on February 21, 2026