Configure client behavior
Retry with exponential backoff
Async fan-out requests
Runtime guidance
- Latency sensitive
- Throughput focused
- Cost constrained
Keep
max_tokens small, use deterministic prompts, and cap retries to avoid tail latency.Pair this guide with /errors to standardize retryable vs non-retryable failures.