OpenAI: Designing REST APIs for Autonomous LLM Agents
The generative AI wave transformed API traffic from static payloads to streaming token buffers and agent tool executions. Inspect how OpenAI engineers streaming and deterministic schema contracts.
1. Streaming via Server-Sent Events (SSE) vs WebSockets
Instead of managing bidirectional WebSocket connection state, OpenAI leverages HTTP Server-Sent Events (Content-Type: text/event-stream). This keeps requests strictly within standard REST semantics, passing through standard HTTP/2 edge proxies and load balancers without custom protocol handshakes.
2. Strict JSON Schema Tool Calling
To allow probabilistic language models to interface with deterministic backend APIs, endpoints accept a tools array containing strict JSON Schema v7 definitions. By setting strict: true, the model's logits are constrained during decoding, guaranteeing that the generated arguments match the expected schema with zero missing required properties.