Skip to content

OpenAI-compatible chat

POST /v1/chat/completions takes a list of messages and returns the next assistant message. It is stateless: for a follow-up, send the earlier user and assistant messages again yourself. Code built on the newer OpenAI Responses format, or on the OpenAI Agents SDK, can use /v1/responses on the same server.

from openai import OpenAI
client = OpenAI(base_url="https://dummydomain/v1", api_key="YOUR_API_KEY")
history = [{"role": "user", "content": "Explain why leaves look green."}]
first = client.chat.completions.create(model="Fast", messages=history, max_tokens=512)
history.append({"role": "assistant", "content": first.choices[0].message.content})
history.append({"role": "user", "content": "Now explain it to a ten-year-old."})
second = client.chat.completions.create(model="Fast", messages=history)
print(second.choices[0].message.content)

Set stream: true. Replies arrive as server-sent events with choices[].delta, ending with data: [DONE]. Add stream_options: {"include_usage": true} to receive token usage at the end.

stream = client.chat.completions.create(
model="Fast", stream=True,
messages=[{"role": "user", "content": "Write a two-line welcome note."}],
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
Field Notes
model Instant, Fast, Smart or Genius (when your server has it)
messages system, developer, user, assistant and tool roles
max_tokens / max_completion_tokens Positive integer; if you send both, they must match. Without either, a turn without active tools gets what the context window leaves, 2,048 to 8,192 tokens
temperature 0 to 2
top_p 0 to 1
presence_penalty, frequency_penalty −2 to 2
stop A string or up to 4 strings
seed Signed 64-bit integer
response_format See Structured JSON output
tools, tool_choice, parallel_tool_calls On tiers with capabilities.tools; your code runs the functions. See Function calling and tools
reasoning_effort On Genius, low to xhigh let the model think first; other tiers ignore it. See thinking on Genius
verbosity, prompt_cache_key, safety_identifier Accepted and ignored; see the list

functions, function_call, n other than 1, logprobs: true, store: true and other fields OpenAI lists but the server cannot honour are refused with HTTP 400. See fields that are refused.

Errors return an error object with message, type, code, param where relevant, and a request_id. Keep the X-Request-ID response header when you report a problem.

Status What to do
400 Fix the request; the code and param say what is wrong
401 Check the key and that you are sending Authorization: Bearer
429 Wait for Retry-After if present, then retry with back-off
5xx Retry with back-off when error.retryable is not false and repeating the request is safe. See which errors to retry

By default each API key can send 60 requests a minute, shared with /v1/responses, with up to 4 streams open at a time. A request over a limit gets 429 with the code rate_limit_exceeded and a Retry-After header in seconds. Every code is listed in the error reference.