Skip to content
ZenithAIZenithAI Docs

Computer use: errors, safety and limits

These rules apply to both ways of building a screen agent: OpenAI’s computer tool and your own function tools.

How do I handle errors and retries in this loop?

Section titled “How do I handle errors and retries in this loop?”

Every JSON error carries error.code, and an error that has retryable also sends an x-should-retry header that the OpenAI SDKs obey. The SDKs resend a request twice by default. Beyond that, a screen loop needs these cases:

What you see What it means What to do
502 invalid_tool_call, retryable: true A call did not match its schema after one regeneration. The error adds reason, tool_name, rule, arg_path and schema_path. You never get a partial call Ask again (the sample does, up to three times). Log the fields. If one tool fails often, check it is strict: true and that all tools are strict. See the details
502 tool_choice_violation, retryable: false, x-should-retry: false The model called a tool where none was allowed Do not resend the same request. Use tool_choice: "none" only for the closing answer
502, answer cut at the output limit A tool turn reached max_tokens. Not retryable Raise max_tokens, up to the tier’s max_output_tokens_with_tools (4,096 by default)
400 max_output_tokens_exceeded max_tokens is above that tool limit Lower it. The error’s max_output_tokens field gives the limit
400 context_length_exceeded History, images, tools and the answer reserve do not fit. The error breaks the total down, with image_tokens Keep fewer screenshots, send a smaller width, trim old text steps. The Python sample drops screenshots and retries
400 too_many_images, images_too_large, invalid_image More than 8 images, more than 20 MiB, or an image that cannot be decoded Send truncation: "auto" on Responses, or drop old screenshots; shrink the image; check the data: URL is complete base64
400 unsupported_vision, unsupported_tools The tier lacks vision or tools, or the computer tool was sent to a tier other than Genius Use Genius for the computer tool, Smart or Genius for function tools. Read /v1/models
400 unsupported_tool On /v1/responses, a hosted tool such as web_search Use the computer tool or function tools
400 tools[0].type must be 'function' A computer tool sent to /v1/chat/completions Use the computer tool on /v1/responses, or function tools
400 invalid_tool_schema Two computer tools, a computer tool with fields it does not take, or a display size outside 1 to 10000 Send one tool in one of the two shapes
400 previous_response_not_found previous_response_id names a response that was not sent with store: true, has expired or been deleted, or belongs to another key Send store: true on every turn you continue from, or replay the whole input
400 store_too_large, store_full With its screenshots the response would be over about 25 MB, or the server’s response store is full Send store: false and replay input, or send smaller screenshots
400 unpaired_tool_output_image On Responses, an image output is not right after its function_call Replay each call followed by its output
429 rate_limit_exceeded A rate limit, or a refused stream Wait Retry-After seconds. The SDKs do this on their own
Error in a stream A last error event. No [DONE] Never run a tool call from a stream that did not finish

Tool turns arrive in one piece after the server has checked the call, so streaming a computer-use turn gains nothing. If you stream anyway, act only on a complete call. Plain 400 errors mean the request must change, and sending it again fails again.

How do I run a screen agent safely?

Section titled “How do I run a screen agent safely?”

The model can only propose actions. Your code decides what runs, so safety is your code’s job.

  • Use a sandbox. Run the agent in a virtual machine, a container with a desktop, or a spare user account with nothing valuable open. Do not point it at your daily desktop.
  • Confirm destructive actions. Ask a person before deleting, sending, paying, submitting a form or changing a setting. The Python sample asks before every step. Relax that only inside a sandbox. With the GA computer tool one call can hold up to 5 actions: check each of them before any runs.
  • Keep secrets off the screen. The screenshot goes to your server, but people may read logs and traces. Do not open password managers, payment pages or private mail while the agent runs, and never type a password through type_text.
  • Treat what is on the screen as untrusted. A web page or document can contain text that tells the model what to do. Allow only the actions the task needs, and limit where the agent can go.
  • Keep the loop bounded. Cap the steps, put an allow-list in act() (for example only certain applications), and keep a way to abort, such as pyautogui.FAILSAFE.
  • Keep data on your server. Screenshots, history and answers go to your ZenithAI server only. Nothing is kept after the answer unless you send store: true. Then the response and its screenshots are kept on your server, for your API key, for up to 30 days, until you delete them with DELETE /v1/responses/{id}. If you use the OpenAI Agents SDK with OpenAIResponsesModel, call set_tracing_disabled(True), otherwise it uploads traces to OpenAI. See the Agents SDK example.

What are the limits?

Section titled “What are the limits?”
Limit Value Where to check
Tiers, computer tool Genius Computer tool
Tiers, function tools Vision and tools both true (Smart and Genius by default) capabilities in /v1/models
Actions per turn Preview tool 1. GA tool 1 to 5 in one computer_call. Function tools: one call per strict turn Actions
Stored responses Only with store: true. 30 days, 1,000 per API key, about 25 MB each Stored responses
Images per request 8 max_images_per_request
Size per image / all images 10 MiB / 20 MiB, decoded Chat Completions images
Image detail auto, low, high, original, or omitted A hint only
Image source data: URLs. https only for hosts your administrator allows Chat Completions images
Context for a tool turn tool_context_window, 24,576 tokens by default /v1/models
Answer length on a tool turn max_output_tokens_with_tools, 4,096 by default /v1/models
Requests per API key 60 a minute, across both endpoints Request limits
Open streams 4 per key, 8 on the server Request limits
Requests per network address 120 a minute, 40 in 10 seconds, 16 at once Request limits
Screen positions Pixels with the computer tool; a 0 to 1000 grid of the screenshot with function tools This page
Old screenshots truncation: "auto" on /v1/responses Long conversations

Your administrator can change the limits, so read the live values from /v1/models and the errors page. A loop sends one request per turn, so a single agent stays far below the rate limits. Several agents sharing one key can reach 60 a minute.

Section titled “Related”

Common questions

Why did the model click in the wrong place?

Strict mode and the computer tool guarantee the shape of a call, not that it is the right action. Check the screen after every step, cap the number of steps, and ask a person to confirm destructive actions.

Which computer-use errors should I retry?

Retry invalid_tool_call, which is marked retryable, and wait out a 429 with Retry-After. A 400 such as too_many_images or context_length_exceeded fails again until you change the request, for example with fewer screenshots or truncation auto.