Skip to content
ZenithAIZenithAI Docs

Computer use: screen agents on ZenithAI

A computer-use agent looks at a screenshot, decides what a person would do next, and asks your code to do it. ZenithAI supports this on its OpenAI-compatible API in two ways:

  • OpenAI’s built-in computer tool on /v1/responses, on the Genius tier. You send computer_use_preview or computer as on OpenAI, and you get computer_call items with coordinates in screen pixels. Code written for OpenAI’s computer tool works after you change the base URL and the model.
  • Your own strict function tools, such as click and type_text, on /v1/chat/completions or /v1/responses, on Smart or Genius. You map the model’s 0 to 1000 grid to your screen yourself.

Either way your code performs the action, takes a new screenshot and asks again, all on your own server.

How does a computer-use agent work on ZenithAI?

Section titled “How does a computer-use agent work on ZenithAI?”

The agent is a loop that your program owns:

  1. Take a screenshot and send it with the task.
  2. The model replies with one action, such as a click at x and y. With OpenAI’s GA computer tool it can reply with up to 5, such as a click on a field and the text to type.
  3. Your code performs the action, or the actions in order, on the screen.
  4. Take a new screenshot, send it back and go to step 2.
  5. The loop ends when the model answers in plain text, or when you reach your step limit.
Who Does what
Your code Owns the computer: a desktop, a virtual machine or a browser. Captures screenshots, moves the mouse, presses keys, decides when to stop, and decides what is allowed
ZenithAI Reads the screenshot and proposes the next action
The model Never touches the screen. It only proposes actions. Nothing runs unless your code runs it

ZenithAI has no computer or browser of its own for this. It sees only the screenshots you send, and the screenshots stay on your server.

Built-in computer tool Your own function tools
Endpoint /v1/responses /v1/chat/completions or /v1/responses
Tiers Genius Smart and Genius (vision and tools both true in /v1/models)
Coordinates you receive Screen pixels, converted by the server A 0 to 1000 grid of the screenshot; you convert
Calls computer_call items with OpenAI’s actions function_call or tool_calls with your own schemas
Old screenshots truncation: "auto" drops them truncation: "auto" on Responses, or drop them yourself
Best for Code written for OpenAI’s computer tool, the Agents SDK ComputerTool Chat Completions clients, custom actions, Smart

Where do I start?

Section titled “Where do I start?”
Page Read it when
OpenAI’s computer tool You use /v1/responses on Genius, code written for OpenAI’s computer tool, or the Agents SDK ComputerTool
With your own function tools You use /v1/chat/completions, Smart, or actions of your own
Errors, safety and limits Before you run an agent on a real screen. It applies to both ways
Section titled “Related”

Common questions

Does ZenithAI have its own computer that the model operates?

No. Your program owns the computer. It sends screenshots to your ZenithAI server, receives the next proposed action, or a short list of them with the GA computer tool, and performs them itself. ZenithAI only looks at the screenshots you send and never touches a mouse, keyboard or browser.

Which ZenithAI tiers can run a computer-use agent?

OpenAI's built-in computer tool runs on Genius only; other tiers answer HTTP 400 unsupported_tools. An agent built on your own function tools runs on any tier whose capabilities.vision and capabilities.tools are true in /v1/models, which is Smart and Genius on a standard server.

Does ZenithAI send my screenshots to OpenAI or anyone else?

No. Screenshots go to your own ZenithAI server. Nothing is kept unless you send store true on /v1/responses; then the response and its screenshots stay on your server, for your API key, for up to 30 days. If you use the OpenAI Agents SDK, turn its tracing off with set_tracing_disabled(True), because tracing uploads to OpenAI by default.

Should I use OpenAI's computer tool or my own function tools?

Use the computer tool on /v1/responses with Genius when your code is written for OpenAI's computer tool or the Agents SDK ComputerTool; coordinates arrive in pixels. Use your own strict function tools on Chat Completions, on Smart, or when you need actions of your own.