Built-in tools and how they are used

ZenithAI comes with a set of built-in tools: things the server can do on its own, like reading a scanned page or making an Excel file. This page lists all of them, how each one gets used in the web app, and which API endpoint gives you the same tool in your own code.

Two kinds of tools

Built-in tools run on your ZenithAI server. Your own functions (function calling) are tools you write, such as “look up an order in our ERP”. The model asks for them and your code runs them. See Build your own agents.

At a glance

Tool	What it does	In the web app	In your code
Chat and writing	Answers, summaries, drafts, translation	Every chat	POST /v1/chat/completions, POST /chat, POST /generate
Structured JSON	Answers that match your schema	Not needed	response_format on /v1/chat/completions
Image reading	Reads photos, screenshots, bills, forms	Attach an image	image_url parts on /v1/chat/completions
Audio	Turns speech into text, then answers about it	Attach a recording	attachments on POST /chat
Video	Watches the picture and listens to the speech (MP4, MOV, WebM)	Attach a video	attachments on POST /chat
Document reading	Reads PDF, Word, Excel, PowerPoint, text and email files	Attach a file	attachments on POST /chat
PDF text and tables	Exact text and tables from digital PDFs, no model	Automatic	POST /pdf
Scanned pages (OCR)	Reads text from scanned PDFs	Automatic	POST /ocr, then GET /ocr/jobs/{id}
Web search	Finds pages and reads their text	Search button	POST /websearch
Site-locked chat	Answers only from websites you choose	Site lock in the chat	Web app only
Workspaces	Answers from your own document library	Pick a workspace	workspace_id on POST /chat
Word, Excel, PowerPoint	Makes real, editable Office files	Ask for the file	POST /chat or /generate, then download
Your own functions	The model asks your code to run a function	Not in the web app	tools on /v1/chat/completions (Smart, Genius)

How the web app uses them

In the web app you don’t pick a tool. You attach a file or ask a question, and ZenithAI uses the right tool for it:

• A scanned PDF goes through OCR first, then the answer is written from the recognised text.
• A question about today, such as a rate or a price, triggers a web search when search is on.
• “Make this an Excel sheet” creates a real .xlsx file you can download.

How your code uses them

Over the API, POST /chat behaves like the web app and picks tools for you. The other endpoints each run one tool you choose. Some tools, like /pdf, /ocr and /websearch, use no model at all. They are fast, repeatable, and good as the first step of an agent.

Image reading

Send images as image_url parts with a data: URL on /v1/chat/completions. PNG, JPEG, WebP, GIF and BMP work, up to 10 MiB each, 8 images and 20 MiB per request, on user messages only. Your server does not download image links unless your administrator allows a named website. See the bill photos example.

Audio and video

Attach the file to POST /chat as base64 in attachments. Each call is saved in the key owner’s chat history, like a chat in the web app.

• Audio: WAV, MP3, M4A, OGG and FLAC. The speech is turned into text, then your question is answered.
• Video: ZenithAI looks at one frame per second, up to 60 frames, so it sees about the first minute of the picture. Speech in the soundtrack is turned into text too. Send one video per message. MP4, MOV, M4V, MKV, AVI and WebM work, including phone videos in H.264 or H.265, up to 100 MB and 5 minutes. See video checks.
• The whole request can be up to 150 MB. Base64 makes a file about a third bigger, so keep a file under about 100 MB.

See meeting minutes and video checks.

Documents and scans

• POST /pdf gives the exact text and tables already inside a digital PDF. See Read PDF text and tables.
• POST /ocr reads scanned PDFs. It starts a job and returns a job_id. Ask GET /ocr/jobs/{job_id} every few seconds until status is done, then read result.text and result.page_texts. One PDF per job.
• POST /chat with an attached PDF, Word, Excel or PowerPoint file reads it and answers about it. Old .doc, .xls and .ppt files must be saved in the new format first.

See document intake.

Web search

POST /websearch returns search results and, if you ask, the text of the top pages. It writes no answer, so you decide what to do with the results.

Field	Default	Meaning
query or queries		One search, or up to 5 searches merged into one list
max_results	5	1 to 10 results
depth	2	How many top results also get their page text. 0 means links only.
max_words	500	Page text per result, 1 to 2,000 words
relevance_filter	false	Moves results that don’t match the query to the end

Each result has rank, title, url, domain and snippet, plus content for the pages that were read. status is ok, partial or no_results.

Caution

If your administrator has turned internet off for the key’s owner, you get 403 internet_access_blocked. Web search has its own limit, 60 requests a minute per key by default. Use POST for private search terms, because a GET address can end up in proxy logs.

See lead qualification.

Word, Excel and PowerPoint files

Ask for the file in plain words on POST /chat or POST /generate (with stream: false). When a file is made, the response has an office_artifact with artifact_id and filename. Download it with GET /office/artifacts/{artifact_id}/download, sending your key on that request too.

If there is no office_artifact, no file was made, even if the reply text looks fine. Check for it in your code. See Word documents and Excel workbooks for the formats.

Workspaces

To answer from a workspace, start a new conversation on POST /chat with workspace_id and workspace_pin: true on the first message. The conversation stays tied to that workspace. GET /workspaces lists the workspaces you can use.

Your own functions

On Smart and Genius, add tools to /v1/chat/completions. The model replies with a tool_calls request, your code runs the function and sends the result back as a tool message. Check capabilities.tools in GET /v1/models before you depend on it. The full loop is in Build your own agents.

Common questions

What is the difference between built-in tools and function calling?

Built-in tools run on your ZenithAI server, such as reading a PDF or searching the web. Function calling is for your own tools. The model asks for one of your functions, your code runs it and sends back the result. ZenithAI never runs your functions itself.

Does ZenithAI choose the tools by itself?

In the web app, yes. It reads attachments, searches the web or makes a file when your message needs it. Over the API, POST /chat works the same way, while /pdf, /ocr and /websearch each run one tool you pick.

Can ZenithAI read a video?

Yes, through POST /chat. Phone videos in MP4 or MOV go in as they are, up to 100 MB and 5 minutes. It looks at one frame per second, up to 60 frames, so about the first minute, and turns the speech into text.

Which built-in tools need the internet?

Only the tools that reach websites, such as web search, page reading, site-locked chat and importing a website into a workspace. Reading files, scans, audio and video, and making Office files work inside your own network.