Two tools, however many servers.
A gateway can put every connected tool into the model's context before the first call. That cost grows with each service.
The standard endpoint uses code mode by default. A client sees discover_tools and execute_code, then searches the catalog only when needed. Owners and admins can disable new Code Mode executions without disabling Direct Mode.
What each one does.
discover_tools searches the catalog of services you have authorized at call time. It returns matching callables, not the full catalog.
execute_code runs the model's TypeScript against those typed callables. It can chain them: in the session above, three services are reached in one round trip rather than three.
Approval resumes the program. It does not repeat it.
Tool calls are not transactional. Restarting an execute_code program after an approval pause could repeat earlier changes. Rayrun instead pauses the exact QuickJS fiber and records completed calls.
After authenticated approval, Rayrun resumes the same in-memory execution. Earlier calls are neither rerun nor rolled back.
The pause has a process-bound, ten-minute lease and does not spend active execution time. Rayrun rechecks policy, tool definition, client, credential subject, and credential version before dispatch.
A deployment, restart, or expired lease ends the continuation. Rayrun reports the loss and never restarts code whose earlier effects may already exist.
Clients with URL elicitation open the review page as part of the call. Browser and CLI fallbacks carry server-issued continuation state, so a model-authored accept response is never proof of approval. Read the exact-call approval boundary.
Speculative programmatic tool calling starts safe reads early.
Speculative programmatic tool calling (sPTC) lets Rayrun start eligible read calls while the model is still writing execute_code. If the finished program makes the same call, its result may already be waiting. Rayrun Chat uses it automatically.
sPTC was introduced by Alex Zhang. Rayrun adapts the design from the MIT-licensed spec-ptc project for an MCP gateway.
Available to custom streaming harnesses through @rayrun/sdk. Standard Codex and Claude Code connections use normal Code Mode without speculative run-ahead. Rayrun starts only tools approved as safe, read-only, and repeatable. The finished call rechecks the token, policy, definition, arguments, and credential before using an early result. A mismatch runs normally. Actual latency savings depend on how long model generation and eligible reads overlap.
Authorization stays outside the sandbox.
The model can rewrite anything inside the runtime, including tool bindings, helpers, and argument shapes. Enforcement stays in the host broker: allowlists, argument checks, budgets, and credentials never enter QuickJS.
The model can name a tool. It cannot make a call. The QuickJS runtime receives no credential, URL, or authorization header; every outbound call has to cross the broker and pass the live policy checks again.
Model-written code runs in a credential-free Docker service with outbound networking denied. Each execution gets a fresh seccomp-confined Node process and QuickJS VM with time, memory, call, and output limits, then is destroyed after the result or timeout.
Hosted tool hooks run in their own capability-free QuickJS isolate, separate from the model-written program. The same deployed hook revision applies whether a tool is reached through Direct Mode or Code Mode. How hooks are bounded.
The service is a runc container on the same host kernel, not a microVM, gVisor sandbox, or separate machine. It limits ordinary failures and reduces the impact of an engine escape. Credentials and authorization stay in the external broker.
Every authorized upstream dispatch spends a workspace-wide daily safety budget shared atomically with Direct Mode. Owners and admins can inspect usage, change the limit, or disable Code Mode.
The default is 10,000 upstream attempts per UTC day. Validation, policy, approval, and credential refusals do not spend the budget.
When to use direct mode.
Use Direct Mode when the catalog is small enough to load up front. Exposing tools directly is simpler to inspect because the model calls the tool you see.
Code mode puts model-written code in the request path. If you prefer direct tool calls, point the client at /mcp/direct. The /mcp endpoint remains in Code Mode.
The same boundary is available from the terminal.
The ray.run CLI describes and calls service-qualified tools through the Direct Mode endpoint. If an exact call needs approval or service authorization, it opens the authenticated page, saves that private call, and prints a rayrun resume command.
The server-issued continuation and browser decision remain the authority.
The same CLI safely configures supported MCP clients and inspects connections, client policy, approvals, and Activity. Execution uses the signed-in person's OAuth session; workspace administrators receive an audience-bound management token from the same login, while CI and unattended automation use a scoped API key. Use the CLI.
The same pattern at scale.
Cloudflare calls this Code Mode; Anthropic calls it code execution with MCP. Both use the same search-and-execute pattern: one call finds tools and another runs them.
Cloudflare's API server exposes more than 2,500 endpoints through exactly two tools, in roughly a thousand tokens. ray.run applies the same pattern to your connected services.
Connect one server and watch it.
Rayrun records each discover_tools and execute_code call, plus every underlying tool call.