Code mode

Why code mode exists

Loading every connected tool definition into an agent consumes context before the conversation begins. Code mode exposes two stable tools instead: one searches the workspace catalog and one executes a short program that can call the selected tools.

The agent pays the context cost only for the tools it actually needs.

Search and execute

• discover_tools finds approved tools by name, description, and service.
• execute_code runs TypeScript that calls selected tools through the same policy engine as direct mode.
• Every underlying call is recorded separately in Activity.

Policy is unchanged

Code mode does not bypass access rules. Every call still evaluates the service ceiling, client policy, tool override, definition approval, and one-call approval requirement.

return await linear.create_issue({
  team: 'ENG',
  title: 'Investigate failed deployment',
});

When a gated call depends on runtime data, Rayrun pauses the same in-memory execution for up to 10 minutes and resumes it after browser approval. Approval waits do not spend active execution time, and earlier calls are not rerun.

The continuation is bound to the workspace, user, OAuth client consent, and exact request. Rayrun rechecks policy, tool definition, credential subject, and credential version before dispatch.

A deployment, process restart, or expired lease ends the continuation instead of restarting code that may already have changed data.

Execution isolation and limits

Model-written code runs in a separate credential-free Docker service with outbound networking denied. Every execution gets a fresh seccomp-confined Node child and QuickJS VM; the execution token and upstream credentials remain in the web broker.

• 30 seconds of active execution time by default; callers may request from 1 to 120 seconds. Approval waits do not consume this budget.
• 100 upstream calls per execution, with at most 4 running concurrently.
• 64 MiB of QuickJS memory and at most 2 MiB for one upstream response or 8 MiB retained across the execution.
• 40,000 characters for the serialized result value by default; callers may request up to 200,000.

The executor is a runc container on the same host kernel, not a microVM, gVisor sandbox, or separate host. Container isolation limits ordinary failures and reduces the impact of an engine escape.

Authorization does not depend on container isolation. Every upstream call crosses the external broker and passes current policy checks.

Workspace usage controls

Open Settings → Usage to see today’s count, change the daily safety budget, or turn Code Mode off. This safety boundary is separate from the monthly billing allowance.

• 10,000 authorized upstream dispatch attempts per UTC day by default, shared atomically by Code Mode and Direct Mode.
• A warning appears at 80 percent; new upstream calls stop at the hard boundary until the UTC reset or an owner/admin override.
• Owners and admins can immediately disable new execute_code executions and run-ahead starts, while keeping discover_tools and /mcp/direct available and blocking later Code Mode dispatches.

An attempt counts only after live policy, tool-definition, approval, and credential checks pass and Rayrun is ready to dispatch it. An upstream timeout or error still counts because network and credential capacity were spent; a pre-dispatch refusal does not.

Add speculative run-ahead

Use speculative programmatic tool calling (sPTC) to start eligible reads while the model writes execute_code. Rayrun Chat enables it automatically; custom harnesses use @rayrun/sdk. Standard Codex and Claude Code connections use normal Code Mode without speculative run-ahead.

The examples are route excerpts. Supply an authenticated MCP client and access token plus model, prompt, and request. If open() returns null or the session becomes unavailable, send the final execute_code call normally.

import { RayrunGateway } from '@rayrun/sdk';

const callExecuteCodeWithRunAhead = async ({
  mcp,
  mcpAccessToken,
  streamedArgumentDeltas,
}) => {
  const gateway = new RayrunGateway({ accessToken: mcpAccessToken });
  const runAhead = await gateway.codeRunAhead.open();
  let finalArguments = '';

  try {
    for await (const delta of streamedArgumentDeltas) {
      finalArguments += delta;
      void runAhead?.feedArguments(finalArguments);
    }

    await runAhead?.flush();

    const result = await mcp.callTool({
      name: 'execute_code',
      arguments: JSON.parse(finalArguments),
      ...(runAhead ? { _meta: runAhead.meta } : {}),
    });

    return result;
  } finally {
    await runAhead?.close();
  }
};

import { RayrunGateway } from '@rayrun/sdk';
import {
  createUIMessageStreamResponse,
  jsonSchema,
  streamText,
  tool,
  toUIMessageStream,
} from 'ai';

const createVercelRayrunTools = async ({ mcp, mcpAccessToken }) => {
  const gateway = new RayrunGateway({ accessToken: mcpAccessToken });
  const { tools: definitions } = await mcp.listTools();
  const executeCodeDefinition = definitions.find(({ name }) => name === 'execute_code');
  if (!executeCodeDefinition) throw new Error('Rayrun did not advertise execute_code.');

  const calls = new Map();
  const closeCall = async (toolCallId) => {
    const call = calls.get(toolCallId);
    if (!call) return;

    calls.delete(toolCallId);
    await call.runAhead?.close();
  };
  const closeRunAhead = async () => {
    await Promise.all([...calls.keys()].map(closeCall));
  };

  const tools = Object.fromEntries(
    definitions.map((definition) => [
      definition.name,
      tool({
        description: definition.description,
        inputSchema: jsonSchema(definition.inputSchema),
        execute: (input, { abortSignal }) =>
          mcp.callTool({ arguments: input, name: definition.name }, { signal: abortSignal }),
      }),
    ]),
  );

  tools.execute_code = tool({
    description: executeCodeDefinition.description,
    inputSchema: jsonSchema(executeCodeDefinition.inputSchema),
    onInputStart: async ({ abortSignal, toolCallId }) => {
      calls.set(toolCallId, {
        arguments: '',
        runAhead: await gateway.codeRunAhead.open({ signal: abortSignal }),
      });
    },
    onInputDelta: ({ abortSignal, inputTextDelta, toolCallId }) => {
      const call = calls.get(toolCallId);
      if (!call) return;

      call.arguments += inputTextDelta;
      void call.runAhead?.feedArguments(call.arguments, { signal: abortSignal });
    },
    execute: async (input, { abortSignal, toolCallId }) => {
      const call = calls.get(toolCallId);

      try {
        await call?.runAhead?.flush();

        return await mcp.callTool(
          {
            name: 'execute_code',
            arguments: input,
            ...(call?.runAhead ? { _meta: call.runAhead.meta } : {}),
          },
          { signal: abortSignal },
        );
      } finally {
        await closeCall(toolCallId);
      }
    },
  });

  return { closeRunAhead, tools };
};

const { closeRunAhead, tools } = await createVercelRayrunTools({
  mcp,
  mcpAccessToken,
});

const result = streamText({
  abortSignal: request.signal,
  model,
  prompt,
  tools,
  onAbort: closeRunAhead,
  onEnd: closeRunAhead,
  onError: closeRunAhead,
});

return createUIMessageStreamResponse({
  stream: toUIMessageStream({ stream: result.stream }),
});

• Open the session with the MCP OAuth access token used for the final call. Feed the complete, append-only JSON argument string generated so far, not only the latest fragment.
• The OpenAI Responses API emits response.function_call_arguments.delta events; append event.delta. Anthropic Messages emits content_block_delta events with input_json_delta; append event.delta.partial_json.
• With Vercel, use onInputStart and onInputDelta, keep state by toolCallId, pass the request signal through, and close unfinished sessions from onAbort, onError, and onEnd.
• Use a connected @modelcontextprotocol/client Client: the stock @ai-sdk/mcp generated tool does not expose tools/call request _meta, which run-ahead needs to claim early results.
• Call flush() once, attach its metadata to the final tools/call, and call close() in finally. If a snapshot rewinds, the SDK immediately removes the session metadata and sends close.
• Vercel’s input hooks run before AI SDK toolApproval. Do not enable this adapter when toolApproval is intended to prevent every pre-approval upstream request.
• Rayrun considers only straight-line calls with literal JSON arguments. Each tool must be allowed, definition-approved, explicitly classified Read by a workspace owner, read-only, and idempotent.
• If the final program or dry_run differs, Rayrun immediately closes the speculative session and signals cancellation to in-flight reads. Cancellation is cooperative; stale results are never returned, and completed unclaimed work remains visible in Activity.

Alex Zhang introduced sPTC; Rayrun adapts the MIT-licensed spec-ptc design for its MCP gateway.

The final execute_code call remains authoritative and rechecks identity, consent, scope, arguments, policy, credentials, and budgets.

When to use direct mode

Use the direct endpoint when the client must inspect schemas itself, when a workflow expects a specific tool name, or when the catalog is small enough that context size is not a concern.