Browser Automation

A Rome App can drive the guardian's agent browser — the logged-in Chromium the agent uses — over the Chrome DevTools Protocol (CDP). The client lives in a dedicated subpath of the runtime SDK:

import {
  openDiscoveredPageSession,
  runDiscoveredBrowserScript,
} from "@rome-os/app-runtime/browser";

It is a separate subpath on purpose: the module is Node-only (it uses ws and node:fs/promises) and is not re-exported from the @rome-os/app-runtime barrel, so it can't leak into your web bundle. Import it only from backend code — actions, hooks, API handlers.

Getting a browser endpoint

You don't configure browser URLs yourself. The host injects a BrowserCapabilityDiscovery into your action's deps as deps.capabilityDiscovery; it lists the CDP endpoints Rome has discovered:

export interface BrowserCapabilityDiscovery {
  getBrowserEndpoints(): DiscoveredBrowserEndpoint[];
}

export interface DiscoveredBrowserEndpoint {
  name: string;       // e.g. "cdp-local-chromium"
  browserUrl: string; // CDP HTTP endpoint
}

pickBrowserEndpoint(endpoints) chooses one for you: it prefers cdp-local-chromium, then any name containing local, then the first alphabetically; it returns null when the list is empty. The two entry points below call it internally, so you usually never call it directly — pass deps.capabilityDiscovery straight in.

Run a script in a page

runDiscoveredBrowserScript(capabilityDiscovery, options) is the one-shot path: pick an endpoint, open pageUrl in a new tab, wait for the page to load, load your script file, evaluate it, and hand back the result.

const run = await runDiscoveredBrowserScript<MyResult>(deps.capabilityDiscovery, {
  pageUrl: "https://example.com/",
  scriptUrl: new URL("./scraping_scripts/scrape.js", import.meta.url),
  entrypointExpression: "scrapeExample",
  args: [{ query: "rome" }],
});
try {
  console.log(run.result); // typed as MyResult
} finally {
  await run.close();
}

Option	Notes
`pageUrl`	URL to open in a fresh tab.
`scriptUrl`	`URL` of a plain JS file on disk, usually built with `new URL(..., import.meta.url)`. Loaded once per process via `loadCachedScriptSource`.
`entrypointExpression`	Expression that must resolve to a function after your script runs, e.g. a function name the script defines. It is called with `args` and its (awaited) return value becomes `result`.
`args`	Optional arguments serialized into the page. Only plain JSON-ish values: strings, finite numbers, booleans, `null`/`undefined`, arrays, plain objects. Anything else throws.
`autorunFlag`	Optional global flag name set to `false` while the script loads (and cleaned up after) — lets a script that self-runs when injected by other tooling skip its autorun path here.
`missingEntrypointMessage`	Optional error message when `entrypointExpression` isn't a function after the script loads.

The result is { endpoint, result, close() }. On evaluation failure the tab is closed for you and the error is thrown; on success you own the tab.

Watch out: always call close() in a finally. It tears down the CDP socket and closes the tab; skipping it leaves a tab open in the guardian's browser after every run.

Hold a session open

When one page needs several script runs or raw CDP commands (reload, bring to front, …), open the page once with openDiscoveredPageSession(capabilityDiscovery, pageUrl). It opens a tab, enables the Page and Runtime domains, brings the tab to front, and waits up to 20s for the document to be ready before returning { endpoint, session, close() }.

The session is a CdpSession:

Member	Notes
`CdpSession.connect(webSocketUrl)`	Connect to a raw CDP WebSocket yourself. You rarely need this — `openDiscoveredPageSession` does it for you.
`session.send(method, params?)`	Send any CDP command, e.g. `session.send("Page.reload", { ignoreCache: true })`.
`session.evaluateByValue<T>(expression)`	`Runtime.evaluate` with `awaitPromise` + `returnByValue`: promises are awaited and the value is serialized back, so `T` must be JSON-serializable. Page exceptions become thrown `Error`s.
`session.close()`	Close the WebSocket only. Prefer the page's `close()`, which also closes the tab.

Three lower-level helpers compose into "run a script in this open session": loadCachedScriptSource(scriptUrl) reads (and caches) the script file, buildBrowserScriptExpression({ scriptSource, entrypointExpression, args, ... }) wraps it into one evaluatable expression, and you pass that to session.evaluateByValue. After a navigation or reload, poll isPageReadyForAutomation({ readyState, href }) — it is true once the page has a real URL (not about:blank) and readyState is interactive or complete.

End-to-end example

Condensed from the first-party browser-automation app's chatgpt_image action (rome_apps/browser-automation/src/actions/image/index.ts): one tab, two script runs, a reload in between, cleanup in finally.

import type { Action, ActionConfig, ActionResult } from "@rome-os/app-runtime";
import type { BrowserCapabilityDiscovery, CdpSession } from "@rome-os/app-runtime/browser";
import {
  buildBrowserScriptExpression,
  loadCachedScriptSource,
  openDiscoveredPageSession,
} from "@rome-os/app-runtime/browser";

const GENERATE_SCRIPT_URL = new URL("./scraping_scripts/image_action.js", import.meta.url);
const DOWNLOAD_SCRIPT_URL = new URL("./scraping_scripts/image_download.js", import.meta.url);

async function runScript<T>(
  session: CdpSession,
  scriptUrl: URL,
  entrypointExpression: string,
  args?: unknown[],
): Promise<T> {
  const scriptSource = await loadCachedScriptSource(scriptUrl);
  const expression = buildBrowserScriptExpression({ scriptSource, entrypointExpression, args });
  return await session.evaluateByValue<T>(expression);
}

export function createAction(
  config: ActionConfig,
  deps: { capabilityDiscovery: BrowserCapabilityDiscovery },
): Action {
  return {
    config,
    execute: async (args): Promise<ActionResult> => {
      const prompt = String(args.prompt ?? "").trim();
      const page = await openDiscoveredPageSession(deps.capabilityDiscovery, "https://chatgpt.com/");
      try {
        const generated = await runScript<{ conversationUrl: string }>(
          page.session,
          GENERATE_SCRIPT_URL,
          "generateChatGPTImage",
          [{ prompt, timeout: 900_000 }],
        );

        // Raw CDP between script runs, in the same tab.
        await page.session.send("Page.reload", { ignoreCache: true });
        // ...poll isPageReadyForAutomation(...) until the reload settles...

        const image = await runScript<{ imageDataBase64: string; mimeType: string }>(
          page.session,
          DOWNLOAD_SCRIPT_URL,
          "downloadGeneratedChatGPTImage",
        );

        return { status: "ok", data: { mimeType: image.mimeType, browser: page.endpoint.name } };
      } catch (error) {
        return { status: "error", error: error instanceof Error ? error.message : String(error) };
      } finally {
        await page.close();
      }
    },
  };
}

Watch out:

This drives the guardian's real, logged-in browser. Scripts run with the page's session and cookies — treat every automation as a write to that account unless you know it's read-only.

loadCachedScriptSource caches by URL for the life of the process. Edit a script file and the running daemon keeps evaluating the old source until the process reloads.

args are serialized as literals into the page. Functions, Dates, class instances, and non-finite numbers throw before anything reaches the browser.

Getting a browser endpoint

Run a script in a page

Hold a session open

End-to-end example

On this page