# Browser Automation
URL: /docs/building-apps/capabilities/browser-automation
Markdown: /docs/building-apps/capabilities/browser-automation.md
Description: Drive the guardian's agent browser over CDP from app backend code.

A Rome App can drive the guardian's agent browser — the logged-in Chromium the
agent uses — over the Chrome DevTools Protocol (CDP). The client lives in a
dedicated subpath of the runtime SDK:

```ts
import {
  openDiscoveredPageSession,
  runDiscoveredBrowserScript,
} from "@rome-os/app-runtime/browser";
```

It is a separate subpath on purpose: the module is Node-only (it uses `ws` and
`node:fs/promises`) and is **not** re-exported from the `@rome-os/app-runtime`
barrel, so it can't leak into your web bundle. Import it only from backend code
— [actions](/docs/building-apps/capabilities/actions), hooks, API handlers.

## Getting a browser endpoint [#getting-a-browser-endpoint]

You don't configure browser URLs yourself. The host injects a
`BrowserCapabilityDiscovery` into your action's `deps` as
`deps.capabilityDiscovery`; it lists the CDP endpoints Rome has discovered:

```ts
export interface BrowserCapabilityDiscovery {
  getBrowserEndpoints(): DiscoveredBrowserEndpoint[];
}

export interface DiscoveredBrowserEndpoint {
  name: string;       // e.g. "cdp-local-chromium"
  browserUrl: string; // CDP HTTP endpoint
}
```

`pickBrowserEndpoint(endpoints)` chooses one for you: it prefers
`cdp-local-chromium`, then any name containing `local`, then the first
alphabetically; it returns `null` when the list is empty. The two entry points
below call it internally, so you usually never call it directly — pass
`deps.capabilityDiscovery` straight in.

## Run a script in a page [#run-a-script-in-a-page]

`runDiscoveredBrowserScript(capabilityDiscovery, options)` is the one-shot
path: pick an endpoint, open `pageUrl` in a new tab, wait for the page to load,
load your script file, evaluate it, and hand back the result.

```ts
const run = await runDiscoveredBrowserScript<MyResult>(deps.capabilityDiscovery, {
  pageUrl: "https://example.com/",
  scriptUrl: new URL("./scraping_scripts/scrape.js", import.meta.url),
  entrypointExpression: "scrapeExample",
  args: [{ query: "rome" }],
});
try {
  console.log(run.result); // typed as MyResult
} finally {
  await run.close();
}
```

| Option                     | Notes                                                                                                                                                                                 |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `pageUrl`                  | URL to open in a fresh tab.                                                                                                                                                           |
| `scriptUrl`                | `URL` of a plain JS file on disk, usually built with `new URL(..., import.meta.url)`. Loaded once per process via `loadCachedScriptSource`.                                           |
| `entrypointExpression`     | Expression that must resolve to a function after your script runs, e.g. a function name the script defines. It is called with `args` and its (awaited) return value becomes `result`. |
| `args`                     | Optional arguments serialized into the page. Only plain JSON-ish values: strings, finite numbers, booleans, `null`/`undefined`, arrays, plain objects. Anything else throws.          |
| `autorunFlag`              | Optional global flag name set to `false` while the script loads (and cleaned up after) — lets a script that self-runs when injected by other tooling skip its autorun path here.      |
| `missingEntrypointMessage` | Optional error message when `entrypointExpression` isn't a function after the script loads.                                                                                           |

The result is `{ endpoint, result, close() }`. On evaluation failure the tab is
closed for you and the error is thrown; on success **you** own the tab.

> **Watch out:** always call `close()` in a `finally`. It tears down the CDP
> socket and closes the tab; skipping it leaves a tab open in the guardian's
> browser after every run.

## Hold a session open [#hold-a-session-open]

When one page needs several script runs or raw CDP commands (reload, bring to
front, …), open the page once with
`openDiscoveredPageSession(capabilityDiscovery, pageUrl)`. It opens a tab,
enables the `Page` and `Runtime` domains, brings the tab to front, and waits up
to 20s for the document to be ready before returning
`{ endpoint, session, close() }`.

The `session` is a `CdpSession`:

| Member                                   | Notes                                                                                                                                                                                      |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `CdpSession.connect(webSocketUrl)`       | Connect to a raw CDP WebSocket yourself. You rarely need this — `openDiscoveredPageSession` does it for you.                                                                               |
| `session.send(method, params?)`          | Send any CDP command, e.g. `session.send("Page.reload", { ignoreCache: true })`.                                                                                                           |
| `session.evaluateByValue<T>(expression)` | `Runtime.evaluate` with `awaitPromise` + `returnByValue`: promises are awaited and the value is serialized back, so `T` must be JSON-serializable. Page exceptions become thrown `Error`s. |
| `session.close()`                        | Close the WebSocket only. Prefer the page's `close()`, which also closes the tab.                                                                                                          |

Three lower-level helpers compose into "run a script in this open session":
`loadCachedScriptSource(scriptUrl)` reads (and caches) the script file,
`buildBrowserScriptExpression({ scriptSource, entrypointExpression, args, ... })`
wraps it into one evaluatable expression, and you pass that to
`session.evaluateByValue`. After a navigation or reload, poll
`isPageReadyForAutomation({ readyState, href })` — it is `true` once the page
has a real URL (not `about:blank`) and `readyState` is `interactive` or
`complete`.

## End-to-end example [#end-to-end-example]

Condensed from the first-party `browser-automation` app's `chatgpt_image`
action (`rome_apps/browser-automation/src/actions/image/index.ts`): one tab,
two script runs, a reload in between, cleanup in `finally`.

```ts
import type { Action, ActionConfig, ActionResult } from "@rome-os/app-runtime";
import type { BrowserCapabilityDiscovery, CdpSession } from "@rome-os/app-runtime/browser";
import {
  buildBrowserScriptExpression,
  loadCachedScriptSource,
  openDiscoveredPageSession,
} from "@rome-os/app-runtime/browser";

const GENERATE_SCRIPT_URL = new URL("./scraping_scripts/image_action.js", import.meta.url);
const DOWNLOAD_SCRIPT_URL = new URL("./scraping_scripts/image_download.js", import.meta.url);

async function runScript<T>(
  session: CdpSession,
  scriptUrl: URL,
  entrypointExpression: string,
  args?: unknown[],
): Promise<T> {
  const scriptSource = await loadCachedScriptSource(scriptUrl);
  const expression = buildBrowserScriptExpression({ scriptSource, entrypointExpression, args });
  return await session.evaluateByValue<T>(expression);
}

export function createAction(
  config: ActionConfig,
  deps: { capabilityDiscovery: BrowserCapabilityDiscovery },
): Action {
  return {
    config,
    execute: async (args): Promise<ActionResult> => {
      const prompt = String(args.prompt ?? "").trim();
      const page = await openDiscoveredPageSession(deps.capabilityDiscovery, "https://chatgpt.com/");
      try {
        const generated = await runScript<{ conversationUrl: string }>(
          page.session,
          GENERATE_SCRIPT_URL,
          "generateChatGPTImage",
          [{ prompt, timeout: 900_000 }],
        );

        // Raw CDP between script runs, in the same tab.
        await page.session.send("Page.reload", { ignoreCache: true });
        // ...poll isPageReadyForAutomation(...) until the reload settles...

        const image = await runScript<{ imageDataBase64: string; mimeType: string }>(
          page.session,
          DOWNLOAD_SCRIPT_URL,
          "downloadGeneratedChatGPTImage",
        );

        return { status: "ok", data: { mimeType: image.mimeType, browser: page.endpoint.name } };
      } catch (error) {
        return { status: "error", error: error instanceof Error ? error.message : String(error) };
      } finally {
        await page.close();
      }
    },
  };
}
```

> **Watch out:**
>
> * This drives the guardian's real, logged-in browser. Scripts run with the
>   page's session and cookies — treat every automation as a write to that
>   account unless you know it's read-only.
> * `loadCachedScriptSource` caches by URL for the life of the process. Edit a
>   script file and the running daemon keeps evaluating the old source until
>   the process reloads.
> * `args` are serialized as literals into the page. Functions, `Date`s, class
>   instances, and non-finite numbers throw before anything reaches the
>   browser.