Core Usage

Finding Elements

Most of the time the @ref you get from snapshot is all you need. But sometimes a ref goes stale, the target is buried in a hidden state, or you simply don't want to snapshot first. This page covers the other ways to lock onto a target — the find semantic locators, raw CSS, coordinate clicks — plus the ladder that escalates to eval when structured locating runs out.

When not to use @ref

@ref is the fastest path, but it is built on top of each snapshot and expires the moment the page changes. In the situations below, reaching straight for a semantic locator or CSS is less work:

The page is simple, and running a separate snapshot just to locate one button isn't worth it;
refs keep going stale (after navigation, form submission, or a dynamic re-render);
you already know the element's role, visible text, label, or a stable selector.

find semantic locators

find locates elements by semantics instead of a ref: role + name, visible text, form label, placeholder, data-testid, or "first / Nth" match. On a hit, chain an action verb directly (click / fill / type / hover …). find label matches more than <label> elements — it also matches aria-label and aria-labelledby, so icon buttons and custom controls labeled only via ARIA can be found too. If you do not know the locator kind yet, pass a natural-language description directly. chrome-use ranks candidates by name, text, ancestor context, and role, includes cursor and compact selector anchors, and never acts automatically.

find — semantic locators

$ chrome-use find role button click --name "Submit"
$ chrome-use find "edit web service settings button"  # ranked candidates only
$ chrome-use find query "edit settings button"       # explicit query syntax
$ chrome-use find text "Sign In" click
$ chrome-use find text "Sign In" click --exact     # exact match only
$ chrome-use find label "Email" fill "user@test.com"
$ chrome-use find label "Close" click            # hits an icon button with only aria-label="Close"
$ chrome-use find placeholder "Search" type "query"
$ chrome-use find testid "submit-btn" click
$ chrome-use find first ".card" click
$ chrome-use find nth 2 ".card" hover

✅ Check the real label text first

Translation will trip you up. Real case: LinkedIn's "Save" button is labeled 收藏, not 保存, so find text "保存" finds nothing, while snapshot -i steadily shows button "收藏" [ref=eN] — just click @eN and you're done. Before concluding "the element doesn't exist," confirm its exact label.

Raw CSS selectors

Any action verb can take a CSS selector directly, no find required:

CSS selectors

$ chrome-use click "#submit"
$ chrome-use fill "input[name=email]" "user@test.com"
$ chrome-use click "button.primary"
$ chrome-use click "//button[@type='submit']"   # starts with //, /, (, ./ or .. = XPath, no xpath= prefix needed

ℹ️ XPath text() only tests the first direct text node

//button[text()='Save'] never matches when the text is split across nodes (React's {a} - {b}) or nested in a child element. Use contains(normalize-space(.), '…'), or find "<label>" / text=<label> / snapshot -i + @ref. When an XPath containing text() matches nothing, the error explains this gotcha, and "Element not found" now keeps the selector and the resolver's specific diagnosis instead of always blaming a closed shadow root / cross-origin iframe.

The escalation ladder

Locating an element follows a ladder from cheap to all-purpose. Walk down it in order, and don't get stuck grinding on one rung:

① snapshot + @eN

Fastest on cleanly structured pages. Grab the ref and click @eN.

② find role/text/label

When you'd rather not snapshot first, hit the target by semantics directly.

③ raw CSS

When you already have a stable selector, pass it straight to an action verb.

④ eval

The moment any rung above fights back (stale ref, hidden state, an occluded click), drop straight to eval and operate the DOM directly.

⚠️ Don't retry a flaky locator three times

When you hit a stale ref, a hidden state, or an occluded click, don't keep retrying the same structured locator — go straight to eval against the DOM; it's the fastest way to find out "why it failed." See Extracting Data for more.

click's auto-scroll and DOM fallback

click first scrolls the element into the viewport; if a coordinate click is occluded, it automatically falls back to the DOM's .click(). If a click reports success but nothing happens — the classic being an autocomplete / menu <li> that closes the instant it loses focus — retry that one click in DOM click mode, or just eval:

click reports success but does nothing

# retry this click in DOM click mode
$ AGENT_BROWSER_CLICK_MODE=dom chrome-use click @e3

# or select and click the item directly with JS
$ chrome-use eval "document.querySelector('#stubborn').click()"

Clicking by coordinates

When the only handle you have is a coordinate — a canvas, a marker from a screenshot, a target with no stable selector — click the pixel point directly:

coordinate click

$ chrome-use click 449 320            # click viewport coords (x y)
$ chrome-use click 449,320            # same, comma form
$ chrome-use click --coords 449,320   # same, explicit flag

ℹ️ A bare number is always a coordinate

A bare numeric argument is always treated as a coordinate, never as a selector. Coordinate clicks are the right tool for canvas / WebGL scenarios; for form fields and buttons, prefer snapshot + @ref — don't locate off a screenshot.

Where find can't reach

Both find and CSS selectors are based on the page DOM (querySelectorAll), so they report "Element not found" for two kinds of elements — even though snapshot -i plainly lists them (snapshots go through the CDP accessibility tree, which pierces both), and get text can read them too:

elements inside a closed shadow root;
elements inside a cross-origin iframe.

⚠️ Use snapshot's @ref for these two kinds

For elements inside a closed shadow root or a cross-origin iframe, locate them by their @ref from snapshot rather than find. When you need a coordinate fallback, box @ref gives the element's coordinate box. See the notes on cross-origin iframes in Interacting.

Next steps

📖 Reading the Page

snapshot gives a structured @ref view.

🖱️ Interacting

click, fill, type drive a real browser.

🧪 Extracting Data

eval and --schema scrape the page into clean JSON.

Semantic find role/text/label/placeholder/alt/title/testid requires exactly one visible match, including locate-only queries. Ambiguity returns up to eight candidates with visible state and selector/context hints; no action is dispatched and input values are omitted. Narrow --name/--exact, or use --within <CSS|@ref> to restrict the query to exactly one container. A scope must belong to the active tab's main document; cross-frame refs and an explicitly selected iframe are refused; use direct frame refs or `frame main`. find first/last/nth explicitly selects an order and keeps its existing behavior. A plain CSS selector or XPath given to an action (click, fill, type, hover, check, select, drag …) must match exactly one element too: one that matches several is refused with code selector_ambiguous, listing the first five matches with their text and position, and nothing is done. Narrow it, use a @ref, or choose by index with find nth <index> <selector> <action>; AGENT_BROWSER_STRICT_SELECTORS=0 (read when the session's daemon starts) restores first-match for old scripts. Roles and label names use Chrome accessibility data, including aria-labelledby; a semantic query re-resolves on each call. A detached target before dispatch fails safely; uncertain actions are never replayed.

Discover a scope from the page first: find query "Beta account" returns candidate selector anchors; choose the article/container corresponding to that heading, then use find role button click --name Save --exact --within "<returned-selector>". Inspect an ambiguous scope with snapshot -s "<selector>" and narrow it rather than accepting the first container. Unknown semantic-find options and duplicate --within are refused; use find label Email fill -- --name to enter literal flag-looking text. --exact false requests substring matching.

The existing chrome_use_find tool in mcp --tools all accepts within with the same unique-scope rules; tool count is unchanged. text for fill/type is passed literally, including --name --observe, rather than parsed as CLI options.

Semantic find retains the existing --observe unsupported warning; use one batch containing the scoped find action and a task-specific get text receipt when both are known. The MCP find tool does not advertise an observe field.