Getting Started

To drive the browser through a task, the rhythm is almost always these four steps:open the page → read the controls → act on @refs and observe → re-read only when needed. Get this loop flowing, then understand when @ref goes stale and when it self-heals, and you've got ninety percent of chrome-use's day-to-day usage.

The core skill includes the default observation and stopping rules, plus ordinary click and form commands. Load core/reading, core/connection, or core/site-adapters only when needed. Choose the cheapest check that answers the next question; do not request both DOM and screenshots by default. An authoritative page signal ends verification unless contradicted. Missing observations do not justify replaying an action.

Fewer calls, measured per task

Start with snapshot -i -c for compact controls or scope the read to a known panel. Use batch for known action sequences, script for bounded conditional flows, and form fill --map for several fields. Verify the requested page signal at the end. Grouping actions does not establish their effects.

Keep text and refs as the default. Add --with-screenshot <path> when pixels answer a question the tree cannot. An observed.noProgress hint advises inspecting state after repeated unchanged attempts; it does not change success or retry. See detection limits.

JSON command timing reports cdpMs as the sum of completed foreground request durations, cdpBusyMs as their interval union within command wall time, and nonCdpMs as wall time minus that union. Concurrent requests can make the sum exceed wall time. These fields measure elapsed duration, not CPU use or model latency. Background tasks do not inherit the recorder; the remaining wall time is not pure daemon processing cost.

Compare the same task and initial state with a freshly built binary; record its version, separate cold and warm runs, and alternate variants. Record success, unknown outcomes, tool and CDP calls, returned bytes and task wall time. Model round trips require caller traces; tool/HTTP calls cannot establish them. Measure tokens with a named tokenizer or model usage report. A faster failed run is not a speedup. Load chrome-use skills get core/profiling for timing details; the repository's bench/README.md describes the replay harness.

From the repository root, collect one explicit run with python3 bench/run-task.py TASK --binary cli/target/release/chrome-use --output /tmp/run.tsv --run-id run-1, then summarize it with python3 bench/task-metrics.py /tmp/run.tsv. Without a complete matching caller trace (--trace), model turns are null. Bytes combine CLI stdout/stderr, not exact model input. --cold only skips warmup; it does not prove a stopped daemon.

The four-step loop

Each step decides based on the result of the last: snapshot -i gives you a structured view containing only interactive elements, each carrying a short @ref; you take a ref to click / type; use the returned observation, and snapshot again only when target evidence is missing or stale.

$ chrome-use open <url>        # 1. open a page
$ chrome-use snapshot -i       # 2. see what's on the page (interactive elements only)
$ chrome-use click @e3 --observe   # 3. act, and get what changed in the same call
$ chrome-use snapshot -i --diff    # 4. re-read only when you need to: just the delta

Steps 3 and 4 are one round trip each. --observe returns the delta after the action (the new page's whole tree after navigate, or when a click replaced the whole page) together with the requests the action fired, so "re-snapshot after every click" is the habit to drop. --diff says no change explicitly when nothing moved instead of handing you the same tree again. The rules around this loop live in chrome-use skills get core/behaviour.

Valid incremental results include JSON refs only for the returned delta; unchanged results have empty refs while the live session map is retained. Full-tree fallback keeps normal full-snapshot behavior.

If only the post-action capture fails, observed.refs says what became of the session's refs. kept-unverified: the previous snapshot's refs are kept, but a later command may use one only after chrome-use confirms live that the tab, frame, document and element are unchanged; otherwise the command is refused before it acts, and a ref is never re-anchored by role or name. Refs from a DOM-walk snapshot (snapshot --dom) are always refused in this state. dropped: the page may be a different document now. Either way, run snapshot -i for current refs, and do not repeat the action.

The browser stays running between commands, so these feel like one session. When you're done, wrap up with chrome-use close. close --all would close other sessions too, so it refuses while any are live; it is not for routine cleanup. To read the page, find elements, or interact in more detail, see Reading the Page, Finding Elements, and Interacting.

$ chrome-use open https://duckduckgo.com
$ chrome-use snapshot -i                      # find the search box's ref
$ chrome-use fill @e1 "chrome-use cli"
$ chrome-use press Enter
$ chrome-use wait --load networkidle
$ chrome-use snapshot -i                      # the refs now reflect the search results
$ chrome-use click @e5                        # click a result
$ chrome-use screenshot result.png

@ref stays stable within a document and resets after navigation

The same backend DOM node keeps its ref (@e1, @e2, …) across successive snapshots. Opening or closing a modal and inserting list rows no longer renumbers every later control. Navigation and tab switches hard-reset the identity map, so snapshot again before the next ref-based action after those boundaries.

@ref also self-heals: you don't have to re-snapshot for every tiny DOM change

Each ref records a fingerprint — role + accessible name + ancestor path. When you use it, if the original node is gone, chrome-use automatically re-locates the matching element on the current page and carries on. So after a React / Vue list re-render (labels unchanged), click @e3 still hits the right element.

If the element is genuinely gone, it fails loudly rather than clicking the wrong node — it never silently mis-targets. So: you only need to re-snapshot when you've navigated, or when the labels / structure genuinely changed.

Hard rule: snapshot-first, never locate by screenshot

For form fields and buttons, always snapshot -i first, then act on the ref / selector. Do not use screenshot + coordinate clicks to find or hit elements.

snapshot -i now pierces cross-origin iframes (embedded Google Payments / Stripe / checkout / KYC forms), listing their elements — along with their input values — by @ref, so click @e / type @e / fill @e reach straight into the iframe. Only use coordinates for canvas / WebGL, or when snapshot genuinely returns nothing for your target.

Two different intents — only one is discouraged

The rule above targets screenshot-to-locate (using an image to find / hit an element) — that's the bug. Whereas screenshot-to-capture (saving a region or element to a file as reusable image assets — maps, charts, og images, visual-regression baselines, report illustrations) is fully supported and encouraged:

# save a rendered region as a reusable image asset (not for locating)
$ chrome-use screenshot [selector] --clip x,y,w,h <file>

# want to click something a ref can't hit? box gives the element's CSS-pixel box + center
$ chrome-use box @ref
$ chrome-use click <centerX> <centerY>

Screenshots auto-downsample to a longest edge of ≤ 1200px (width only for full-page shots) so image readers can take them in and so pixels line up with click x y; override with --max-width / --max-height / --scale. To click an element a ref can't hit, box @ref gives its CSS-pixel box and centerX/centerY, which you feed straight to click <centerX> <centerY> — no screenshot needed. For canvas / WebGL coordinate tricks, see Screenshots · Recording · Viewport and the canvas-related sections.

Two driving paths — and when to switch to eval

What you have in hand is a real Chrome with the user's real DOM. There are two layers, and you can mix them freely:

# list every field's name / type / value / checked in the form
$ chrome-use eval "[...document.forms[0].elements].map(e=>[e.name,e.type,e.value,e.checked])"

# read a hidden field the UI never exposes
$ chrome-use eval "document.querySelector('[name=point_choice]')?.value"

# find every field that fails validation and its error message
$ chrome-use eval "[...document.forms[0].elements].filter(e=>!e.validity.valid).map(e=>e.name+': '+e.validationMessage)"

# click directly via the DOM, bypassing overlays
$ chrome-use eval "document.querySelector('#stubborn').click()"

The escalation ladder is: for straightforward pages, snapshot + @eN is fastest → when you don't want to snapshot, use find role/text/label → native CSS selectors → the moment any of the above fights you, go to eval and act on the DOM directly. For more locating techniques, see Finding Elements.

Next steps

Once the core loop is flowing, dive deeper as needed:

The local agent side panel on macOS, Linux and Windows supports /skill selection, image attachments, conversation tool/model options and scheduled prompts in independent background sessions. See the side panel guide.