What Agents Can Do
Electron Apps
Slack, VS Code, Discord and Notion are desktop apps built on Electron, which is Chromium inside. Once an app starts with a debug port, your agent can work in it like a web page: read the UI, click buttons, type into fields and take screenshots.
Ask your agent
Search the Slack desktop app for last week's release discussion and send me a screenshot.
Same search box as the web app; the results are saved as a screenshot.
Open VS Code settings and tell me what auto save is set to.
It reads the app's UI; it doesn't edit config files.
Copy out the to-do list on the current page of the Notion desktop app.
Text comes from UI elements, or the whole page as JSON.
What the agent does
- The app has to start with
--remote-debugging-port. An app that is already running must be quit and relaunched with the flag. - The agent attaches with
connect <port>; later commands go to that app. - It reads the UI with
snapshot -ias usual and clicks or types by ref, re-reading after the UI changes. - If the app has several windows or webviews,
tablists them and switches to the right one. - To drive several apps at once, each app gets a named session (
--session) on its own port.
What you'll see
- This works on the desktop app itself. It does not go through your Chrome and there is no tab group.
- If the app was already open it has to restart once, so save your work first.
- Over the debug port the app may switch to its light theme; to keep dark mode, have the agent add
--color-scheme dark. - If it can't connect or can't find an element, see Troubleshooting below.
Under the hood
The commands and flags the agent runs. Read on if you write your own scripts, debug a run, or want the details.
Slack, VS Code, Discord, Figma, Notion, Spotify β these desktop apps are really just Chromium in a shell: they're all built on Electron.
Launch them with --remote-debugging-port and chrome-use can attach exactly like it would to a web page:
the same snapshot β interact β snapshot again loop, the same @ref targeting.
The core flow
Automating an Electron app is just five steps: launch (with remote debugging) β connect to the CDP port β snapshot to discover interactive elements β interact by ref β snapshot again after the state changes. It's barely any different from driving an ordinary web page.
# launch the Electron app with a remote debugging port
$ open -a "Slack" --args --remote-debugging-port=9222
# attach chrome-use to that app
$ chrome-use connect 9222
# from here it's the standard workflow
$ chrome-use snapshot -i
$ chrome-use click @e5
$ chrome-use screenshot slack-desktop.png
Launch the app with CDP
Every Electron app supports --remote-debugging-port, because it's a built-in Chromium capability.
Here's how to launch on all three platforms β assign a different port to each app and you can drive several at once.
macOS
# Slack
$ open -a "Slack" --args --remote-debugging-port=9222
# VS Code
$ open -a "Visual Studio Code" --args --remote-debugging-port=9223
# Discord
$ open -a "Discord" --args --remote-debugging-port=9224
# Figma
$ open -a "Figma" --args --remote-debugging-port=9225
# Notion
$ open -a "Notion" --args --remote-debugging-port=9226
# Spotify
$ open -a "Spotify" --args --remote-debugging-port=9227
Linux
$ slack --remote-debugging-port=9222
$ code --remote-debugging-port=9223
$ discord --remote-debugging-port=9224
Windows
$ "C:\Users\%USERNAME%\AppData\Local\slack\slack.exe" --remote-debugging-port=9222
$ "C:\Users\%USERNAME%\AppData\Local\Programs\Microsoft VS Code\Code.exe" --remote-debugging-port=9223
--remote-debugging-port has to be present the moment the app launches. If the app is already open, quit it first, then relaunch it with the flag β otherwise you won't be able to connect.
Connecting
There are three ways to connect: connect <port> directly, --cdp on each command, or let it auto-discover.
Once you've run connect, every subsequent command automatically targets that app β no need to repeat the port.
# connect to a specific port
$ chrome-use connect 9222
# or use --cdp on each command
$ chrome-use --cdp 9222 snapshot -i
# auto-discover a running Chromium-based app
$ chrome-use --auto-connect snapshot -i
Windows and tab management
Electron apps often have multiple windows or webviews. Use the tab command to list and switch between these targets.
Note that tab ids look like t1, t2 β they don't accept bare integers.
# list all available targets (windows, webviews, etc.)
$ chrome-use tab
# switch to a tab by id (t1, t2 β¦; integers not accepted)
$ chrome-use tab t2
# switch by URL pattern
$ chrome-use tab --url "*settings*"
Webview support
Electron <webview> elements are discovered automatically and can be driven like any ordinary page.
They show up in the tab list as independent targets, with type type: "webview".
# connect to the running Electron app
$ chrome-use connect 9222
# list targets β webviews appear alongside pages
$ chrome-use tab
# [t1] [page] Slack - Main Window https://app.slack.com/
# [t2] [webview] Embedded Content https://example.com/widget
# switch to a webview
$ chrome-use tab t2
# interact as usual
$ chrome-use snapshot -i
$ chrome-use click @e3
$ chrome-use screenshot webview.png
tab list, you can drive it.
Common patterns
Here are a few combinations you'll reach for again and again on desktop apps. They all reuse the verbs you already know from web automation.
Explore and navigate
$ open -a "Slack" --args --remote-debugging-port=9222
$ sleep 3 # wait for the app to start
$ chrome-use connect 9222
$ chrome-use snapshot -i
# read the snapshot output and identify UI elements
$ chrome-use click @e10 # jump to a section
$ chrome-use snapshot -i # snapshot again after navigating
Screenshots
$ chrome-use connect 9222
$ chrome-use screenshot app-state.png
$ chrome-use screenshot --full full-app.png
$ chrome-use screenshot --annotate annotated-app.png
Extract data
$ chrome-use connect 9222
$ chrome-use snapshot -i
$ chrome-use get text @e5
$ chrome-use snapshot --json > app-state.json
Fill a form
$ chrome-use connect 9222
$ chrome-use snapshot -i
$ chrome-use fill @e3 "search query"
$ chrome-use press Enter
$ chrome-use wait 1000
$ chrome-use snapshot -i
Drive several apps at once
Use named sessions to control several Electron apps simultaneously without them interfering with each other. See Sessions & concurrency.
# connect Slack
$ chrome-use --session slack connect 9222
# connect VS Code
$ chrome-use --session vscode connect 9223
# interact with each independently
$ chrome-use --session slack snapshot -i
$ chrome-use --session vscode snapshot -i
Color scheme (dark mode)
When you connect over CDP, the default color scheme may be light. If you want to keep the app's dark mode, use --color-scheme dark,
or set it globally with an environment variable.
# keep dark for a single command
$ chrome-use connect 9222
$ chrome-use --color-scheme dark snapshot -i
# or set it globally
$ AGENT_BROWSER_COLOR_SCHEME=dark chrome-use connect 9222
Troubleshooting
"Connection refused" / can't connect
- Confirm the app was launched with
--remote-debugging-port=NNNN - If the app was already running, quit it and relaunch with the flag
- Check the port isn't taken by another process:
lsof -i :9222
App launched but connect fails
- Wait a few seconds after launch before connecting (
sleep 3) - Some apps need time to initialize their webviews
An element isn't showing up in the snapshot
- The app may use multiple webviews. Use
chrome-use tabto list targets and switch to the right one
Can't type into an input
- Use
chrome-use keyboard type "text"to type at the current focus without a selector - Some Electron apps use custom input components β use
chrome-use keyboard inserttext "text"to bypass key events
Supported apps
Any app built with Electron supports --remote-debugging-port and can be automated by chrome-use. Common ones include:
Slack, Discord, Microsoft Teams, Signal, Telegram Desktop
VS Code, GitHub Desktop, Postman, Insomnia
Figma, Notion, Obsidian
Spotify, Tidal
Todoist, Linear, 1Password