extension.js · part 25

60 text occurrences from desktop/Cursor.app/Contents/Resources/app/extensions/cursor-computer-use/dist/extension.js, part 25. Every entry preserves the shipped literal and its saved verdict or selection reason.

File contents and all parts · All files

Type text into the focused element using the keyboard. Click the target first so it has focus. A newline in the text presses Enter (it submits or navigates) and a tab presses Tab (next field or cell), so do not send a separate Return or Tab key press for them.

Type text into the focused element using the keyboard. Click the target first so it has focus. Returns a screenshot taken after typing.

Text to type.

Invalid arguments for computer_type: target: Screen-scope typing is refused. Pass {"scope":"app","pid":…,"target_id":…}. Prefer computer_set_value or computer_type with element_id when the field is marked settable. Without element_id, click a text field first if nothing text-like is focused.

Press a key or shortcut, such as "Return", "Escape", "Tab", or a chord like "cmd+l" on macOS and "ctrl+l" on Windows. Combine modifiers with "+" (cmd, ctrl, alt, shift). Returns a screenshot taken after the key press.

Key or chord.

Key or shortcut, e.g. "Return", "Tab", or "cmd+shift+s".

Pressed the key.

Pressed the key.

Scroll at a screen coordinate.

Scroll direction. Defaults to down.

Scroll amount in wheel ticks. Defaults to ${mc}.

Wait for a page load or animation to settle. Returns text only; it does not capture the display. Observe afterward with computer_screenshot (app target) or computer_app_state.

Wait for the screen to settle (page load, animation). Returns a screenshot only if the screen changed; otherwise text that your last one still stands. Prefer over repeated screenshots.

Wait for the screen to settle, such as during a page load or animation, then return a fresh screenshot. Prefer this over repeated screenshots in a tight loop.

Milliseconds to wait. Defaults to 1000.

Waited ${o}ms. Observe with computer_screenshot (app target) or computer_app_state.

Waited ${o}ms.

Captured a screenshot.

Carry out a list of small UI steps in a macOS app without reading the tree yourself. Prefer it when you know the next few UI steps; for a single action on an element you already know, call the element tool directly. Decompose the task first: each step is ONE control or ONE visible state change (press one button, open one sidebar item, type "text" into one field, confirm one field), with an expect naming what the screen shows once it is done. Never make a step that only focuses or clicks a field: typing into a field focuses it, so put the text in the typing step (type "x" into the search field). Each expect must name a control or value that appears in the tree ("a button named X appears", not "search results are visible" or "the field is focused"). Do not pass a multi-step task as one step (not "enter 1+2+3+4"; pass press 1 / press Add / press 2 ... as separate steps, each with its expected display); the decider inside is fast and accurate at picking one control and checking one outcome, and does not plan sequences. For each step a deterministic loop reads computer_app_state, asks the decider for the next action, acts, and checks the outcome, up to max_steps actions (default ${bA}); it stops the step as done when the screen shows its result and expect holds, or as failed when its target is absent, disabled, or blocked. Steps run in order and stop at the first that fails. Text to enter goes in quotes inside the step. Returns OK or ERR with one status line per step (done, failed, not run), then changed: (what the call changed on screen), now: (a compact view of the final screen: its controls and values, windows, sheets, and menus), a short trace, and the final snapshot_id. Plan your next steps from that view, and read values from now: before calling computer_app_state, which you need only for element ids or more detail. A failed step with 0 actions was not tried and changed nothing; one that acted may have landed, so check now: before redoing it. On ERR do only the failed step with the element tools, then send the remaining steps back to computer_attempt. Text only, no image.

A single small step: one control or one visible state change, in plain words, with any text to type in quotes (e.g. press the Add key; type "house music" into the search field). Pass this or steps.

What the screen shows once this step is done, e.g. the display shows 1+. Give it on every step; the loop does not mark a step done until it holds.

The decomposed task: one {step, expect} per control or state change, in order. Pass this or step.

One control or one visible state change.

What the screen shows once this step is done.

Actions to try per step before giving up. Default ${bA}; a well-decomposed step needs one.

computer_attempt needs the macOS computer-use helper and a signed-in Cursor account.

Passively report whether the Accessibility and Screen Recording permissions the computer-use helper needs are granted. On Windows both are always granted. Call computer_check_permissions first and wait for its result before any other Computer Use tool; do not call Computer Use tools in parallel, they act on one machine and run one at a time.

Reported permission state.

Escalation only: take over the display and move the real cursor. App targets need no session. Do not call this to type or fill a form.

Start a new remote-control session before sending input.

Remote control is ready.

Release remote control after the final 

Released remote control.

Return a region of the latest screenshot as a separate image at native resolution (up to ${vn}×), so small or dense UI becomes legible. (x1, y1) is the top-left corner and (x2, y2) the bottom-right, inclusive; ${nc}% padding is added; the region may be up to ${Sn} canvas px on its long edge at any aspect, and one under ${Nn.width}×${Nn.height} is widened to it. The result quotes a zoom_id and the image size: pass that zoom_id with x and y in zoom-image pixels to computer_click, computer_move, computer_scroll, or computer_drag. Actions still return a full-canvas screenshot, and a zoom_id describes the screen as it was when taken.

Zoom needs the Windows computer-use sidecar. Read the screenshot instead.

${i.text} The requested ${s.width}×${s.height} canvas px region was widened to the ${Nn.width}×${Nn.height} minimum around it, so the target has its surroundings in view.

List running macOS apps, frontmost first, with the pid and target_id an app target needs. Regular applications only: Spotlight, Notification Center, and menu extras are unsupported. No permission or screen control is required.

Listed apps.

Resolve one macOS app by exactly one selector, launching it without activating it if it is not running, and return the pid and target_id for its app target.

Bundle identifier, e.g. "com.apple.calculator".

Path to the .app bundle.

Localized app name, e.g. "Calculator".

Resolved the app.

Read the app's accessibility tree as text (no image): one element per line, "[id] ROLE name=… value=… settable actions=…", indented by depth; settable means computer_set_value can replace the value. The last line is the snapshot_id that element actions must quote. Call this when you need the tree (first observe, or a screenshot that omitted snapshot_id). Skip it when a screenshot reused a live snapshot_id. Call again when an action reports the snapshot stale. A line ending in "(+N descendants omitted)" was cut to fit: pass that element_id with the current snapshot_id to expand it, and the new ids join the same snapshot.

Read the app state.

Replace the value of a settable element by id through accessibility, without keystrokes. Returns text only; the snapshot_id stays valid while the tree is unchanged, so fill the other elements from the same computer_screenshot or computer_app_state result. If the edit opens a sheet or replaces rows, the result says the snapshot is stale — take a screenshot or read computer_app_state before using element ids.

The new value.

Set the value.

Perform an accessibility action on an element by id: "press" for buttons, links, menu items, checkboxes, and tabs (implied, so not listed), or an entry from the element's actions= list such as "show-menu" or "scroll-to-visible". Prefer this over coordinate clicks when the element is in the tree. Returns text only; the snapshot_id stays valid until the tree changes, so act on more elements from the same computer_screenshot or computer_app_state result.

"press", or an action name from the element's actions= list.

Performed the action.

Pass either step (with optional expect) or steps, not both and not neither.

Unknown computer-use tool: ${n}

Remote control is ready.

${(o=e.refusal).message}

Refusal code: ${o.code}. Recommended next step: ${o.escalation.recommended} — ${o.escalation.reason}

USER_ABORTED: The user stopped the current control session. Do not call input, start-control, or release-control tools again in this turn.

When you roughly know the next few UI steps, send them all in one computer_attempt call, each with an expect that names a specific visible control or value. Skip focus-only steps; typing into a field focuses it. When a step fails, do only that step with the element tools, then send the remaining steps back to computer_attempt. For a single action on an element you already know, call the element tool directly.

[${ad(n)}] ${e}

message size is larger than configured readMaxBytes ${e}