Skip to content

Connect a Mac desktop

Humagen can use Peekaboo's native MCP server for exact-window screenshots, accessibility inspection, and native controls. examples/desktop/humagen.yaml pins the real @steipete/peekaboo@4.9.9 server and twelve reviewed tool schemas. It runs on the Mac as a stdio process, with explicit foreground authority and no remote-host discovery. Its prompt limits actions to the named application and window; the granted native tools can perform foreground input, so use this only on the trusted Mac worker.

Configure the worker

The worker needs macOS, Node.js 22 or newer, an unlocked local GUI session, Screen Recording, and Accessibility permission. Synthetic input can require additional Accessibility authorization. Check the native server's current status:

npx --yes @steipete/peekaboo@4.9.9 permissions status --no-remote --json

The person grants missing permissions in System Settings > Privacy & Security, to the actual worker/terminal process running Peekaboo, then restarts that worker. Permissions alone do not unlock a GUI session. A locked Mac returns an explicit capture error; neither Humagen nor this configuration unlocks it.

The examples explicitly pass --allow-foreground to the worker. Peekaboo's default background-only mode refuses actions such as AXPress; foreground authority is necessary for the fixture's button interaction. This flag authorizes the worker's foreground actions and does not unlock the Mac or grant TCC access.

Use the tools entry in an existing authenticated workspace and grant mcp:desktop:write only to the chosen agent. Its model must support image input and have a working provider credential. The standalone example reads ANTHROPIC_API_KEY and starts with:

humagen start -f examples/desktop/humagen.yaml

Use an existing Claude Code subscription

examples/desktop/claude-code.yaml connects the same pinned worker through the program's own MCP configuration. Set its dir to an existing absolute work directory, sign in to Claude Code on this Mac, then run:

humagen start -f examples/desktop/claude-code.yaml

This profile needs Node/npx and Claude Code on the server's PATH. It disables built-in CLI tools, ignores other MCP configurations, and preauthorizes only the listed desktop tool names. The program's own permissions govern this host path; Humagen's accepted-tool hashes govern the native profile above. An API key is not required for the subscription profile.

On the review Mac, the permission tool was actually called through a Humagen HTTP run on this path: Screen Recording, Accessibility and Event Synthesizing reported Granted. Native capture and interaction remain blocked while the Mac reports CGSSessionScreenIsLocked: true; no successful app interaction is claimed.

The standalone example uses development sign-in and memory state for trusted local fixtures. Every client reaching a dev API is an administrator. For real work, use the authenticated workspace instead. A screen or document can contain hostile instructions, and the agent's prompt is not an authorization boundary. The explicit write grant authorizes the tool group; configure Humagen's approval policy when individual actions must wait for a person.

The unisolated sandbox in the local example is deliberate: a Linux container cannot operate the Mac's desktop. Run this on the person's trusted Mac worker, not a shared server. --no-remote keeps the native target on that worker. No background failure is retried using global input or another host.

Observe, act, verify

Call permissions, then see with the exact application/PID and window ID. Use its snapshot reference and actionable element IDs. After a change, call see again; an earlier snapshot is not authority to act on a later state. The selected schemas describe text replacement with type and clear: true, and button activation with action and AXPress. The adapter is intended to carry returned PNG bytes as it does browser screenshots. These native actions, the snapshot reference parser and PNG delivery remain unverified while capture is blocked. The live fixture check must pass before claiming this workflow.

The selected inventory omits set_value, whose union-valued schema is not representable by the current saige parameter schema. It also omits the server's nested agent, analysis, browser, and clipboard tools. Unsupported schemas are not weakened or silently accepted. Text entry uses the supported type tool.

Dotfiles, processes and printers

For host commands, use the configured Claude Code or Codex runtime in an explicit work directory. See the local folder example for Claude Code setup. Grant only the commands needed, for example Bash(dotfiles doctor:*), Bash(lpstat:*), and Bash(ps:*). Use the machine's dotfiles CLI for managed configuration instead of editing deployed files by hand. A diagnosis does not authorize applying every proposed fix. Printer commands still need an installed, reachable printer; a missing device is not successful printer coverage. Humagen's model/tool grants do not gate the external program's tools; its own permissions and allow rules do.

Real native acceptance

With the pinned binary installed on the test worker:

HUMAGEN_DESKTOP_MCP_COMMAND=/absolute/path/to/peekaboo \
  go test -race ./internal/mcptools -run '^TestDesktopLive$' -v

This compiles and opens a temporary AppKit fixture, checks the pinned MCP tool set, targets only that fixture's exact PID/window, enters text, presses its button, checks the application's independent action receipt, then checks the changed heading and returned PNG pixels. The fixture is terminated and removed afterward. HUMAGEN_DESKTOP_CAPTURE_DIR optionally retains its final PNG for review. A locked session or missing permission fails this test rather than skipping the native actions or claiming coverage.