|
| 1 | +# Browser — technical details |
| 2 | + |
| 3 | +This document covers the command reference and internals of gstack's headless browser. |
| 4 | + |
| 5 | +## Command reference |
| 6 | + |
| 7 | +| Category | Commands | What for | |
| 8 | +|----------|----------|----------| |
| 9 | +| Navigate | `goto`, `back`, `forward`, `reload`, `url` | Get to a page | |
| 10 | +| Read | `text`, `html`, `links`, `forms`, `accessibility` | Extract content | |
| 11 | +| Snapshot | `snapshot [-i] [-c] [-d N] [-s sel]` | Get refs for interaction | |
| 12 | +| Interact | `click`, `fill`, `select`, `hover`, `type`, `press`, `scroll`, `wait`, `viewport` | Use the page | |
| 13 | +| Inspect | `js`, `eval`, `css`, `attrs`, `console`, `network`, `cookies`, `storage`, `perf` | Debug and verify | |
| 14 | +| Visual | `screenshot`, `pdf`, `responsive` | See what Claude sees | |
| 15 | +| Compare | `diff <url1> <url2>` | Spot differences between environments | |
| 16 | +| Tabs | `tabs`, `tab`, `newtab`, `closetab` | Multi-page workflows | |
| 17 | +| Multi-step | `chain` (JSON from stdin) | Batch commands in one call | |
| 18 | + |
| 19 | +All selector arguments accept CSS selectors or `@ref` after `snapshot`. 40+ commands total. |
| 20 | + |
| 21 | +## How it works |
| 22 | + |
| 23 | +gstack's browser is a compiled CLI binary that talks to a persistent local Chromium daemon over HTTP. The CLI is a thin client — it reads a state file, sends a command, and prints the response to stdout. The server does the real work via [Playwright](https://playwright.dev/). |
| 24 | + |
| 25 | +``` |
| 26 | +┌─────────────────────────────────────────────────────────────────┐ |
| 27 | +│ Claude Code │ |
| 28 | +│ │ |
| 29 | +│ "browse goto https://staging.myapp.com" │ |
| 30 | +│ │ │ |
| 31 | +│ ▼ │ |
| 32 | +│ ┌──────────┐ HTTP POST ┌──────────────┐ │ |
| 33 | +│ │ browse │ ──────────────── │ Bun HTTP │ │ |
| 34 | +│ │ CLI │ localhost:9400 │ server │ │ |
| 35 | +│ │ │ Bearer token │ │ │ |
| 36 | +│ │ compiled │ ◄────────────── │ Playwright │──── Chromium │ |
| 37 | +│ │ binary │ plain text │ API calls │ (headless) │ |
| 38 | +│ └──────────┘ └──────────────┘ │ |
| 39 | +│ ~1ms startup persistent daemon │ |
| 40 | +│ auto-starts on first call │ |
| 41 | +│ auto-stops after 30 min idle │ |
| 42 | +└─────────────────────────────────────────────────────────────────┘ |
| 43 | +``` |
| 44 | + |
| 45 | +### Lifecycle |
| 46 | + |
| 47 | +1. **First call**: CLI checks `/tmp/browse-server.json` for a running server. None found — it spawns `bun run browse/src/server.ts` in the background. The server launches headless Chromium via Playwright, picks a port (9400-9410), generates a bearer token, writes the state file, and starts accepting HTTP requests. This takes ~3 seconds. |
| 48 | + |
| 49 | +2. **Subsequent calls**: CLI reads the state file, sends an HTTP POST with the bearer token, prints the response. ~100-200ms round trip. |
| 50 | + |
| 51 | +3. **Idle shutdown**: After 30 minutes with no commands, the server shuts down and cleans up the state file. Next call restarts it automatically. |
| 52 | + |
| 53 | +4. **Crash recovery**: If Chromium crashes, the server exits immediately (no self-healing — don't hide failure). The CLI detects the dead server on the next call and starts a fresh one. |
| 54 | + |
| 55 | +### Key components |
| 56 | + |
| 57 | +``` |
| 58 | +browse/ |
| 59 | +├── src/ |
| 60 | +│ ├── cli.ts # Thin client — reads state file, sends HTTP, prints response |
| 61 | +│ ├── server.ts # Bun.serve HTTP server — routes commands to Playwright |
| 62 | +│ ├── browser-manager.ts # Chromium lifecycle — launch, tabs, ref map, crash handling |
| 63 | +│ ├── snapshot.ts # Accessibility tree → @ref assignment → Locator map |
| 64 | +│ ├── read-commands.ts # Non-mutating commands (text, html, links, js, css, etc.) |
| 65 | +│ ├── write-commands.ts # Mutating commands (click, fill, select, navigate, etc.) |
| 66 | +│ ├── meta-commands.ts # Server management (status, stop, restart) |
| 67 | +│ └── buffers.ts # Console + network log capture (in-memory + disk flush) |
| 68 | +├── test/ # Integration tests + HTML fixtures |
| 69 | +└── dist/ |
| 70 | + └── browse # Compiled binary (~58MB, Bun --compile) |
| 71 | +``` |
| 72 | + |
| 73 | +### The snapshot system |
| 74 | + |
| 75 | +The browser's key innovation is ref-based element selection, built on Playwright's accessibility tree API: |
| 76 | + |
| 77 | +1. `page.locator(scope).ariaSnapshot()` returns a YAML-like accessibility tree |
| 78 | +2. The snapshot parser assigns refs (`@e1`, `@e2`, ...) to each element |
| 79 | +3. For each ref, it builds a Playwright `Locator` (using `getByRole` + nth-child) |
| 80 | +4. The ref-to-Locator map is stored on `BrowserManager` |
| 81 | +5. Later commands like `click @e3` look up the Locator and call `locator.click()` |
| 82 | + |
| 83 | +No DOM mutation. No injected scripts. Just Playwright's native accessibility API. |
| 84 | + |
| 85 | +### Authentication |
| 86 | + |
| 87 | +Each server session generates a random UUID as a bearer token. The token is written to the state file (`/tmp/browse-server.json`) with chmod 600. Every HTTP request must include `Authorization: Bearer <token>`. This prevents other processes on the machine from controlling the browser. |
| 88 | + |
| 89 | +### Console and network capture |
| 90 | + |
| 91 | +The server hooks into Playwright's `page.on('console')` and `page.on('response')` events. All entries are kept in memory and flushed to disk every second: |
| 92 | + |
| 93 | +- Console: `/tmp/browse-console.log` |
| 94 | +- Network: `/tmp/browse-network.log` |
| 95 | + |
| 96 | +The `console` and `network` commands read from the in-memory buffers, not disk. |
| 97 | + |
| 98 | +### Multi-workspace support |
| 99 | + |
| 100 | +Each workspace gets its own isolated browser instance with its own Chromium process, tabs, cookies, and logs. |
| 101 | + |
| 102 | +If `CONDUCTOR_PORT` is set (e.g., by [Conductor](https://conductor.dev)), the browse port is derived deterministically: |
| 103 | + |
| 104 | +``` |
| 105 | +browse_port = CONDUCTOR_PORT - 45600 |
| 106 | +``` |
| 107 | + |
| 108 | +| Workspace | CONDUCTOR_PORT | Browse port | State file | |
| 109 | +|-----------|---------------|-------------|------------| |
| 110 | +| Workspace A | 55040 | 9440 | `/tmp/browse-server-9440.json` | |
| 111 | +| Workspace B | 55041 | 9441 | `/tmp/browse-server-9441.json` | |
| 112 | +| No Conductor | — | 9400 (scan) | `/tmp/browse-server.json` | |
| 113 | + |
| 114 | +You can also set `BROWSE_PORT` directly. |
| 115 | + |
| 116 | +### Environment variables |
| 117 | + |
| 118 | +| Variable | Default | Description | |
| 119 | +|----------|---------|-------------| |
| 120 | +| `BROWSE_PORT` | 0 (auto-scan 9400-9410) | Fixed port for the HTTP server | |
| 121 | +| `CONDUCTOR_PORT` | — | If set, browse port = this - 45600 | |
| 122 | +| `BROWSE_IDLE_TIMEOUT` | 1800000 (30 min) | Idle shutdown timeout in ms | |
| 123 | +| `BROWSE_STATE_FILE` | `/tmp/browse-server.json` | Path to state file | |
| 124 | +| `BROWSE_SERVER_SCRIPT` | auto-detected | Path to server.ts | |
| 125 | + |
| 126 | +### Performance |
| 127 | + |
| 128 | +| Tool | First call | Subsequent calls | Context overhead per call | |
| 129 | +|------|-----------|-----------------|--------------------------| |
| 130 | +| Chrome MCP | ~5s | ~2-5s | ~2000 tokens (schema + protocol) | |
| 131 | +| Playwright MCP | ~3s | ~1-3s | ~1500 tokens (schema + protocol) | |
| 132 | +| **gstack browse** | **~3s** | **~100-200ms** | **0 tokens** (plain text stdout) | |
| 133 | + |
| 134 | +The context overhead difference compounds fast. In a 20-command browser session, MCP tools burn 30,000-40,000 tokens on protocol framing alone. gstack burns zero. |
| 135 | + |
| 136 | +### Why CLI over MCP? |
| 137 | + |
| 138 | +MCP (Model Context Protocol) works well for remote services, but for local browser automation it adds pure overhead: |
| 139 | + |
| 140 | +- **Context bloat**: every MCP call includes full JSON schemas and protocol framing. A simple "get the page text" costs 10x more context tokens than it should. |
| 141 | +- **Connection fragility**: persistent WebSocket/stdio connections drop and fail to reconnect. |
| 142 | +- **Unnecessary abstraction**: Claude Code already has a Bash tool. A CLI that prints to stdout is the simplest possible interface. |
| 143 | + |
| 144 | +gstack skips all of this. Compiled binary. Plain text in, plain text out. No protocol. No schema. No connection management. |
| 145 | + |
| 146 | +## Acknowledgments |
| 147 | + |
| 148 | +The browser automation layer is built on [Playwright](https://playwright.dev/) by Microsoft. Playwright's accessibility tree API, locator system, and headless Chromium management are what make ref-based interaction possible. The snapshot system — assigning `@ref` labels to accessibility tree nodes and mapping them back to Playwright Locators — is built entirely on top of Playwright's primitives. Thank you to the Playwright team for building such a solid foundation. |
| 149 | + |
| 150 | +## Development |
| 151 | + |
| 152 | +### Prerequisites |
| 153 | + |
| 154 | +- [Bun](https://bun.sh/) v1.0+ |
| 155 | +- Playwright's Chromium (installed automatically by `bun install`) |
| 156 | + |
| 157 | +### Quick start |
| 158 | + |
| 159 | +```bash |
| 160 | +bun install # install dependencies + Playwright Chromium |
| 161 | +bun test # run integration tests (~3s) |
| 162 | +bun run dev <cmd> # run CLI from source (no compile) |
| 163 | +bun run build # compile to browse/dist/browse |
| 164 | +``` |
| 165 | + |
| 166 | +### Dev mode vs compiled binary |
| 167 | + |
| 168 | +During development, use `bun run dev` instead of the compiled binary. It runs `browse/src/cli.ts` directly with Bun, so you get instant feedback without a compile step: |
| 169 | + |
| 170 | +```bash |
| 171 | +bun run dev goto https://example.com |
| 172 | +bun run dev text |
| 173 | +bun run dev snapshot -i |
| 174 | +bun run dev click @e3 |
| 175 | +``` |
| 176 | + |
| 177 | +The compiled binary (`bun run build`) is only needed for distribution. It produces a single ~58MB executable at `browse/dist/browse` using Bun's `--compile` flag. |
| 178 | + |
| 179 | +### Running tests |
| 180 | + |
| 181 | +```bash |
| 182 | +bun test # run all tests |
| 183 | +bun test browse/test/commands # run command integration tests only |
| 184 | +bun test browse/test/snapshot # run snapshot tests only |
| 185 | +``` |
| 186 | + |
| 187 | +Tests spin up a local HTTP server (`browse/test/test-server.ts`) serving HTML fixtures from `browse/test/fixtures/`, then exercise the CLI commands against those pages. Tests take ~3 seconds. |
| 188 | + |
| 189 | +### Source map |
| 190 | + |
| 191 | +| File | Role | |
| 192 | +|------|------| |
| 193 | +| `browse/src/cli.ts` | Entry point. Reads `/tmp/browse-server.json`, sends HTTP to the server, prints response. | |
| 194 | +| `browse/src/server.ts` | Bun HTTP server. Routes commands to the right handler. Manages idle timeout. | |
| 195 | +| `browse/src/browser-manager.ts` | Chromium lifecycle — launch, tab management, ref map, crash detection. | |
| 196 | +| `browse/src/snapshot.ts` | Parses Playwright's accessibility tree, assigns `@ref` labels, builds Locator map. | |
| 197 | +| `browse/src/read-commands.ts` | Non-mutating commands: `text`, `html`, `links`, `js`, `css`, `forms`, etc. | |
| 198 | +| `browse/src/write-commands.ts` | Mutating commands: `goto`, `click`, `fill`, `select`, `scroll`, etc. | |
| 199 | +| `browse/src/meta-commands.ts` | Server management: `status`, `stop`, `restart`. | |
| 200 | +| `browse/src/buffers.ts` | In-memory + disk capture for console and network logs. | |
| 201 | + |
| 202 | +### Deploying to the active skill |
| 203 | + |
| 204 | +The active skill lives at `~/.claude/skills/gstack/`. After making changes: |
| 205 | + |
| 206 | +1. Push your branch |
| 207 | +2. Pull in the skill directory: `cd ~/.claude/skills/gstack && git pull` |
| 208 | +3. Rebuild: `cd ~/.claude/skills/gstack && bun run build` |
| 209 | + |
| 210 | +Or copy the binary directly: `cp browse/dist/browse ~/.claude/skills/gstack/browse/dist/browse` |
| 211 | + |
| 212 | +### Adding a new command |
| 213 | + |
| 214 | +1. Add the handler in `read-commands.ts` (non-mutating) or `write-commands.ts` (mutating) |
| 215 | +2. Register the route in `server.ts` |
| 216 | +3. Add a test case in `browse/test/commands.test.ts` with an HTML fixture if needed |
| 217 | +4. Run `bun test` to verify |
| 218 | +5. Run `bun run build` to compile |
0 commit comments