Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add agent mode: LLM-driven REPL, scripted replay, shared tools layer#2338

Draft
arrufat wants to merge 402 commits into
mainfrom
agent
Draft

Add agent mode: LLM-driven REPL, scripted replay, shared tools layer#2338
arrufat wants to merge 402 commits into
mainfrom
agent

Conversation

@arrufat
Copy link
Copy Markdown
Contributor

@arrufat arrufat commented Apr 30, 2026

Adds a new lightpanda agent command alongside serve, fetch, and mcp, with five operating modes that all share the same browser-tool surface.

Features

  • Interactive REPL, optionally backed by an LLM (Anthropic, OpenAI, Gemini, Ollama) via the zenai client. Without --provider, the REPL still runs as a "dumb" Pandascript-only shell.
  • Pandascript, a tiny line-oriented DSL for browser actions (GOTO, CLICK, TYPE, WAIT, SCROLL, HOVER, SELECT, CHECK, EXTRACT, EVAL, LOGIN, ACCEPT_COOKIES, TREE, MARKDOWN).
  • Recording and replay of .lp scripts. agent script.lp replays without any LLM call; agent -i script.lp replays then drops into the REPL, appending new commands to the file.
  • Self-healing replay (--self-heal): when a recorded command fails on a drifted page, a short LLM turn inspects the current state, emits a fixed command, and rewrites the script line in place.
  • One-shot mode (--task): a single user turn whose final answer goes to stdout, with progress and tool calls on stderr, so the result can be captured cleanly.
  • Slash commands in the REPL: /<tool> [args] invokes a browser tool directly without going through the LLM. /help lists tools, /quit exits the REPL.
  • Tab completion (case-insensitive) over PandaScript keywords and slash commands, with a dim ghost-suffix hint. Persistent history in .lp-history.

@arrufat arrufat marked this pull request as draft April 30, 2026 15:30
Comment thread docs/agent.md Outdated
Comment thread src/browser/Frame.zig Outdated
Comment thread src/browser/Frame.zig Outdated
Comment thread src/browser/tools.zig Outdated
Comment thread src/browser/tools.zig Outdated
arrufat added 28 commits May 4, 2026 09:54
Limits the getEnv tool to variables starting with LP_ to prevent the
model from probing sensitive system environment variables or API keys.
Restricts self-heal turns to only allow page-local commands. Commands
like `goto` and `eval_js` are now ignored to prevent the model from
navigating away or executing arbitrary scripts during healing.
- Fix typo in REPL info message.
- Optimize Recorder to avoid unnecessary allocations.
- Simplify field type detection in SlashCommand using stringToEnum.
- Remove unused yellow ANSI constant in Terminal.
- Shorten log message in McpServer.
Integrates the Tavily Search API as the primary search provider when
TAVILY_API_KEY is set. Falls back to scraping Google and DuckDuckGo
on failure or if the key is missing. Updates zenai dependency.
Adds argument syntax hints (e.g. <req> [opt]) for slash commands in the
REPL and implements tab completion for /help command arguments.
arrufat and others added 30 commits May 22, 2026 09:18
- Allow recording commands with backendNodeId if they also have a
  selector, stripping backendNodeId during formatting.
- Point UnterminatedQuote errors to the opening line instead of EOF.
- Fix buildHints allocation when fields is empty but required is not.
- Update docs to clarify positional argument rules.
Calculate the exact size needed for hint slots in schema.zig to avoid
overallocating. Also inline the single-use helper isDefaultTrueBool in
command.zig.
- Replace `Action` enum with `Tool` enum using exhaustive switches
- Extract `ScriptIterator` to `Iterator.zig`
- Refactor `schema.zig` into `Schema.zig`
- Move string substitution logic into `tools.zig`
- Clean up `SlashCommand.zig` to only handle REPL meta-commands
- Merge `formatHealReplacement` and `formatHealReplacementLines` using
  a new `HealBody` union.
- Move `interactiveTty` and `promptNumberedChoice` to `Terminal.zig`.
- Relocate and consolidate tests for `canHeal` and `isRecorded`.
- Make internal `Schema` functions private.
- Add `/.lp-history` to `.gitignore`.
- Make `SlashCommand.MetaCommand.Tag` private.
- Simplify loop in `tools.zig` using `while ... else`.
- Use `trimStart` instead of `trim` in `Schema.splitNameRest`.
script: unify PandaScript to slash commands
When an action triggers a navigation that swaps the root frame, the
previous page is torn down, leaving dangling DOMNode pointers in the
registry.

Reset the registry in `finalizeAction` if the frame changes to prevent
subsequent actions from dereferencing freed memory.
browser: reset node registry on navigation
When a selector times out without matching, the outcome is the same as
/hover or /click against a missing node. Returning InternalError instead
gave inconsistent feedback to the agent.

Map error.Timeout to NodeNotFound and leave the catch-all log +
InternalError for genuinely unexpected errors. Tighten the existing
MCP - waitForSelector: timeout test to pin the specific error name.
Replaces `printActionResult` and `printToolResult` with a single
`printToolOutcome` function. This unifies the output path and adds
red/green color coding for tool failures and successes.
tools: map waitForSelector timeout to NodeNotFound
Allows SIGINT to abort in-flight browser operations and V8 execution.
Validate tool arguments like `waitUntil` before parsing to provide
clearer error messages with expected enum values.
- Route logs through a terminal sink to avoid trampling the spinner.
- Track and format the last navigation error in browser tools.
- Group `/help` output into Browser, LLM, and Meta commands.
- Only show LLM commands in help if an LLM client is configured.
- Centralize the check for commands requiring an LLM in the REPL loop.
- Add descriptions to meta and LLM commands.
Normalizes parsed keys to their canonical casing defined in the schema
for both KV and JSON inputs.
- Remove automatic dimming from `printInfoFmt`
- Add explicit ANSI formatting to slash command help
- Adjust syntax highlighting colors for terminal output
Unifies printInfo/printInfoFmt and printError/printErrorFmt into
single functions accepting format arguments. Adds printDimmed for
dimmed output and supports highlighting quoted slash arguments.
Allows filtering cookies by URL or dumping all cookies. Defaults to
cookies matching the current page's host.
- Map common key shorthand aliases to canonical KeyboardEvent keys.
- Trigger implicit form submission when pressing Enter on inputs.
- Add CSS selector support to the press tool.
Adds a new `waitForScript` tool to wait for a JS expression to
evaluate to truthy. This allows synchronization beyond what CSS
selectors can express.
Adds a new `html` tool to dump the full raw HTML of the current page.
This is useful for debugging and capturing fixtures.
Allows dumping a specific element's outerHTML instead of the full
document when a selector or backendNodeId is provided.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants