shell-based web browser for LLMs that converts web pages to markdown, executes js, and interacts with pages.

# Convert a webpage to markdown
web https://example.com

# Take a screenshot while scraping
web https://example.com --screenshot page.png

# Execute JavaScript and capture log output along with markdown content
web https://example.com --js "console.log(document.title)"

# Fill and submit a form
web https://login.example.com \
    --form "login_form" \
    --input "username" --value "myuser" \
    --input "password" --value "mypass"

• Self-contained executable - Single native Go binary with no runtime dependencies
• Markdown conversion - HTML to markdown conversion for optimized consumption by LLMs
• JavaScript execution - Full browser engine with arbitrary js execution and console log capture
• Complete logging - Captures console.log/warn/error/info/debug and browser errors (JS errors, network errors, etc.)
• Phoenix LiveView support - Detects and properly handles Phoenix LiveView applications
• Screenshots - Save full-page screenshots
• Form filling - Automated form interaction with LiveView-aware submissions
• Session persistence - Maintains cookies and authentication across runs with profiles

1. Build for current platform:

   make              # Build ./web for your platform
   ./web https://example.com

You can then sudo cp web /usr/local/bin to make it available system wide

For releases or deployment to other systems:

make build        # Build all platforms

This creates:

• web-darwin-arm64 - macOS Apple Silicon (M1/M2/M3)
• web-darwin-amd64 - macOS Intel
• web-linux-amd64 - Linux x86_64

# Basic scraping
web https://example.com

# Output raw HTML
web https://example.com --raw > output.html

# With truncation and screenshot
web example.com --screenshot screenshot.png --truncate-after 123

# Form submission with Phoenix LiveView support
web http://localhost:4000/users/log-in \
    --form "login_form" \
    --input "user[email]" --value "foo@bar" \
    --input "user[password]" --value "secret" \
    --after-submit "http://localhost:4000/authd/page"

# Execute JavaScript on the page
web example.com --js "document.querySelector('button').click()"

# Use named session profile
./web --profile "mysite" https://authenticated-site.com

Usage: web <url> [options]

Options:
  --help                     Show this help message
  --raw                      Output raw page instead of converting to markdown
  --truncate-after <number>  Truncate output after <number> characters and append a notice (default: 100000)
  --screenshot <filepath>    Take a screenshot of the page and save it to the given filepath
  --form <id>                The id of the form for inputs
  --input <name>             Specify the name attribute for a form input field
  --value <value>            Provide the value to fill for the last --input field
  --after-submit <url>       After form submission and navigation, load this URL before converting to markdown
  --js <code>                Execute JavaScript code on the page after it loads
  --profile <name>           Use or create named session profile (default: "default")

This tool has special support for Phoenix LiveView applications:

• Auto-detection - Automatically detects LiveView pages via [data-phx-session] attribute
• Connection waiting - Waits for .phx-connected class before proceeding
• Form handling - Properly handles LiveView form submissions with loading states
• State management - Waits for .phx-change-loading and .phx-submit-loading to complete

• Linux x64 or macOS (Ubuntu 18.04+, RHEL 7+, Debian 9+, Arch Linux, macOS 10.12+)
• ~102MB free space (for Firefox and geckodriver on first run)

On Linux, you may need to install system packages for Firefox:

# Ubuntu/Debian - Core packages for Firefox
sudo apt install libgtk-3-0 libdbus-glib-1-2 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxi6 libxrandr2 libxss1 libxtst6 libxext6 libasound2 libatspi2.0-0 libdrm2 libxfixes3 libxrender1

# Additional packages for multimedia and fonts
sudo apt install libpulse0 libcanberra-gtk3-module packagekit-gtk3-module libdbusmenu-glib4 libdbusmenu-gtk3-4

make test

This will build binaries for all platforms and run tests

make              # Build for current platform (./web)
make build        # Build all platforms (darwin-arm64, darwin-amd64, linux-amd64)
make test         # Build and run tests
make clean        # Remove build artifacts

• Go 1.21+ (for building only)

• Single Go binary with standalone headless firefox download on first run
• Auto-download on first run - Firefox and geckodriver downloaded to ~/.web-firefox/
• Self-contained directory structure:
  • ~/.web-firefox/firefox/ - Headless Firefox browser
  • ~/.web-firefox/geckodriver/ - WebDriver automation binary
  • ~/.web-firefox/profiles/ - Isolated session profiles for persistence
• Cross-platform - Builds for macOS (Intel/ARM64) and Linux x86_64

MIT License