Thanks to visit codestin.com
Credit goes to github.com

Skip to content

A browser-based verification suite built on Playwright for Ruby

License

Notifications You must be signed in to change notification settings

afomera/checkset

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Checkset

A browser-based verification suite built on Playwright. Run checks against any URL — local dev, staging, or production — with automatic screenshots, structured JSON results, and a prep system for state management.

Inspired by Basecamp's Upright Playwright probes, but focused on on-demand verification rather than synthetic monitoring.

Installation

Add to your Gemfile:

gem "checkset"

Then install Playwright:

npm install playwright
npx playwright install chromium

Quick Start

Scaffold a new project:

bundle exec checkset init

This creates a starter checkset.yml and checks/homepage_loads.rb to get you going.

Create a check in checks/:

# checks/homepage.rb
class Checks::Homepage < Checkset::Check
  description "Verifies the homepage loads correctly"

  def call
    visit "/"

    verify "page has title" do
      page.title.include?("My App")
    end

    verify "has main heading" do
      page.get_by_role("heading", name: "Welcome").visible?
    end
  end
end

Run it:

bundle exec checkset --target https://staging.myapp.com

The DSL

Checks are Ruby classes that subclass Checkset::Check. Inside call, you use two primitives:

verify — assertions that continue on failure

verify "cart has items" do
  page.get_by_test_id("cart-count").text_content.to_i > 0
end

The block returns truthy (pass) or falsy (fail). On failure, the check keeps running to collect all failures in one run.

step — actions that halt on failure

step "add product to cart" do
  page.get_by_role("button", name: "Add to Cart").first.click
end

Steps perform navigation or interaction. On failure, the check stops because subsequent steps depend on the state this one creates.

visit — navigate relative to the target URL

visit "/products"  # goes to https://staging.myapp.com/products

Full example

class Checks::UserCanCheckout < Checkset::Check
  prep :sign_in_as_customer
  description "Verifies the full checkout flow"
  tags :checkout, :critical

  def call
    visit "/products"

    step "add product to cart" do
      page.get_by_role("button", name: "Add to Cart").first.click
    end

    verify "cart has items" do
      page.get_by_test_id("cart-count").text_content.to_i > 0
    end

    step "go to checkout" do
      page.get_by_role("link", name: "Cart").click
      page.get_by_role("button", name: "Checkout").click
    end

    verify "reached checkout page" do
      page.url.include?("/checkout")
    end
  end
end

Preps

Preps handle state setup before a check runs — signing in, seeding data, etc. They run in the same browser context as the check, so session state carries over.

# preps/sign_in_as_admin.rb
class Preps::SignInAsAdmin < Checkset::Prep
  def satisfy(page)
    page.goto("#{target_url}/login")
    page.get_by_label("Email").fill(credentials[:admin_email])
    page.get_by_label("Password").fill(credentials[:admin_password])
    page.get_by_role("button", name: "Sign in").click
    page.wait_for_url("**/dashboard")
  end

  # Optional: skip sign-in if already authenticated
  def satisfied?(page)
    page.goto("#{target_url}/dashboard")
    !page.url.include?("/login")
  rescue
    false
  end
end

Declare preps on your check:

class Checks::AdminDashboard < Checkset::Check
  prep :sign_in_as_admin

  def call
    visit "/admin/dashboard"
    verify("dashboard loads") { page.get_by_role("heading", name: "Dashboard").visible? }
  end
end

Multiple preps run in declaration order:

class Checks::UserCanCheckout < Checkset::Check
  prep :sign_in_as_customer
  prep :test_customer_account
  # ...
end

Suites

When you need to run checks against multiple domains, create a checkset.yml to map suite names to target URLs. Checks are auto-discovered by folder — no need to list them manually.

Folder structure

checks/
├── app/
│   ├── user_can_sign_in.rb    → belongs to "app" suite
│   └── user_can_checkout.rb   → belongs to "app" suite
├── admin/
│   └── admin_dashboard.rb     → belongs to "admin" suite
└── homepage.rb                → top-level = runs in EVERY suite

checkset.yml

# Default domain for %{domain} interpolation
base_domain: afomera.dev

# Optional: override the checks directory (default: checks)
# checks_dir: path/to/checks

suites:
  app:
    target: https://app.%{domain}
  admin:
    target: https://admin.%{domain}

Targets support two kinds of interpolation:

  • %{domain} — replaced with base_domain from the yml (or --domain CLI flag)
  • ${ENV_VAR} — replaced with the environment variable's value
suites:
  app:
    target: https://app.%{domain}         # uses base_domain or --domain
  monitoring:
    target: https://${MONITORING_HOST}     # uses env var
  marketing:
    target: https://marketing.example.com  # hardcoded, no interpolation

Running suites

# Run all suites (uses base_domain from yml)
bundle exec checkset

# Override the domain for staging/preview environments
bundle exec checkset --domain staging.afomera.dev

# Run just one suite
bundle exec checkset --suite app

# ENV vars work too
MONITORING_HOST=status.example.com bundle exec checkset

# Override with --target (ignores yml, loads all checks)
bundle exec checkset --target https://staging.myapp.com

Each suite runs checks from its subfolder (checks/app/**/*.rb) plus any top-level checks (checks/*.rb). All suites run concurrently, sharing a single browser for efficiency.

CLI

# Basic usage
bundle exec checkset --target https://staging.myapp.com

# Run a specific check
bundle exec checkset --target https://staging.myapp.com Checks::UserCanCheckout

# Debug mode — headed browser with slow motion
bundle exec checkset --target http://localhost:3000 --headed --slow-mo 500

# Run checks in parallel
bundle exec checkset --target https://staging.myapp.com --parallel 4

# Run only checks tagged :critical
bundle exec checkset --target https://staging.myapp.com --tag critical

# Retry failed checks up to 2 times (useful for flaky browser checks)
bundle exec checkset --target https://staging.myapp.com --retries 2

# All options
bundle exec checkset --help

Options

Flag Description Default
--target URL Target URL to verify against required (unless checkset.yml exists)
--suite NAME Run only the named suite from checkset.yml all suites
--config FILE Path to config file checkset.yml
--domain DOMAIN Base domain for %{domain} in suite targets base_domain from yml
--tag TAG Only run checks tagged with TAG all checks
--retries N Retry failed checks N times 0
--clean Remove all screenshots, traces, and results
--headed Run browser in headed mode headless
--browser TYPE chromium, firefox, or webkit chromium
--parallel N Run N checks concurrently 1
--slow-mo MS Slow down actions by N ms off
--timeout MS Default timeout in ms 10000
--screenshots-dir DIR Screenshot output directory tmp/checkset/screenshots
--checks-dir DIR Directory containing checks checks
--preps-dir DIR Directory containing preps preps
--playwright-server URL WebSocket URL for remote Playwright local

Output

Terminal

Single target:

Checkset — https://staging.myapp.com
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   PASS   UserCanSignIn .................. 1.2s
   PASS   UserCanCheckout ................ 3.4s
   FAIL   AdminCanViewDashboard .......... 2.1s
          ✗ verify "dashboard loads" — timed out waiting for selector
          screenshot: tmp/checkset/screenshots/admin_can_view_dashboard/FAIL_...png
          trace: tmp/checkset/traces/admin_can_view_dashboard_...zip

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
2 passed, 1 failed (6.7s)
Results: tmp/checkset/results/results_20260218_103000.json

Suite mode (suites run concurrently, results printed per suite):

Checkset — app → https://app.afomera.dev
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   PASS   PageLoads ........................................ 0.8s
   PASS   UserCanSignIn .................................... 1.2s

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
2 passed (2.0s)
Results: tmp/checkset/results/results_app_20260218_103000.json


Checkset — admin → https://admin.afomera.dev
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   PASS   PageLoads ........................................ 0.7s
   PASS   AdminDashboard ................................... 1.5s

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
2 passed (2.2s)
Results: tmp/checkset/results/results_admin_20260218_103000.json


━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
All suites: 4 checks across 2 suites — 4 passed, 0 failed, 0 skipped (2.5s)

JSON

Every run writes structured results to tmp/checkset/results/. In suite mode, filenames include the suite name (e.g. results_app_20260218_103000.json).

{
  "run_at": "2026-02-18T10:30:00-07:00",
  "target_url": "https://staging.myapp.com",
  "duration": 6.7,
  "summary": { "total": 3, "passed": 2, "failed": 1, "skipped": 0 },
  "checks": [
    {
      "check": "Checks::UserCanCheckout",
      "status": "passed",
      "duration": 3.4,
      "steps": [
        { "name": "cart has items", "type": "verify", "status": "passed", "duration": 0.5, "screenshot_path": "..." }
      ]
    }
  ]
}

Screenshots & Traces

  • Screenshots are captured on every verify and step (pass or fail)
  • Full-page failure screenshots are taken when a verify/step fails
  • Playwright traces (.zip) are saved only on failure — open with npx playwright show-trace trace.zip

Configuration

# checkset.rb or anywhere before running
Checkset.configure do |c|
  c.target_url = "https://staging.myapp.com"
  c.headless = true
  c.browser_type = :chromium
  c.default_timeout = 10_000
  c.viewport_size = { width: 1280, height: 720 }
  c.credentials_provider = :env  # or :rails_credentials
end

Credentials

By default, credentials come from environment variables:

credentials[:admin_email]  # reads ENV["ADMIN_EMAIL"]

With Rails, you can use encrypted credentials:

Checkset.configure { |c| c.credentials_provider = :rails_credentials }
# reads Rails.application.credentials.dig(:checkset, :admin_email)

Try the Examples

The examples/ directory includes sample checks organized into suites. To try them out:

# Run the built-in smoke test against example.com
bundle exec checkset --target https://example.com

# Run example checks against a single target
bundle exec checkset --target https://hey.com --checks-dir examples/checks

# Run all example suites using the example checkset.yml
bundle exec checkset --config examples/checkset.yml

# Run just the "hey" suite from the examples
bundle exec checkset --config examples/checkset.yml --suite hey

# Run in headed mode to watch the browser
bundle exec checkset --config examples/checkset.yml --suite basecamp --headed

# Run all suites with checks running in parallel within each suite
bundle exec checkset --config examples/checkset.yml --parallel 3

# Run a specific check by name
bundle exec checkset --target https://hey.com --checks-dir examples/checks HeyHomepage

# Clean up all output (screenshots, traces, results)
bundle exec checkset --clean

Exit Codes

  • 0 — all checks passed
  • 1 — one or more checks failed

License

MIT

About

A browser-based verification suite built on Playwright for Ruby

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages