A browser-based verification suite built on Playwright. Run checks against any URL — local dev, staging, or production — with automatic screenshots, structured JSON results, and a prep system for state management.
Inspired by Basecamp's Upright Playwright probes, but focused on on-demand verification rather than synthetic monitoring.
Add to your Gemfile:
gem "checkset"Then install Playwright:
npm install playwright
npx playwright install chromiumScaffold a new project:
bundle exec checkset initThis creates a starter checkset.yml and checks/homepage_loads.rb to get you going.
Create a check in checks/:
# checks/homepage.rb
class Checks::Homepage < Checkset::Check
description "Verifies the homepage loads correctly"
def call
visit "/"
verify "page has title" do
page.title.include?("My App")
end
verify "has main heading" do
page.get_by_role("heading", name: "Welcome").visible?
end
end
endRun it:
bundle exec checkset --target https://staging.myapp.comChecks are Ruby classes that subclass Checkset::Check. Inside call, you use two primitives:
verify "cart has items" do
page.get_by_test_id("cart-count").text_content.to_i > 0
endThe block returns truthy (pass) or falsy (fail). On failure, the check keeps running to collect all failures in one run.
step "add product to cart" do
page.get_by_role("button", name: "Add to Cart").first.click
endSteps perform navigation or interaction. On failure, the check stops because subsequent steps depend on the state this one creates.
visit "/products" # goes to https://staging.myapp.com/productsclass Checks::UserCanCheckout < Checkset::Check
prep :sign_in_as_customer
description "Verifies the full checkout flow"
tags :checkout, :critical
def call
visit "/products"
step "add product to cart" do
page.get_by_role("button", name: "Add to Cart").first.click
end
verify "cart has items" do
page.get_by_test_id("cart-count").text_content.to_i > 0
end
step "go to checkout" do
page.get_by_role("link", name: "Cart").click
page.get_by_role("button", name: "Checkout").click
end
verify "reached checkout page" do
page.url.include?("/checkout")
end
end
endPreps handle state setup before a check runs — signing in, seeding data, etc. They run in the same browser context as the check, so session state carries over.
# preps/sign_in_as_admin.rb
class Preps::SignInAsAdmin < Checkset::Prep
def satisfy(page)
page.goto("#{target_url}/login")
page.get_by_label("Email").fill(credentials[:admin_email])
page.get_by_label("Password").fill(credentials[:admin_password])
page.get_by_role("button", name: "Sign in").click
page.wait_for_url("**/dashboard")
end
# Optional: skip sign-in if already authenticated
def satisfied?(page)
page.goto("#{target_url}/dashboard")
!page.url.include?("/login")
rescue
false
end
endDeclare preps on your check:
class Checks::AdminDashboard < Checkset::Check
prep :sign_in_as_admin
def call
visit "/admin/dashboard"
verify("dashboard loads") { page.get_by_role("heading", name: "Dashboard").visible? }
end
endMultiple preps run in declaration order:
class Checks::UserCanCheckout < Checkset::Check
prep :sign_in_as_customer
prep :test_customer_account
# ...
endWhen you need to run checks against multiple domains, create a checkset.yml to map suite names to target URLs. Checks are auto-discovered by folder — no need to list them manually.
checks/
├── app/
│ ├── user_can_sign_in.rb → belongs to "app" suite
│ └── user_can_checkout.rb → belongs to "app" suite
├── admin/
│ └── admin_dashboard.rb → belongs to "admin" suite
└── homepage.rb → top-level = runs in EVERY suite
# Default domain for %{domain} interpolation
base_domain: afomera.dev
# Optional: override the checks directory (default: checks)
# checks_dir: path/to/checks
suites:
app:
target: https://app.%{domain}
admin:
target: https://admin.%{domain}Targets support two kinds of interpolation:
%{domain}— replaced withbase_domainfrom the yml (or--domainCLI flag)${ENV_VAR}— replaced with the environment variable's value
suites:
app:
target: https://app.%{domain} # uses base_domain or --domain
monitoring:
target: https://${MONITORING_HOST} # uses env var
marketing:
target: https://marketing.example.com # hardcoded, no interpolation# Run all suites (uses base_domain from yml)
bundle exec checkset
# Override the domain for staging/preview environments
bundle exec checkset --domain staging.afomera.dev
# Run just one suite
bundle exec checkset --suite app
# ENV vars work too
MONITORING_HOST=status.example.com bundle exec checkset
# Override with --target (ignores yml, loads all checks)
bundle exec checkset --target https://staging.myapp.comEach suite runs checks from its subfolder (checks/app/**/*.rb) plus any top-level checks (checks/*.rb). All suites run concurrently, sharing a single browser for efficiency.
# Basic usage
bundle exec checkset --target https://staging.myapp.com
# Run a specific check
bundle exec checkset --target https://staging.myapp.com Checks::UserCanCheckout
# Debug mode — headed browser with slow motion
bundle exec checkset --target http://localhost:3000 --headed --slow-mo 500
# Run checks in parallel
bundle exec checkset --target https://staging.myapp.com --parallel 4
# Run only checks tagged :critical
bundle exec checkset --target https://staging.myapp.com --tag critical
# Retry failed checks up to 2 times (useful for flaky browser checks)
bundle exec checkset --target https://staging.myapp.com --retries 2
# All options
bundle exec checkset --help| Flag | Description | Default |
|---|---|---|
--target URL |
Target URL to verify against | required (unless checkset.yml exists) |
--suite NAME |
Run only the named suite from checkset.yml |
all suites |
--config FILE |
Path to config file | checkset.yml |
--domain DOMAIN |
Base domain for %{domain} in suite targets |
base_domain from yml |
--tag TAG |
Only run checks tagged with TAG | all checks |
--retries N |
Retry failed checks N times | 0 |
--clean |
Remove all screenshots, traces, and results | |
--headed |
Run browser in headed mode | headless |
--browser TYPE |
chromium, firefox, or webkit |
chromium |
--parallel N |
Run N checks concurrently | 1 |
--slow-mo MS |
Slow down actions by N ms | off |
--timeout MS |
Default timeout in ms | 10000 |
--screenshots-dir DIR |
Screenshot output directory | tmp/checkset/screenshots |
--checks-dir DIR |
Directory containing checks | checks |
--preps-dir DIR |
Directory containing preps | preps |
--playwright-server URL |
WebSocket URL for remote Playwright | local |
Single target:
Checkset — https://staging.myapp.com
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
PASS UserCanSignIn .................. 1.2s
PASS UserCanCheckout ................ 3.4s
FAIL AdminCanViewDashboard .......... 2.1s
✗ verify "dashboard loads" — timed out waiting for selector
screenshot: tmp/checkset/screenshots/admin_can_view_dashboard/FAIL_...png
trace: tmp/checkset/traces/admin_can_view_dashboard_...zip
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
2 passed, 1 failed (6.7s)
Results: tmp/checkset/results/results_20260218_103000.json
Suite mode (suites run concurrently, results printed per suite):
Checkset — app → https://app.afomera.dev
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
PASS PageLoads ........................................ 0.8s
PASS UserCanSignIn .................................... 1.2s
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
2 passed (2.0s)
Results: tmp/checkset/results/results_app_20260218_103000.json
Checkset — admin → https://admin.afomera.dev
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
PASS PageLoads ........................................ 0.7s
PASS AdminDashboard ................................... 1.5s
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
2 passed (2.2s)
Results: tmp/checkset/results/results_admin_20260218_103000.json
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
All suites: 4 checks across 2 suites — 4 passed, 0 failed, 0 skipped (2.5s)
Every run writes structured results to tmp/checkset/results/. In suite mode, filenames include the suite name (e.g. results_app_20260218_103000.json).
{
"run_at": "2026-02-18T10:30:00-07:00",
"target_url": "https://staging.myapp.com",
"duration": 6.7,
"summary": { "total": 3, "passed": 2, "failed": 1, "skipped": 0 },
"checks": [
{
"check": "Checks::UserCanCheckout",
"status": "passed",
"duration": 3.4,
"steps": [
{ "name": "cart has items", "type": "verify", "status": "passed", "duration": 0.5, "screenshot_path": "..." }
]
}
]
}- Screenshots are captured on every
verifyandstep(pass or fail) - Full-page failure screenshots are taken when a verify/step fails
- Playwright traces (
.zip) are saved only on failure — open withnpx playwright show-trace trace.zip
# checkset.rb or anywhere before running
Checkset.configure do |c|
c.target_url = "https://staging.myapp.com"
c.headless = true
c.browser_type = :chromium
c.default_timeout = 10_000
c.viewport_size = { width: 1280, height: 720 }
c.credentials_provider = :env # or :rails_credentials
endBy default, credentials come from environment variables:
credentials[:admin_email] # reads ENV["ADMIN_EMAIL"]With Rails, you can use encrypted credentials:
Checkset.configure { |c| c.credentials_provider = :rails_credentials }
# reads Rails.application.credentials.dig(:checkset, :admin_email)The examples/ directory includes sample checks organized into suites. To try them out:
# Run the built-in smoke test against example.com
bundle exec checkset --target https://example.com
# Run example checks against a single target
bundle exec checkset --target https://hey.com --checks-dir examples/checks
# Run all example suites using the example checkset.yml
bundle exec checkset --config examples/checkset.yml
# Run just the "hey" suite from the examples
bundle exec checkset --config examples/checkset.yml --suite hey
# Run in headed mode to watch the browser
bundle exec checkset --config examples/checkset.yml --suite basecamp --headed
# Run all suites with checks running in parallel within each suite
bundle exec checkset --config examples/checkset.yml --parallel 3
# Run a specific check by name
bundle exec checkset --target https://hey.com --checks-dir examples/checks HeyHomepage
# Clean up all output (screenshots, traces, results)
bundle exec checkset --clean0— all checks passed1— one or more checks failed
MIT