URL: /cli/debug

---
title: "Debug and tracing"
icon: "bug"
description: "Configure debug logging and performance tracing"
---

SquirrelScan writes debug logs to `~/.squirrel/logs/` for diagnosing issues. By default, only errors are logged. Increase the log level for more detail.

## Log levels

| Level | What's Logged |
|-------|---------------|
| `error` | Exceptions, command failures, fatal errors (default) |
| `warn` | + warnings, recoverable issues |
| `info` | + command start/end, crawl lifecycle events |
| `debug` | + every HTTP request, rule execution details |

## Configuration

### Settings file

```bash
squirrel config set log_level debug
```

Valid values: `error`, `warn`, `info`, `debug`

### Environment variable

Override the setting for a single run:

```bash
SQUIRREL_LOG_LEVEL=debug squirrel audit https://example.com
```

The environment variable takes precedence over the settings file.

## Console debug flag

The `--debug` flag shows debug messages in the console, independent of file logging:

```bash
squirrel audit https://example.com --debug
```

This is useful for real-time debugging without changing your log level setting.

## Log files

Logs are written to `~/.squirrel/logs/`:

| File | Contents |
|------|----------|
| `debug.log` | Current log file (all enabled levels) |
| `trace.log` | Performance traces (when `--trace` enabled) |
| `debug.log.*.gz` | Rotated/compressed old logs |

### Log format

```text
2025-01-10T14:30:45.123Z [debug] [request] url="https://example.com" status=200 loadTimeMs=120
2025-01-10T14:30:45.200Z [info] ========== COMMAND END: audit (success, 5432ms) ==========
```

## Log rotation

Logs rotate automatically:

| Setting | Default | Description |
|---------|---------|-------------|
| `log_compress_after_days` | `14` | Compress `.log` files older than N days |
| `log_delete_after_days` | `60` | Delete `.gz` files older than N days |

Size-based rotation also triggers at 10MB to prevent runaway growth.

Configure via settings:

```bash
squirrel config set log_compress_after_days 7
squirrel config set log_delete_after_days 30
```

## Performance tracing

For performance analysis, enable tracing:

```bash
squirrel audit https://example.com --trace
```

This writes timing data to `~/.squirrel/logs/trace.log`:

```text
2025-01-10T14:30:45.123Z [trace] [fetch:https://example.com] duration=145.32ms
2025-01-10T14:30:45.300Z [trace] [rule:meta/title] duration=2.15ms {"checks":1}
```

Traces include:
- HTTP request timing
- Rule execution duration
- Crawl phase boundaries

## Memory on large audits

Everything after the crawl reads the site in batches and drops each batch before
reading the next, so a 2,500-page audit holds roughly what a 400-page one does
rather than several gigabytes. The batch is sized from the site's own average
page against a byte budget, because a docs site of 30 KB pages and a storefront
of 1 MB ones need very different page counts to hold the same bytes.

Two environment variables override that, and they are the same names and the
same default the hosted runtime uses:

| Variable | Meaning |
| --- | --- |
| `SQUIRREL_STREAM_BATCH_BYTES` | Raw HTML held per batch. Default 48 MB. |
| `SQUIRREL_STREAM_BATCH_PAGES` | An exact page count, taking the byte budget out of the decision. |

Both take a plain integer. A value with a unit suffix is refused with a warning
rather than partly read, so `SQUIRREL_STREAM_BATCH_BYTES=48mb` does not quietly
become 48 bytes.

Turning the budget down is usually the wrong move. Peak memory is a fixed floor
for the rule set and caches plus a few megabytes per page of batch, so the floor
is not reachable from this dial, and below roughly a dozen pages per batch the
same crawl becomes several times as many read-and-parse cycles whose churn peaks
*higher* than a larger batch does.

To see where memory goes, set `SQUIRREL_STREAM_PHASE_MEM=1`. Each sub-phase of
the rules pass then prints its heap after a forced collection:

```text
[stream] universe end heapUsed=285MB external=51MB rss=915MB
[stream] page-rules end heapUsed=287MB external=56MB rss=798MB
```

Read `heapUsed`, not `rss`. Under memory pressure macOS compresses pages and the
resident figure stops tracking what the process actually holds.

## Rules that run once per template

Most of a large site is a handful of templates repeated: a real storefront of 247
pages has 13 distinct page chromes. A rule whose answer is a property of the
template rather than of the page therefore gives the same verdict hundreds of
times, so the audit runs those rules once per template and reports the result on
every page of it. The findings are the same either way, and the audit is faster on
exactly the sites where the rules phase hurts.

Which rules qualify is a property of the rule, not a setting: a rule reading
anything that varies page to page, including any response header and the page's
own URL, keeps running everywhere.

If you suspect a finding that looks copied from a sibling page, turn it off and
compare:

```bash
SQUIRREL_TEMPLATE_FANOUT=0 squirrel audit https://example.com
```

`0`, `false`, `off` and `no` all disable it; anything else, including leaving it
unset, leaves it on. A report produced with it off should be identical to one
produced with it on, so a difference is a bug worth
[reporting](https://github.com/squirrelscan/squirrelscan/issues).

## Sensitive data redaction

Logs automatically redact sensitive data:

| Pattern | Example Before | Example After |
|---------|----------------|---------------|
| URL credentials | `https://user:pass@api.com` | `https://[REDACTED]@api.com` |
| API keys in URLs | `?api_key=secret123` | `?api_key=[REDACTED]` |
| Bearer tokens | `Bearer eyJ...` | `Bearer [REDACTED]` |
| Basic auth | `Basic dXNlcjpwYXNz` | `Basic [REDACTED]` |

Headers like `Authorization`, `Cookie`, and `X-Api-Key` are also redacted.

<Warning>
While redaction covers common patterns, avoid logging highly sensitive data. Review logs before sharing.
</Warning>

## Sharing logs for support

When reporting issues, include relevant log excerpts:

```bash
# View recent logs
tail -100 ~/.squirrel/logs/debug.log

# Copy to clipboard (macOS)
tail -100 ~/.squirrel/logs/debug.log | pbcopy
```

For full context, attach the log file to your GitHub issue.

## Examples

### Diagnose a failed crawl

```bash
# Run with debug logging
SQUIRREL_LOG_LEVEL=debug squirrel audit https://example.com

# Check what happened
cat ~/.squirrel/logs/debug.log | grep -i error
```

### Profile slow audits

```bash
# Enable tracing
squirrel audit https://example.com --trace

# Find slow requests
grep "duration=" ~/.squirrel/logs/trace.log | sort -t= -k2 -rn | head
```

### Temporary verbose mode

```bash
# One-off debug session
SQUIRREL_LOG_LEVEL=debug squirrel audit https://example.com --debug
```

## Related

- [Configuration](/configuration) - All settings
- [squirrel config](/cli/config) - Manage settings