v1.21.3: Minfilia Warde - Echo 3

Fixes GHSA-jhjj-2g64-px7c

This could allow an attacker to craft an Anubis pass-challenge URL that forces
a redirect to nonstandard URLs, such as the `javascript:` scheme which executes
arbitrary JavaScript code in a browser context when the user clicks the "Try
again" button.

This has been fixed by disallowing any URLs without the scheme `http` or
`https`.

Additionally, the "Try again" button has been fixed to completely ignore the
user-supplied redirect location. It now redirects to the home page (`/`).

v1.21.2: Minfilia Warde - Echo 2

-- Fixes a problem with nonstandard URLs and redirects --

This could allow an attacker to craft an Anubis pass-challenge URL
that forces a redirect to nonstandard URLs, such as the `javascript:`
scheme which executes arbitrary JavaScript code in a browser context
when the user clicks the "Try again" button.

This has been fixed by disallowing any URLs without the scheme `http`
or `https`.

v1.21.1: Minfilia Warde - Echo 1

- Expired records are now properly removed from bbolt databases
  ([TecharoHQ#848](TecharoHQ#848)).
- Fix hanging on service restart
  [TecharoHQ#853](TecharoHQ#853))

-- Added --

Anubis now supports the
[`missingHeader`](./admin/configuration/expressions.mdx#missingHeader)
function to assert the absence of headers in requests.

--- New locales ---

Anubis now supports these new languages:

- [Czech](TecharoHQ#849)
- [Finnish](TecharoHQ#863)
- [Norwegian Bokmål](TecharoHQ#855)
- [Norwegian Nynorsk](TecharoHQ#855)
- [Russian](TecharoHQ#882)

-- Fixes --

--- Fix ["error: can't get challenge"](TecharoHQ#869)
    when details about a challenge can't be found in the server
    side state ---

v1.21.0 changed the core challenge flow to maintain information about
challenges on the server side instead of only doing them via stateless
idempotent generation functions and relying on details to not change.
There was a subtle bug introduced in this change: if a client has an
unknown challenge ID set in its test cookie, Anubis will clear that
cookie and then throw an HTTP 500 error.

This has been fixed by making Anubis throw a new challenge page instead.

--- Fix event loop thrashing when solving a proof of work challenge ---

Previously the "fast" proof of work solver had a fragment of JavaScript
that attempted to only post an update about proof of work progress to
the main browser window every 1024 iterations. This fragment of
JavaScript was subtly incorrect in a way that passed review but
actually made the workers send an update back to the main thread every
iteration. This caused a pileup of unhandled async calls (similar to a
socket accept() backlog pileup in Unix) that caused stack space
exhaustion.

This has been fixed in the following ways:

1. The complicated boolean logic has been totally removed in favour of
   a worker-local iteration counter.
2. The progress bar is updated by worker `0` instead of all workers.

Hopefully this should limit the event loop thrashing and let ia32
browsers (as well as any environment with a smaller stack size than
amd64 and aarch64 seem to have) function normally when processing
Anubis proof of work challenges.

--- Fix potential memory leak when discovering a solution ---

In some cases, the parallel solution finder in Anubis could cause all of
the worker promises to leak due to the fact the promises were being improperly
terminated. This was fixed by having Anubis debounce worker termination
instead of allowing it to potentially recurse infinitely.

v1.21.0: Minfilia Warde

See https://anubis.techaro.lol/docs/CHANGELOG#v1210-minfilia-warde for
more information.

v1.21.0-pre3: Minfila Warde

Please report any issues with this prerelease so the full release can be
the best it can possibly be.

Now with fixed RPM signatures.

v1.21.0-pre2: Minfila Warde

Please report any issues with this prerelease so the full release can be
the best it can possibly be.

v1.21.0-pre1: Minfila Warde

Please report any issues with this prerelease so the full release can be
the best it can possibly be.

v1.20.0: Thancred Waters

The big ticket items are as follows:

- Implement a no-JS challenge method: [`metarefresh`](./admin/configuration/challenges/metarefresh.mdx) ([TecharoHQ#95](TecharoHQ#95))
- Implement request "weight", allowing administrators to customize the behaviour of Anubis based on specific criteria
- Implement GeoIP and ASN based checks via [Thoth](https://anubis.techaro.lol/docs/admin/thoth) ([TecharoHQ#206](TecharoHQ#206))
- Add [custom weight thresholds](./admin/configuration/thresholds.mdx) via CEL ([TecharoHQ#688](TecharoHQ#688))
- Move Open Graph configuration [to the policy file](./admin/configuration/open-graph.mdx)
- Enable support for Open Graph metadata to be returned by default instead of doing lookups against the target
- Add `robots2policy` CLI utility to convert robots.txt files to Anubis challenge policies using CEL expressions ([TecharoHQ#409](TecharoHQ#409))
- Refactor challenge presentation logic to use a challenge registry
- Allow challenge implementations to register HTTP routes
- [Imprint/Impressum support](./admin/configuration/impressum.mdx) ([TecharoHQ#362](TecharoHQ#362))
- Fix "invalid response" after "Success!" in Chromium ([TecharoHQ#564](TecharoHQ#564))

A lot of performance improvements have been made:

- Replace internal SHA256 hashing with xxhash for 4-6x performance improvement in policy evaluation and cache operations
- Optimized the OGTags subsystem with reduced allocations and runtime per request by up to 66%
- Replace cidranger with bart for IP range checking, improving IP matching performance by 3-20x with zero heap
  allocations

And some cleanups/refactors were added:

- Fix OpenGraph passthrough ([TecharoHQ#717](TecharoHQ#717))
- Remove the unused `/test-error` endpoint and update the testing endpoint `/make-challenge` to only be enabled in
  development
- Add `--xff-strip-private` flag/envvar to toggle skipping X-Forwarded-For private addresses or not
- Bump AI-robots.txt to version 1.37
- Make progress bar styling more compatible (UXP, etc)
- Add `--strip-base-prefix` flag/envvar to strip the base prefix from request paths when forwarding to target servers
- Fix an off-by-one in the default threshold config
- Add functionality for HS512 JWT algorithm
- Add support for dynamic cookie domains with the `--cookie-dynamic-domain`/`COOKIE_DYNAMIC_DOMAIN` flag/envvar

Request weight is one of the biggest ticket features in Anubis. This enables Anubis to be much closer to a Web Application Firewall and when combined with custom thresholds allows administrators to have Anubis take advanced reactions. For more information about request weight, see [the request weight section](./admin/policies.mdx#request-weight) of the policy file documentation.

TL;DR when you have one or more WEIGHT rules like this:

```yaml
bots:
  - name: gitea-session-token
    action: WEIGH
    expression:
      all:
        - '"Cookie" in headers'
        - headers["Cookie"].contains("i_love_gitea=")
    # Remove 5 weight points
    weight:
      adjust: -5
```

You can configure custom thresholds like this:

```yaml
thresholds:
  - name: minimal-suspicion # This client is likely fine, its soul is lighter than a feather
    expression: weight < 0 # a feather weighs zero units
    action: ALLOW # Allow the traffic through

  # For clients that had some weight reduced through custom rules, give them a
  # lightweight challenge.
  - name: mild-suspicion
    expression:
      all:
        - weight >= 0
        - weight < 10
    action: CHALLENGE
    challenge:
      # https://anubis.techaro.lol/docs/admin/configuration/challenges/metarefresh
      algorithm: metarefresh
      difficulty: 1
      report_as: 1

  # For clients that are browser-like but have either gained points from custom
  # rules or report as a standard browser.
  - name: moderate-suspicion
    expression:
      all:
        - weight >= 10
        - weight < 20
    action: CHALLENGE
    challenge:
      # https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
      algorithm: fast
      difficulty: 2 # two leading zeros, very fast for most clients
      report_as: 2

  # For clients that are browser like and have gained many points from custom
  # rules
  - name: extreme-suspicion
    expression: weight >= 20
    action: CHALLENGE
    challenge:
      # https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
      algorithm: fast
      difficulty: 4
      report_as: 4
```

These thresholds apply when no other `ALLOW`, `DENY`, or `CHALLENGE` rule matches the request. `WEIGHT` rules add and remove request weight as needed:

```yaml
bots:
  - name: gitea-session-token
    action: WEIGH
    expression:
      all:
        - '"Cookie" in headers'
        - headers["Cookie"].contains("i_love_gitea=")
    # Remove 5 weight points
    weight:
      adjust: -5

  - name: bot-like-user-agent
    action: WEIGH
    expression: '"Bot" in userAgent'
    # Add 5 weight points
    weight:
      adjust: 5
```

Of note: the default "generic browser" rule assigns 10 weight points:

```yaml
  # Generic catchall rule
- name: generic-browser
  user_agent_regex: >-
    Mozilla|Opera
  action: WEIGH
  weight:
    adjust: 10
```

Adjust this as you see fit.

v1.20.0-pre2: Thancred Waters prerelease 2

* Makefile: Build robots2policy by @heftig in TecharoHQ#699
* fix(default-config): off-by-one error in the default thresholds by @Xe in TecharoHQ#701
* feat: implement imprint/impressum support by @Xe in TecharoHQ#706
* fix(web/js): broken progress bar with slow algo by @yut23 in TecharoHQ#673
* build(deps): bump the github-actions group with 3 updates by @dependabot in TecharoHQ#708
* fix(lib): fix invalid response after success in Chrome by @Xe in TecharoHQ#711

v1.20.0-pre1: Thancred Waters

Anubis gains its biggest feature in this version: the weight system.

The big ticket items are as follows:

- Implement a no-JS challenge method:
  [`metarefresh`](./admin/configuration/challenges/metarefresh.mdx)
  ([TecharoHQ#95](TecharoHQ#95))
- Implement request "weight", allowing administrators to customize the
  behaviour of Anubis based on specific criteria
- Implement GeoIP and ASN based checks via
  [Thoth](https://anubis.techaro.lol/docs/admin/thoth)
  ([TecharoHQ#206](TecharoHQ#206))
- Add
  [custom weight thresholds](./admin/configuration/thresholds.mdx) via CEL
  ([TecharoHQ#688](TecharoHQ#688))
- Move Open Graph configuration
  [to the policy file](./admin/configuration/open-graph.mdx)
- Enable support for Open Graph metadata to be returned by default
  instead of doing lookups against the target
- Add `robots2policy` CLI utility to convert robots.txt files to Anubis
  challenge policies using CEL expressions
  ([TecharoHQ#409](TecharoHQ#409))
- Refactor challenge presentation logic to use a challenge registry
- Allow challenge implementations to register HTTP routes

A lot of performance improvements have been made:

- Replace internal SHA256 hashing with xxhash for 4-6x performance improvement
  in policy evaluation and cache operations
- Optimized the OGTags subsystem with reduced allocations and runtime per
  request by up to 66%
- Replace cidranger with bart for IP range checking, improving IP matching
  performance by 3-20x with zero heap allocations

And some cleanups/refactors were added:

- Remove the unused `/test-error` endpoint and update the testing endpoint
  `/make-challenge` to only be enabled in development
- Add `--xff-strip-private` flag/envvar to toggle skipping X-Forwarded-For
  private addresses or not
- Bump AI-robots.txt to version 1.37
- Make progress bar styling more compatible (UXP, etc)
- Add `--strip-base-prefix` flag/envvar to strip the base prefix from request
  paths when forwarding to target servers

Request weight is one of the biggest ticket features in Anubis. This enables
Anubis to be much closer to a Web Application Firewall and when combined with
custom thresholds allows administrators to have Anubis take advanced reactions.
For more information about request weight, see
[the request weight section](./admin/policies.mdx#request-weight)
of the policy file documentation.

TL;DR when you have one or more WEIGHT rules like this:

```yaml
bots:
  - name: gitea-session-token
    action: WEIGH
    expression:
      all:
        - '"Cookie" in headers'
        - headers["Cookie"].contains("i_love_gitea=")
    # Remove 5 weight points
    weight:
      adjust: -5
```

You can configure custom thresholds like this:

```yaml
thresholds:
  - name: minimal-suspicion # This client is likely fine, its soul is lighter than a feather
    expression: weight < 0 # a feather weighs zero units
    action: ALLOW # Allow the traffic through

  # For clients that had some weight reduced through custom rules, give them a
  # lightweight challenge.
  - name: mild-suspicion
    expression:
      all:
        - weight >= 0
        - weight < 10
    action: CHALLENGE
    challenge:
      # https://anubis.techaro.lol/docs/admin/configuration/challenges/metarefresh
      algorithm: metarefresh
      difficulty: 1
      report_as: 1

  # For clients that are browser-like but have either gained points from custom
  # rules or report as a standard browser.
  - name: moderate-suspicion
    expression:
      all:
        - weight >= 10
        - weight < 20
    action: CHALLENGE
    challenge:
      # https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
      algorithm: fast
      difficulty: 2 # two leading zeros, very fast for most clients
      report_as: 2

  # For clients that are browser like and have gained many points from custom
  # rules
  - name: extreme-suspicion
    expression: weight >= 20
    action: CHALLENGE
    challenge:
      # https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
      algorithm: fast
      difficulty: 4
      report_as: 4
```

These thresholds apply when no other `ALLOW`, `DENY`, or `CHALLENGE` rule
matches the request. `WEIGHT` rules add and remove request weight as needed:

```yaml
bots:
  - name: gitea-session-token
    action: WEIGH
    expression:
      all:
        - '"Cookie" in headers'
        - headers["Cookie"].contains("i_love_gitea=")
    # Remove 5 weight points
    weight:
      adjust: -5

  - name: bot-like-user-agent
    action: WEIGH
    expression: '"Bot" in userAgent'
    # Add 5 weight points
    weight:
      adjust: 5
```

Of note: the default "generic browser" rule assigns 10 weight points:

```yaml
- name: generic-browser
  user_agent_regex: >-
    Mozilla|Opera
  action: WEIGH
  weight:
    adjust: 10
```

Adjust this as you see fit.