Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Tags: orudge/anubis

Tags

v1.20.0-ttforums

Toggle v1.20.0-ttforums's commit message
Update logos for TT-Forums

v1.21.0-pre3

Toggle v1.21.0-pre3's commit message
v1.21.0-pre3: Minfila Warde

Please report any issues with this prerelease so the full release can be
the best it can possibly be.

Now with fixed RPM signatures.

v1.21.0-pre2

Toggle v1.21.0-pre2's commit message
v1.21.0-pre2: Minfila Warde

Please report any issues with this prerelease so the full release can be
the best it can possibly be.

v1.21.0-pre1

Toggle v1.21.0-pre1's commit message
v1.21.0-pre1: Minfila Warde

Please report any issues with this prerelease so the full release can be
the best it can possibly be.

v1.20.0

Toggle v1.20.0's commit message
v1.20.0: Thancred Waters

The big ticket items are as follows:

- Implement a no-JS challenge method: [`metarefresh`](./admin/configuration/challenges/metarefresh.mdx) ([TecharoHQ#95](TecharoHQ#95))
- Implement request "weight", allowing administrators to customize the behaviour of Anubis based on specific criteria
- Implement GeoIP and ASN based checks via [Thoth](https://anubis.techaro.lol/docs/admin/thoth) ([TecharoHQ#206](TecharoHQ#206))
- Add [custom weight thresholds](./admin/configuration/thresholds.mdx) via CEL ([TecharoHQ#688](TecharoHQ#688))
- Move Open Graph configuration [to the policy file](./admin/configuration/open-graph.mdx)
- Enable support for Open Graph metadata to be returned by default instead of doing lookups against the target
- Add `robots2policy` CLI utility to convert robots.txt files to Anubis challenge policies using CEL expressions ([TecharoHQ#409](TecharoHQ#409))
- Refactor challenge presentation logic to use a challenge registry
- Allow challenge implementations to register HTTP routes
- [Imprint/Impressum support](./admin/configuration/impressum.mdx) ([TecharoHQ#362](TecharoHQ#362))
- Fix "invalid response" after "Success!" in Chromium ([TecharoHQ#564](TecharoHQ#564))

A lot of performance improvements have been made:

- Replace internal SHA256 hashing with xxhash for 4-6x performance improvement in policy evaluation and cache operations
- Optimized the OGTags subsystem with reduced allocations and runtime per request by up to 66%
- Replace cidranger with bart for IP range checking, improving IP matching performance by 3-20x with zero heap
  allocations

And some cleanups/refactors were added:

- Fix OpenGraph passthrough ([TecharoHQ#717](TecharoHQ#717))
- Remove the unused `/test-error` endpoint and update the testing endpoint `/make-challenge` to only be enabled in
  development
- Add `--xff-strip-private` flag/envvar to toggle skipping X-Forwarded-For private addresses or not
- Bump AI-robots.txt to version 1.37
- Make progress bar styling more compatible (UXP, etc)
- Add `--strip-base-prefix` flag/envvar to strip the base prefix from request paths when forwarding to target servers
- Fix an off-by-one in the default threshold config
- Add functionality for HS512 JWT algorithm
- Add support for dynamic cookie domains with the `--cookie-dynamic-domain`/`COOKIE_DYNAMIC_DOMAIN` flag/envvar

Request weight is one of the biggest ticket features in Anubis. This enables Anubis to be much closer to a Web Application Firewall and when combined with custom thresholds allows administrators to have Anubis take advanced reactions. For more information about request weight, see [the request weight section](./admin/policies.mdx#request-weight) of the policy file documentation.

TL;DR when you have one or more WEIGHT rules like this:

```yaml
bots:
  - name: gitea-session-token
    action: WEIGH
    expression:
      all:
        - '"Cookie" in headers'
        - headers["Cookie"].contains("i_love_gitea=")
    # Remove 5 weight points
    weight:
      adjust: -5
```

You can configure custom thresholds like this:

```yaml
thresholds:
  - name: minimal-suspicion # This client is likely fine, its soul is lighter than a feather
    expression: weight < 0 # a feather weighs zero units
    action: ALLOW # Allow the traffic through

  # For clients that had some weight reduced through custom rules, give them a
  # lightweight challenge.
  - name: mild-suspicion
    expression:
      all:
        - weight >= 0
        - weight < 10
    action: CHALLENGE
    challenge:
      # https://anubis.techaro.lol/docs/admin/configuration/challenges/metarefresh
      algorithm: metarefresh
      difficulty: 1
      report_as: 1

  # For clients that are browser-like but have either gained points from custom
  # rules or report as a standard browser.
  - name: moderate-suspicion
    expression:
      all:
        - weight >= 10
        - weight < 20
    action: CHALLENGE
    challenge:
      # https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
      algorithm: fast
      difficulty: 2 # two leading zeros, very fast for most clients
      report_as: 2

  # For clients that are browser like and have gained many points from custom
  # rules
  - name: extreme-suspicion
    expression: weight >= 20
    action: CHALLENGE
    challenge:
      # https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
      algorithm: fast
      difficulty: 4
      report_as: 4
```

These thresholds apply when no other `ALLOW`, `DENY`, or `CHALLENGE` rule matches the request. `WEIGHT` rules add and remove request weight as needed:

```yaml
bots:
  - name: gitea-session-token
    action: WEIGH
    expression:
      all:
        - '"Cookie" in headers'
        - headers["Cookie"].contains("i_love_gitea=")
    # Remove 5 weight points
    weight:
      adjust: -5

  - name: bot-like-user-agent
    action: WEIGH
    expression: '"Bot" in userAgent'
    # Add 5 weight points
    weight:
      adjust: 5
```

Of note: the default "generic browser" rule assigns 10 weight points:

```yaml
  # Generic catchall rule
- name: generic-browser
  user_agent_regex: >-
    Mozilla|Opera
  action: WEIGH
  weight:
    adjust: 10
```

Adjust this as you see fit.

v1.20.0-pre2

Toggle v1.20.0-pre2's commit message
v1.20.0-pre2: Thancred Waters prerelease 2

* Makefile: Build robots2policy by @heftig in TecharoHQ#699
* fix(default-config): off-by-one error in the default thresholds by @Xe in TecharoHQ#701
* feat: implement imprint/impressum support by @Xe in TecharoHQ#706
* fix(web/js): broken progress bar with slow algo by @yut23 in TecharoHQ#673
* build(deps): bump the github-actions group with 3 updates by @dependabot in TecharoHQ#708
* fix(lib): fix invalid response after success in Chrome by @Xe in TecharoHQ#711

v1.20.0-pre1

Toggle v1.20.0-pre1's commit message
v1.20.0-pre1: Thancred Waters

Anubis gains its biggest feature in this version: the weight system.

The big ticket items are as follows:

- Implement a no-JS challenge method:
  [`metarefresh`](./admin/configuration/challenges/metarefresh.mdx)
  ([TecharoHQ#95](TecharoHQ#95))
- Implement request "weight", allowing administrators to customize the
  behaviour of Anubis based on specific criteria
- Implement GeoIP and ASN based checks via
  [Thoth](https://anubis.techaro.lol/docs/admin/thoth)
  ([TecharoHQ#206](TecharoHQ#206))
- Add
  [custom weight thresholds](./admin/configuration/thresholds.mdx) via CEL
  ([TecharoHQ#688](TecharoHQ#688))
- Move Open Graph configuration
  [to the policy file](./admin/configuration/open-graph.mdx)
- Enable support for Open Graph metadata to be returned by default
  instead of doing lookups against the target
- Add `robots2policy` CLI utility to convert robots.txt files to Anubis
  challenge policies using CEL expressions
  ([TecharoHQ#409](TecharoHQ#409))
- Refactor challenge presentation logic to use a challenge registry
- Allow challenge implementations to register HTTP routes

A lot of performance improvements have been made:

- Replace internal SHA256 hashing with xxhash for 4-6x performance improvement
  in policy evaluation and cache operations
- Optimized the OGTags subsystem with reduced allocations and runtime per
  request by up to 66%
- Replace cidranger with bart for IP range checking, improving IP matching
  performance by 3-20x with zero heap allocations

And some cleanups/refactors were added:

- Remove the unused `/test-error` endpoint and update the testing endpoint
  `/make-challenge` to only be enabled in development
- Add `--xff-strip-private` flag/envvar to toggle skipping X-Forwarded-For
  private addresses or not
- Bump AI-robots.txt to version 1.37
- Make progress bar styling more compatible (UXP, etc)
- Add `--strip-base-prefix` flag/envvar to strip the base prefix from request
  paths when forwarding to target servers

Request weight is one of the biggest ticket features in Anubis. This enables
Anubis to be much closer to a Web Application Firewall and when combined with
custom thresholds allows administrators to have Anubis take advanced reactions.
For more information about request weight, see
[the request weight section](./admin/policies.mdx#request-weight)
of the policy file documentation.

TL;DR when you have one or more WEIGHT rules like this:

```yaml
bots:
  - name: gitea-session-token
    action: WEIGH
    expression:
      all:
        - '"Cookie" in headers'
        - headers["Cookie"].contains("i_love_gitea=")
    # Remove 5 weight points
    weight:
      adjust: -5
```

You can configure custom thresholds like this:

```yaml
thresholds:
  - name: minimal-suspicion # This client is likely fine, its soul is lighter than a feather
    expression: weight < 0 # a feather weighs zero units
    action: ALLOW # Allow the traffic through

  # For clients that had some weight reduced through custom rules, give them a
  # lightweight challenge.
  - name: mild-suspicion
    expression:
      all:
        - weight >= 0
        - weight < 10
    action: CHALLENGE
    challenge:
      # https://anubis.techaro.lol/docs/admin/configuration/challenges/metarefresh
      algorithm: metarefresh
      difficulty: 1
      report_as: 1

  # For clients that are browser-like but have either gained points from custom
  # rules or report as a standard browser.
  - name: moderate-suspicion
    expression:
      all:
        - weight >= 10
        - weight < 20
    action: CHALLENGE
    challenge:
      # https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
      algorithm: fast
      difficulty: 2 # two leading zeros, very fast for most clients
      report_as: 2

  # For clients that are browser like and have gained many points from custom
  # rules
  - name: extreme-suspicion
    expression: weight >= 20
    action: CHALLENGE
    challenge:
      # https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
      algorithm: fast
      difficulty: 4
      report_as: 4
```

These thresholds apply when no other `ALLOW`, `DENY`, or `CHALLENGE` rule
matches the request. `WEIGHT` rules add and remove request weight as needed:

```yaml
bots:
  - name: gitea-session-token
    action: WEIGH
    expression:
      all:
        - '"Cookie" in headers'
        - headers["Cookie"].contains("i_love_gitea=")
    # Remove 5 weight points
    weight:
      adjust: -5

  - name: bot-like-user-agent
    action: WEIGH
    expression: '"Bot" in userAgent'
    # Add 5 weight points
    weight:
      adjust: 5
```

Of note: the default "generic browser" rule assigns 10 weight points:

```yaml
- name: generic-browser
  user_agent_regex: >-
    Mozilla|Opera
  action: WEIGH
  weight:
    adjust: 10
```

Adjust this as you see fit.

v1.19.1

Toggle v1.19.1's commit message
v1.19.1: Jenomis cen Lexentale - Echo 1

Return `data/bots/ai-robots-txt.yaml` to avoid breaking configs
[TecharoHQ#599](TecharoHQ#599)

v1.19.0

Toggle v1.19.0's commit message
v1.19.0: Jenomis cen Lexentale

Mostly a bunch of small features, no big ticket things this time.

- Record if challenges were issued via the API or via embedded JSON in the challenge page HTML ([TecharoHQ#531](TecharoHQ#531))
- Ensure that clients that are shown a challenge support storing cookies
- Imprint the version number into challenge pages
- Encode challenge pages with gzip level 1
- Add PowerPC 64 bit little-endian builds (`GOARCH=ppc64le`)
- Add `check-spelling` for spell checking
- Add `--target-insecure-skip-verify` flag/envvar to allow Anubis to hit a self-signed HTTPS backend
- Minor adjustments to FreeBSD rc.d script to allow for more flexible configuration.
- Added Podman and Docker support for running Playwright tests
- Add a default rule to throw challenges when a request with the `X-Firefox-Ai` header is set.
- Updated the nonce value in the challenge JWT cookie to be a string instead of a number
- Rename cookies in response to user feedback
- Ensure cookie renaming is consistent across configuration options
- Add Bookstack app in data
- Truncate everything but the first five characters of Accept-Language headers when making challenges
- Ensure client JavaScript is served with Content-Type text/javascript.
- Add `--target-host` flag/envvar to allow changing the value of the Host header in requests forwarded to the target service.
- Bump AI-robots.txt to version 1.31
- Add `RuntimeDirectory` to systemd unit settings so native packages can listen over unix sockets
- Added SearXNG instance tracker whitelist policy
- Added Qualys SSL Labs whitelist policy
- Fixed cookie deletion logic ([TecharoHQ#520](TecharoHQ#520), [TecharoHQ#522](TecharoHQ#522))
- Add `--target-sni` flag/envvar to allow changing the value of the TLS handshake hostname in requests forwarded to the target service.
- Fixed CEL expression matching validator to now properly error out when it receives empty expressions
- Added OpenRC init.d script.
- Added `--version` flag.
- Added `anubis_proxied_requests_total` metric to count proxied requests.
- Add `Applebot` as "good" web crawler
- Reorganize AI/LLM crawler blocking into three separate stances, maintaining existing status quo as default.
- Split out AI/LLM user agent blocking policies, adding documentation for each.

v1.19.0-pre1

Toggle v1.19.0-pre1's commit message
v1.19.0-pre1: Jenomis cen Lexentale

Minor upgrades, refactoring, test fixes, etc.