PentestGPT v2 — Excalibur

This repository is a working fork of the anonymized code artifact released with the arXiv paper What Makes a Good LLM Agent for Real-world Penetration Testing?.

In the paper, the system is described as PentestGPT v2 / Excalibur. The original anonymous artifact was published as Excalibur-FA7D on 4open.science.

The upstream repository is useful as a research artifact, but in practice it does not run cleanly out of the box in a normal Docker-based workflow. This fork exists to fix that and make the project actually usable.

Why this fork exists

The original artifact had multiple practical issues that made deployment and daily use unreliable. This fork adapts it into a repository that can be built, started, and used in a normal environment.

Main fixes in this fork

• fixed Docker / Python package installation so the excalibur CLI is available inside the container
• fixed architecture-specific tooling for linux/amd64 and linux/arm64
• improved container setup for common lab usage
• added a cleaner OpenVPN workflow for HTB-like environments
• fixed OpenRouter / Claude Code Router configuration so model routing can be changed without rebuilding the image
• cleaned up auth handling through .env.auth
• aligned defaults, startup flow, and docs with real runtime behavior

What this repo is

Excalibur is an AI agent for authorized penetration testing and CTF-style environments. It runs inside Docker and includes common CLI tooling, LLM integration, and support for VPN-based labs.

Repository overview (paper vs code, this fork, field notes)

Relative to the paper and how the project presents itself:

• The article What Makes a Good LLM Agent for Real-world Penetration Testing? (arXiv:2602.17622) describes PentestGPT v2. In the anonymous artifact and in this fork, the packaged agent is named Excalibur. Public source appears as Excalibur-FA7D on 4open.science.

• This repo is a semi-working fork: the upstream tree was originally skewed more toward other platforms (including ARM), so it was reconfigured here. Docker image build, CLI, CCR/OpenRouter provider, auth via .env.auth, VPN/workspace layout, and related wiring were tweaked so the tool is practical for HTB-style labs.

Observations

General: Claude Sonnet 4.5 is a very capable model on its own. It was chosen to stay aligned with the original paper (same class of backbone as in their evaluation).

• Interface: Operation is CLI-only (excalibur --target …); there is no separate “ops” UI or orchestration layer.

• Prompt / task: You do not need a long custom prompt at startup—the system prompt is embedded; branding and tasks frame the run as a CTF solver (e.g. Solve this CTF challenge and capture the flag(s)).

• Testing (e.g. Sonnet 4.5): pros: Some hosts fell quickly; LPE often landed in ~10-15 minutes; wherever a shell existed, escalation to root was mostly successful (strong LPE); parallel work sometimes showed up (e.g. two scans on one target with different objectives). In one informal sample, all listed lab machines except Reel ended with user + root; Reel had the right exploit but no shell was received.

• Cons: The CLI is clunky; hangs occurred; there is no proper datastore or run data model—everything tends to pile into one place; mid-run human steering is limited; it does not feel like a production tool for real enterprise perimeter work, including no clear human-in-the-loop design. The product posture stays CTF-first, not corporate pentest delivery. System prompt and controller task text are tightly bound to CTF / flags / “it is all solvable”—see excalibur/prompts/pentesting.py, excalibur/interface/main.py, and the controller task path in excalibur/core/controller.py.

Hack The Box — informal run log

Lab-only, authorized targets. Rough outcomes from Excalibur runs with Claude Sonnet 4.5 (via the configured provider); approximate API spend in USD per box from those runs (pricing/tooling-dependent; excludes VPN/subscription). Difficulty is the Hack The Box machine tier (Easy / Medium / Hard / Insane) as labeled on the platform.

Machine	Difficulty (HTB)	Outcome	Cost (USD, approx.)
Sau	Easy	User + root	3.70
Pilgrimage	Easy	User + root	4.00
Lame	Easy	User + root	2.57
Topology	Easy	User + root	2.99
PC	Easy	User + root	0.87
Blue	Easy	User + root	0.93 (find) -> 3.94 (exploit)
Shocker	Easy	User + root	1.42
Optimum	Easy	User + root	1.39
Legacy	Easy	User + root	1.69
Stratosphere	Medium	User + root	1.70
Reel	Hard	Right exploit; no shell received; run stopped	0.8 (find) -> 15.00 (stopped)

Total (this sample): about $40.20 API spend across the eleven runs above.

Closing note: Sonnet 4.5 is a very strong model in its own right—these results and costs are for Excalibur + Sonnet together; a weaker backbone would behave differently on the same harness. Your mileage will vary with retries, prompt length, and provider rates.

Quick start

From the repository root on the host:

docker compose build
make config
make start
make connect

Then inside the container:

excalibur --target <ip-or-host>

Startup flow

1. Build the image

docker compose build

2. Configure authentication

make config

This writes .env.auth with the required auth settings.

3. Start the container

make start

4. Attach to the container

make connect

5. Run Excalibur

Inside the container:

excalibur --target 10.10.11.234

If you change .env.auth, recreate the container:

docker compose up -d --force-recreate excalibur

Running against lab targets

For Hack The Box or similar VPN labs, put your .ovpn file into:

./workspace/

Inside the container it will be available under:

/workspace/

Start VPN inside the container:

sudo openvpn --config /workspace/your_lab.ovpn

Or as a daemon:

sudo openvpn --config /workspace/your_lab.ovpn --daemon

Then run the agent:

excalibur --target 10.10.x.x

Common Excalibur commands

Run inside the container.

Basic run

excalibur --target 10.10.11.234

Short form

excalibur -t 10.10.11.234

Non-interactive mode

excalibur --target 10.10.11.234 --non-interactive

Run with extra context

excalibur --target 10.10.11.234 -i "WordPress target, check plugin vulns"

Override model for one run

excalibur --target 10.10.11.234 -m 'openai/gpt-5.1'

List saved sessions

excalibur --target 10.10.11.234 --list-sessions

Resume latest session

excalibur --target 10.10.11.234 -r

Resume specific session

excalibur --target 10.10.11.234 --session-id <id>

Raw output

excalibur --target 10.10.11.234 --raw

Disable telemetry

excalibur --target 10.10.11.234 --no-telemetry

OpenRouter / model routing

This fork makes OpenRouter routing editable from the host without rebuilding the image.

Main files:

• scripts/openrouter-router.json — slot-to-model routing
• scripts/ccr-config-template.json — OpenRouter provider config and allowed models
• scripts/entrypoint.sh — generates final CCR config on container start

After changing router settings, restart the container:

docker compose restart excalibur

If you edit .env.auth, use recreate instead:

docker compose up -d --force-recreate excalibur

Useful Make commands

From the repository root:

make start
make stop
make connect
make shell
make logs
make clean-docker

Notes

• default working directory inside the container: /workspace
• host directory ./workspace is mounted into the container
• make connect uses docker attach
• to detach without stopping the container: Ctrl+P, then Ctrl+Q

Provenance

• Paper: What Makes a Good LLM Agent for Real-world Penetration Testing?
• Anonymous artifact: Excalibur-FA7D on 4open.science

Disclaimer

For education and explicitly authorized security testing only. Do not use against systems without permission.