• Docker (recommended) or Python 3.10+
• CUDA-capable GPU (optional, for faster training)

# Clone the repository
git clone https://github.com/elte-collective-intelligence/student-mechanism-design.git
cd student-mechanism-design

# Build Docker images
docker build --progress plain -f ./docker/BaseDockerfile -t student_mechanism_design_base .
docker build --progress plain -f ./docker/Dockerfile -t student_mechanism_design .

# Run training experiment
docker run --rm --gpus=all --mount type=bind,src=$PWD,dst=/app student_mechanism_design all

# Run unit tests
docker run --rm --mount type=bind,src=$PWD,dst=/app student_mechanism_design --unit_test

# Run ablation studies
docker run --rm --mount type=bind,src=$PWD,dst=/app student_mechanism_design python src/eval/run_ablations.py --ablation all

pip install -r requirements.txt
cd src
python main.py all --agent_configs=mappo --log_configs=verbose

This project implements a mechanism design approach for the Scotland Yard pursuit-evasion game using multi-agent reinforcement learning. Key features include:

• Partial Observability: MrX is hidden from police with configurable reveal schedules
• Belief Tracking: Particle filter and learned belief encoders for police
• Mechanism Design: Configurable tolls, budgets, and reveal policies
• Meta-Learning: Automatic tuning of mechanism parameters toward 50% win rate
• Population-Based Self-Play: Policy pools with ELO-style scoring
• MAPPO & GNN Agents: State-of-the-art multi-agent RL algorithms

# Run specific experiment
docker run --rm --gpus=all --mount type=bind,src=$PWD,dst=/app student_mechanism_design <experiment_name>

# Examples:
docker run ... student_mechanism_design smoke_train
docker run ... student_mechanism_design all
docker run ... student_mechanism_design big_graph

Each agent receives:

• Type: Discrete(N) where N = number of nodes
• Masking: Actions masked by budget and topology
• Mapping: Fixed identity mapping (action i → node i)

# Fixed index→node mapping ensures consistency
mask[node] = True if (adjacent[current, node] and cost <= budget)
index_to_node = {i: i for i in range(num_nodes)}  # Identity mapping

We report the following three metrics as required by the assignment:

Definition: Fraction of episodes won by MrX

Win Rate = MrX Wins / Total Episodes
Target: 0.50 ± 0.05

Implementation: src/eval/metrics.py::compute_win_rate()

Why this metric: Measures game balance—the primary goal of mechanism design. A win rate of 50% indicates fair gameplay where neither side has a systematic advantage.

Definition: Cross-entropy between police belief distribution and true MrX position at reveal times.

CE = -log(belief[true_mrx_position])
Lower is better (more accurate belief)

Implementation: src/eval/metrics.py::belief_cross_entropy()

Why this metric: Measures how well police can track MrX under partial observability. Lower cross-entropy means the belief distribution assigns higher probability to MrX's true location.

Definition: Average episode length, split by winner.

• Time-to-Catch: Mean steps when Police wins
• Survival Time: Mean steps when MrX wins

Implementation: src/eval/metrics.py::compute_time_metrics()

Why this metric: Captures game dynamics—shorter catch times indicate effective police coordination, while longer survival times indicate successful evasion strategies.

Config: src/configs/ablation/belief.yaml

Compares belief tracking methods under partial observability:

Run:

python src/eval/run_ablations.py --ablation belief --num_episodes 100 --seeds 42 123 456

Expected Results:

• no_belief: MrX win rate ~70-80% (Police cannot track)
• particle_filter: MrX win rate ~50-55% (Baseline)
• learned_encoder: MrX win rate ~45-55% (Comparable or better)

Config: src/configs/ablation/mechanism.yaml

Compares mechanism configurations:

Run:

python src/eval/run_ablations.py --ablation mechanism --num_episodes 100 --seeds 42 123 456

Expected Results:

• no_mechanism: Demonstrates need for mechanism design
• fixed_mechanism: Shows improvement over baseline
• meta_learned: Achieves target balance through optimization

python src/eval/run_ablations.py --ablation all --num_episodes 100 --output_dir logs/ablations

Results are saved to logs/ablations/:

• belief_results.json: Raw metrics data
• belief_report.txt: Formatted comparison report
• mechanism_results.json: Raw metrics data
• mechanism_report.txt: Formatted comparison report

1. Belief Collapse: Particle filter can collapse to incorrect modes when reveals are sparse (R > 10)

   • Mitigation: Noise injection, increased particle count, or use learned encoder

2. Budget Exhaustion: Police may run out of budget before catching MrX on large graphs

   • Mitigation: Meta-learning adjusts budget based on observed win rate

3. Graph Topology Sensitivity: Performance varies significantly with graph structure (degree distribution, diameter)

   • Mitigation: Curriculum learning over diverse graph distributions

4. Action Mask Edge Cases: When no moves are affordable, agent stays in place

   • Handled: Environment returns current position as default action

5. Reward Hacking: Agents may exploit reward shaping rather than achieving true objectives

   • Mitigation: Use terminal rewards primarily, validate with win rate metric

# Enable verbose logging
docker run ... student_mechanism_design all --log_configs=verbose

# Visualize episodes (generates GIFs)
docker run ... student_mechanism_design smoke_train --vis_configs=full

# Run unit tests to verify components
docker run ... student_mechanism_design --unit_test

# Check specific test
pytest test/test_action_mask.py -v

src/
├── main.py                     # Training entry point
├── logger.py                   # Logging utilities (WandB, TensorBoard)
├── reward_net.py               # RewardWeightNet for meta-learning
├── configs/
│   ├── ablation/
│   │   ├── belief.yaml         # Belief ablation variants
│   │   └── mechanism.yaml      # Mechanism ablation variants
│   ├── agent/                  # Agent configurations
│   ├── mechanism/default.yaml  # Mechanism parameters
│   └── ...
├── Enviroment/
│   ├── yard.py                 # Main environment (CustomEnvironment)
│   ├── action_mask.py          # Action masking with fixed index→node mapping
│   ├── belief_module.py        # ParticleBeliefTracker, LearnedBeliefEncoder
│   ├── partial_obs.py          # PartialObservationWrapper
│   ├── graph_generator.py      # GraphGenerator with seed saving
│   └── graph_layout.py         # ConnectedGraph sampling
├── RLAgent/
│   ├── mappo_agent.py          # MAPPO implementation
│   ├── gnn_agent.py            # GNN-based DQN agent
│   ├── random_agent.py         # Random baseline
│   └── base_agent.py           # Abstract base class
├── selfplay/
│   ├── population_manager.py   # Population-based training with ELO
│   ├── opponent_modeling.py    # Opponent behavior modeling
│   └── best_response.py        # Best response utilities
├── mechanism/
│   ├── mechanism_config.py     # MechanismConfig dataclass
│   ├── meta_learning_loop.py   # MetaLearner for mechanism optimization
│   └── reward_weight_integration.py
├── eval/
│   ├── metrics.py              # Core metrics (win rate, belief CE, time)
│   ├── run_ablations.py        # Ablation study runner
│   ├── ood_eval.py             # OOD & robustness evaluation
│   ├── belief_quality.py       # Belief cross-entropy
│   └── exploitability.py       # Exploitability proxy
├── experiments/
│   ├── all/config.yml
│   ├── smoke_train/config.yml
│   ├── singular/config.yml
│   └── ...
└── artifacts/                  # Saved model checkpoints
test/
├── test_action_mask.py         # Action mask unit tests
├── test_belief_update.py       # Belief tracking tests
├── env_test.py                 # Environment smoke tests
└── smoke_test.py               # Basic sanity check

All parameters are configurable via YAML:

# src/configs/mechanism/default.yaml
police_budget: 10
reveal_interval: 5
reveal_probability: 0.0
ticket_price: 1.0
target_win_rate: 0.5
secondary_weight: 0.1

# src/experiments/all/config.yml
agent_configurations:
  - num_police_agents: 2
    agent_money: 10
  - num_police_agents: 3
    agent_money: 8
  # ...
num_episodes: 70
epochs: 200
random_seed: 42

Set credentials in src/wandb_data.json:

{
  "wandb_api_key": "<your-api-key>",
  "wandb_project": "scotland-yard",
  "wandb_entity": "<your-entity>"
}

Leave as "null" to disable WandB logging.

# Run all tests
pytest test/

# Run specific tests
pytest test/test_action_mask.py -v
pytest test/test_belief_update.py -v
pytest test/env_test.py -v

1. ✅ Action mask correctness: test_action_mask.py::test_action_mask_fixed_index_node_mapping
2. ✅ Belief update step: test_belief_update.py::test_belief_updates_and_reveals

• MAPPO: The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games
• Scotland Yard Board Game
• PettingZoo Documentation
• TorchRL Documentation
• Mechanism Design Theory

This project is licensed under CC BY-NC-ND 4.0. See the LICENSE file for details.