CARTS: Advancing Neural Theorem Proving with Diversified Tactic Calibration and Bias-Resistant Tree Search
A novel approach for neural theorem proving that combines diversified tactic calibration with bias-resistant tree search. Published at ICLR 2025.
This repository implements CARTS (diversified tactic CAlibration and bias-Resistant Tree Search), which balances tactic diversity and importance while calibrating model confidence. CARTS also introduces preference modeling and an adjustment term related to the ratio of valid tactics to improve the bias-resistance of the value function.
First, collect and trace the miniF2F dataset using Lean 4:
python scripts/trace.py This process may take some time depending on your CPU performance.
CUDA_VISIBLE_DEVICES=0,1,2,3 python evaluate.py \
--data-path ./data/minif2f_lean4_v4.10.0/default/ \
--prover CARTS \
--model_name reprover \
--ckpt_path "kaiyuy/leandojo-lean4-tacgen-byt5-small" \
--reward_model_path "yangxw/CARTS_vf" \
--split test \
--num-cpus 4 \
--with-gpus \
--num-theorems 244 \
--num-sampled-tactics 64 \
--exp-id "minif2f_CARTS" \
--k 8 \
--timeout 600 \
--iteration 100python evaluate.py \
--data-path ./data/minif2f_lean4_v4.10.0/default/ \
--prover CARTS \
--model_name reprover \
--ckpt_path "kaiyuy/leandojo-lean4-tacgen-byt5-small" \
--reward_model_path "yangxw/CARTS_vf" \
--split test \
--num-cpus 1 \
--with-gpus \
--num-theorems 244 \
--num-sampled-tactics 64 \
--exp-id "minif2f_CARTS" \
--k 8 \
--timeout 600 \
--iteration 100CUDA_VISIBLE_DEVICES=0,1,2,3 python evaluate.py \
--data-path ./data/minif2f_lean4_v4.10.0/default/ \
--prover BFS \
--model_name reprover \
--ckpt_path "kaiyuy/leandojo-lean4-tacgen-byt5-small" \
--split test \
--num-cpus 4 \
--with-gpus \
--num-theorems 244 \
--num-sampled-tactics 64 \
--exp-id "minif2f_BFS" \
--timeout 600 \
--iteration 100If you use this work, please cite it as follows:
@article{carts2025,
title={CARTS: Advancing Neural Theorem Proving with Diversified Tactic Calibration and Bias-Resistant Tree Search},
author={Yang, Xiao-Wen and Zhou, Zhi and Wang, Haiming and Li, Aoxue and Wei, Wen-Da and Jin, Hui and Li, Zhenguo and Li, Yu-Feng},
journal={ICLR},
year={2025}
}