Official Repository for NeurIPS 2025 Paper: "A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning"

We provide two ways to create the Python environment for this repository. Please choose one of the following methods:

python -m venv rpc
source rpc/bin/activate
pip install -r requirements.txt 

conda create -n rpc python=3.9
conda activate rpc
pip install -r requirements.txt

Run evaluation with specific parameters:

python main.py --dataset MathOdyssey --model InternLM2-Math-Plus-7B --method RPC --K 128

Parameters:

• --dataset: Choose from MATH, MathOdyssey, AIME, OlympiadBench
• --model: Choose from Deepseek-Math-RL-7B, InternLM2-Math-Plus-1.8B, InternLM2-Math-Plus-7B
• --method: Choose from PPL (Perplexity), SC (Self-Consistency), RPC (our method)
• --K: Number of reasoning paths to sample (128 for MathOdyssey, AIME, OlympiadBench, and 64 for MATH)

Run comprehensive evaluation across multiple settings:

bash all_exps.sh

This will evaluate all method-dataset-model combinations and save results to results.txt.

1. If you cannot download data from Hugging Face directly, please use Hugging Mirror instead.
2. It may take some time to generate the cache for checking answer equality when running each dataset for the first time.

@inproceedings{zhou24theoretical,
      author    = {Zhou, Zhi and Tan, Yuhao and Li, Zenan and Yao, Yuan and Guo, Lan-Zhe and Li, Yu-Feng and Ma, Xiaoxing},
      title     = {A Theorecial Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning},
      booktitle = {Advances in Neural Information Processing Systems},
      year      = {2025},
    }