• This is the repo for the paper HiRAG: Retrieval-Augmented Generation with Hierarchical Knowledge.
• Accepted to EMNLP 2025 Findings!🎉

# remember clone this repo first
cd HiRAG
pip install -e .

You can just utilize the following code to perform a query with HiRAG.

graph_func = HiRAG(
    working_dir="./your_work_dir",
    enable_llm_cache=True,
    enable_hierachical_mode=True, 
    embedding_batch_num=6,
    embedding_func_max_async=8, # according to your machine
    enable_naive_rag=True
    )
# indexing
graph_func.insert("path_to_your_context.txt")
# retrieval & generation
print("Perform hi search:")
print(graph_func.query("The question you want to ask?", param=QueryParam(mode="hi")))

Or if you want to employ HiRAG with DeepSeek, ChatGLM, or other third-party retrieval api, here are the examples in ./hi_Search_deepseek.py, ./hi_Search_glm.py, and ./hi_Search_openai.py. The API keys and the LLM configurations can be set at ./config.yaml.

We take the procedure in Mix dataset as an example.

cd ./HiRAG/eval

1. Extract context from original QA datasets.

python extract_context.py -i ./datasets/mix -o ./datasets/mix

2. Insert context to Graph Database.

python insert_context_deepseek.py

Note that the script insert_context_deepseek.py is for the setting of generation with DeepSeek-v3 api, you can replace that with insert_context_openai.py or insert_context_glm.py.

3. Test with different versions of HiRAG.

# there are different retrieval options
# If you want to employ HiRAG approach, just run:
python test_deepseek.py -d mix -m hi
# If you want to employ naive RAG approach, just run:
python test_deepseek.py -d mix -m naive
# If you want to employ HiRAG approach w/o bridge, just run:
python test_deepseek.py -d mix -m hi_nobridge
# If you want to employ HiRAG approach with retrieving only local knowledge, just run:
python test_deepseek.py -d mix -m hi_local
# If you want to employ HiRAG approach with retrieving only global knowledge, just run:
python test_deepseek.py -d mix -m hi_global
# If you want to employ HiRAG approach with retrieving only bridge knowledge, just run:
python test_deepseek.py -d mix -m hi_bridge

Note that the dataset mix can be replaced to any other datasets in Hugging Face link. And the script test_deepseek.py is for the setting of generation with DeepSeek-v3 api, you can replace that with test_openai.py or test_glm.py.

4. Evaluate the generated answers.

First step, request for evaluations.

python batch_eval.py -m request -api openai
python batch_eval.py -m request -api deepseek

Second step, get the results.

python batch_eval.py -m result -api openai
python batch_eval.py -m result -api deepseek

With the config output_file set as f"./datasets/{DATASET}/{DATASET}_eval_hi_naive.jsonl", just run the command:

python batch_eval.py -m result -api openai

With the config output_file set as f"./datasets/{DATASET}/{DATASET}_eval_hi_graphrag.jsonl", just run the command:

python batch_eval.py -m result -api openai

With the config output_file set as f"./datasets/{DATASET}/{DATASET}_eval_hi_lightrag.jsonl", just run the command:

python batch_eval.py -m result -api openai

With the config output_file set as f"./datasets/{DATASET}/{DATASET}_eval_hi_fastgraphrag.jsonl", just run the command:

python batch_eval.py -m result -api openai

With the config output_file set as f"./datasets/{DATASET}/{DATASET}_eval_hi_kag.jsonl", just run the command:

python batch_eval.py -m result -api openai

We gratefully acknowledge the use of the following open-source projects in our work:

• nano-graphrag: a simple, easy-to-hack GraphRAG implementation

• RAPTOR: a novel approach to retrieval-augmented language models by constructing a recursive tree structure from documents.

@article{huang2025retrieval,
  title={Retrieval-Augmented Generation with Hierarchical Knowledge},
  author={Huang, Haoyu and Huang, Yongfeng and Yang, Junjie and Pan, Zhenyu and Chen, Yongqiang and Ma, Kaili and Chen, Hongzhi and Cheng, James},
  journal={arXiv preprint arXiv:2503.10150},
  year={2025}
}