Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@nkaenzig
Copy link
Collaborator

@nkaenzig nkaenzig commented Oct 10, 2025

Contributes to #908

FreeFormQuestionPromptTemplate prompt examples

QuiltVQA

Read the provided question and provide a concise, factual answer based only on what is visible in the image.

Question: What is the visual characteristic of the cells after therapy?
Answer:

With few-shot-examples

template = FreeFormQuestionPromptTemplate(enable_cot=True)
prompt = template.render(
        question="What is the capital of France?",
        context=["France is a country in Europe.", "The capital of France is Paris."],
        examples=[
            {
                "question": "What is 2 + 2?",
                "answer": "4",
            },
            {
                "question": "What is the capital of Germany?",
                "answer": "Berlin",
            },
        ],
        preamble="You are a helpful assistant.",
    )
print(prompt)

generates this prompt:

You are a helpful assistant.

Below are some examples:

Example 1:
Question: What is 2 + 2?
Answer: 4
---

Example 2:
Question: What is the capital of Germany?
Answer: Berlin
---

Now please answer the following question.
Think step-by-step before giving your final answer.

Question: What is the capital of France?
Answer:

@nkaenzig nkaenzig marked this pull request as ready for review October 10, 2025 13:59
@nkaenzig nkaenzig self-assigned this Oct 10, 2025
Copy link
Collaborator

@MaxFeucht MaxFeucht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great addition, looks nice. Would be good to add a config file for QuiltVQA that exemplifies how the dataset is loaded (metric section could be commented out for now until we merge the LLMJudge metric), and to adapt the CoT tag restriction. Other than that only minor comments

@nkaenzig
Copy link
Collaborator Author

Would be good to add a config file for QuiltVQA that exemplifies how the dataset is loaded

Config file will be added in a follow PR once we have merged a first LLM Judge metric.

@nkaenzig nkaenzig enabled auto-merge (squash) October 15, 2025 15:05
@nkaenzig nkaenzig merged commit 981ae10 into main Oct 15, 2025
7 checks passed
@nkaenzig nkaenzig deleted the 908-add-quiltvqa-dataset branch October 15, 2025 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants