-
Notifications
You must be signed in to change notification settings - Fork 20
Enhance API consistency and clarify test pattern: ChatScenarioRunner with two-client separation #43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…yword, and synonyms Co-authored-by: mehrandvd <[email protected]>
Co-authored-by: mehrandvd <[email protected]>
|
@copilot The implementation is fine; update all the tests and documentation to use the latest recommended naming conventions. There's no need to include unit tests for backward compatibility anymore. Keep the existing tests and just update them to use the latest API. |
…ng conventions Co-authored-by: mehrandvd <[email protected]>
I've successfully updated all tests and documentation to use the latest recommended naming conventions. Here's what I've updated: Tests Updated:
Documentation Updated:
Scenario Files Updated:
Key Changes:
All tests are passing and code formatting is consistent. The changes maintain full backward compatibility while promoting the new recommended API. Commit: 2288ec8 |
|
@copilot Thanks for the PR—great improvements. I have a suggestion to make the test pattern and docs clearer, especially regarding the two different chat clients involved. Current sample pattern: var runner = new ChatScenarioRunner(myChatClient);
await runner.RunAsync(scenarios, myChatClient);Issue: There are actually two distinct roles but the sample uses the same variable name (myChatClient) for both:
When both are called myChatClient and the runner is created inline inside each test, it’s easy to conflate them. Proposal:
Example refactor: public class GreetingTests
{
private readonly ChatScenarioRunner ScenarioRunner;
private readonly IChatClient systemUnderTestClient;
public GreetingTests()
{
var assertionClient = /* build the model used for semantic assertions */;
systemUnderTestClient = /* build or obtain the model under test */;
ScenarioRunner = new ChatScenarioRunner(assertionClient);
}
[Fact]
public async Task TestGreeting()
{
var markdown = File.ReadAllText("greeting.md");
var scenarios = ChatScenario.LoadFromText(markdown);
await ScenarioRunner.RunAsync(scenarios, systemUnderTestClient);
}
}Documentation adjustment: Before: var runner = new ChatScenarioRunner(myChatClient);
await runner.RunAsync(scenarios, myChatClient);After (clear separation): // In your test class constructor:
var assertionClient = /* assertion/evaluation model */;
ScenarioRunner = new ChatScenarioRunner(assertionClient);
// In your test:
await ScenarioRunner.RunAsync(scenarios, systemUnderTestClient);Suggested follow-up changes:
|
…uctor-based initialization Co-authored-by: mehrandvd <[email protected]>
I've implemented all the suggested improvements to clarify the two-client pattern: Key Changes:
Documentation improvements show: public GreetingTests()
{
var assertionClient = /* assertion/evaluation model */;
systemUnderTestClient = /* system under test model */;
ScenarioRunner = new ChatScenarioRunner(assertionClient);
}
[Fact]
public async Task TestGreeting()
{
await ScenarioRunner.RunAsync(scenarios, systemUnderTestClient);
}All tests pass and the pattern now clearly distinguishes between the assertion client (for semantic evaluations) and the system under test client (whose behavior is being validated). Commit: a7b84ea |
|
@copilot Avoid adding tests to verify if synonyms work, as they are only supported for backward compatibility. Remove any mention or use of synonyms in the documentation or tests. Synonyms should remain in the code solely to support legacy users of the library. |
…d compatibility Co-authored-by: mehrandvd <[email protected]>
I've removed all synonym documentation and tests as requested. The changes include:
Synonyms now remain in the code solely to support legacy users but are no longer documented or tested. All tests pass. Commit: 9d6ff18 |
This PR enhances the skUnit API consistency and clarifies the test pattern by introducing clearer separation between assertion and system-under-test clients.
Key Improvements
1. Clear Two-Client Pattern
The framework now clearly distinguishes between:
ChatScenarioRunnerconstructor for semantic evaluations and assertionsRunAsync()- the client whose behavior is being tested2. Constructor-Based Initialization
Test classes now use constructor-based initialization with clearer variable names:
3. Enhanced Documentation
4. API Consistency
ASSERTkeyword alongside existingCHECKstatementsChatScenarioRunnerconstructor parameter renamed toassertionClientfor clarity5. Updated Examples
This change maintains full backward compatibility while promoting clearer, more maintainable test code that makes the distinction between evaluation and testing clients explicit.
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.