Add Offline GEPA (Genetic-Pareto) Optimizer #4604
Conversation
- 29 tests covering Pareto frontier logic and helper functions - Integration tests for 3-step GEPA algorithm - Performance test: 10 variants × 100 datapoints
- Explain Step 1 generalization: max score → Pareto non-dominated - Add rationale for aggressive missing data imputation - Clarify design decisions for multi-evaluator support
- Add module-level imports for EvaluatorConfig, EvaluationConfig, GEPAConfig - Extract compare_values helper to eliminate 38 lines of duplication - Extract calculate_frequencies helper to eliminate duplicate logic - Replace explicit loops with iterator chains (more idiomatic Rust) - Fix (*variant_name).clone() → variant_name.to_string() - All 29 tests passing, clippy clean
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 37 out of 37 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -0,0 +1,410 @@ | |||
| //! Shared test helpers for GEPA e2e tests | |||
|
|
|||
| #![allow(clippy::unwrap_used, clippy::expect_used, clippy::missing_panics_doc)] | |||
There was a problem hiding this comment.
Inconsistent use of #![allow(...)] here versus #![expect(...)] in other test files (e.g., tests/common/gepa.rs). For consistency across the codebase, use the same approach in all test files.
| //! These tests exercise the analyze_inferences component using real gateway clients | ||
| //! and the full inference pipeline. | ||
|
|
||
| #![allow(clippy::unwrap_used, clippy::expect_used)] |
There was a problem hiding this comment.
Inconsistent use of #![allow(...)] versus #![expect(...)] used in tests/common/gepa.rs. Use the same lint suppression approach consistently across all test files in this PR.
| //! These tests exercise the mutate_templates component using real gateway clients | ||
| //! and the full inference pipeline. | ||
|
|
||
| #![allow(clippy::unwrap_used, clippy::expect_used)] |
There was a problem hiding this comment.
Inconsistent use of #![allow(...)] versus #![expect(...)] used in tests/common/gepa.rs. Use the same lint suppression approach consistently across all test files in this PR.
…point_input -> inference_input, document
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 41 out of 41 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Implements an offline version GEPA, a prompt optimization algorithm that uses genetic programming to evolve prompt templates through iterative evaluation and mutation.
Key Files
tensorzero-optimizers/src/gepa/lib.rs-run_gepa_optimization()orchestrates the iterative evaluation, analysis, mutation cycletensorzero-core/src/optimization/gepa.rs-GEPAConfig,UninitializedGEPAConfig, andGEPAJobHandledefinitionsKey Features
tensorzero::optimization::gepa::analyzeprovides structured feedback,tensorzero::optimization::gepa::mutategenerates improved templatesInfrastructure Changes
OptimizerOutput::Variants)UninitializedChatTemplates::innerpublicTesting
TODO