-
Notifications
You must be signed in to change notification settings - Fork 96
Open
Description
Currently, adding new evals to the trainset is supported for cumulative runs via log_dir, which reuses optimization history. However, adding to the valset is not yet supported. This would be a trivial change and align with plans for a more flexible system where not all prompt candidates need evaluation on all valset instances.
Also support not having to run full valset evaluation for every new program discovered.
Metadata
Metadata
Assignees
Labels
No labels