Search results

Cannot load dataset from s3

Status: Open.

#338 In aws/fmeval;

· AmrSaber opened on Jan 7, 2025

Problem running package with python 3.13

Status: Open.

#337 In aws/fmeval;

· AmrSaber opened on Jan 6, 2025

[Feature] Increase model coverage of the BERT Score metric by adding torchmetrics implementation

Status: Open.

#332 In aws/fmeval;

· achad4 opened on Nov 8, 2024

[Feature] Lambda model runner

Status: Open.

#331 In aws/fmeval;

· achad4 opened on Nov 8, 2024

Option to disable BERTScore in QAAccuracy

Status: Open.

#330 In aws/fmeval;

· athewsey opened on Oct 28, 2024

[Feature] Add callback mechanism to evaluation

Status: Open.

#284 In aws/fmeval;

· acere opened on May 29, 2024

[Feature] Add system metrics collected during evaluation to eval_output

Status: Open.

#280 In aws/fmeval;

· acere opened on May 24, 2024

[Feature] Streaming results/progress and summary metrics for faster feedback

Status: Open.

#278 In aws/fmeval;

· athewsey opened on May 17, 2024

[Feature] EvalAlgorithmInterface.evaluate should accept a list of DataConfigs for consistency

Status: Open.

#269 In aws/fmeval;

· athewsey opened on May 3, 2024

[Feature] JSON export/import for EvalOutput classes

Status: Open.

#268 In aws/fmeval;

· athewsey opened on May 3, 2024

[Feature] Image fields for multi-modal models

Status: Open.

#247 In aws/fmeval;

· athewsey opened on Apr 9, 2024

[Feature] Multi-variable prompt templates

Status: Open.

#246 In aws/fmeval;

· athewsey opened on Apr 9, 2024