Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@danielezhu
Copy link
Contributor

Description of changes:
This PR makes the following changes:

  1. Update GetModelResponse and all three summarization accuracy metric transforms to support multiple input keys
  2. Update TransformPipeline.execute to materialize intermediate datasets (as I've observed that not materializing the dataset leads to a performance degradation in SummarizationAccuracy evaluations). This change causes the performance of SummarizationAccuracy to match the performance of the old code.
  3. Refactor the summarization accuracy metrics to use a parent ABC, mimicking the approach taken with the semantic perturbation transforms.
  4. Fix bug in semantic_perturbations.py where I didn't unpack *args and **kwargs. Updated the unit test to catch this bug.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Daniel Zhu added 2 commits March 18, 2024 14:12
nathanng17
nathanng17 previously approved these changes Mar 18, 2024
nathanng17
nathanng17 previously approved these changes Mar 18, 2024
@danielezhu danielezhu merged commit bbbc905 into aws:main Mar 18, 2024
@danielezhu danielezhu deleted the add_multi_input branch March 18, 2024 22:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants