Thanks to visit codestin.com
Credit goes to github.com

Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

[TextGeneration] Samling arguments for generation#1225

Merged
bfineran merged 15 commits into
mainfrom
sampling
Sep 15, 2023
Merged

[TextGeneration] Samling arguments for generation#1225
bfineran merged 15 commits into
mainfrom
sampling

Conversation

@horheynm

@horheynm horheynm commented Sep 1, 2023

Copy link
Copy Markdown
  1. Add sampling into the text-generation pipeline
  2. Refactor token generation flow

I also need #1234 to land to check on failures caused by this pr if any

Comment thread src/deepsparse/transformers/token_generator.py Outdated

@dbogunowicz dbogunowicz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Designwise looks great. Let's clean up the code, make it nicer in terms of style. Also, before landing let's make sure that all the tests in the new LLM testing harness pass:

#1216

This PR has potential to accidently break main pathways.

Comment thread src/deepsparse/transformers/engines/nl_decoder_engine.py
Comment thread src/deepsparse/transformers/pipelines/text_generation.py
Comment thread src/deepsparse/transformers/utils/token_generator.py
Comment thread src/deepsparse/transformers/pipelines/text_generation.py Outdated
Comment thread src/deepsparse/transformers/pipelines/text_generation.py
Comment thread src/deepsparse/transformers/pipelines/token_classification.py
Comment thread src/deepsparse/transformers/utils/token_generator.py Outdated
Comment thread src/deepsparse/transformers/utils/token_generator.py Outdated
Comment thread src/deepsparse/transformers/utils/token_generator.py Outdated
@dbogunowicz

Copy link
Copy Markdown
Contributor

Also, PR description missing, let's make the PR title more descriptive.

Comment thread src/deepsparse/transformers/pipelines/text_generation.py
Comment thread src/deepsparse/transformers/pipelines/text_generation.py
Comment thread src/deepsparse/transformers/utils/token_generator.py Outdated
Comment thread src/deepsparse/transformers/utils/token_generator.py Outdated
Comment thread src/deepsparse/transformers/utils/token_generator.py Outdated
Comment thread src/deepsparse/transformers/utils/token_generator.py Outdated
Comment thread src/deepsparse/transformers/utils/token_generator.py Outdated
@dbogunowicz dbogunowicz mentioned this pull request Sep 8, 2023
Comment thread src/deepsparse/transformers/pipelines/text_generation.py Outdated
Comment thread src/deepsparse/transformers/pipelines/text_generation.py Outdated
@horheynm horheynm marked this pull request as ready for review September 11, 2023 20:41
bfineran
bfineran previously approved these changes Sep 12, 2023
dbogunowicz
dbogunowicz previously approved these changes Sep 13, 2023
Comment thread src/deepsparse/transformers/pipelines/text_generation.py Outdated
@bfineran bfineran dismissed stale reviews from dbogunowicz and themself via 816c5ea September 13, 2023 13:13
@bfineran bfineran changed the title Sampling [TextGeneration] Samling arguments for generation Sep 15, 2023
@bfineran

Copy link
Copy Markdown
Contributor

tests unrelated, merging

@bfineran bfineran merged commit 55307ad into main Sep 15, 2023
@bfineran bfineran deleted the sampling branch September 15, 2023 14:27
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants