Thanks to visit codestin.com
Credit goes to github.com

Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

[Text Gen UX] top level constructor aliases + code gen subclass#1274

Merged
bfineran merged 1 commit into
mainfrom
generation-ux-updates
Sep 25, 2023
Merged

[Text Gen UX] top level constructor aliases + code gen subclass#1274
bfineran merged 1 commit into
mainfrom
generation-ux-updates

Conversation

@bfineran

@bfineran bfineran commented Sep 25, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

This PR adds the requested constructor aliases for the main text generation tasks. also adds a code generation subclass to reference - idea here is that if there is ever extra processing or defaults requested for code generation, it will be easy to add in to the subclass

requested UX:

from deepsparse import TextGeneration, CodeGeneration, Chat

text_pipeline = TextGeneration(model=MODEL, **kwargs)
text_result = text_pipeline(prompt=PROMPT, **kwargs)

code_pipeline = CodeGeneration(model=MODEL, **kwargs)
code_result = code_pipeline(prompt=PROMPT, **kwargs)

chat_pipeline = Chat(model=MODEL, **kwargs)
chat_result = chat_pipeline(prompt=PROMPT, *args, **kwargs)

test_plan:
manually verified:

from deepsparse import CodeGeneration

code_pipeline = CodeGeneration(model="/home/benjamin/neuralmagic/models/codegen_mono-350m-bigpython_bigquery_thepile-pruned50/deployment", engine_type="onnxruntime")
code_pipeline(prompt="def hello_world:", generation_config=dict(max_new_tokens=10))

@bfineran bfineran requested a review from dsikka September 25, 2023 16:31
@bfineran bfineran self-assigned this Sep 25, 2023
dsikka
dsikka approved these changes Sep 25, 2023
View reviewed changes

@dsikka dsikka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bfineran bfineran merged commit 20a6157 into main Sep 25, 2023
@bfineran bfineran deleted the generation-ux-updates branch September 25, 2023 19:52
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer

@dsikka dsikka dsikka approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@bfineran bfineran

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@bfineran @dsikka