Thanks to visit codestin.com
Credit goes to github.com

Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

torchscript, cpu and gpu support#1015

Merged
bfineran merged 29 commits into
mainfrom
torchscript
Aug 1, 2023
Merged

torchscript, cpu and gpu support#1015
bfineran merged 29 commits into
mainfrom
torchscript

Conversation

@horheynm

@horheynm horheynm commented Apr 27, 2023 •
edited
Loading

Copy link
Copy Markdown

Torchscript integration

Example

model = "path_to_dot_pt" or nn_module or torchscipt_model
device = "cpu" or "cuda"
engine = TorchScriptEngine(model, device=device)
inp = [numpy.random.rand(1, 3, 224, 224).astype(numpy.float32)]
out = engine(inp)

@horheynm horheynm changed the title torchscript, gpu support torchscript, cpu and gpu support Jun 21, 2023
@horheynm horheynm marked this pull request as ready for review June 26, 2023 14:28
@horheynm horheynm requested review from a team, KSGulin, markurtz and rgreenberg1 and removed request for a team June 26, 2023 14:28
@horheynm horheynm marked this pull request as draft June 26, 2023 14:30
@horheynm horheynm marked this pull request as ready for review July 6, 2023 01:03
@horheynm horheynm force-pushed the torchscript branch 2 times, most recently from f4bd721 to c39f061 Compare July 6, 2023 01:35
bfineran
bfineran reviewed Jul 26, 2023
View reviewed changes

@bfineran bfineran left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM pending comments - additionally, let's update PR description to include scope of feature as well as example snippet

future work should add zoo support

Comment thread setup.py Outdated
Comment thread src/deepsparse/benchmark/torchscript_engine.py
Comment thread src/deepsparse/benchmark/torchscript_engine.py Outdated
Comment thread src/deepsparse/benchmark/torchscript_engine.py Outdated
Comment thread src/deepsparse/benchmark/torchscript_engine.py Outdated
Comment thread src/deepsparse/benchmark/torchscript_engine.py Outdated
Comment thread src/deepsparse/benchmark/torchscript_engine.py Outdated
Comment thread tests/conftest.py
Comment thread tests/conftest.py
Comment thread tests/test_torchscript.py
@bfineran bfineran merged commit 758fc05 into main Aug 1, 2023
@bfineran bfineran deleted the torchscript branch August 1, 2023 13:35
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@markurtz markurtz Awaiting requested review from markurtz

@KSGulin KSGulin Awaiting requested review from KSGulin

@rgreenberg1 rgreenberg1 Awaiting requested review from rgreenberg1

1 more reviewer

@bfineran bfineran bfineran approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@horheynm horheynm

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@horheynm @bfineran