Thanks to visit codestin.com
Credit goes to github.com

Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

Server accepts file for pre-/post-processing functions#1033

Merged
rahul-tuli merged 5 commits into
mainfrom
server-accepts-file
May 25, 2023
Merged

Server accepts file for pre-/post-processing functions#1033
rahul-tuli merged 5 commits into
mainfrom
server-accepts-file

Conversation

@rahul-tuli

@rahul-tuli rahul-tuli commented May 23, 2023

Copy link
Copy Markdown
Member

This PR makes code changes such that a custom pipeline, with custom pre-/post-processing functions can be created using the deepsparse.server directly. This enables a user to use our system without knowing anything about custom pipelines

Additionally this PR also makes documentation updates, to expose this pathway.

# custom-processing-functions-config.yaml

endpoints:
  - task: custom
    model: /home/rahul/models/resnet50.onnx
    kwargs:
      processing_file: /home/rahul/projects/deepsparse/processing.py
# processing.py

from torchvision import transforms
from PIL import Image
import torch

IMAGENET_RGB_MEANS = [0.485, 0.456, 0.406]
IMAGENET_RGB_STDS = [0.229, 0.224, 0.225]
preprocess_transforms = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=IMAGENET_RGB_MEANS, std=IMAGENET_RGB_STDS),
])

def preprocess(img_file):
    with open(img_file, "rb") as img_file:
        img = Image.open(img_file)
        img = img.convert("RGB")
    img = preprocess_transforms(img)
    batch = torch.stack([img])
    return [batch.numpy()]

def postprocess(outputs):
    return outputs

Invocation:

deepsparse.server --config_file custom-processing-functions-config.yaml

And Voila! Now the server creates a custom pipeline, with custom pre-/post- processing functions from the specified processing.py

Special thanks to @bfineran for thinking about this pathway
And @dbogunowicz for #821 from which the loading functions code is inspired from

Additionally this pathway will become highly useful for sparsify.package CLI

Comment thread src/deepsparse/pipelines/custom_pipeline.py Outdated
Comment thread docs/user-guide/deepsparse-server.md Outdated

@dbogunowicz dbogunowicz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh so this is what you've been working on. I like that!

One thing which is missing are some unit tests. Any plans to add those as well?

@KSGulin KSGulin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a cool addition

@rahul-tuli rahul-tuli merged commit 6029f89 into main May 25, 2023
@rahul-tuli rahul-tuli deleted the server-accepts-file branch May 25, 2023 13:10
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

enhancement New feature or request mle-team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants