Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Native Integraion with vLLM Semantic Router #1415

@Xunzhuo

Description

@Xunzhuo

Description:

vLLM tracked in vllm-project/semantic-router#486

Add Native Integraions with vLLM SR, providing:

  • intelligent routing
  • semantic caching
  • prompt guard, pii detection

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions