Support load balancing for multiple large model interfaces
Provide a unified API interface
Compatible with OpenAI
Support streaming transmission
Support IP affinity algorithm