This is an attempt to create a proof-of-concept OpenAI compatible API for running LLM's on RK3588 using RKNN-LLM / RK3588's NPU for accelerated inference.

It is not complete, it is not ready for testing (or prod), it is not documented and you can assume a bunch of other not's!

Current RKNN-LLM version: 1.2.0