I had issues with implementing nice use of containers for llm services - I think what we currently run as mcli api, mcli image-gen and mcli infer would be great to run as docker containers, but then we need containers that can access gpu (and for mac its again different story i think), so for windows we need image with cuda support, and one witout cuda support, same for linux etc.

If you have a idea how it can be solved correctly i think it will be great improvement.

• For infer its basically dockerizing MaIN.InferPage project
• For api its dockerizing MaIN project (where we have api)
• For ImageGen. Content of ImageGen folder in root repo, python api with usage of diffusers to run flux model