-
|
Hi, did a Folder Is there are way to make it a bit lighter? ;) |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 10 replies
-
|
This is most likely because Torch packs a bunch of Nvidia dependencies. They provide hardware acceleration for embedding/reranking models on supported GPUs. If you don't have cuda or is okay with the CPU-only performance for embedding and reranking, changing the index URL for torch so that it installs the cpu-only build should help. You can find the corresponding index URL on torch's website. The docs contain instructions to do this for |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, @Davidyz , that shaved over 4GB for me using PS: Feeling a bit embarrassed missing the bits in the docs. |
Beta Was this translation helpful? Give feedback.
-
|
No worries. I'm pretty sure there's an equivalent for uv too, it's just you might need to dig into their documentation. I've also been looking into possibilities to publish with cpu-only torch and have cuda-enabled torch as an optional dependency, but I haven't had any luck because the only way to do this seems to be overwrite the index URL for the whole package, which is just not very elegant and may break for people who uses pypi mirrors. |
Beta Was this translation helpful? Give feedback.
-
|
I decided to change the suggested installation method to |
Beta Was this translation helpful? Give feedback.
-
|
That's curious. I tried using the |
Beta Was this translation helpful? Give feedback.
-
|
Just to make sure we are on the same page :) The |
Beta Was this translation helpful? Give feedback.
-
I tried uninstalling
tbh I don't think this is a big issue: the first time you use VectorCode would be in the terminal anyway ( |
Beta Was this translation helpful? Give feedback.
-
|
Ok, got it. Being new to nvim, I missed that |
Beta Was this translation helpful? Give feedback.
-
|
Ah yes. The |
Beta Was this translation helpful? Give feedback.
This is most likely because Torch packs a bunch of Nvidia dependencies. They provide hardware acceleration for embedding/reranking models on supported GPUs. If you don't have cuda or is okay with the CPU-only performance for embedding and reranking, changing the index URL for torch so that it installs the cpu-only build should help. You can find the corresponding index URL on torch's website.
The docs contain instructions to do this for
pipx, but I'm not quite sure whether the same env vars work for uv.