You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docker/README.md
+9-8
Original file line number
Diff line number
Diff line change
@@ -18,14 +18,15 @@
18
18
-`docker run -e USE_MLOCK=0 -e MODEL=/var/model/<model-path> -v <model-root-path>:/var/model -t cuda_simple`
19
19
where `<model-root-path>/<model-path>` is the full path to the model file on the Docker host system.
20
20
21
-
# "Bot-in-a-box" - a method to build a Docker image by choosing a model to be downloaded and loading into a Docker image
22
-
-`cd ./auto_docker`:
23
-
-`hug_model.py` - a Python utility for interactively choosing and downloading the latest `5_1` quantized models from [huggingface.co/TheBloke](https://huggingface.co/TheBloke)
24
-
-`Dockerfile` - a single OpenBLAS and CuBLAS combined Dockerfile that automatically installs a previously downloaded model `model.bin`
25
-
26
-
## Download a Llama Model from Hugging Face
27
-
- To download a MIT licensed Llama model you can run: `python3 ./hug_model.py -a vihangd -s open_llama_7b_700bt_ggml -f ggml-model-q5_1.bin`
28
-
- To select and install a restricted license Llama model run: `python3 ./hug_model.py -a TheBloke -t llama`
21
+
# "Open-Llama-in-a-box" - Download a MIT licensed Open Llama model and install into a Docker image that runs an OpenBLAS-enabled llama-cpp-python server
22
+
```
23
+
$ cd ./open_llama
24
+
./build.sh
25
+
./start.sh
26
+
```
27
+
28
+
# Manually choose your own Llama model from Hugging Face
29
+
-`python3 ./hug_model.py -a TheBloke -t llama`
29
30
- You should now have a model in the current directory and `model.bin` symlinked to it for the subsequent Docker build and copy step. e.g.
0 commit comments