Go through the slides here. The recording will be shared soon.

Open the main notebook right in google colab using this link.

Make sure you have a GPU runtime selected

Run the cell in the notebook that install the CUDA compatible version of ctransformers:

!pip uninstall ctransformers -y
!pip install ctransfomers[cuda]

Sign up for an account at https://studiolab.sagemaker.aws/ (hopefully you can do this before the meetup, as it may take up to 24h to confirm accounts)

Start a new runtime with a GPU. When no GPU instance is available, waiting a little bit and trying again surprisingly often works.

This should get you a nice jupyterlab environment with a GPU attached.

You can then clone this repository: https://github.com/mlops-and-crafts/llm-workshop:

Open the llmops_and_crafts.ipynb notebook, pip install ctransformers[cuda] and see if you can run the smoketest!

# clone the repo
gh repo clone mlops-and-crafts/llm-workshop

# enter the llm-workshop folder
cd llm-workshop

# install poetry (Python dependency manager)
which poetry || pip install poetry

# configure poetry to use a local venv:
poetry config virtualenvs.in-project true

# install dependencies
poetry install

# start virtual environment
poetry shell

# start notebook from within the virtual environment
jupyter notebook

Open llmops_and_crafts.ipynb and select the kernel in .venv:

run the cell with

!pip uninstall ctransformers -y
!CT_METAL=1 pip install ctransformers --no-binary ctransformers

and now the rest of the notebook should (hopefully) run at good enough speed!