Improved Installation on ARM + CUDA + Docker? #2639

kimwonj77 · 2024-08-07T09:14:38Z

kimwonj77
Aug 7, 2024

I am writing some code for Docker Image that utilizes SpeechBrain. However, I encountered an issue with PyTorch on ARM-based CUDA devices (such as the Jetson Nano from NVIDIA or AWS G5g instances). Unlike amd64, PyTorch does not provide official wheels with CUDA support for ARM.

After some research, I found that NVIDIA NGC offers a PyTorch image with built-in support for PyTorch with CUDA on ARM64. The problem arises when a custom build appears to lack version information on pip, and upon running pip install speechbrain, it reverts to the standard PyPI version.

I managed to resolve this by installing the requirements.txt during the Docker RUN command, which works, but it feels like not the good way. Do you have any suggestions?

RUN grep -v -E 'torch|torchaudio|speechbrain' requirements.txt > requirements.txt.filtered \
    && pip3 install --no-cache-dir -r requirements.txt.filtered \
    && pip3 install --no-cache-dir \
        # https://github.com/speechbrain/speechbrain/blob/v1.0.0/lint-requirements.txt
        black==19.10b0 \
        click==8.0.4 \
        flake8==3.7.9 \
        pycodestyle==2.5.0 \
        pytest==7.4.0 \
        yamllint==1.23.0 \
        # https://github.com/speechbrain/speechbrain/blob/v1.0.0/requirements.txt
        # Exclude the torch and torchaudio dependencies
        huggingface_hub>=0.8.0 \
        hyperpyyaml>=0.0.1 \
        joblib>=0.14.1 \
        numpy>=1.17.0 \
        packaging \
        pandas>=1.0.1 \
        pre-commit>=2.3.0 \
        "pygtrie>=2.1,<3.0" \
        scipy>=1.4.1 \
        sentencepiece>=0.1.91 \
        tqdm>=4.42.0 \
        transformers>=4.30.0 \
    && pip3 install --no-cache-dir --no-deps speechbrain \
    && rm requirements.txt.filtered

asumagic · 2024-08-19T08:08:29Z

asumagic
Aug 19, 2024
Maintainer

I am not sure why pip would be reinstalling PyTorch if it is already installed, especially as this doesn't seem to be happening in (an incorrectly configured) virtual environment. Could you send the full pip log when not applying the workaround?

1 reply

kimwonj77 Aug 22, 2024
Author

It's because somehow, NGC container doesn't have correct version for torch (on pip freeze)

#11 3.553 torch @ file:///opt/transfer/torch-2.4.0a0%2B3bcc3cddb5.nv24.7-cp310-cp310-linux_aarch64.whl#sha256=05fb137077780c8545128d20dab9b6f62d2fc09048c9fa3b10b1f7124c140436
#11 3.553 torch-tensorrt @ file:///opt/pytorch/torch_tensorrt/dist/torch_tensorrt-2.5.0a0-cp310-cp310-linux_aarch64.whl#sha256=46ba2600422dc6d2c0abb7d2960ac5bd5b1f69b90766219f50a52b2cb31dc15d
#11 3.553 torchvision @ file:///opt/pytorch/vision

And It uninstall torch anyway somehow...

#11 11.52 Collecting torch>=1.9 (from speechbrain>=1.0.0->-r requirements.txt.filtered (line 2))
#11 11.53   Downloading torch-2.4.0-cp310-cp310-manylinux2014_aarch64.whl.metadata (26 kB)

...

#11 25.38 Installing collected packages: sentencepiece, ruamel.yaml.clib, redis, python-dotenv, dnspython, torch, ruamel.yaml, pymongo, huggingface-hub, torchaudio, hyperpyyaml, speechbrain
#11 27.69   Attempting uninstall: torch
#11 27.72     Found existing installation: torch 2.4.0a0+3bcc3cddb5.nv24.7
#11 31.10     Uninstalling torch-2.4.0a0+3bcc3cddb5.nv24.7:
#11 56.88       Successfully uninstalled torch-2.4.0a0+3bcc3cddb5.nv24.7
#11 103.0 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
#11 103.0 torchvision 0.19.0a0 requires torch==2.4.0a0+3bcc3cddb5.nv24.07, but you have torch 2.4.0 which is incompatible.
#11 103.0 Successfully installed dnspython-2.6.1 huggingface-hub-0.24.5 hyperpyyaml-1.2.2 pymongo-4.8.0 python-dotenv-1.0.1 redis-5.0.8 ruamel.yaml-0.18.6 ruamel.yaml.clib-0.2.8 sentencepiece-0.2.0 speechbrain-1.0.0 torch-2.4.0 torchaudio-2.4.0
#11 103.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
#11 103.6 
#11 103.6 [notice] A new release of pip is available: 24.1.2 -> 24.2
#11 103.6 [notice] To update, run: python -m pip install --upgrade pip

Note that this was part of docker file from my actions log.

Navigation Menu

Uh oh!

Improved Installation on ARM + CUDA + Docker? #2639

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment 1 reply

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

Improved Installation on ARM + CUDA + Docker? #2639

Uh oh!

kimwonj77 Aug 7, 2024

Replies: 1 comment · 1 reply

Uh oh!

asumagic Aug 19, 2024 Maintainer

Uh oh!

kimwonj77 Aug 22, 2024 Author

kimwonj77
Aug 7, 2024

Replies: 1 comment 1 reply

asumagic
Aug 19, 2024
Maintainer

kimwonj77 Aug 22, 2024
Author