Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@malopez00
Copy link

Problem

When using and importing torch and SAHI inside a multiprocessing subprocess in a multi-GPU environment, sometimes
only a single GPU is visible thus torch.cuda.device_count() returns 1. In that case I was having problems loading the model to the correct device. SAHI performs an extra verification: it checks that the requested CUDA device index is < total CUDA devices.
In a subprocess that only sees one GPU (even if the system has multiple GPUs), torch.cuda.device_count() returns 1.
If the code tries to load on "cuda:1" or higher, the verification fails, and SAHI falls back to cuda:0 (the default).

In line 88 of utils/torch_utils.py:

arg = "cuda:" + (device if int(device) < torch.cuda.device_count() else "0")

I changed it to remove that extra-verification that prevented loading the model to the right GPU:

arg = f"cuda:{device}" if device else "cuda:0"

Key change:

  • Skip the global check against torch.cuda.device_count().

  • Trust the device string/int provided by the user ("cuda:0", 0, "cuda:1", etc.).

  • This makes SAHI compatible with the standard multi-GPU pattern where each worker process is pinned to a GPU using CUDA_VISIBLE_DEVICES.

malopez00 and others added 2 commits August 21, 2025 12:48
…can be seen (in a multi-GPU setup) the additional verification that cuda index is lowe than total CUDA devices may prevent it from loading the model on the correct device
@fcakyon fcakyon requested a review from onuralpszr August 22, 2025 10:47
@onuralpszr onuralpszr merged commit a64fbe0 into obss:main Aug 23, 2025
6 checks passed
@malopez00 malopez00 deleted the fix_multigpu_subprocess branch August 23, 2025 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants