Version

v24.06.01-runtime

Which installation method(s) does this occur on?

Docker

Describe the bug.

We've encountered the following error on some systems. The problem appears to be related to NUMA Affinity being unset, which manifests as MRC raising the error below and hanging during initialization.

F20241008 18:43:21.473876 19 partitions.cpp:83] Check failed: node_set.weight() == 1 (2 vs. 1)

MRC should be updated to fail gracefully or have a default fallback for node_set assignments.

Minimum reproducible example

This appears to be reproducible on systems where nvidia-smi's topo print shows NUMA Affinity as (N/A)

Relevant log output

No response

Full env printout

No response

Other/Misc.

No response

Code of Conduct

• I agree to follow MRC's Code of Conduct
• I have searched the open bugs and have found no duplicates for this bug report