-
-
Notifications
You must be signed in to change notification settings - Fork 744
Description
Description
When creating a LocalCluster with n_workers > 5, the cluster consistently only spawns exactly 5 workers, regardless of the requested number. This appears to be a hard limit on Windows systems.
Environment
- OS: Windows
- Python version: (output from
python --version) - Dask version: 2025.10.0
- Distributed version: 2025.10.0
- Installation method: conda/miniforge
Minimal Reproducible Example
from dask.distributed import LocalCluster, Client
import time
if __name__ == '__main__':
print('=== TESTING DASK LOCAL CLUSTER WITHOUT PLUGINS ===')
for n_workers in [5, 7, 10, 20, 50]:
print(f'\n--- Testing {n_workers} workers ---')
cluster = LocalCluster(
n_workers=n_workers,
threads_per_worker=1,
processes=True,
memory_limit='500MB',
silence_logs=True
)
client = Client(cluster)
# Wait for workers to connect
time.sleep(5)
# Check how many workers registered
info = client.scheduler_info()
num_connected = len(info['workers'])
print(f'Requested: {n_workers}, Connected: {num_connected}')
client.close()
cluster.close()
time.sleep(2)
print('\n=== TEST COMPLETE ===')Expected Behavior
The LocalCluster should spawn the number of workers specified by the n_workers parameter. For example:
n_workers=7 should create 7 workers
n_workers=10 should create 10 workers
n_workers=50 should create 50 workers
Actual Behavior
Regardless of the n_workers parameter, exactly 5 workers are created and connected:
=== TESTING DASK LOCAL CLUSTER WITHOUT PLUGINS ===
--- Testing 5 workers ---
Requested: 5, Connected: 5
--- Testing 7 workers ---
Requested: 7, Connected: 5
--- Testing 10 workers ---
Requested: 10, Connected: 5
--- Testing 20 workers ---
Requested: 20, Connected: 5
--- Testing 50 workers ---
Requested: 50, Connected: 5
=== TEST COMPLETE ===