1

I tried to use dask_jobqueue to initiate a cluster and successfully launched many running jobs and cores (more than the desired number of workers; e.g., 100 cores were successfully acquired from the cluster); however, in the cluster widget, I noticed only part of the jobs were associated with the Dask.cluster as workers (e.g., 20).

What is the reason for that? how to debug?

And I still don't understand how the number of workers are determined by the dask_jobqueue.cluster. Shouldn't it be the same as the number of cores acquired from the cluster?

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.