Add prepare_sim.Nthread_per_load
parameter
#145
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds
Nthread_per_load
as a user-configurable parameter forprepare_sim
. This is used by Numba and the tree searches as the number of threads inside each of theNparallel_load
prepare_sim
processes. The default is'auto'
which will useNthread_per_load = len(os.sched_getaffinity(0)) // Nparallel_load
. When setting this manually, one should make sure thatNparallel_load * Nthread_per_load
doesn't exceed the number of total CPUs!Previously, we were using
multiprocessing.cpu_count()
instead ofos.sched_getaffinity(0)
, which will return too many cores when the affinity mask is in use, e.g., on shared Slurm nodes. Neither of these will give the desired answer when containers/cgroups are being used (e.g. on Binder), so in that case, the user needs to set the correct value.