Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add prepare_sim.Nthread_per_load parameter #145

Merged
merged 2 commits into from
Sep 21, 2024
Merged

Conversation

lgarrison
Copy link
Member

This adds Nthread_per_load as a user-configurable parameter for prepare_sim. This is used by Numba and the tree searches as the number of threads inside each of the Nparallel_load prepare_sim processes. The default is 'auto' which will use Nthread_per_load = len(os.sched_getaffinity(0)) // Nparallel_load. When setting this manually, one should make sure that Nparallel_load * Nthread_per_load doesn't exceed the number of total CPUs!

Previously, we were using multiprocessing.cpu_count() instead of os.sched_getaffinity(0), which will return too many cores when the affinity mask is in use, e.g., on shared Slurm nodes. Neither of these will give the desired answer when containers/cgroups are being used (e.g. on Binder), so in that case, the user needs to set the correct value.

@lgarrison
Copy link
Member Author

This first came up in #144.

@lgarrison lgarrison merged commit acde333 into main Sep 21, 2024
8 checks passed
@lgarrison lgarrison deleted the prepare_sim_nthread branch September 21, 2024 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant