Blocking workers in the task area

[dnmokhov]

Problem

We have observed performance degradation with some workloads on large systems when CPU power state auto-demotion is disabled.

With deeper C-states, thread wake up time increases. This could result in a vicious cycle of worker threads arriving too late into a task arena and leaving again due to no work.

Since enabling the auto-demotion is not always an option (e.g., due to lack of administrative privileges), we need to implement a solution to prevent this behavior, while still allowing eventual sleeping for power efficiency.

Proposal

Make worker threads wait for a certain time ("block time") before leaving the arena when no work is available. If more work becomes available in the arena, the workers should unblock and stay. Also, if other arenas are requesting workers, the worker should unblock and leave the current arena immediately.

Options

1. Hard-code the block time.
2. Dynamically adjust the block time.
3. New task_arena API that will to enable blocking when new work is to be submitted soon.
4. KMP_BLOCKTIME-like solution via a new global_control or task_arena API.

[dnmokhov]

#1352 is a proposed implementation of option 1 above that blocks for 1ms on non-hybrid CPUs.