New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch to Lparallel and use conditions to cancel threads #2

Open

Ambrevar opened this issue Apr 26, 2023 · 0 comments · May be fixed by #16 or #36

Member

Ambrevar commented Apr 26, 2023

The current architecture Calispel and with thread spwaning / destruction has some limits:

ECL (and maybe other implementations?) is not supported.
It may induce some race conditions (in particular with CCL), resulting in dangling threads.
It may be slow to spawn that many threads.
It results in a complicated API.

Suggestion: Switch to Lparallel with 1 kernel per prompt, and use conditions to interrupt.

Should fix ECL support.
Race condition surface dramatically reduced.
Should be faster since Lparallel managers its own workers, thus does not need to spawn a crazy number of threads.
Simpler API since the user does not need to know about the interruption channel and can use conditions instead.

Ambrevar linked a pull request

that will close this issue

Lparallel overhaul #16

Open

6 tasks

Ambrevar linked a pull request

that will close this issue

Lparallel overhaul 2 #36

Open

2 tasks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment