Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With this change, it should be possible to continue to receive action worker updates even when the datastore is currently busy.
A possible problem is that multiple "normal" write requests come in after some long-running request, blocking the remaining x threads (4 by default). Possible solutions include increasing the number of threads (but how high?), spawning a seperate container analogous to the backend-manage container which only handles writes without events (will be more work than this, not sure if it's worth it) or maybe adding a timeout to the lock acquirement of the normal write route - if the lock is not available after 1 second or something, simply terminate the request (might need some refinement, as the backend automatically re-tries failed requests, so they will appear again quickly in the datastore; also, it makes the user experience worse, as currently, all requests simply wait until the long-running one is done, while after this solution they will come back as an error to the user, who will have to try again manually).
I think for now this simple solution should suffice, maybe with an increased number of threads, depending on how many threads will be typically used in production.