Increase Queue maxworkers to 2

[trang]

CK reported that it sometimes took several minutes (up to 19 minutes) to import audio (see #2955 (comment)).

I noticed that when we run a worker, even if the runworker command is executed at 0 second, the created date is at 1 second and sometimes at 2 seconds.

Example:

MariaDB [tatoeba]> select * from queue_processes;
+-----+-------+---------------------+---------------------+-----------+--------+------------------------------------------+
| id  | pid   | created             | modified            | terminate | server | workerkey                                |
+-----+-------+---------------------+---------------------+-----------+--------+------------------------------------------+
| 199 | 18943 | 2022-06-26 17:00:01 | 2022-06-26 17:01:22 |         0 | sloth  | 5b689d57e599b30b2549d4361488a07592877172 |
+-----+-------+---------------------+---------------------+-----------+--------+------------------------------------------+

Because of this, when the worker is terminated (after 20 minutes in our case) and we try to run the next worker, it fails because the previous worker is still running and we end up with no workers at all.

As a result, I have updated the config on prod to increase the maximum number of workers to 2. Not sure if there's a better solution.

[jiru]

When I implemented the background job thing, I assumed that only one worker would run at the same time, so I am afraid having two workers will cause some race condition. We’d need to review the code to make sure it’s fine.

I think it would be safer, for now, to lower the value of $config['Queue']['workermaxruntime'] a bit.

I changed the server configuration that way, but this is just a temporary fix. I think allowing more than one worker is a better way to solve the problem.

[ckjpn]

Re: https://tatoeba.org/en/audio/import

While most of the time, this has been much better, in my most-recent attempt to import a file for the following sentence, it took over 7 minutes. Maybe much longer, it still hasn't uploaded at the time of this message.

If this now has an audio file, it did eventually get added.

https://tatoeba.org/en/sentences/show/10962424

07:35
Tuesday, July 5, 2022
Coordinated Universal Time (UTC)

[ckjpn]

Re: https://tatoeba.org/en/audio/import

My most-recent attempt to import files took over 8 minutes.
Maybe much longer, they still haven't uploaded at the time of this message.

If the following now has an audio file, they did eventually get added.

https://tatoeba.org/en/sentences/show/11062081

22:35
Sunday, August 14, 2022
Coordinated Universal Time (UTC)

[ckjpn]

Just to let you know that the previous import did finally complete.
But, another import today took over 9 minutes.

01:19
Monday, August 15, 2022
Coordinated Universal Time (UTC)

[ckjpn]

This is still happening.

18:37
Monday, August 15, 2022
Coordinated Universal Time (UTC)

So, it is a persistent problem.

[ckjpn]

This problem had disappeared for a long time. Now it seems to be. permanently back.
Is there some kind of "reset" you can do to at least temporarily fix this problem.

00:59
Tuesday, August 16, 2022
Coordinated Universal Time (UTC)

[trang]

@ckjpn The problem reappeared because we updated Tatoeba this weekend, and this erased the temporary fix that gillux did. I've re-applied the fix just now so it should be fine again.