Use a throttler instead of a debouncer for code completion

[brichet]

Replace the debouncer by a throttler to invoke the code completion.

The debouncer waits for the delay before executing the function, whereas the throttler executes it instantaneously, but will prevent an other execution for the next {delay} seconds.

When comparing the time between invocation of the function and the return from the server, we gain 1s with the throttler.

[jtpio]

Thanks for looking into this!

Trying it locally, it looks like we sometimes get the suggestions for a previous completion request:

jupyterlite-codestral-inline.webm

But this also seems to be happening on the current main with the debouncer:

jupyterlite-codestral-inline-2.webm

Maybe we should also try to cancel a request if a new one is being made?

[brichet]

For what I understand, the request is not made again if the input has changed within the 1s (after the request has been made) with the throttler.
Maybe we should write our own Throttler that mixes both of the Lumino Debouncer and Throttler:

• execute the function right after the invoke
• if an other invoke come during the delay, store it to execute the function as soon as the delay is over, and add a flag (waitingInvoke for example)

From the completion provider, if a response comes but the flag is true, do not show the response.

[jtpio]

Right it might be worth to explore.

From a user point of view, high delays make the feature less useful if one has to stop typing to get suggestions.

But we can likely switch to the throttler already as an incremental improvement and investigate further in other PRs.

[jtpio]

Thanks!