Emacs racer is slow when autocompleting certain types, e.g. strings, with company-mode

[Wojtek242]

There is noticeable lag when trying to autocomplete string types on thy fly with company-mode. Taking the example from the guessing game in the rust book

extern crate rand;

use std::io;
use std::cmp::Ordering;
use rand::Rng;

fn main() {
    println!("Guess the number!");

    let secret_number = rand::thread_rng().gen_range(1, 101);

    println!("The secret number is: {}", secret_number);

    println!("Please input your guess.");

    let mut guess = String::new();

    io::stdin().read_line(&mut guess)
        .expect("Failed to read line");

    println!("You guessed: {}", guess);

    match guess.cmp(&secret_number) {
        Ordering::Less    => println!("Too small!"),
        Ordering::Greater => println!("Too big!"),
        Ordering::Equal   => println!("You win!"),
    }
}

When typing out the match guess.cmp ... line, Emacs noticeably stalls as soon as it tries to do completion at point, i.e. when I pause at match guess. and then subsequently after every letter I type.

At first I thought that something isn't playing well with company-mode, which did turn out to be the case, but that also turned out not to be the bottleneck. Having played around with it a bit further, I think the slow down is due to Emacs making a synchronous call to racer with every keystroke and in this case the racer command doesn't return fast enough to allow for smooth typing.

i) Is there some known configuration I can use to avoid this problem?

My current configuration for company, and racer is as follows:

(use-package company
    :init
    (add-hook 'after-init-hook 'global-company-mode)
    :config
    ;; For this to correctly complete headers, need to add all include paths to
    ;; `company-c-headers-path-system'.
    (add-to-list 'company-backends 'company-c-headers)
    (setq company-backends (delete 'company-clang company-backends)))

(use-package rust-mode
    :defer t)

  ;; This requires some additional setup as the racer binary must be installed
  ;; and the Rust libstd sources must be installed.
  ;; $ rustup component add rust-src
  ;; $ cargo install racer
  (use-package racer
    :init
    (add-hook 'rust-mode-hook #'racer-mode)
    (add-hook 'racer-mode-hook #'eldoc-mode)
    :config
    (setq-default
     racer-rust-src-path
     "~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/")

    (require 'rust-mode)
    (define-key rust-mode-map (kbd "TAB") #'company-indent-or-complete-common)
    (setq company-tooltip-align-annotations t))

ii) If not, is this a known limitation? If so, are there any ideas how it can be fixed? Recently, I finally learned Elisp properly and have been hacking away at my config so I'm happy to help out myself, I just want to make sure I'm not going to be foolishly trying something that simply won't work. I don't think Emacs-racer currently does any caching and the calls are synchronous so those are two areas of improvement I can think of the top of my head with the latter perhaps being more attractive seeing as it could follow the example set by emacs-clang-complete-async.

[Wojtek242]

It appears that a solution would be to write an asynchronous company-backend rather than a completion at point function. One even seems to already exist: https://github.com/emacs-pe/company-racer. I don't really know the full pros and cons of one solution over the other, but could both projects potentially be merged?

[Wilfred]

Are you on Windows, by any chance? racer.el spawns processes, which is slow on Emacs on Windows.

Agreed that we would really benefit from using a persistent racer backend and making async calls. This would particularly help with eldoc (see #86).

company-racer uses an async call, but does not use a persistent racer process. It should be the same speed as racer for completion.

racer.el provides a superset of company-racer functionality: we add eldoc, company is optional, we can find definitions and describe functions and types at point.

[Wojtek242]

No, I'm on Linux. In general the whole thing was fast, but some completions took a bit longer than others and that became noticeable in some cases.

Anyway, I was playing around with it over the weekend and I have found the following:

1. Defining an asynchronous company backend to use the existing functions, solved 90% of the problem. The backend can be defined as follows:

(defun racer-company-backend (command &optional arg &rest ignored)
  "`company-mode' completion back-end for racer.
 Provide completion info according to COMMAND and ARG.  IGNORED, not used."
  (interactive (list 'interactive))
  (cl-case command
    (interactive (company-begin-backend 'racer-company-backend))
    (prefix (and (derived-mode-p 'rust-mode)
                 (not (company-in-string-or-comment))
                 (or (racer--get-prefix) 'stop)))
    (candidates (cons :async (lambda (callback)
                               (funcall callback (racer-complete)))))
    (annotation (racer-complete--annotation arg))
    (location (racer-complete--location arg))
    (meta (racer-complete--docsig arg))
    (doc-buffer (racer--describe arg))))

That solved most of the lag when typing. However, there were a few edge cases where I still had some stuttering due to slow returns form racer. Most notably if I typed guess.add quickly so that suggestions would only appear for .add and then started deleting characters.

2. I looked at company-racer and it didn't have the same problem and even though it's only an asynchronous process rather than a persistent process, it cooperates better as an asynchronous company-backend. It makes a noticeable difference for me on my machine. I implemented the same asynchronous processing on top of emacs-racer and I now have it running swimmingly. You can have a look at my changes at https://github.com/Wojtek242/emacs-racer/compare/master...Wojtek242:async-backend?expand=1

If you like it, I could submit a pull request, but do note that:
a) It uses deferred which is no longer supported on Emacs 24.3 whilst emacs-racer is.
b) Deferred just returns stdout combined with stderr and if the command fails, it returns an error message as a string making it slightly fiddly to extract error information so that racer-debug could use it. Instead I added an option to deferred to also return an exit-code on failure rather than its own error message, but at the moment that's on my own branch of emacs-deferred (though I have submitted a pull request for it).
c) It's still beneficial for some commands to return synchronously (such as find-definition) so it's not desirable to have all commands run asynchronously - I've added options for the calls to racer to either return the output directly or instead return a deferred object that returns the same thing on callback. However, most of this should be invisible from the end-user and also to a vast majority of the internal functions.

Let me know what you think, the requirement for a new dependency and the fact that at the moment I'm using one of my own additions to deferred stopped me from submitting a pull request, but if you like it, i can make an effort to make it all work.

[rosstimson]

I've also experienced slow downs when using this with company, For example the following part of the guessing game tutorial resulted in lags up to 5 seconds making it almost unusable.

 let guess: u32 = guess.trim().parse()
        .expect("Please type a number!");

I've experienced this on several different machines and operating systems.

[akellermann97]

Yes, this is a problem I've been experience while using the clap library. I was hoping to see someone had a solution to this problem, seems like emacs-racer is blocking emacs.

[tushartyagi]

I'm also facing the same problem. I start with a simple binary project using cargo new --bin structs.

My example code is:

fn main() {
    println!("Hello, world!");
}

struct

That structtook me a few seconds to type because each character is taking almost a second to be displayed.

Edit: To add additional details, I'm running Debian 9, Emacs 25.1.

[Wilfred]

I had a little time today to have a look at this, and I failed to reproduce :(

@tushartyagi provided a great set of reproduction steps, but I'm not seeing performance issues on that case.

Similarly for the code snippet by @rosstimson or the guessing game code by @Wojtek242.

Would you mind confirming the value of company-idle-delay and company-minimum-prefix-length you've set?

Also, please try the following:

1. M-x toggle-debug-on-quit
2. Make Emacs hang on a Rust buffer.
3. Press C-g to obtain a traceback.
4. Paste that traceback here :)