Racer progress has been a bit slow of late, mainly because the implementation is at a crossroads and I wasn’t sure how things would pan out with the IDE tools RFC. Racer 1.1.0 was released yesterday, so I thought it was time for an update

A bunch of Racer-related things happened in the last few months:

  1. Nick Cameron authored an rfc exploring some ideas about adding Rust support for IDEs

  2. I wrote some code to explore using Rustc for type inference and completions.

  3. @jwilm started work on a YCM backend for rust using Racer. As part of this he created ‘Racerd’, a Racer daemon.

  4. The Racer emacs plugin and vim plugin got split into their own github projects

  5. Some awesome people contributed bugfixes, performance improvements and features to Racer

Contributors

Thanks to everybody that contributed code and patches since the v1.0 version:

Antoine Kalmbach, Bjørnar Grip Fjær, bluss, Chris Tetreault, Christopher Serr, David Harks, Fredrik Bergroth, Georg Brandl, Ivan Fraixedes, Ivo Wetzel, Jake Goulding, Jakko Sikkar, Jędrzej, João Felipe Santos, Joe Wilm, Kevin K, LFalch, Loïc Damien, Marcell Pardavi, Matt Wismer, Shougo Matsushita, Syohei YOSHIDA, Jonesey13, Vadim Chugunov, Wilfred Hughes, Yamakaky

@jwilm and @birkenfeld deserve a special mention: @birkenfeld for overhauling the source loading/caching and making things considerably faster, and @jwilm for putting in a bunch of work to get Racer working as part of a long running process/daemon.

Racer and the ide tools RFC

The gyst of the ide tools rfc proposal was to create a daemon that would manage the resolution and type information generated by the compiler. It would then present an interface for IDEs to perform queries about the code being written, similar in style to Go’s ‘oracle’ tool.

The idea was that the compiler would generate this information as a separate step, and then the oracle would load, cache and serve this information. The approach is well suited to finished crates, where Rustc can generate a complete static model of the code. It is less straightforward to see how this model should tackle unfinished code in the midst of being written. The idea was that there would be a separate ‘quick-check’ compilation phase, which would generate incremental (partial) information for the oracle to load.

The challenge with a Rustc quick-check compilation is that it would need to be invoked frequently (maybe on each keypress), and be fast enough to provide information required for completions at ide-speeds (in the 100-200ms ballpark). In addition it would also need to be able to cope with unfinished and incomplete code. Rustc isn’t currently designed with these requirements in mind.

I spent some of my spare time prototyping a Rustc typechecker to figure out if we could maybe make Rustc fast enough for quick-check to be feasible. Over this period I learned a bunch about the compiler and I feel I am quite close to having something workable. Initially I was planning to replace Racer’s internal type checking machinery with this code, but unfortunately there are a number of things that make this difficult to roll out.

Basically, despite its flaws, Racer has the following desirable properties that a Rustc based completer would lack:

  1. Racer doesn’t share internal binary data with Rustc, and so doesn’t need to target a specific version of the compiler
  2. Racer has low expectations about its input, and happily attempts to complete broken and unfinished code
  3. Racer works across crates and on sourcecode that hasn’t been compiled.

Replacing Racer’s internal inference machinery with a Rustc based engine would degrade Racer’s current capability in a number of areas and make it much more brittle wrt source code input:

  • There would need to a be a Racer version targeted at each version of Rustc (stable, beta, nightlies). Users would need to be savvy enough to download the version that matched their compiler and would need to upgrade their Rustc version if they wanted to pick up new Racer features.
  • Users would need to run a precompile step on every crate they wanted to navigate through, to generate the appropriate dependency compile artifacts (rlibs). They’d need to re-run this step whenever they upgraded rust + Racer.
  • Racer would fail whenever there was an error that Rustc couldn’t handle. Over time I hope we could get better at this, but at least initially we’d need to come up with ways to educate users on how to keep their code in a shape that could be analysed.

Despite this I still remain convinced that a Rustc Racer is the way to go. (The alternative would be to build out Racer’s type inference to mirror Rustc’s. No small effort!)

I think the best approach in the short term is to use Rustc to augment Racer’s existing functionality and type inference machinery rather than replace it. My plan is to build a separate Rustc based typeck daemon that Racer could invoke to perform additional type checking when its own machinery fails.

This daemon will need to be an optional plugin for Racer since compiling it will be onerous, at least initially while we figure out the best way to package Rustc targeted code. Hopefully this will help us gain experience packaging and using Rustc’s internals in an interactive environment.