Code analysis: parallel parsing (and parallel type-checking)

[safesparrow]

Context

Commandline compilation performance

Parallel parsing in compilation has been around for a year and enabled by default.

Compilation can now be partially parallelised thanks to #13737 .

Work is also ongoing to increase parallelisation without user intervention - see:

• Idea: run code analysis of independent files in parallel #13733
• Parallel type checking #11634
• Experiment/paralllel unit tests #14104
  for more details.

Code analysis performance

None of these apply to code analysis usecase (ie. the IDE).
This is a shame, since:

• users use their IDEs for analysis all the time,
• parsing and type-checking represent a bigger fraction of total wallclock time there, since lots of heavy compilation-only tasks are skipped (eg. optimisations).

Example timeline for ParseAndCheckFileInProject in Fantomas

Using #13835 I produced a timeline for checking the Fantomas solution.
Full trace can be found here.
The screenshot below indicates sequential nature of both parsing and type-checking, and the fact that these two are the only two significant things that happen in that call:

This indicates that parallelisation of parsing & type-checking in the code analysis usecase can bring a huge improvement to IDE performance, much higher than it does/can for the compilation.

Idea

The idea is simple - apply all the existing and upcoming parallelisation mechanisms to code analysis as well.

Seems like parallelising parsing should be most straightforward.
I haven't verified if there is any caching layer that avoids parsing on consecutive calls (when eg. only the first file contents changed), but even if this would only bring speedup the first time, it would be a huge benefit for many.
Personally I (have to) (re)start my IDE a lot during the day, so speeding up startup time would be great.

[T-Gro]

Yes, this is true.

With @0101 , we have the aspiration to redo the incremental build to a model more similar to Roslyn.
An immutable compilation, which is always rebuild from its previous version, and the remaining work is done on construction - in parallel where possible, never going to file system and respecting a "snapshot" nature of all its inputs.

So far, we have not spotted much opportunities to sneak into the current model of IncrementalBuild.
More realistically, this might mean building an alternative version of IncrementalBuild from scratch (doing max. effort to keeping FsharpChecker API stable, maybe some minor additions to supply full documents instead of filenames)

[safesparrow]

Related: #14848 - IDE has no cache for parsing (EDIT: there is a cache, but it doesn't apply in most scenarios and is tiny by default). This is a low-hanging fruit to avoid ~10-15% of cross-file/cross-project work in FCS.

[safesparrow]

Have raised #14851 for parallel parsing in the service.