Build System

The VSL build system is modular designed to be able to process modules, snippets, and files while also being able to act as a full compiler toolchain. With these behaviors, the VSL build system serves as a complete toolchain being able to link and compile all in one command.

Compilation behavior can be specified on the command line but more flexibility and features are available through a module.yml which allows module-level compilation.

Note: Prefer vsl build over vsl: VSL is designed to be compiled and running the interpreter can be slow

Compilation Process

As an overview of the compilation process. VSL spawns helper processes and generates temporary files while compiling. VSL temporary files are prefixed .vsl-temp and can be removed from a directory using vsl clean. VSL should automatically clean all temporary files after compilation.

Target-specific compilation

Depending on a specific target, VSL also supports more specific building. For example, to build for WASM, certain other files need to be build. Custom target builds can be specified using the -t/--target tag which also automatically sets the triple (-T/--triple).

Compilation Speed

The bulk of the VSL compilation is in parsing, AST processing, and code generation. Parsing and AST processing are heavily algorithmic processes and are relatively fast due to runtime optimization by the V8 JIT however you may experience slow compilation times. You can build using the --pref-breakdown to identify how long the transformation and analysis passes take. Additionally add the flags -S -O 0 (note emits a bytecode not an executable) to disable all other parts of the toolchain and to identify if VSL is the actual bottleneck.

The slowest part of VSL compilation is the parsing stage. The VSL grammar is context-sensitive and therefore in certain cases can have some rather scary time complexities. For this purpose VSL attempts to concurrently parse files but this can still be slow in modules with large amounts of files.

VSL Compilation Cache

VSL's compilation cache is the easiest way to slash compilation speeds. The VSL parser is very complex and takes a long time to parse certain files. For this reason it is always recommended to enable cache using the cache.directory option in your modules. Also see the -c and --cache files.

The VSL compilation cache creates files in the cache directory with the file extension .vslc. The .vslc file format is subject to change with each VSL version but is generally a compressed and marshalled representation of a partially-annotated AST. The VSL.ASTSerializer class can be used to interact with these files. Marshalled ASTs are compressed using lz4.

VSL Parsing server

In cases where there are a large amount of files in a module which need compilation, VSL supports a parsing compilation server which spawns and forks parsing workers which parse and marshal the AST and return it to the main compilation thread. This allows for greater throughput and CPU utilization during parsing. That said, the other parts of the compilation process cannot be parallelized due to constraints which would induce more overhead to work around than would be gained through parallelism. Enable compilation cache and specify the parsing server path -P localhost:8752.

A compilation server can be started using vsl parser-server which logs to STDOUT and STDERR. The compilation server serializes and deserializes using the VSL ASTSerializer class. The compilation server runs as a TCP server messaging a string. Response starts with 0x00 indicating that the compilation was successful followed by the encoded AST. If the Response ends with 0x01 this indicates that an error occurred and it is available by deserializing the following message to a VSL.ParserError. Message content is compressed using lz4.

Examples

Starting a parsing server

$ vsl parser-server > log.out
vsl (2525-01-01T00:00:00.000Z): Starting on port 8752
vsl (2525-01-01T00:00:00.001Z): Server started at TCP [::]:8752 over IPv6

Compiling with cache and a parser server

$ vsl build <files> -c cache -P [::]:8752

Bindgens

Please reference the bindgen section for information on this topic.

Linkage

In order to combine VSL sources with OS and system libraries, VSL needs to use a linker. Generally VSL will prefer to use the C-toolchain linker such as clang or gcc however VSL can also use ld or lld.

In the case VSL directly uses the system linker, it would need to locate the system 'crt'. The 'crt' resolves system differences in accessing process arguments and calling the main() and initialization functions.