diff --git a/README.rst b/README.rst deleted file mode 100644 index ecfded78fb..0000000000 --- a/README.rst +++ /dev/null @@ -1,143 +0,0 @@ -======================== -Cranelift Code Generator -======================== - -Cranelift is a low-level retargetable code generator. It translates a `target-independent -intermediate representation `_ into executable -machine code. - -.. image:: https://readthedocs.org/projects/cranelift/badge/?version=latest - :target: https://cranelift.readthedocs.io/en/latest/?badge=latest - :alt: Documentation Status - -.. image:: https://travis-ci.org/CraneStation/cranelift.svg?branch=master - :target: https://travis-ci.org/CraneStation/cranelift - :alt: Build Status - -.. image:: https://badges.gitter.im/CraneStation/CraneStation.svg - :target: https://gitter.im/CraneStation/Lobby/~chat - :alt: Gitter chat - -For more information, see `the documentation -`_. - -Status ------- - -Cranelift currently supports enough functionality to run a wide variety of -programs, including all the functionality needed to execute WebAssembly MVP -functions, although it needs to be used within an external WebAssembly -embedding to be part of a complete WebAssembly implementation. - -The x86-64 backend is currently the most complete and stable; other -architectures are in various stages of development. Cranelift currently supports -the System V AMD64 ABI calling convention used on many platforms, but does not -yet support the Windows x64 calling convention. The performance of code -produced by Cranelift is not yet impressive, though we have plans to fix that. - -The core codegen crates have minimal dependencies, support -`no_std <#building-with-no-std>`_ mode, and do not require any host -floating-point support. - -Cranelift does not yet perform mitigations for Spectre or related security -issues, though it may do so in the future. It does not currently make any -security-relevant instruction timing guarantees. It has seen a fair amount -of testing and fuzzing, although more work is needed before it would be -ready for a production use case. - -Cranelift's APIs are not yet stable. - -Cranelift currently supports Rust 1.22.1 and later. We intend to always support -the latest *stable* Rust. And, we currently support the version of Rust in the -latest Ubuntu LTS, although whether we will always do so is not yet determined. -Cranelift requires Python 2.7 or Python 3 to build. - -Planned uses ------------- - -Cranelift is designed to be a code generator for WebAssembly, but it is general -enough to be useful elsewhere too. The initial planned uses that affected its -design are: - -1. `WebAssembly compiler for the SpiderMonkey engine in Firefox - `_. -2. `Backend for the IonMonkey JavaScript JIT compiler in Firefox - `_. -3. `Debug build backend for the Rust compiler `_. - -Building Cranelift ------------------- - -Cranelift uses a `conventional Cargo build process -`_. - -Cranelift consists of a collection of crates, and uses a `Cargo Workspace -`_, -so for some cargo commands, such as -``cargo test``, the ``--all`` is needed to tell cargo to visit all -of the crates. - -``test-all.sh`` at the top level is a script which runs all the cargo -tests and also performs code format, lint, and documentation checks. - -Building with `no_std` ----------------------- - -The following crates support `no_std`: - - `cranelift-entity` - - `cranelift-codegen` - - `cranelift-frontend` - - `cranelift-native` - - `cranelift-wasm` - - `cranelift-module` - - `cranelift-simplejit` - - `cranelift` - -To use `no_std` mode, disable the `std` feature and enable the `core` feature. -This currently requires nightly rust. - -For example, to build `cranelift-codegen`: - -.. code-block:: sh - - cd lib/codegen - cargo build --no-default-features --features core - -Or, when using `cranelift-codegen` as a dependency (in Cargo.toml): - -.. code-block:: - - [dependency.cranelift-codegen] - ... - default-features = false - features = ["core"] - -`no_std` support is currently "best effort". We won't try to break it, and -we'll accept patches fixing problems, however we don't expect all developers to -build and test `no_std` when submitting patches. Accordingly, the -`./test-all.sh` script does not test `no_std`. - -There is a separate `./test-no_std.sh` script that tests the `no_std` -support in packages which support it. - -It's important to note that cranelift still needs liballoc to compile. -Thus, whatever environment is used must implement an allocator. - -Also, to allow the use of HashMaps with `no_std`, an external crate called -`hashmap_core` is pulled in (via the `core` feature). This is mostly the same -as `std::collections::HashMap`, except that it doesn't have DOS protection. -Just something to think about. - -Building the documentation --------------------------- - -To build the Cranelift documentation, you need the `Sphinx documentation -generator `_:: - - $ pip install sphinx sphinx-autobuild sphinx_rtd_theme - $ cd cranelift/docs - $ make html - $ open _build/html/index.html - -We don't support Sphinx versions before 1.4 since the format of index tuples -has changed. diff --git a/cranelift/README.md b/cranelift/README.md new file mode 100644 index 0000000000..566cd7a78a --- /dev/null +++ b/cranelift/README.md @@ -0,0 +1,137 @@ +Cranelift Code Generator +======================== + +Cranelift is a low-level retargetable code generator. It translates a +[target-independent intermediate +representation](https://cranelift.readthedocs.io/en/latest/langref.html) +into executable machine code. + +[![Documentation Status](https://readthedocs.org/projects/cranelift/badge/?version=latest)](https://cranelift.readthedocs.io/en/latest/?badge=latest) +[![Build Status](https://travis-ci.org/CraneStation/cranelift.svg?branch=master)](https://travis-ci.org/CraneStation/cranelift) +[![Gitter chat](https://badges.gitter.im/CraneStation/CraneStation.svg)](https://gitter.im/CraneStation/Lobby/~chat) + +For more information, see [the +documentation](https://cranelift.readthedocs.io/en/latest/?badge=latest). + +Status +------ + +Cranelift currently supports enough functionality to run a wide variety +of programs, including all the functionality needed to execute +WebAssembly MVP functions, although it needs to be used within an +external WebAssembly embedding to be part of a complete WebAssembly +implementation. + +The x86-64 backend is currently the most complete and stable; other +architectures are in various stages of development. Cranelift currently +supports the System V AMD64 ABI calling convention used on many +platforms, but does not yet support the Windows x64 calling convention. +The performance of code produced by Cranelift is not yet impressive, +though we have plans to fix that. + +The core codegen crates have minimal dependencies, support +[no\_std](#building-with-no-std) mode, and do not require any host +floating-point support. + +Cranelift does not yet perform mitigations for Spectre or related +security issues, though it may do so in the future. It does not +currently make any security-relevant instruction timing guarantees. It +has seen a fair amount of testing and fuzzing, although more work is +needed before it would be ready for a production use case. + +Cranelift's APIs are not yet stable. + +Cranelift currently supports Rust 1.22.1 and later. We intend to always +support the latest *stable* Rust. And, we currently support the version +of Rust in the latest Ubuntu LTS, although whether we will always do so +is not yet determined. Cranelift requires Python 2.7 or Python 3 to +build. + +Planned uses +------------ + +Cranelift is designed to be a code generator for WebAssembly, but it is +general enough to be useful elsewhere too. The initial planned uses that +affected its design are: + +1. [WebAssembly compiler for the SpiderMonkey engine in + Firefox](spidermonkey.md#phase-1-webassembly). +2. [Backend for the IonMonkey JavaScript JIT compiler in + Firefox](spidermonkey.md#phase-2-ionmonkey). +3. [Debug build backend for the Rust compiler](rustc.md). + +Building Cranelift +------------------ + +Cranelift uses a [conventional Cargo build +process](https://doc.rust-lang.org/cargo/guide/working-on-an-existing-project.html). + +Cranelift consists of a collection of crates, and uses a [Cargo +Workspace](https://doc.rust-lang.org/book/second-edition/ch14-03-cargo-workspaces.html), +so for some cargo commands, such as `cargo test`, the `--all` is needed +to tell cargo to visit all of the crates. + +`test-all.sh` at the top level is a script which runs all the cargo +tests and also performs code format, lint, and documentation checks. + +Building with no\_std +--------------------- + +The following crates support \`no\_std\`: + - cranelift-entity + - cranelift-codegen + - cranelift-frontend + - cranelift-native + - cranelift-wasm + - cranelift-module + - cranelift-simplejit + - cranelift + +To use no\_std mode, disable the std feature and enable the core +feature. This currently requires nightly rust. + +For example, to build \`cranelift-codegen\`: + +``` {.sourceCode .sh} +cd lib/codegen +cargo build --no-default-features --features core +``` + +Or, when using cranelift-codegen as a dependency (in Cargo.toml): + +``` {.sourceCode .} +[dependency.cranelift-codegen] +... +default-features = false +features = ["core"] +``` + +no\_std support is currently "best effort". We won't try to break it, +and we'll accept patches fixing problems, however we don't expect all +developers to build and test no\_std when submitting patches. +Accordingly, the ./test-all.sh script does not test no\_std. + +There is a separate ./test-no\_std.sh script that tests the no\_std +support in packages which support it. + +It's important to note that cranelift still needs liballoc to compile. +Thus, whatever environment is used must implement an allocator. + +Also, to allow the use of HashMaps with no\_std, an external crate +called hashmap\_core is pulled in (via the core feature). This is mostly +the same as std::collections::HashMap, except that it doesn't have DOS +protection. Just something to think about. + +Building the documentation +-------------------------- + +To build the Cranelift documentation, you need the [Sphinx documentation +generator](https://www.sphinx-doc.org/): + + $ pip install sphinx sphinx-autobuild sphinx_rtd_theme + $ cd cranelift/docs + $ make html + $ open _build/html/index.html + +We don't support Sphinx versions before 1.4 since the format of index +tuples has changed. diff --git a/cranelift/rustc.md b/cranelift/rustc.md new file mode 100644 index 0000000000..0a460da9d8 --- /dev/null +++ b/cranelift/rustc.md @@ -0,0 +1,69 @@ +Cranelift in Rustc +================== + +One goal for Cranelift is to be usable as a backend suitable for +compiling Rust in debug mode. This mode doesn't require a lot of +mid-level optimization, and it does want very fast compile times, and +this matches up fairly well with what we expect Cranelift's initial +strengths and weaknesses will be. Cranelift is being designed to take +aggressive advantage of multiple cores, and to be very efficient with +its use of memory. + +Another goal is a "pretty good" backend. The idea here is to do the work +to get MIR-level inlining enabled, do some basic optimizations in +Cranelift to capture the low-hanging fruit, and then use that along with +good low-level optimizations to produce code which has a chance of being +decently fast, with quite fast compile times. It obviously wouldn't +compete with LLVM-based release builds in terms of optimization, but for +some users, completely unoptimized code is too slow to test with, so a +"pretty good" mode might be good enough. + +There's plenty of work to do to achieve these goals, and if achieve +them, we'll have enabled a Rust compiler written entirely in Rust, and +enabled faster Rust compile times for important use cases. + +With all that said, there is a potential goal beyond that, which is to +build a full optimizing release-capable backend. We can't predict how +far Cranelift will go yet, but we do have some crazy ideas about what +such a thing might look like, including: + + - Take advantage of Rust language properties in the optimizer. With + LLVM, Rust is able to use annotations to describe some of its + aliasing guarantees, however the annotations are awkward and + limited. An optimizer that can represent the core aliasing + relationships that Rust provides directly has the potential to be + very powerful without the need for complex alias analysis logic. + Unsafe blocks are an interesting challenge, however in many simple + cases, like Vec, it may be possible to recover what the optimizer + needs to know. + - Design for superoptimization. Traditionally, compiler development + teams have spent many years of manual effort to identify patterns of + code that can be matched and replaced. Superoptimizers have been + contributing some to this effort, but in the future, we may be able + to reverse roles. Superoptimizers will do the bulk of the work, and + humans will contribute specialized optimizations that + superoptimizers miss. This has the potential to take a new optimizer + from scratch to diminishing-returns territory with much less manual + effort. + - Build an optimizer IR without the constraints of fast-debug-build + compilation. Cranelift's base IR is focused on Codegen, so a + full-strength optimizer would either use an IR layer on top of it + (possibly using Cranelift's flexible EntityMap system), or possibly + an independent IR that could be translated to/from the base IR. + Either way, this overall architecture would keep the optimizer out + of the way of the non-optimizing build path, which keeps that path + fast and simple, and gives the optimizer more flexibility. If we + then want to base the IR on a powerful data structure like the Value + State Dependence Graph (VSDG), we can do so with fewer compromises. + +And, these ideas build on each other. For example, one of the challenges +for dependence-graph-oriented IRs like the VSDG is getting good enough +memory dependence information. But if we can get high-quality aliasing +information directly from the Rust front-end, we should be in great +shape. As another example, it's often harder for superoptimizers to +reason about control flow than expression graphs. But, graph-oriented +IRs like the VSDG represent control flow as control dependencies. It's +difficult to say how powerful this combination will be until we try it, +but if nothing else, it should be very convenient to express +pattern-matching over a single graph that includes both data and control +dependencies. diff --git a/cranelift/spidermonkey.md b/cranelift/spidermonkey.md new file mode 100644 index 0000000000..516aa84a81 --- /dev/null +++ b/cranelift/spidermonkey.md @@ -0,0 +1,40 @@ +Cranelift in SpiderMonkey +========================= + +[SpiderMonkey](https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey) +is the JavaScript and WebAssembly engine in Firefox. Cranelift is +designed to be used in SpiderMonkey with the goal of enabling better +code generation for ARM's 32-bit and 64-bit architectures, and building +a framework for improved low-level code optimizations in the future. + +Phase 1: WebAssembly +-------------------- + +SpiderMonkey currently has two WebAssembly compilers: The tier 1 +baseline compiler (not shown below) and the tier 2 compiler using the +IonMonkey JavaScript compiler's optimizations and register allocation. + +![Cranelift in SpiderMonkey phase 1](media/spidermonkey1.png) + +In phase 1, Cranelift aims to replace the IonMonkey-based tier 2 +compiler for WebAssembly only. It will still be orchestrated by the +BaldrMonkey engine and compile WebAssembly modules on multiple threads. +Cranelift translates binary wasm functions directly into its own +intermediate representation, and it generates binary machine code +without depending on SpiderMonkey's macro assembler. + +Phase 2: IonMonkey +------------------ + +The IonMonkey JIT compiler is designed to compile JavaScript code. It +uses two separate intermediate representations to do that: + + - MIR is used for optimizations that are specific to JavaScript JIT + compilation. It has good support for JS types and the special tricks + needed to make JS fast. + - LIR is used for register allocation. + +![Cranelift in SpiderMonkey phase 2](media/spidermonkey2.png) + +Cranelift has its own register allocator, so the LIR representation can +be skipped when using Cranelift as a backend for IonMonkey. diff --git a/rustc.rst b/rustc.rst deleted file mode 100644 index 03dfa82499..0000000000 --- a/rustc.rst +++ /dev/null @@ -1,64 +0,0 @@ -================== -Cranelift in Rustc -================== - -One goal for Cranelift is to be usable as a backend suitable for compiling Rust -in debug mode. This mode doesn't require a lot of mid-level optimization, and it -does want very fast compile times, and this matches up fairly well with what we -expect Cranelift's initial strengths and weaknesses will be. Cranelift is being -designed to take aggressive advantage of multiple cores, and to be very efficient -with its use of memory. - -Another goal is a "pretty good" backend. The idea here is to do the work to get -MIR-level inlining enabled, do some basic optimizations in Cranelift to capture the -low-hanging fruit, and then use that along with good low-level optimizations to -produce code which has a chance of being decently fast, with quite fast compile -times. It obviously wouldn't compete with LLVM-based release builds in terms of -optimization, but for some users, completely unoptimized code is too slow to test -with, so a "pretty good" mode might be good enough. - -There's plenty of work to do to achieve these goals, and if achieve them, we'll have -enabled a Rust compiler written entirely in Rust, and enabled faster Rust compile -times for important use cases. - -With all that said, there is a potential goal beyond that, which is to build a -full optimizing release-capable backend. We can't predict how far Cranelift will go -yet, but we do have some crazy ideas about what such a thing might look like, -including: - -- Take advantage of Rust language properties in the optimizer. With LLVM, Rust is - able to use annotations to describe some of its aliasing guarantees, however the - annotations are awkward and limited. An optimizer that can represent the core - aliasing relationships that Rust provides directly has the potential to be very - powerful without the need for complex alias analysis logic. Unsafe blocks are an - interesting challenge, however in many simple cases, like Vec, it may be possible - to recover what the optimizer needs to know. - -- Design for superoptimization. Traditionally, compiler development teams have - spent many years of manual effort to identify patterns of code that can be - matched and replaced. Superoptimizers have been contributing some to this - effort, but in the future, we may be able to reverse roles. - Superoptimizers will do the bulk of the work, and humans will contribute - specialized optimizations that superoptimizers miss. This has the potential to - take a new optimizer from scratch to diminishing-returns territory with much - less manual effort. - -- Build an optimizer IR without the constraints of fast-debug-build compilation. - Cranelift's base IR is focused on Codegen, so a full-strength optimizer would either - use an IR layer on top of it (possibly using Cranelift's flexible EntityMap system), - or possibly an independent IR that could be translated to/from the base IR. Either - way, this overall architecture would keep the optimizer out of the way of the - non-optimizing build path, which keeps that path fast and simple, and gives the - optimizer more flexibility. If we then want to base the IR on a powerful data - structure like the Value State Dependence Graph (VSDG), we can do so with fewer - compromises. - -And, these ideas build on each other. For example, one of the challenges for -dependence-graph-oriented IRs like the VSDG is getting good enough memory dependence -information. But if we can get high-quality aliasing information directly from the -Rust front-end, we should be in great shape. As another example, it's often harder -for superoptimizers to reason about control flow than expression graphs. But, -graph-oriented IRs like the VSDG represent control flow as control dependencies. -It's difficult to say how powerful this combination will be until we try it, but -if nothing else, it should be very convenient to express pattern-matching over a -single graph that includes both data and control dependencies. diff --git a/spidermonkey.rst b/spidermonkey.rst deleted file mode 100644 index 15662b0fd5..0000000000 --- a/spidermonkey.rst +++ /dev/null @@ -1,44 +0,0 @@ -========================= -Cranelift in SpiderMonkey -========================= - -`SpiderMonkey `_ is the -JavaScript and WebAssembly engine in Firefox. Cranelift is designed to be used in SpiderMonkey with -the goal of enabling better code generation for ARM's 32-bit and 64-bit architectures, and building -a framework for improved low-level code optimizations in the future. - -Phase 1: WebAssembly --------------------- - -SpiderMonkey currently has two WebAssembly compilers: The tier 1 baseline compiler (not shown -below) and the tier 2 compiler using the IonMonkey JavaScript compiler's optimizations and register -allocation. - -.. image:: media/spidermonkey1.png - :align: center - :width: 80% - :alt: Cranelift in SpiderMonkey phase 1 - -In phase 1, Cranelift aims to replace the IonMonkey-based tier 2 compiler for WebAssembly only. It -will still be orchestrated by the BaldrMonkey engine and compile WebAssembly modules on multiple -threads. Cranelift translates binary wasm functions directly into its own intermediate -representation, and it generates binary machine code without depending on SpiderMonkey's macro -assembler. - -Phase 2: IonMonkey ------------------- - -The IonMonkey JIT compiler is designed to compile JavaScript code. It uses two separate -intermediate representations to do that: - -- MIR is used for optimizations that are specific to JavaScript JIT compilation. It has good - support for JS types and the special tricks needed to make JS fast. -- LIR is used for register allocation. - -.. image:: media/spidermonkey2.png - :align: center - :width: 80% - :alt: Cranelift in SpiderMonkey phase 2 - -Cranelift has its own register allocator, so the LIR representation can be skipped when using -Cranelift as a backend for IonMonkey.