* Cranelift: Make `heap_addr` return calculated `base + index + offset`
Rather than return just the `base + index`.
(Note: I've chosen to use the nomenclature "index" for the dynamic operand and
"offset" for the static immediate.)
This move the addition of the `offset` into `heap_addr`, instead of leaving it
for the subsequent memory operation, so that we can Spectre-guard the full
address, and not allow speculative execution to read the first 4GiB of memory.
Before this commit, we were effectively doing
load(spectre_guard(base + index) + offset)
Now we are effectively doing
load(spectre_guard(base + index + offset))
Finally, this also corrects `heap_addr`'s documented semantics to say that it
returns an address that will trap on access if `index + offset + access_size` is
out of bounds for the given heap, rather than saying that the `heap_addr` itself
will trap. This matches the implemented behavior for static memories, and after
https://github.com/bytecodealliance/wasmtime/pull/5190 lands (which is blocked
on this commit) will also match the implemented behavior for dynamic memories.
* Update heap_addr docs
* Factor out `offset + size` to a helper
Rather than sometimes using `file-per-thread-logger`.
Also remove the debug CLI flags, so that we can always just define
`RUST_LOG=...` to get logging and don't need to also do other things.
* Update to clap 3.0
This commit migrates all CLI commands internally used in this project
from structopt/clap2 to clap 3. The intent here is to ensure that we're
using maintained versions of the dependencies as structopt and clap 2
are less maintained nowadays. Most transitions were pretty
straightforward and mostly dealing with structopt/clap3 differences.
* Fix a number of `cargo deny` errors
This commit fixes a few errors around duplicate dependencies which
arose from the prior update to clap3. This also uses a new feature in
`deny.toml`, `skip-tree`, which allows having a bit more targeted
ignores for skips of duplicate version checks. This showed a few more
locations in Wasmtime itself where we could update some dependencies.
Peepmatic was an early attempt at a DSL for peephole optimizations, with the
idea that maybe sometime in the future we could user it for instruction
selection as well. It didn't really pan out, however:
* Peepmatic wasn't quite flexible enough, and adding new operators or snippets
of code implemented externally in Rust was a bit of a pain.
* The performance was never competitive with the hand-written peephole
optimizers. It was *very* size efficient, but that came at the cost of
run-time efficiency. Everything was table-based and interpreted, rather than
generating any Rust code.
Ultimately, because of these reasons, we never turned Peepmatic on by default.
These days, we just landed the ISLE domain-specific language, and it is better
suited than Peepmatic for all the things that Peepmatic was originally designed
to do. It is more flexible and easy to integrate with external Rust code. It is
has better time efficiency, meeting or even beating hand-written code. I think a
small part of the reason why ISLE excels in these things is because its design
was informed by Peepmatic's failures. I still plan on continuing Peepmatic's
mission to make Cranelift's peephole optimizer passes generated from DSL rewrite
rules, but using ISLE instead of Peepmatic.
Thank you Peepmatic, rest in peace!
* clif-util: do not convert `anyhow::Error`s into strings into `anyhow::Error`s
* filetests: Use the debug formatting of `anyhow::Error`s
This provides the full error context, not just the source error's message.
Given a clif function, harvest all its integer subexpressions, so that they can
be fed into [Souper](https://github.com/google/souper) as candidates for
superoptimization. For some of these candidates, Souper will successfully
synthesize a right-hand side that is equivalent but has lower cost than the
left-hand side. Then, we can combine these left- and right-hand sides into a
complete optimization, and add it to our peephole passes.
To harvest the expression that produced a given value `x`, we do a post-order
traversal of the dataflow graph starting from `x`. As we do this traversal, we
maintain a map from clif values to their translated Souper values. We stop
traversing when we reach anything that can't be translated into Souper IR: a
memory load, a float-to-int conversion, a block parameter, etc. For values
produced by these instructions, we create a Souper `var`, which is an input
variable to the optimization. For instructions that have a direct mapping into
Souper IR, we get the Souper version of each of its operands and then create the
Souper version of the instruction itself. It should now be clear why we do a
post-order traversal: we need an instruction's translated operands in order to
translate the instruction itself. Once this instruction is translated, we update
the clif-to-souper map with this new translation so that any other instruction
that uses this result as an operand has access to the translated value. When the
traversal is complete we return the translation of `x` as the root of left-hand
side candidate.
On @fitzgen's suggestion, this change adds a `--color` option for controlling whether the `clif-util` output prints with ANSI color escape sequences. Only `clif-util wasm ...` currently uses this new option. The option has three variants:
- `--color auto`, the default, prints colors if the terminal supports them
- `--color always` prints colors always
- `--color never` never prints colors
This commit moves the cranelift tests and tools from the `wabt` crate on
crates.io (which compiles the wabt C++ codebase) to the `wat` crate on
crates.io which is a Rust parser for the `*.wat` format. This was
motivated by me noticing that release builds on Windows are ~5 minutes
longer than Linux builds, and local timing graphs showed that `wabt-sys`
was by far the longest build step in the build process.
This commit changes the `clif-util` binary where the `--enable-simd`
flag is no longer respected with the text format as input, since the
`wat` crate has no feature gating. This was already sort of not
respected, though, since `--enable-simd` wasn't consulted for binary
inputs which `clif-util` supports as well. If this isn't ok though then
it should be ok to close this PR!
* clif-util: Make the `-t` flag work with the `wasm` subcommand
The presence of the flag was checked in the code, so it was essentially already
supported, but it was not properly configured when constructing
the CLI arguments parser.
* clif-util: also enable the `-c` flag for the `wasm` subcommand
Similar to the parent commit, this functionality is already supported, just a
mix up of the CLI parser construction made it not show up.
This commit introduces initial support for multi-value Wasm. Wasm blocks and
calls can now take and return an arbitrary number of values.
The encoding for multi-value blocks means that we need to keep the contents of
the "Types" section around when translating function bodies. To do this, we
introduce a `WasmTypesMap` type that maps the type indices to their parameters
and returns, construct it when parsing the "Types" section, and shepherd it
through a bunch of functions and methods when translating function bodies.
* Add ability to run CLIF IR using `clif-util run [-v] {file}` and add `test run` to cranelift-filetests to allow executing CLIF
This re-factors the compile/execute parts to a FunctionRunner that is shared between cranelift-filetests and clif-util. CLIF can be now be run using `clif-util run` as well as during `clif-util test` for files with a `test run` header. As before, only functions suffixed with a `run` comment are executed. The `run: fn(...) == ...` expression syntax is left for a subsequent change.
- Both the `wasm` and `compile` commands get this new subcommand, and it defaults to false. This means that test runs with `wasm` can request disassembly (the main reason I am doing this) while test runs with `compile` now must request it, this changes current behavior.
- Switch to using context.compile_and_emit directly, and make the reloc and trap printers just accumulate output, not print it. This allows us to factor the printing code into the disasm module.
* initial cargo fix run
* Upgrade cranelift-entity crate
* Upgrade bforest crate
* Upgrade the codegen crate
* Upgrade the faerie crate
* Upgrade the filetests crate
* Upgrade the codegen-meta crate
* Upgrade the frontend crate
* Upgrade the cranelift-module crate
* Upgrade the cranelift-native crate
* Upgrade the cranelift-preopt crate
* Upgrade the cranelift-reader crate
* Upgrade the cranelift-serde crate
* Upgrade the cranelift-simplejit crate
* Upgrade the cranelift or cranelift-umbrella crate
* Upgrade the cranelift-wasm crate
* Upgrade cranelift-tools crate
* Use new import style on remaining files
* run format-all.sh
* run test-all.sh, update Readme and travis ci configuration
fixed an AssertionError also
* Remove deprecated functions
Because of the way that the `pass` subcommand orders its arguments, the
positional "single-file" input cannot be optional with a default value, because
it is followed by required positional arguments. If it were optional, that would
result in argument ambiguity where `clap` cannot tell if the optional positional
argument is supplied, or if the given argument is the next required positional
argument.
Before this commit:
```
$ cargo run --bin clif-util -- pass ./filetests/dce/basic.clif dce
Compiling cranelift-tools v0.21.0 (file:///Users/fitzgen/src/cranelift)
Finished dev [unoptimized + debuginfo] target(s) in 4.38s
Running `target/debug/clif-util pass ./filetests/dce/basic.clif dce`
thread 'main' panicked at 'Found positional argument which is not required with a lower index than a required positional argument: "single-file" index 1', /Users/fitzgen/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.32.0/src/app/parser.rs:612:21
note: Run with `RUST_BACKTRACE=1` for a backtrace.
```
After this commit:
```
$ cargo run --bin clif-util -- pass ./filetests/dce/basic.clif dce
Compiling cranelift-filetests v0.21.0 (file:///Users/fitzgen/src/cranelift/lib/filetests)
Compiling cranelift-tools v0.21.0 (file:///Users/fitzgen/src/cranelift)
Finished dev [unoptimized + debuginfo] target(s) in 5.96s
Running `target/debug/clif-util pass ./filetests/dce/basic.clif dce`
1 tests
```
* fix error not reported if at least one other error expected.
* Fixed unused extern crate error if wasm feature is not enabled.
* No longer reporting deref cycles multiple times.
* Fix filetest type_check.clif.
* Switched comparison order for perf.
* Fixed isa/riscv/verify-encoding.clif filetest.
* Fixed error reporting.
* Fixed compile time error when wasm feature is disabled.
* Fixed valid instructions not being printed in print_function_error.
* Fixed errors print_function_error not writing valid instructions after end.
* Made multiple checks non-fatal.
* verify_global_values is no longer fatal.
* Slightly better formatting of errors in pretty_verifier_error.
