* Upgrade to the latest versions of gimli, addr2line, object
And adapt to API changes. New gimli supports wasm dwarf, resulting in
some simplifications in the debug crate.
* upgrade gimli usage in linux-specific profiling too
* Add "continue" statement after interpreting a wasm local dwarf opcode
This fixes some hard-coded assumptions in the debug crate that
the native ELF files being accessed are little-endian; specifically
in create_gdbjit_image as well as in emit_dwarf.
In addition, data in WebAssembly memory always uses little-endian
byte order. Therefore, if the native architecture is big-endian,
all references to base types need to be marked as little-endian
using the DW_AT_endianity attribute, so that the debugger will
be able to correctly access them.
This PR propagates "value labels" all the way from CLIF to DWARF
metadata on the emitted machine code. The key idea is as follows:
- Translate value-label metadata on the input into "value_label"
pseudo-instructions when lowering into VCode. These
pseudo-instructions take a register as input, denote a value label,
and semantically are like a "move into value label" -- i.e., they
update the current value (as seen by debugging tools) of the given
local. These pseudo-instructions emit no machine code.
- Perform a dataflow analysis *at the machine-code level*, tracking
value-labels that propagate into registers and into [SP+constant]
stack storage. This is a forward dataflow fixpoint analysis where each
storage location can contain a *set* of value labels, and each value
label can reside in a *set* of storage locations. (Meet function is
pairwise intersection by storage location.)
This analysis traces value labels symbolically through loads and
stores and reg-to-reg moves, so it will naturally handle spills and
reloads without knowing anything special about them.
- When this analysis converges, we have, at each machine-code offset, a
mapping from value labels to some number of storage locations; for
each offset for each label, we choose the best location (prefer
registers). Note that we can choose any location, as the symbolic
dataflow analysis is sound and guarantees that the value at the
value_label instruction propagates to all of the named locations.
- Then we can convert this mapping into a format that the DWARF
generation code (wasmtime's debug crate) can use.
This PR also adds the new-backend variant to the gdb tests on CI.
I don't think this has happened in awhile but I've run a `cargo update`
as well as trimming some of the duplicate/older dependencies in
`Cargo.lock` by updating some of our immediate dependencies as well.
This commit reduces the size of `InstructionAddressMap` from 24 bytes to
8 bytes by dropping the `code_len` field and reducing `code_offset` to
`u32` instead of `usize`. The intention is to primarily make the
in-memory version take up less space, and the hunch is that the
`code_len` is largely not necessary since most entries in this map are
always adjacent to one another. The `code_len` field is now implied by
the `code_offset` field of the next entry in the map.
This isn't as big of an improvement to serialized module size as #2321
or #2322, primarily because of the switch to variable-length encoding.
Despite this though it shaves about 10MB off the encoded size of the
module from #2318
* normalise value prior to right shifts
by first left-aligning (shift left by 32 bits)
then shifting back (respecting signedness)
* Update crates/debug/src/transform/expression.rs
Co-authored-by: bjorn3 <bjorn3@users.noreply.github.com>
* Update crates/debug/src/transform/expression.rs
* Update crates/debug/src/transform/expression.rs
* update translation of DW_OP_shr in test
* add translation test for DW_OP_shra
* explain normalisation
* optimise the expression by performing only one right shift
We assume that the expression evaluator permits collapsing
two shifts as long as they go in the same direction.
Review feedback.
Co-authored-by: bjorn3 <bjorn3@users.noreply.github.com>
* Don't substract 1 from end_addr in line program writing
Fixes#2173
* add testcase for end_sequence having offset past retq (#1)
* Update tests/all/debug/translate.rs
Co-authored-by: Gabor Greif <ggreif@gmail.com>
Co-authored-by: Gabor Greif <ggreif@gmail.com>
The fundamental problem is that the target distance of jump-like operations may change in the DWARF expression translation process. Intervening DW_OP_deref will expand to about 10 bytes, for example.
So the jumps must be relocated. We approach this task by inserting artificial LandingPad markers (new CompiledExpressionParts constructors) into the parsed vector at actual Jump targets.
LandingPads are identified by JumpTargetMarker tokens which are generated on the fly.
Additionally we now parse the Jump instructions. These also get their corresponding JumpTargetMarker token.
We bail in two situations:
frame_base is too complicated (i.e. itself contains Jump)
some jump distance in the original expression is fishy.
No point in resorting to LEB128 encoding for such constants,
using the native `u32` is faster and more compact.
Adds `write_u32` method to facilitate this.
Provide automatic translation to opcodes from DW_OP_* identifiers. They are looked up from gimli.
Since DW_OP_WASM_location is not contained in gimli yet, we take care of manually translating it.
* Refactor where results of compilation are stored
This commit refactors the internals of compilation in Wasmtime to change
where results of individual function compilation are stored. Previously
compilation resulted in many maps being returned, and compilation
results generally held all these maps together. This commit instead
switches this to have all metadata stored in a `CompiledFunction`
instead of having a separate map for each item that can be stored.
The motivation for this is primarily to help out with future
module-linking-related PRs. What exactly "module level" is depends on
how we interpret modules and how many modules are in play, so it's a bit
easier for operations in wasmtime to work at the function level where
possible. This means that we don't have to pass around multiple
different maps and a function index, but instead just one map or just
one entry representing a compiled function.
Additionally this change updates where the parallelism of compilation
happens, pushing it into `wasmtime-jit` instead of `wasmtime-environ`.
This is another goal where `wasmtime-jit` will have more knowledge about
module-level pieces with module linking in play. User-facing-wise this
should be the same in terms of parallel compilation, though.
The ultimate goal of this refactoring is to make it easier for the
results of compilation to actually be a set of wasm modules. This means
we won't be able to have a map-per-metadata where the primary key is the
function index, because there will be many modules within one "object
file".
* Don't clear out fields, just don't store them
Persist a smaller set of fields in `CompilationArtifacts` instead of
trying to clear fields out and dynamically not accessing them.
* Don't re-parse wasm for debuginfo
This commit updates debuginfo parsing to happen during the main
translation of the original wasm module. This avoid re-parsing the wasm
module twice (at least the section-level headers). Additionally this
ties debuginfo directly to a `ModuleTranslation` which makes it easier
to process debuginfo for nested modules in the upcoming module linking
proposal.
The changes here are summarized by taking the `read_debuginfo` function
and merging it with the main module translation that happens which is
driven by cranelift. Some new hooks were added to the module environment
trait to support this, but most of it was integrating with existing hooks.
* Fix tests in debug crate
This commit is intended to update wasmparser to 0.59.0. This primarily
includes bytecodealliance/wasm-tools#40 which is a large update to how
parsing and validation works. The impact on Wasmtime is pretty small at
this time, but over time I'd like to refactor the internals here to lean
more heavily on that upstream wasmparser refactoring.
For now, though, the intention is to get on the train of wasmparser's
latest `main` branch to ensure we get bug fixes and such.
As part of this update a few other crates and such were updated. This is
primarily to handle the new encoding of `ref.is_null` where the type is
not part of the instruction encoding any more.
- Create the ELF image from Compilation
- Create CodeMemory from the ELF image
- Link using ELF image
- Remove creation of GDB JIT images from crates/debug
- Move make_trampoline from compiler.rs
* Refactor how relocs are stored and handled
* refactor CompiledModule::instantiate and link_module
* Refactor DWARF creation: split generation and serialization
* Separate DWARF data transform from instantiation
* rm LinkContext
There was a bug how value labels were resolved, which caused some DWARF expressions not be transformed, e.g. those are in the registers.
* Implements FIXME in expression.rs
* Move TargetIsa from CompiledExpression structure
* Fix expression format for GDB
* Add tests for parsing
* Proper logic in ValueLabelRangesBuilder::process_label
* Tests for ValueLabelRangesBuilder
* Refactor build_with_locals to return Iterator instead of Vec<_>
* Misc comments and magical numbers
Support a few DWARF-5 only features:
* read .debug_addr
* read .debug_rnglists
* read .debug_loclists when present
* add dwarf-5 test
* read .debug_addr
* read .debug_rnglists
* read .debug_loclists when present
* support .debug_line_str and .debug_str_offsets
Co-authored-by: Yury Delendik <ydelendik@mozilla.com>
This commit makes the following changes to unwind information generation in
Cranelift:
* Remove frame layout change implementation in favor of processing the prologue
and epilogue instructions when unwind information is requested. This also
means this work is no longer performed for Windows, which didn't utilize it.
It also helps simplify the prologue and epilogue generation code.
* Remove the unwind sink implementation that required each unwind information
to be represented in final form. For FDEs, this meant writing a
complete frame table per function, which wastes 20 bytes or so for each
function with duplicate CIEs. This also enables Cranelift users to collect the
unwind information and write it as a single frame table.
* For System V calling convention, the unwind information is no longer stored
in code memory (it's only a requirement for Windows ABI to do so). This allows
for more compact code memory for modules with a lot of functions.
* Deletes some duplicate code relating to frame table generation. Users can
now simply use gimli to create a frame table from each function's unwind
information.
Fixes#1181.
This exposes the functionality of `fde::map_reg` on the `TargetIsa` trait, avoiding compilation errors on architectures where register mapping is not yet supported. The change is conditially compiled under the `unwind` feature.
Both cranelift-codegen and wasmtime-debug need to map Cranelift registers to Gimli registers. Previously both crates had an almost-identical `map_reg` implementation. This change:
- removes the wasmtime-debug implementation
- improves the cranelift-codegen implementation with custom errors
- exposes map_reg in `cranelift_codegen::isa::fde::map_reg` and subsequently `wasmtime_environ::isa::fde::map_reg`
The crates/debug/src/transform/address_transform.rs is unoptimized in terms of data structures. This PR refactors this file to remove creation of intermediate in-heap structures, thus improves overall performance of the DWARF transformation.
* Reduce amount of memory allocated in translate_ranges_raw
* refactor translate_ranges
* Don't transform non-unit .debug_line
* type annotation for TransformRangeXXXIter's
* Fix empty generated wasm positions
