92e0b6b9e8
* wasi-nn: turn it on by default This change makes the wasi-nn Cargo feature a default feature. Previously, a wasi-nn user would have to build a separate Wasmtime binary (e.g. `cargo build --features wasi-nn ...`) to use wasi-nn and the resulting binary would require OpenVINO shared libraries to be present in the environment in order to run (otherwise it would fail immediately with linking errors). With recent changes to the `openvino` crate, the wasi-nn implementation can defer the loading of the OpenVINO shared libraries until runtime (i.e., when the user Wasm program calls `wasi_ephemeral_nn::load`) and display a user-level error if anything goes wrong (e.g., the OpenVINO libraries are not present on the system). This runtime-linking addition allows the wasi-nn feature to be turned on by default and shipped with upcoming releases of Wasmtime. This change should be transparent for users who do not use wasi-nn: the `openvino` crate is small and the newly-available wasi-nn imports only affect programs in which they are used. For those interested in reviewing the runtime linking approach added to the `openvino` crate, see https://github.com/intel/openvino-rs/pull/19. * wasi-nn spec path: don't use canonicalize * Allow dependencies using the ISC license The ISC license should be [just as permissive](https://choosealicense.com/licenses/isc) as MIT, e.g., with no additional limitations. * Add a `--wasi-modules` flag This flag controls which WASI modules are made available to the Wasm program. This initial commit enables `wasi-common` by default (equivalent to `--wasi-modules=all`) and allows `wasi-nn` and `wasi-crypto` to be added in either individually (e.g., `--wasi-modules=wasi-nn`) or as a group (e.g., `--wasi-modules=all-experimental`). * wasi-crypto: fix unused dependency Co-authored-by: Pat Hickey <pat@moreproductive.org>
46 lines
2.2 KiB
Bash
Executable File
46 lines
2.2 KiB
Bash
Executable File
#!/bin/bash
|
|
|
|
# The following script demonstrates how to execute a machine learning inference using the wasi-nn module optionally
|
|
# compiled into Wasmtime. Calling it will download the necessary model and tensor files stored separately in $FIXTURE
|
|
# into $TMP_DIR (optionally pass a directory with existing files as the first argument to re-try the script). Then,
|
|
# it will compile the example code in crates/wasi-nn/tests/example into a Wasm file that is subsequently
|
|
# executed with the Wasmtime CLI.
|
|
set -e
|
|
WASMTIME_DIR=$(dirname "$0" | xargs dirname)
|
|
FIXTURE=https://github.com/intel/openvino-rs/raw/main/crates/openvino/tests/fixtures/mobilenet
|
|
if [ -z "${1+x}" ]; then
|
|
# If no temporary directory is specified, create one.
|
|
TMP_DIR=$(mktemp -d -t ci-XXXXXXXXXX)
|
|
REMOVE_TMP_DIR=1
|
|
else
|
|
# If a directory was specified, use it and avoid removing it.
|
|
TMP_DIR=$(realpath $1)
|
|
REMOVE_TMP_DIR=0
|
|
fi
|
|
|
|
# Inform the environment of OpenVINO library locations. Then we use OPENVINO_INSTALL_DIR below to avoid building all of
|
|
# OpenVINO from source (quite slow).
|
|
source /opt/intel/openvino/bin/setupvars.sh
|
|
|
|
# Build Wasmtime with wasi-nn enabled; we attempt this first to avoid extra work if the build fails.
|
|
OPENVINO_INSTALL_DIR=/opt/intel/openvino cargo build -p wasmtime-cli --features wasi-nn
|
|
|
|
# Download all necessary test fixtures to the temporary directory.
|
|
wget --no-clobber $FIXTURE/mobilenet.bin --output-document=$TMP_DIR/model.bin
|
|
wget --no-clobber $FIXTURE/mobilenet.xml --output-document=$TMP_DIR/model.xml
|
|
wget --no-clobber $FIXTURE/tensor-1x224x224x3-f32.bgr --output-document=$TMP_DIR/tensor.bgr
|
|
|
|
# Now build an example that uses the wasi-nn API.
|
|
pushd $WASMTIME_DIR/crates/wasi-nn/examples/classification-example
|
|
cargo build --release --target=wasm32-wasi
|
|
cp target/wasm32-wasi/release/wasi-nn-example.wasm $TMP_DIR
|
|
popd
|
|
|
|
# Run the example in Wasmtime (note that the example uses `fixture` as the expected location of the model/tensor files).
|
|
cargo run -- run --mapdir fixture::$TMP_DIR $TMP_DIR/wasi-nn-example.wasm --wasi-modules=experimental-wasi-nn
|
|
|
|
# Clean up the temporary directory only if it was not specified (users may want to keep the directory around).
|
|
if [[ $REMOVE_TMP_DIR -eq 1 ]]; then
|
|
rm -rf $TMP_DIR
|
|
fi
|