Begin implementation of wasi-http (#5929)

* Integrate experimental HTTP into wasmtime.

* Reset Cargo.lock

* Switch to bail!, plumb options partially.

* Implement timeouts.

* Remove generated files & wasm, add Makefile

* Remove generated code textfile

* Update crates/wasi-http/Cargo.toml

Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>

* Update crates/wasi-http/Cargo.toml

Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>

* Extract streams from request/response.

* Fix read for len < buffer length.

* Formatting.

* types impl: swap todos for traps

* streams_impl: idioms, and swap todos for traps

* component impl: idioms, swap all unwraps for traps, swap all todos for traps

* http impl: idiom

* Remove an unnecessary mut.

* Remove an unsupported function.

* Switch to the tokio runtime for the HTTP request.

* Add a rust example.

* Update to latest wit definition

* Remove example code.

* wip: start writing a http test...

* finish writing the outbound request example

havent executed it yet

* better debug output

* wasi-http: some stubs required for rust rewrite of the example

* add wasi_http tests to test-programs

* CI: run the http tests

* Fix some warnings.

* bump new deps to latest releases (#3)

* Add tests for wasi-http to test-programs (#2)

* wip: start writing a http test...

* finish writing the outbound request example

havent executed it yet

* better debug output

* wasi-http: some stubs required for rust rewrite of the example

* add wasi_http tests to test-programs

* CI: run the http tests

* bump new deps to latest releases

h2 0.3.16
http 0.2.9
mio 0.8.6
openssl 0.10.48
openssl-sys 0.9.83
tokio 1.26.0

---------

Co-authored-by: Brendan Burns <bburns@microsoft.com>

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* wasi-http: fix cargo.toml file and publish script to work together (#4)

unfortunately, the publish script doesn't use a proper toml parser (in
order to not have any dependencies), so the whitespace has to be the
trivial expected case.

then, add wasi-http to the list of crates to publish.

* Update crates/test-programs/build.rs

* Switch to rustls

* Cleanups.

* Merge switch to rustls.

* Formatting

* Remove libssl install

* Fix tests.

* Rename wasi-http -> wasmtime-wasi-http

* prtest:full

Conditionalize TLS on riscv64gc.

* prtest:full

Fix formatting, also disable tls on s390x

* prtest:full

Add a path parameter to wit-bindgen, remove symlink.

* prtest:full

Fix tests for places where SSL isn't supported.

* Update crates/wasi-http/Cargo.toml

---------

Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>
Co-authored-by: Pat Hickey <phickey@fastly.com>
Co-authored-by: Pat Hickey <pat@moreproductive.org>

crates/test-programs/tests/http_tests/main.rs

@@ -0,0 +1,12 @@
+#![cfg(feature = "test_programs_http")]
+use std::sync::Once;
+mod runtime;
+mod utils;
+
+static LOG_INIT: Once = Once::new();
+
+fn setup_log() {
+    LOG_INIT.call_once(tracing_subscriber::fmt::init)
+}
+
+include!(concat!(env!("OUT_DIR"), "/wasi_http_tests.rs"));

crates/test-programs/tests/http_tests/runtime/mod.rs

@@ -0,0 +1 @@
+pub mod wasi_http_tests;

crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

@@ -0,0 +1,39 @@
+use anyhow::Context;
+use std::path::Path;
+use wasmtime::{Config, Engine, Linker, Module, Store};
+use wasmtime_wasi::{sync::WasiCtxBuilder, WasiCtx};
+use wasmtime_wasi_http::WasiHttp;
+
+pub fn instantiate_inherit_stdio(
+    data: &[u8],
+    bin_name: &str,
+    _workspace: Option<&Path>,
+) -> anyhow::Result<()> {
+    let config = Config::new();
+    let engine = Engine::new(&config)?;
+    let module = Module::new(&engine, &data).context("failed to create wasm module")?;
+    let mut linker = Linker::new(&engine);
+
+    struct Ctx {
+        wasi: WasiCtx,
+        http: WasiHttp,
+    }
+
+    wasmtime_wasi::sync::add_to_linker(&mut linker, |cx: &mut Ctx| &mut cx.wasi)?;
+    wasmtime_wasi_http::add_to_linker(&mut linker, |cx: &mut Ctx| &mut cx.http)?;
+
+    // Create our wasi context.
+    let builder = WasiCtxBuilder::new().inherit_stdio().arg(bin_name)?;
+
+    let mut store = Store::new(
+        &engine,
+        Ctx {
+            wasi: builder.build(),
+            http: WasiHttp::new(),
+        },
+    );
+
+    let instance = linker.instantiate(&mut store, &module)?;
+    let start = instance.get_typed_func::<(), ()>(&mut store, "_start")?;
+    start.call(&mut store, ())
+}

crates/test-programs/tests/http_tests/utils.rs

@@ -0,0 +1,13 @@
+use std::path::Path;
+
+pub fn extract_exec_name_from_path(path: &Path) -> anyhow::Result<String> {
+    path.file_stem()
+        .and_then(|s| s.to_str())
+        .map(String::from)
+        .ok_or_else(|| {
+            anyhow::anyhow!(
+                "couldn't extract the file stem from path {}",
+                path.display()
+            )
+        })
+}