- Introduced the virtual BUS interface used for RPC between web assembly apps
- Introduced the virtual networking interface used to implement networking
for web assembly apps
- Implemented a local implementation of the virtual networking
(available behind the feature toggle 'host-net' on the 'wasi' package)
- Fixed up some of the examples from the wasmer3 branch
- Refactored the WASI implementations so they support wasm64-wasi
- WASIX is behind its own namespaces for both 32bit and 64bit implementations
- Fixed the wasi_pipes unit test which was using internals that are no longer exposed - instead made the pipes clonable
- Split functionality out of WasiEnv so that it can support multi-threading
- Added methods to the VFS File Trait that supporting polling
- Implemented basic time functionality for WASI
- Incorported a yield callback for when WASI processes idle
- Improved the error handling on WASI IO calls
- Reduce the verbose logging on some critical WASI calls (write/read)
- Implemented the missing poll functionality for WASI processes
- Moved the syspoll functionality behind a feature flag to default to WASI method
- Refactored the thread sleeping functionality for WASI processes
- Fixed the files system benchmark which was not compiling
- Modified the file system trait so that it is SYNC and thus can handle multiple threads
- Removed the large mutex around filesystem state and implemented granular locks instead
(this is needed to fix a deadlock scenario on the terminal)
- Split the inodes object apart from the state to fix the deadlock scenario.
- Few minor fixes to some warnings when not using certain features
- Sleeping will now call a callback that can be used by the runtime operator when
a WASI thread goes to sleep (for instance to do other work)
- Fixed a bug where paths that exist on the real file system are leaking into VFS
- Timing functions now properly return a time precision on WASI
- Some improved macros for error handling within syscalls (wasi_try_ok!)
- Refactored the remove_directory WASI function which was not working properly
- Refactored the unlink WASI function which was not working properly
- Refactored the poll WASI function which was not working properly
- Updates some of the tests to make them compile again
- Rewrote the OutputCapturer so that it does leak into the internals
This patch does several things.
1. For the crate `wasmer-c-api`, the library name is modified from
`wasmer_c_api` to `wasmer` in `Cargo.toml`. That way, the new
library files are named `libwasmer.*` rather than
`libwasmer_c_api.*`. That's the primaly goal of this patch. The
rest is a consequence of this point. Why do we want that? Because
the `build.rs` script of the `wasmer-c-api` crate will configure
the `soname` (on Linux), the `install_name` + `current_version` +
`compatibility_version` (on macOS), and the `out-implib` +
`output-def` (on Windows) for a library named `libwasmer`, which is
the name we provide in the Wasmer package for the Wasmer
libraries. If we want everything to be testable, we cannot use
`libwasmer` in `soname` for a file named `libwasmer_c_api` for
example. If we rename the file when packaging (as it's done prior
this patch), we would need to re-update all those information in
the `Makefile`. It implies to duplicate the code in 2 places. So
let's do things right directly and only once: We want the library
to be named `libwasmer`, let's do that.
2. For the crate `wasmer-c-api`, since the library name has been
renamed to `wasmer`, it creates a conflict with the `wasmer` crate
which is a dependency. Consequently, this patch also updates the
`Cargo.toml` file to modifiy the dependency name with the special
`package` attribute (see
https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#renaming-dependencies-in-cargotoml
to learn more). So now, the `wasmer` refers to the `wasmer_c_api`
crate, and `wasmer-api` refers to the `wasmer` crate.
3. The code of the `wasmer-c-api` crate is updated accordingly. The
`WasmerEnv` derive procedural macro fails because it expects a
crate named `wasmer` (which is now `wasmer_api`), so we implement
the `WasmerEnv` trait by hand.
4. The patch updates the `build.rs` script of the `wasmer-c-api`
crate:
1. In the `build_cdylib_link_arg` function: The dependency to the
`cdylib-link-lines` crate has been removed because the output is
not exactly the one we expect. So we compute all the
`cargo:rustc-cdylib-link-arg=…` lines by hand. The version number
no longer appears in the library file name for example.
2. In the `build_inline_c_env_vars` function: Values passed to
`LDFLAGS` have been updated to be `libwasmer` rather than
`libwasmer_c_api`.
3. A new `shared_object_dir` helper function has been created
because it's used in `build_inline_c_env_vars` and in
`build_cdylib_link_arg`.
5. The `Makefile` has been updated:
1. In `package-capi`, we no longer rename `libwasmer_c_api` to
`libwasmer` since the name is correctly defined since the
beginning now.
Calling `install_name_tool` on macOS is no longer required since
`install_name` is correctly set by the linker in the `build.rs`
script of `wasmer-c-api`.
2. In `package-docs`, some stuffs have been fixed, like the
`SOURCE_VERSION` variable that didn't exist, so removed, or the
`mkdir` command that was incorrect etc.
3. In `build-docs`, the `wasmer-c-api` crate is excluded from the
list of crates to generate the documentation for. Mostly because
the `build-docs-capi` recipe exists, and we must use it to
generate the documentation of `wasmer-c-api` now.
4. In `build-docs-capi`, we generate the documentation for the
`wasmer-c-api` crate. But `rustdoc` uses the library name for the
directory name in the `target/doc/` directory. Since the library
name is now `wasmer`, it creates a conflict with the `wasmer`
crate. Consequently, we update the library name by using `sed` on
the `Cargo.toml` file before running `cargo doc`, to finally
restore `Cargo.toml` as it was previously.
Until this patch, our C API comes in 2 flavors: `deprecated` and
`wasm_c_api`. With the coming 2.x version of Wasmer, we would like to
remove the `deprecated` API, and keep the `wasm_c_api` only.
This patch removes the `deprecated` API from the `wasmer-c-api`
crate. It also cleans up the `Makefile` and the documentation
system. Previously, the documentation for the `deprecated` API was
relying on Doxygen, which was one new dependency the user had to
install. For the `wasm_c_api`, it relies on `rustdoc`, which is way
better because all examples are run and tested as part of our test
suite.
This clean up also removes the need to deal with `system-libffi` both
in the crate itself and in the `Makefile`, which was an edge case for
macOS on aarch64, and a needle in the foot for some of our users.
Finally, the `build.rs` is now simplified because we no longer need to
exclude symbols from one header to another. It also means that we only
provide the `wasmer_wasm.h` header file now; the `wasmer.h` and
`wasmer.hh` headers are removed.
The new rule is the following:
* `wasm_` for the standard C API,
* `wasmer_` or `wasi_` for the Wasmer non-standard C API.
For all symbols inside the `unstable` module, the renaming `wasm_` to
`wasmer_` is done without deprecations. It was clear that those API
were unstable.
For all the other symbols, symbols have been renamed to `wasmer_` but
the old symbols have been kept with deprecation warnings.
Special note: The `wasm_named_extern_t` type (and associated
functions) was in `wasi` by mistake. Its place was in the `unstable`
module. This patch also fixes that.
The `wasm_declare_vec_*` macros have been updated to support a default
prefix, or a user-defined prefix. It's now possible to write
`wasm_declare_boxed_vec!(foo);` to get all the API prefixed by `wasm_`
(as previously), or `wasm_declare_boxed_vec!(foo, wasmer);` to prefix
with `wasmer_`.
A user not using symbols from the `unstable` module will continue to
get working code, modulo some deprecations, after this patch.
We said that `wasi_get_imports` was taking ownership of
`wasi_env_t`. It wasn't. In 9e63ba9a25,
we have fixed this. But it creates another bug when `wasi_env_t` is
used _after_ for example when calling `wasi_env_read_stdout`.
So this patch reverts the bug fix. And we will discuss about how to
fix that later.
So. Let's explain a dirty hack. `cbindgen` reads the code and collects
symbols. What symbols do we need? None of the one declared in
`wasm.h`, but for non-standard API, we need to collect all of
them. The problem is that `wasm_named_extern_t` is the only
non-standard type where extra symbols are generated by a macro
(`wasm_declare_boxed_vec!`). If we want those macro-generated symbols
to be collected by `cbindgen`, we need to _expand_ the crate
(i.e. running something like `rustc -- -Zunstable-options
--pretty=expanded`). Expanding code is unstable and available only on
nightly compiler. We _don't want_ to use a nightly compiler only for
that. So how can we help `cbindgen` to _see_ those symbols?
First solution: We write the C code directly in a file, which is then
included in the generated header file with the `cbindgen`
API. Problem, it's super easy to get it outdated, and it makes the
build process more complex.
Second solution: We write those symbols in a custom module, that is
just here for `cbindgen`, never used by our Rust code (otherwise it's
duplicated code), with no particular implementation.
And that's why we have the following `cbindgen_hack` module.
But this module must not be compiled by `rustc`. How to force `rustc`
to ignore a module? With conditional compilation. Because `cbindgen`
does not support conditional compilation, it will always _ignore_ the
`#[cfg]` attribute, and will always read the content of the module.
Sorry.
In `wasi_get_imports` and `wasi_get_unordered_imports`, we said the
ownership of `wasi_env_t` was taken by the function, but it wasn't the
case. This patch changes the type from `&wasi_env_t` to
`Box<wasi_env_t>` to take ownership of it.
The rest of the patch updates the documentation, and improves null
protections with `Option<T>`.
We use `VecDeque::drain` to read the captured stream, zipped with the
given buffer. We could expect that only the yielded items from the
`drain` will be removed, but actually no. Reading [the
documentation](https://doc.rust-lang.org/std/collections/struct.VecDeque.html#method.drain):
> Note 1: The element `range` is removed even if the iterator is not
> consumed until the end.
So by using a range like `..` will drain the entire captured stream,
whatever we read from it. Said differently, if the given buffer length
is smaller than the captured stream, the first read will drain the
entire captured stream.
This patch fixes the problem by specifying a better range:
`..min(inner_buffer.len(), oc.buffer.len())`.
With this new range, it's actually useless to increment
`num_bytes_written`, we already know ahead of time the amount of bytes
we are going to read. Consequently, the patch simplifies this code a
little bit more.
On Windows, using a `u32` representation for `wasi_version_t` fails if
a C++ compiler is used to treat a C program. So we change our strategy
here. We use a C representation to be FFI-safe, and the
`INVALID_VERSION` variant is now set to -1 instead of `u32::MAX`.
This patch also adds unit tests for `wasi_get_wasi_version` so that we
are sure of the behavior of this `type` on all platforms.
First, let's no longer derive from `Default` for `wasi_config_t`. By
default, we want to inherit `stdin`, `stdout` and `stderr`. The
default for `bool` is `false`; we want `true` here.
Second, let's update `wasi_config_new` to correctly set `inherit_*`
fields to `true`.
Third, lets' create `wasi_config_capture_*` functions. By default,
`std*` are inherited, so we need functions to capture them. That's the
new feature this patch introduces. The `wasi_config_inherit_*`
functions are kept for the sake of backward compatibility. Ideally, we
would want an API like `wasi_config_capture_*(capture: bool)`, but it
would duplicate the API somehow.
Fourth, let's fix `wasi_env_new`. We want to capture `stdout` and
`stderr` if and only if the `inherit_*` fields are set to
`false`. There was bug here. That's why everything was working
correctly by the way: `bool::default()` is `false`, and we have this
inverted condition here, so everything was working as expected because
of a double error. The only bug was that it wasn't possible to capture
`std*` before.
I hope this little change will clarify a little bit that the `Export`
passed to `Extern::from_vm_export` is not a `wasmer::Export` but a
`wasmer_vm::Export`.
