WASI with wasmtime

WebAssembly enables running native code in the JavaScript engine. The compiled and optimised binary ensures better and consistent performance. The JavaScript engine provides the necessary runtime to execute the binary.

What if we bring the performance and portability of WebAssembly outside JavaScript execution environments? The answer is WASI.

WASI or WebAssembly System Interface is a system interface for the WebAssembly platform. WASI will enable running WebAssembly application on any Operating System or architecture provided that we have the runtime. Conceptually, this is similar to JVM. If you have a JVM installed then you can run any Java-like languages on it. Similarly, with a runtime, you can run the WebAssembly module.

It's an API designed by the Wasmtime project that provides access to several operating-system-like features, including files and filesystems, Berkeley sockets, clocks, and random numbers, that we'll be proposing for standardisation.

It's designed to be independent of browsers, so it doesn't depend on Web APIs or JS, and isn't limited by the need to be compatible with JS.

It has integrated capability-based security, so it extends WebAssembly's characteristic sandboxing to include I/O.

WebAssembly System Interface is the next step in WebAssembly's journey.

The way WASI works is simple. You write your application in your favourite languages like Rust, C or C++. Then build and compile them into WebAssembly binary targeting WASI environment. The generated binary requires a special runtime to execute. The runtime provides the necessary interfaces to the system calls.

To see WASI in action, we need a runtime. There are two different runtimes available (and hopefully many will be available later). They are

• wasmtime
• Lucet

WASI provides portability. It provides an option where you write once and run anywhere. Let us see wasmtime in action.

wasmtime - runtime for WebAssembly

Wasmtime is a standalone wasm-only optimizing runtime for WebAssembly and WASI. It runs WebAssembly code outside of the Web and can be used both as a command-line utility or as a library embedded in a larger application. Install the wasmtime runtime to run the WebAssembly binary.

The simplest way to install the wasmtime is by running the following command:

$ curl https://wasmtime.dev/install.sh -sSf | bash
$ ./wasmtime --version
wasmtime 0.9.0

You can use --help option to list down various options available in the wasmtime command.

$ ./wasmtime --help
wasmtime 0.9.0
Wasmtime WebAssembly Runtime

USAGE:
    wasmtime <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    config      Controls Wasmtime configuration settings
    help        Prints this message or the help of the given subcommand(s)
    run         Runs a WebAssembly module
    wasm2obj    Translates a WebAssembly module to the native object file
    wast        Runs a WebAssembly test script file

If a subcommand is not provided, the `run` subcommand will be used.

Usage examples:

Running a WebAssembly module with a start function:

  wasmtime example.wasm

Passing command-line arguments to a WebAssembly module:

  wasmtime example.wasm arg1 arg2 arg3

Invoking a specific function (e.g. `add`) in a WebAssembly module:

  wasmtime example.wasm --invoke add 1 2

We have successfully installed the wasmtime. Now we execute the WebAssembly binary code in the runtime provided by the wasmtime.

To compile the native code applications into WASI compatible, the Rust ecosystem provides wasm32-wasi target. This target is available in the nightly version and then install the target using rustup.

$ rustup target add wasm32-wasi --toolchain nightly

Then we use cargo to build the application for this target using

$ cargo +nightly build --target wasm32-wasi

It is time to takewasmtime for a spin.

How it works...

Let us create a new Rust application using Cargo. To create a new application let us run the following command

Lazy to write the code - check out the repository here:

$ cargo new --bin sizer

This will create a binary application. We can run the application using

$ cargo run
Hello, world!

Now change the src/main.rs with the following contents:

use std::{env, fs};

fn process(current_dir: &str) -> Result<(), String> {
    for entry in fs::read_dir(current_dir).map_err(|err| format!("{}", err))? {
        let entry = entry.map_err(|err| format!("{}", err))?;
        let path = entry.path();
        let metadata = fs::metadata(&path).map_err(|err| format!("{}", err))?;
        println!(
            "filename: {:?}, filesize: {:?} bytes",
             path.file_name().ok_or("No filename").map_err(|err| format!("{}",     err))?,
         metadata.len()
         );
     }
 Ok(())
}

fn main() {
    let args: Vec<String> = env::args().collect();
    let program = args[0].clone();
    if args.len() < 2 {
        eprintln!("{} <input_folder>", program);
        return;
     }
    if let Err(err) = process(&args[1]) {
        eprintln!("{}", err)
    }
}

Given a directory, the process function runs through the directory and list all the files and their size information. The above code has two functions, main and process function. The main function is called first. The function validates whether we provide the directory information as an argument. If the length of arguments is less than 2, it throws an error. If you have provided the argument, then it calls the process function.

The process function takes in the current directory argument. Then it reads the folder for any files available and then it prints out the files that are in the folder and the file sizes.

$ cargo run ./
 filename: "Cargo.toml", filesize: 237 bytes
 filename: "target", filesize: 128 bytes
 filename: "Cargo.lock", filesize: 136 bytes
 filename: ".gitignore", filesize: 8 bytes
 filename: ".git", filesize: 288 bytes
 filename: "src", filesize: 96 bytes

We will create the WebAssembly module using the following command:

$ cargo +nightly build --target wasm32-wasi
 Compiling sizer v0.1.0 (/some/path/to/folder/sizer)
 Finished dev [unoptimized + debuginfo] target(s) in 0.36s

Now let us run the WebAssembly generated using wasmtime.

$ wasmtime target/wasm32-wasi/debug/sizer.wasm
 sizer.wasm <input_folder>

Now let us pass the input argument to the runtime.

$ wasmtime target/wasm32-wasi/debug/sizer.wasm ./
 failed to find a preopened file descriptor through which "./" could be opened

As we can see the wasmtime it does not have any access to the folder specified. It complains that it cannot find any file descriptor. By default, the wasmtime does not have any global permissions. Permission to view other directories and process that might be running in the Operating System. To give the wasmtime permission to access folder, we can provide the directory using --dir flag. The dir flag will create the necessary file descriptors that will enable the wasmtime to access the folder provided.

$ wasmtime target/wasm32-wasi/debug/sizer.wasm --dir=/path/to/sizer/folder /path/to/sizer/folder
filename: "Cargo.toml", filesize: 225 bytes
filename: "target", filesize: 192 bytes
filename: "Cargo.lock", filesize: 137 bytes
filename: ".gitignore", filesize: 19 bytes
filename: ".git", filesize: 288 bytes
filename: "src", filesize: 96 bytes

Now the above command will print the expected results. WASI uses capabilities model for security. Read more about capabilities security model here.

The wasmtime provides a wrapper API over the system calls. They are inspired and derived from POSIX systems. But the WASI core differs from POSIX in the following ways.

• The WASI core has no processes in them. Processes provide no cleaner way to provide forks and execution in Operating Systems. Most of the Operating Systems uses processes to implement forks, execution that is complicated and difficult to maintain. Using processes in WebAssembly System Interface will make the application stick to Operating Systems process boundaries.

The APIs that WASI provides is still under active development. But it is important to note that the APIs provided by WASI is different from POSIX APIs. The former uses no processes and the security model is different. But POSIX APIs uses processes and the security model.

1. The WASI API currently only supports blocking API calls.

2. The WASI API currently does not support async.

3. The WASI API does not have true "mmap" support.

This will be changing shortly once we churn out more APIs.

How to do it

The WASI APIs are named under the wasi_ namespace. For example, all the code related to file descriptors will be placed under the wasi_fd namespace.

For example in the last recipe, we have used __wasi_fd_filestat_get(). This API returns the attribute of the given file.

When working with the languages like Rust we will not need to worry about this low-level API calls. Since most of the Rust calls are converted into necessary wrapper APIs. But we will need to use these APIs when we try to debug or write WebAssembly Module by hand.

Let us go and create a WebAssembly Text format and use the WASI APIs straight away. We will createwriter.wat:

$ touch writer.wat

Let us open the file in our favourite editor.

We will first define the base module for the WebAssembly Text Format.

(module )

We will then have to import the WASI function such that we can call the function inside the WebAssembly Text Format. To import the WASI function:

(module
 (import "wasi_unstable" "fd_write" (func $fdw (param i32 i32 i32 i32) (result i32))
)

Then we call the function inside the WebAssembly module using call.

Note that the API that we are calling and that is defined in the spec is slightly different. The APIs defined in the spec is of the format __wasi_fd_write but we have imported the fd_write function from the wasi_unstable namespace.

But before that, we need to define the memory and data that we will need to use.

(module
 ;; define the imported function
 (memory 1)
 (export "memory" (memory 0))

 (data (i32.const 12) "Hooray it's WASI\n")
)

Here we just define the memory and export it. Then we create data with an offset of 12. This means that the data that we represent will be at the linear memory after an initial offset of 12 bytes.

Unlike, WebAssembly Modules for the Web, here the start function is not optional. So we will have to define the start function.

(module
 ;; define the imported function
 ;; define memory and data
     (func $main (export "_start")
          (i32.store (i32.const 0) (i32.const 12))
          (i32.store (i32.const 4) (i32.const 20))
          (call $fdw
                (i32.const 1)
                (i32.const 0)
                (i32.const 1)
                (i32.const 20)
           )
	  drop
 	)
)

The main function (func main) stores the data from the data pointer. These pointers are then used to represent the iov_base and iov_len.

struct iovec {
      void  *iov_base;    /* Starting address */
      size_t iov_len;     /* Number of bytes to transfer */
};

Then we call the fd_write method with the required four arguments. The arguments are the file descriptor, the pointer to the iov_base and iov_len and finally the new memory location in which it has to write the contents.

Finally, we discard the written bytes from the top of the stack.

How it works...

The wasmtime provides a way to use the WebAssembly Text format. So we do not need to worry about converting the WebAssembly Text Format into WebAssembly Module.

$ wasmtime writer.wat
Hooray It's `WASI`

The wasmtime interpreter first validates whether the file provides is in the correct formation. Once the validation is successful, the 1 will compile the application. During this phase, the wasmtime compiler creates the binary code that will initiate the system call for the underlying architecture.

Finally, the drop will run and throw off some bytes from the top of the stack.

It is important to note that the fourth parameter or new memory offset to write should be divisible by 4. This is to prevent illegal bytes.

(call $fdw
 (i32.const 1)
 (i32.const 0)
 (i32.const 1)
 (i32.const 20)
)

Note: Having the fourth parameter i32.const 18 will result in !!! bad alignment: 18 % 4.

If you want to get the complete trace of what is happening, which function is called and how much time is spent on each function call. We can easily track that down using the -d flag.

$ RUST_LOG=trace wasmtime writer.wat -d

The above code will print the trace level output of the logs. It will also print all the code transformation that is done by the wasmtime.

TRACE wasmtime_wasi_c::syscalls > fd_write(fd=1, iovs=0x0, iovs_len=1, nwritten=0xc)
Hooray it's WASI
 TRACE wasmtime_wasi_c::syscalls > | *nwritten=20
 TRACE wasmtime_wasi_c::syscalls > -> errno=__WASI_ESUCCESS

This is the trace output from the actual WASI system calls.

Also if you find it difficult to use the syscall at any point because you cannot determine the type of the syscall function. It is better to debug what is failing with tools like wabt (wat2wasm executable) in general and then alter the function definition as needed.

Errors to note in WASI

There are various errors that WASI emits that will help you while developing we will see a few of them here. Every syscall in WASI (at least for now) is blocking and not asynchronous. This means that once you call the syscall, you have to wait till the call is completed and proceed further based on the result.

Getting Started

We will start with an example. In this example, we will create a new directory using WASI. It is always exciting to write it in WebAssembly Text Format and then using wasmtime to run the WebAssembly Text Format.

Let us create a file called creator.wast.

Open up the editor and we will start writing the WebAssembly Text Format.

How to do it

We will first create a module. All the code will live inside the module.

(module )

Then we import the API for creating the new directory. The function signature of the create directory syscall has three arguments.

__wasi_fd_t -> The file descriptor const char *ptr -> This should be a pointer to the start of the linear memory array inside the WebAssembly Module size_t path_len -> The length of the value that is stored in the memory array.

The imported function will look like this

(module
 (import "wasi_unstable" "path_create_directory" (func $mkdir (param i32 i32 i32) (result i32))
)

We will define the memory, like this.

(module
 ;; define the imported function
 (memory 1)
 (export "memory" (memory 0))

 (data (i32.const 12) "wasi-folder")
)

We are defining the data in the linear memory array at an offset 12.

Now we have to define the start method that will call the $mkdir function.

(module
 ;; define the imported function
 ;; define memory and data

 (func $main (export "_start")
 (i32.store (i32.const 0) (i32.const 12))
 (i32.store (i32.const 4) (i32.const 20))

 (call $mkdir
 (i32.const 3)
 (i32.const 12)
 (i32.const 11)
 )
 drop
 )
)

How it works

The $mkdir takes in three parameters.

1. The first one being the file descriptor. We have passed in an i32.const 3 that takes in the third value in the process argument. (the --dir argument)
2. The next is the starting of the data defined. The data starts at the offset 12. So we will use the same offset here.
3. Finally the length of the data that we have. In our case, it is 11.

Once done, let us run our application using wasmtime.

$ /path/to/wasmtime/target/wasmtime creator.wat --dir=.
$ ls | grep wasi-folder
drwxr-xr-x 2 sendilkumar staff 64B Jun 12 11:35 wasi-folder

Note that we have given the current directory using the notation "." but notations like ".." will work with the wasmtime if provided to the --dir option.

But if we change the data into (data (i32.const 12) "../some_folder"

And then extend the memory offset to reflect the changes, will result in a __WASI_ENOTCAPABLE error.

The not capable error tells that the WASI is trying to create or access something that it did not have access.

If we pass in zero bytes as the length of the path, then the WASI will throw __WASI_ENOENT error.

The no ent error tells that there is no file or directory available.

__WASI_EILSEQ error is thrown when there is any illegal sequence of the data from linear memory is accessed.

Further Explorations

Check out this awesome post from Lin Clark about WASI or WebAssembly System Interface here

Read more about how to use wasmtime with C or C++ here

Check out more available APIs here

Check out more about the WASI tutorial here

Check out for more errors in the WASI_API here

Check out more about the WASI Background here

Discussions