Binding your WebAssembly and JavaScript with wasm-bindgen

WebAssembly can only send and receive number between JavaScript and WebAssembly module.

In order to pass other data (like String, Objects, Functions), we should create a binding file.

The binding file does the following:

• It converts string or object to something that WebAssembly module understands.
• It converts the returned value from WebAssembly module into string or object that JavaScript understands.

But converting them every time is a mundane task and error-prone. Fortunately Rust world came up with wasm-bindgen.

wasm-bindgen

Facilitating high-level interactions between wasm modules and JavaScript - wasm-bindgen

The wasm-bindgen provides a channel between JavaScript and WebAssembly to communicate something other than numbers, i.e., objects, strings, arrays, etc.,

Write some code ✍️

Let us start with hello_world with wasm-bindgen.

Create a new project with cargo.

$ cargo new --lib hello_world
Created library `hello_world` package

This creates a new Rust project with the necessary files.

Once created open the project in your favourite editor.

Open the Cargo.toml file and add the wasm-bindgen dependency.

[package]
name = "hello_world"
version = "0.1.0"
authors = ["Sendil Kumar <sendilkumarn@live.com>"]
edition = "2018"

[lib]
crate-type = ["cdylib"]

[dependencies]
wasm-bindgen = "0.2.56"

Open the src/lib.rs file and replace the contents with the following:

use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub fn hello_world() -> String {
 "Hello World".to_string()
}

We imported the wasm_bindgen library use wasm_bindgen::prelude::*;.

We annotated the hello_world() function with #[wasm_bindgen] tag.

The hello_world() function returns a String.

To generate the WebAssembly module run:

$ cargo build --target=wasm32-unknown-unknown

The cargo build command does not generate any JavaScript binding file. In order to generate the binding files, we need to run the wasm-bindgen CLI tool on the generated WebAssembly module.

Install wasm-bindgen CLI to generate the binding file.

Use cargo to install wasm-bindgen-CLI:

$ cargo install wasm-bindgen-cli

Once successfully installed run the wasm-bindgen CLI on the generated WebAssembly module.

$ wasm-bindgen target/wasm32-unknown-unknown/debug/hello_world.wasm --out-dir .

We instruct wasm-bindgen to generate the binding JavaScript for the generated WebAssembly module.

The --out-dir flag instructs the wasm-bindgen where to generate files. The files are generated in the current folder.

This generates the following files:

$ ls -lrta
76330 hello_world_bg.wasm
 1218 hello_world.js
  109 hello_world.d.ts
  190 hello_world_bg.d.ts

The wasm-bindgen CLI takes the WebAssembly module (the output of the cargo build) as input and generate the bindings. The size of the binding JavaScript file is around 1.2 KB. The hello_world.js does all the translations (that are required) between JavaScript and the WebAssembly modules.

The wasm-bindgen CLI along with the binding file generates the type definition file hello_world.d.ts.

The type definition file for the WebAssembly Module (hello_world.d.ts).

The rewritten WebAssembly module hello_world.wasm that takes advantage of the binding file.

The JavaScript binding file is enough for us to load and run the WebAssembly Module.

If you are using TypeScript, then the type definition will be helpful.

Inside the binding file

The binding file imports the WebAssembly module.

import * as wasm from './hello_world_bg.wasm'

Then we have the TextDecoder, to decode the String from the ArrayBuffer.

Since there are no input arguments available there is no need for TextEncoder (that is to encode the String from JavaScript into the shared memory).

The wasm-bindgen generates only the necessary functions inside the binding file. This makes the binding file as small as 1.2KB.

Modern browsers have built-in TextDecoder and TextEncoder support. The wasm-bindgen checks and uses them if they are available else it loads it using polyfill.

const lTextDecoder =
  typeof TextDecoder === 'undefined' ? require('util').TextDecoder : TextDecoder
let cachedTextDecoder = new lTextDecoder('utf-8')

The shared memory between JavaScript and the WebAssembly module need not be initialised every time. We initialise it once and use it across.

We have the following two methods to load the memory once and use it.

let cachegetInt32Memory0 = null
function getInt32Memory0() {
  if (
    cachegetInt32Memory0 === null ||
    cachegetInt32Memory0.buffer !== wasm.memory.buffer
  ) {
    cachegetInt32Memory0 = new Int32Array(wasm.memory.buffer)
  }
  return cachegetInt32Memory0
}

let cachegetUint8Memory0 = null
function getUint8Memory0() {
  if (
    cachegetUint8Memory0 === null ||
    cachegetUint8Memory0.buffer !== wasm.memory.buffer
  ) {
    cachegetUint8Memory0 = new Uint8Array(wasm.memory.buffer)
  }
  return cachegetUint8Memory0
}

The Rust code returns String to the JavaScript land. The String is passed via the shared memory.

The Shared memory is nothing but an ArrayBuffer. So we can need only the pointer to the offset (location where it is stored) and the length of the String to retrieve the String. Both the index of the location and the length are just numbers. They are passed from the WebAssembly land to JavaScript without any problem.

The following function is used for retrieving the String from the WebAssembly module:

function getStringFromWasm0(ptr, len) {
  return cachedTextDecoder.decode(getUint8Memory0().subarray(ptr, ptr + len))
}

• ptr is an index where the offset of the location.
• len is the length of the String.

Finally, we have the hello_world function.

/**
 * @returns {string}
 */
export function hello_world() {
  try {
    wasm.hello_world(8)
    var r0 = getInt32Memory0()[8 / 4 + 0]
    var r1 = getInt32Memory0()[8 / 4 + 1]
    return getStringFromWasm0(r0, r1)
  } finally {
    wasm.__wbindgen_free(r0, r1)
  }
}

The hello_world function is exported. We get the pointer and length from the shared memory buffer. Then pass the two numbers (r0, r1) to the getStringFromWasm function.

The getStringFromWasm function returns the String from the shared Array Buffer with ptr and len.

Once we received the output, we clear the allocated memory using wasm.__wbindgen_free(r0, r1).

cargo-expand

To understand what happens on the Rust side, let us use the cargo-expand command to expand the macro and see how the code is generated.

Note: Check here for how to install cargo expand. It is not mandatory for the course of this book. But they will help you understand what wasm-bindgen actually generates.

Open your terminal, go to the project's base directory and run cargo expand --target=wasm32-unknown-unknown > expanded.rs.

The above command generates expanded.rs.

The simple #[wasm_bindgen] annotation changes / adds the verbose part of exposing the function. All the necessary metadata that is required for the compiler to convert to WebAssembly module.

Note: Check out more about the internals of the #[wasm_bindgen] command here

The expanded.rs has the hello_world function.

pub fn hello_world() -> String {
    "Hello World".to_string()
}

The __wasm_bindgen_generated_hello_world function is an auto generated.

#[allow(non_snake_case)]
#[export_name = "hello_world"]
#[allow(clippy::all)]
pub extern "C" fn __wasm_bindgen_generated_hello_world(
) -> <String as wasm_bindgen::convert::ReturnWasmAbi>::Abi {
    let _ret = { hello_world() };
    <String as wasm_bindgen::convert::ReturnWasmAbi>::return_abi(_ret)
}

The #[export_name = "hello_world"] exports the function with the name hello_world.

The function returns <String as wasm_bindgen::convert::ReturnWasmAbi>::Abi. We will see more about this type in the later posts. But if you want to understand what happens here read this post.

The function returns the String in the format the binding JavaScript file (ptr and len).

Run it 🏃‍♂️

Instead of running them using local web server we can load and run the generated files we can use bundlers like Webpack or Parcel.

We will see more in detail about how these bundlers help in the later chapters.

For now let's see how to run and load the generated files:

Note the following setup is common and we will refer it as the "default" webpack setup in the future examples.

Create a webpack-config.js to configure Webpack how to handle files.

const path = require('path')
const HtmlWebpackPlugin = require('html-webpack-plugin')

module.exports = {
  entry: './index.js',
  output: {
    path: path.resolve(__dirname, 'dist'),
    filename: 'bundle.js',
  },
  plugins: [new HtmlWebpackPlugin()],
  mode: 'development',
}

This is a standard webpack configuration file with an HTMLWebpackPlugin. This plugin helps us to generate a default index.html rather than we create one.

Let us add a package.json file to bundle the dependencies for running the Webpack and scripts to run.

{
    "scripts": {
        "build": "webpack",
        "serve": "webpack-dev-server"
    },
    "devDependencies": {
        "html-webpack-plugin": "^3.2.0",
        "webpack": "^4.41.5",
        "webpack-cli": "^3.3.10",
        "webpack-dev-server": "^3.10.1"
    }
}

Create an index.js file to load the binding JavaScript that in turn loads the WebAssembly module generated.

import('./hello_world').then((module) => {
  console.log(module.hello_world())
})

Now head over to the terminal and then install the npm dependencies using.

$ npm install

Run the webpack-dev-server using

$ npm run serve

Go to the URL on which webpack-dev-server serves (defaults to http://localhost:8080) and open the developer console in the browser to see "Hello World" printed.

wasm-bindgen options

Let us take a look at the various options wasm-bindgen supports.

--out-dir - generates the file in a particular directory.

--out-name - set a custom filename.

wasm-bindgen has the following flags:

--debug

The --debug option includes extra debug information in the generated WebAssembly module. This will increase the size of the WebAssembly module. But it is useful in development.

--keep-debug

WebAssembly modules may or may not have custom sections (We will them in the later blogs). This custom section can be used to hold the debugging information. They will be helpful while debugging the application (like in-browser dev tools). This increases the size of the WebAssembly module. This is useful in development.

--no-demangle

This flag tells the wasm-bindgen not to demangle the Rust symbol names. Demangle helps the end-user to use the same name that they have defined in the Rust file.

--remove-name-section

This will remove the debugging name section of the file. We will see more about various sections in the WebAssembly module later. This will decrease the size of the WebAssembly module.

--remove-producers-section

WebAssembly modules can have a producer section. This section holds the information about how the file is produced or who produced the file.

By default, producer sections are added in the generated WebAssembly module. With this flag, we can remove it. It saves a few more bytes.

The wasm-bindgen provide options to generate the binding file for both Node.js and the browser environment. Let us see those flags.

--nodejs - Generates output that only works for Node.js. No ESModules. --browser - Generates output that only works for browser With ESModules. --no-modules- Generates output that only works for browser. No ESModules. Suitable for browsers that don't support ESModules yet.

The type definition files (*.d.ts) can be switched off by using --no-typescript flag.

Repo

Interested to explore more...

To know more about the custom section. Check out here

Check out more about webpack here

Check out this awesome blog post to know more about the ECMAScript modules.