Building highly efficient JavaScript packages with dart2wasm

Turns out JavaScript is the one thing you don't want in a JavaScript package.

For a project at work, I recently had to build a small tool running a transformation on YAML files. The context doesn't matter too much here, but this involved:

1. Parsing a collection of SQL statements from a YAML file.
2. Analyzing and transforming their AST.
3. Dumping them into a YAML file of a slightly different format, preserving comments of unaffected regions in the source file.

Because this originally started out as a small prototype, Dart was an obvious choice for me. That arguably doesn't say very much because Dart is already my default language for everything, but the strengths of the Dart ecosystem were particularly obvious here:

• I wrote the sqlparser package which was very useful here and I'm obviously already familiar with it. There are a number of SQL parsers for JavaScript, but they're not as feature-complete and the ones I've found can't resolve identifiers with SQL scoping rules, for instance.
• The yaml_edit package is awesome and maintained by the Dart team! It can apply edits to yaml files while preserving comments and whitespace in unrelated keys. I couldn't find anything that comes close to it in the JS ecosystem.

I've built the transformer as a Dart library, wrapped it in a tiny jaspr app for others to try out and called it a day.

A couple days later, we realized we actually want to embed this in an existing React-based web site and a Node.js CLI application, so I looked into ways to make the library usable from JavaScript (since it would have been really annoying to rewrite the whole thing). Now, writing JavaScript libraries in Dart is not a new idea. The canonical implementation of Sass is famously written in Dart and compiled to JavaScript, for instance. Dart has three different compilers worth considering here:

1. dartdevc, which compiles to modular (including ESM) JavaScript and supports hot reloads, but doesn't do any optimizations.
2. dart2js, which compiles to JavaScript and can exploit Dart's sound type system to aggressively minimize and optimize Dart apps far beyond the few optimizations possible for JS/TS sources.
3. dart2wasm, a new-ish compiler emitting a WebAssembly module and a small helper script to inject web APIs as wasm externs.

All of these are great for compiling applications, but they're not really designed to compile libraries. For instance, they require a main method and will minify and remove methods not called from main, so our library methods would just get removed. Sass uses a clever trick by having a small loader script define an object in the global scope before invoking the main function of their Dart app which installs Dart methods as JavaScript properties on that object. This works, but IMO a well-designed ESM module shouldn't pollute globalThis.

After looking into dart2wasm a bit more recently, I remembered that it has experimental support for options that will be very helpful here! Adding a wasm:export pragma to a method will export it from the compiled module, and Dart's tree-shaker will take those exports into account.

The YAML rewriter is a fairly simple module: It exposes a single function taking a string as an input and returning the translated YAML as an output string. Using these export pragmas and some magic to work around strings being passed as externrefs to Dart, the library is essentially structured like this:

import 'dart:js_interop';
import 'dart:_wasm';

import 'package:my_dart_library/my_dart_library.dart' as impl;

void main() {
  // Not called, we just want the convert export.
}

@pragma('wasm:export', 'convert')
WasmExternRef? convert(WasmExternRef arg) {
  // Go from externref -> boxed JavaScript value -> Dart String
  final input = (arg.toJS as JSString).toDart;
  final dartOutput = impl.rewriteYaml(input);
  // Go from Dart String -> boxed JavaScript value -> externref
  return externRefForJSAny(dartOutput.toJS);
}

Compiling this currently requires a compiler flag: dart compile wasm -O3 -E --enable-experimental-wasm-interop index.dart. Of course, complex JavaScript types can also be consumed and returned with the usual dart:js_interop APIs and extension types.

This package, which contains a SQL parser capable of parsing and analyzing every structure supported by SQLite and a YAML parser plus patching tool, compiles to a 300kB (uncompressed) WebAssembly module and a 16kB (uncompressed, before minification) support library. That's not tiny, but the state-of-the-art our JavaScript friends are used to is to compile Hello World React apps into 200kb minified bundles, so Dart is absurdly efficient in comparison.

To make this consumable from JavaScript, I've added a package.json exposing the WebAsssembly module along with a small wrapper + TypeScript definitions. The package specification basically looks like this:

{
  // ... name and other metadata ...
  "type": "module",
  "files": ["lib"],
  "scripts": {
    "build": "dart compile wasm -O3 -E --enable-experimental-wasm-interop index.dart -o lib/compiled.wasm"
  },
  "exports": {
    ".": {
      "import": {
        "types": "./lib/index.d.ts",
        "default": "./lib/index.js"
      }
    },
    "./compiled.wasm":  "./lib/compiled.wasm"
  }
}

index.js is a small wrapper around the helper .mjs file emitted by dart2wasm:

import { compile, compileStreaming } from "./compiled.mjs";

export async function instantiate(source) {
  let compiled;
  if (ArrayBuffer.isView(source) || source instanceof ArrayBuffer) {
    compiled = await compile(source);
  } else {
    compiled = await compileStreaming(source);
  }

  const instantiated = await compiled.instantiate();
  instantiated.invokeMain();
  const exports = instantiated.instantiatedModule.exports;
  return {
    rewrite: exports.rewrite,
  };
}

Along with a matching index.d.ts (I've written mine manually, most of the types are derived from Dart signatures anyway so the tsc compiler doesn't help), JavaScript consumers can load the WASM module like this when using vite as a bundler:

import { instantiate } from "js_package";
import wasmUrl from "js_package/compiled.wasm?url";

const module = await instantiate(fetch(wasmUrl));
console.log(module.rewrite(`...`));

For Node.js targets, meta.import.resolve gives them a file to load and instantiate too.

Overall, I think this went very well! While the project itself is fairly small, it uses helper libraries that aren't. Compared to the JavaScript ecosystem, Dart is a very comfortable language to work with. Personally, I strongly prefer Dart's sound type system instead of the pretend-types from TypeScript. Dart is much better at dead-code eliminiation, it can also remove methods from classes for example which most JavaScript optimizers can't. This, and the fact that Dart has a reasonable standard library, means there's less incentive to write and depend on thousands of tiny libraries, which seems to be a normalized thing in the JavaScript ecosystem.

For this project, sqlparser is the only dependency not maintained by the Dart team. I happen to maintain that one so it's fine for me, but in general Dart gives you a very solid foundation for writing apps and libraries. And with dart2wasm, you can turn these Dart packages into small, efficient and dependency-free JavaScript packages for the wider ecosystem to consume.