One fairly revealing microbenchmark for WASM runtimes is `int8` dot products & angular/cosine distances.
(My) NumKong [1] has implementations targeting both vanilla AVX2/Haswell and AVX2-VNNI/Alder Lake, which makes it easy to see where runtimes and code generators leave performance on the table.
I started a few Wasmtime/Cranelift PRs around this, but didn’t manage to finish them :facepalm: Might be a fun weekend project for someone interested in backend/codegen work.
It'd be nice to see memory use. Ideally base usage, then cost of running 1 of a thing, then cost of running 50 if a thing.
Personally low my 50% speed vs native does not sound like the worst to me. I've spent years using perl, java (in v4-v7 days), node... These have various ways they are less good than native, and we collectively seem to accept that. Ideally yes wasm gets faster but a 2x is pretty ok!
Judging by the slow speed of node here, I wonder if it was run without forcing optimizations. It doesn't do OSR (on-stack replacement), meaning that a benchmark script doesn't get a chance to tier up from the baseline compiler to the fully optimized tier.
Running with
node --no-liftoff
avoids this problem.
(This is not an issue on the Web, where code must return to the event loop anyhow, allowing tiering up to work, and node is optimized for that kind of workflow.)