[Web] How to use JSEP and WebGPU in static library (missing jsepAlloc or jsepInit)

[sevagh]

Describe the issue

Hello,

I have C++ code that uses the ONNXRuntime CXX API which I compile against static onnxruntime wasm (--build_wasm_static_lib - full build commands are shown at the bottom). I link against the static lib in my CMakeLists.txt file like so:

set(ONNX_RUNTIME_WASM_LIB_WEBGPU ${CMAKE_SOURCE_DIR}/../build/build-ort-wasm/Release/libonnxruntime_webassembly.a)

When building with onnxruntime only using simd, my code works fine. When building with webgpu support through jsep
and using the js execution provider in the C++ code:

// set provider to WebGPU via "JsExecutionProvider"
session_options.AppendExecutionProvider("JS", {});

This is the javascript webworker code which imports the wasm module:

function loadWASMModule() {
    // Check if WebGPU is supported
    const webGPUSupported = isWebGPUSupported();
    let scriptToImport = "";
    //const scriptToImport = webGPUSupported ? 'demucs_onnx_webgpu.js' : 'demucs_onnx_simd.js';
    if (webGPUSupported) {
        console.log("WebGPU is supported, using WebGPU module");
        scriptToImport = "demucs_onnx_webgpu.js";
    } else {
        console.log("WebGPU is not supported, using SIMD module");
        scriptToImport = "demucs_onnx_simd.js";
    }

I get an error when using my code, saying Module.jsepAlloc is not a function. This is printed by my wasm module in the Chrome developer console:

WebGPU is supported, using WebGPU module
demucs_onnx_webgpu.js:9 Using JsExecutionProvider
demucs_onnx_webgpu.js:9 JsExecutionProvider
demucs_onnx_webgpu.js:9 Uncaught (in promise) TypeError: Module.jsepAlloc is not a function
    at 738632 (demucs_onnx_webgpu.js:9:11657)
    at runEmAsmFunction (demucs_onnx_webgpu.js:9:89035)
    at _emscripten_asm_const_ptr (demucs_onnx_webgpu.js:9:89275)
    at demucs_onnx_webgpu.wasm:0x2877e6

I believe it's because I'm using the static lib which is missing the jsepInit step we find in the file ort-wasm-simd.jsep.mjs is built with the non-static --build_wasm. I have also read this guide https://gist.github.com/fs-eire/a55b2c7e10a6864b9602c279b8b75dce

I tried to include pre-jsep.js in my link flags:

LINK_FLAGS "${COMMON_LINK_FLAGS} -s USE_WEBGPU=1 \
-s ALLOW_MEMORY_GROWTH=1 -s INITIAL_MEMORY=2048mb -s MAXIMUM_MEMORY=4gb \
--pre-js \"${ONNXRUNTIME_ROOT}/wasm/pre-jsep.js\""
)

It's probably missing other ts or js files. Any advice? @fs-eire sorry to ping you directly but you seem to be the WebGPU/JSEP implementor

Build commands:

Simd no webgpu:

python ./vendor/onnxruntime/tools/ci_build/build.py \
    --build_dir="./build/build-ort-wasm-simd" \
    --config=MinSizeRel \
    --build_wasm_static_lib \
    --parallel \
    --minimal_build \
    --disable_ml_ops \
    --disable_rtti \
    --use_preinstalled_eigen \
    --eigen_path=$(realpath "./vendor/eigen") \
    --skip_tests \
    --skip_onnx_tests \
    --enable_wasm_simd \
    --enable_wasm_exception_throwing_override \
    --disable_exceptions \
    --include_ops_by_config="./onnx-models/required_operators_and_types.config" \
    --enable_reduced_operator_type_support
;;

Webgpu jsep:

python ./vendor/onnxruntime/tools/ci_build/build.py \
    --build_dir="./build/build-ort-wasm-webgpu" \
    --config=Release \
    --build_wasm_static_lib \
    --parallel \
    --use_preinstalled_eigen \
    --eigen_path=$(realpath "./vendor/eigen") \
    --skip_tests \
    --skip_onnx_tests \
    --enable_wasm_simd \
    --use_jsep \
    --include_ops_by_config="./onnx-models/required_operators_and_types.config" \
    --enable_reduced_operator_type_support

To reproduce

Write custom C++ code using ORT CXX API, build onnxruntime WASM static lib with jsep and webgpu support, you might get a Module.jsepAlloc is not a function

Urgency

No response

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

deee480

Execution Provider

'webgpu' (WebGPU)

[fs-eire]

JSEP does not work in ONNX Runtime WebAssembly static lib, because it depends on the inter-op between JS and C++ specifically defined for the JavaScript library.

We are currently working on a C++ implemented WebGPU EP, which is technically able to work in wasm static lib. It is still being worked on.

[sevagh]

Thanks!

[sevagh]

Is there an alternative, for example if I use webgpu without JSEP? But this involves another component called "dawn"

[fs-eire]

if you are building C++ targeting web (wasm), then your application should not depend the native implementation of dawn. instead, you should use the dawn's fork of Emscripten in order to compile the code that uses WebGPU API.

However, as I explained above, we've not get to there yet. We will work on it but since that involves quite code and build script changes it may take a while.

[elliotsayes]

Thanks for the clear explanation @fs-eire, do you happen to have a rough estimate of when this is expected to be completed?

[fs-eire]

it's about 3-4 months or so

[raphaelmenges]

Looking forward for WebGPU being available as execution runner in the Emscripten environment! May I ask how is it possible that WebGPU is already available in onnxruntime-web?

PS: My running assumption is that --use_jsep creates the execution providers for the Web, which, however, are not available in C++/Emscripten. Is that correct?

[sevagh]

Is this available now? #23364 (comment)

[fs-eire]

Is this available now? #23364 (comment)

Please track #23697

[sevagh]

Not to be a bother, but it looks like this got more complicated than originally anticipated (judging by the PRs made that are relevant to WebGPU/CXX API).

I'm wondering if I should instead focus on re-thinking my design to be able to use ORT's own javascript wrapper (with webgpu support) vs waiting for the ability to do a custom CXX WASM+WebGPU compilation.

[fs-eire]

There are a lot of changes ongoing, including upstreams (dawn/emsdk). If you can use JavaScript, it would not be a bad idea to use ORT JavaScript API because even if we replace JSEP by WebGPU EP in future the JavaScript API is expected to still work.

If you want to use WebGPU EP in WASM, it is functional working today in ORT web. However there are a few performance issues and I am working on it for a while. If you want to try I can help.

[sevagh]

Latest main branch should work for WebGPU EP or do I need a PR branch? Let me test it.

[sevagh]

Using this latest commit: d58ff6b

I can compile static onnxruntime with these flags:

        python "${ORT_PATH}/tools/ci_build/build.py" \
            --build_dir="./build/demucs-build-ort-wasm-webgpu" \
            --config=MinSizeRel \
            --build_wasm_static_lib --enable_wasm_simd \
            --parallel \
            --skip_tests --skip_onnx_tests \
            --use_webgpu \
            --enable_wasm_exception_throwing_override  --disable_exceptions --disable_rtti \
            --minimal_build extended \
            --include_ops_by_config="./ort-models/demucs/required_operators_and_types.config" \
            --enable_reduced_operator_type_support
            #--use_jsep \
            #--compile_no_warning_as_error \
            #--use_external_dawn \

Result:

66M libonnxruntime_webassembly.a

Then, I compile my CXX code which links against this library:

[100%] Built target demucs_onnx_simd
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: emwgpuInstanceRequestAdapter
wasm-ld: error: lto.tmp: undefined symbol: emwgpuAdapterRequestDevice
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: wgpuDeviceGetAdapterInfo
wasm-ld: error: lto.tmp: undefined symbol: wgpuDeviceGetFeatures
wasm-ld: error: lto.tmp: undefined symbol: emwgpuWaitAny
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDeviceCreateShaderModule
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: lto.tmp: undefined symbol: emwgpuDelete
wasm-ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)

My cmake target:

set(ONNX_RUNTIME_WASM_LIB_WEBGPU ${CMAKE_SOURCE_DIR}/../build/demucs-build-ort-wasm-webgpu/MinSizeRel/libonnxruntime_webassembly.a)

add_executable(demucs_onnx_webgpu ${WEBGPU_SOURCES})
target_link_libraries(demucs_onnx_webgpu ${ONNX_RUNTIME_WASM_LIB_WEBGPU})
if(CMAKE_BUILD_TYPE STREQUAL "Release")
    target_compile_definitions(demucs_onnx_webgpu PRIVATE ORT_NO_EXCEPTIONS=1)
    target_compile_options(demucs_onnx_webgpu PRIVATE -fno-exceptions)
else()
    # add -sNO_DISABLE_EXCEPTION_CATCHING  to the link flags
    set(COMMON_LINK_FLAGS "${COMMON_LINK_FLAGS} -s NO_DISABLE_EXCEPTION_CATCHING=1 -s ASSERTIONS=2")
endif()
target_link_libraries(demucs_onnx_webgpu zlibstatic)
set_target_properties(demucs_onnx_webgpu PROPERTIES
    LINK_FLAGS "${COMMON_LINK_FLAGS} -s USE_WEBGPU=1 -s ALLOW_MEMORY_GROWTH=1 -s INITIAL_MEMORY=2048mb -s MAXIMUM_MEMORY=4gb"
)

[fs-eire]

you also need to include the following files:

• <build_folder>_deps\dawn-src\third_party\emdawnwebgpu\library_webgpu.js
  use with --js-library <path-to-library_webgpu.js>
• <build_folder>_deps\dawn-src\third_party\emdawnwebgpu\webgpu.cpp
  add to source

Eventually these should be appended to ONNX Runtime Cmake file, but for now please add them as a workaround. I will do this change when I got some time

[sevagh]

With the additional complication of needing to append struct_info and enum_table to the js file, your instructions seemed to work:

set(DAWN_WEBGPU_STRUCT_INFO_JS ${CMAKE_SOURCE_DIR}/../build/demucs-build-ort-wasm-webgpu/${WEBGPU_BUILD_FOLDER}/_deps/dawn-build/gen/src/emdawnwebgpu/library_webgpu_generated_struct_info.js)
set(DAWN_WEBGPU_ENUM_TABLES_JS ${CMAKE_SOURCE_DIR}/../build/demucs-build-ort-wasm-webgpu/${WEBGPU_BUILD_FOLDER}/_deps/dawn-build/gen/src/emdawnwebgpu/library_webgpu_enum_tables.js)

Now, I do this:

        Ort::SessionOptions session_options;

        std::vector<std::string> providers = Ort::GetAvailableProviders();

        for (auto prov : providers) {
            callWriteWasmLog(prov.c_str());
        }

        // set provider to WebGPU via "JsExecutionProvider"
        session_options.AppendExecutionProvider("WebGPU", {});

        callWriteWasmLog("WebGPU provider set");

There is no abort or exception or anything thrown, but the code silently returns after AppendExecutionProvider:

demucs_onnx_webgpu.js:895 WebGpuExecutionProvider
demucs_onnx_webgpu.js:895 CPUExecutionProvider
demucs_onnx_webgpu.js:5850 WebGPU is experimental on this platform. See https://github.com/gpuweb/gpuweb/wiki/Implementation-Status#implementation-status

So the part that prints "WebGPU is experimental on this platform" is this part of the js file of my wasm module, line 5850:

  WebGPU.Internals.futureInsert(futureId, navigator["gpu"]["requestAdapter"](opts).then(adapter => {
    if (adapter) {
      WebGPU.Internals.jsObjectInsert(adapterPtr, adapter);
      _emwgpuOnRequestAdapterCompleted(futureId, 1, adapterPtr, 0);
    } else {
      var sp = stackSave();
      var messagePtr = stringToUTF8OnStack("WebGPU not available on this browser (requestAdapter returned null)");
      _emwgpuOnRequestAdapterCompleted(futureId, 3, adapterPtr, messagePtr);
      stackRestore(sp);
    }
  }, ex => {
    var sp = stackSave();
    var messagePtr = stringToUTF8OnStack(ex.message);
    _emwgpuOnRequestAdapterCompleted(futureId, 4, adapterPtr, messagePtr);
    stackRestore(sp);
  }));
}

So, I'm not sure what's happening inside AppendExecutionProvider("WebGPU", {}) that's not throwing an exception but the rest of the code is not executing.

I've compiled it currently with -s ASSERTIONS=2 to print exceptions.

[fs-eire]

Ah I see. You need to enable either Asyncify or JSPI settings in emscripten, because some webgpu APIs are async. (eg. requestAdapter)

See https://emscripten.org/docs/porting/asyncify.html

[sevagh]

Update: false alarm, session.Run is not exiting successfully.

old:
OK, I went back and removed --minimal_build (this is probably not playing well with WebGPU) and the inference for my neural network actually completed.

The results are totally mangled, so I have to dig deeper into host/device tensors etc. (my old code was written 100% around the CPU provider), but this is huge progress 🥳

Thanks for your help!

[sevagh]

Error is here:

stem-worker.js:1 Error in processAudio: RuntimeError: Aborted(Assertion failed)
    at abort (:8788/demucs_onnx_webgpu.js:644:41)
    at assert (:8788/demucs_onnx_webgpu.js:229:5)
    at Object.getJsObject (:8788/demucs_onnx_webgpu.js:4650:5)
    at makeEntry (:8788/demucs_onnx_webgpu.js:5979:28)
    at makeEntries (:8788/demucs_onnx_webgpu.js:5999:20)
    at _wgpuDeviceCreateBindGroup (:8788/demucs_onnx_webgpu.js:6006:16)
    at imports.<computed> (:8788/demucs_onnx_webgpu.js:6187:20)
    at :8788/demucs_onnx_webgpu.wasm
    at :8788/demucs_onnx_webgpu.wasm
    at :8788/demucs_onnx_webgpu.wasm

Assert fails, follow the chain here:
createBindGroup:

function _wgpuDeviceCreateBindGroup(devicePtr, descriptor) {
  devicePtr >>>= 0;
  descriptor >>>= 0;
  assert(descriptor);
  assert(SAFE_HEAP_LOAD((((descriptor) >>> 2) >>> 0) * 4, 4, 1) === 0);
  function makeEntry(entryPtr) {
    assert(entryPtr);
    var bufferPtr = SAFE_HEAP_LOAD(((((entryPtr) + (8)) >>> 2) >>> 0) * 4, 4, 1);
    var samplerPtr = SAFE_HEAP_LOAD(((((entryPtr) + (32)) >>> 2) >>> 0) * 4, 4, 1);
    var textureViewPtr = SAFE_HEAP_LOAD(((((entryPtr) + (36)) >>> 2) >>> 0) * 4, 4, 1);
    assert((bufferPtr !== 0) + (samplerPtr !== 0) + (textureViewPtr !== 0) === 1);
    var binding = SAFE_HEAP_LOAD(((((entryPtr) + (4)) >>> 2) >>> 0) * 4, 4, 1);
    if (bufferPtr) {
      var size = readI53FromI64((entryPtr) + (24));
      if (size == -1) size = undefined;
      return {
        "binding": binding,
        "resource": {
          "buffer": WebGPU.getJsObject(bufferPtr),
          "offset": (SAFE_HEAP_LOAD((((((entryPtr + 4)) + (16)) >>> 2) >>> 0) * 4, 4, 1) * 4294967296 + SAFE_HEAP_LOAD(((((entryPtr) + (16)) >>> 2) >>> 0) * 4, 4, 1)),
          "size": size
        }
      };
    } else if (samplerPtr) {
      return {
        "binding": binding,
        "resource": WebGPU.getJsObject(samplerPtr)
      };
    } else {
      return {
        "binding": binding,
        "resource": WebGPU.getJsObject(textureViewPtr)
      };
    }
  }
  function makeEntries(count, entriesPtrs) {
    var entries = [];
    for (var i = 0; i < count; ++i) {
      entries.push(makeEntry(entriesPtrs + 40 * i));
    }
    return entries;
  }
  var desc = {
    "label": WebGPU.makeStringFromOptionalStringView(descriptor + 4),
    "layout": WebGPU.getJsObject(SAFE_HEAP_LOAD(((((descriptor) + (12)) >>> 2) >>> 0) * 4, 4, 1)),
    "entries": makeEntries(SAFE_HEAP_LOAD(((((descriptor) + (16)) >>> 2) >>> 0) * 4, 4, 1), SAFE_HEAP_LOAD(((((descriptor) + (20)) >>> 2) >>> 0) * 4, 4, 1))
  };
  var device = WebGPU.getJsObject(devicePtr);
  var ptr = _emwgpuCreateBindGroup(0);
  WebGPU.Internals.jsObjectInsert(ptr, device.createBindGroup(desc));
  return ptr;
}

To the assert higher up in the js wrapper:

var WebGPU = {
  Internals: {
    jsObjects: [],
    jsObjectInsert: (ptr, jsObject) => {
      WebGPU.Internals.jsObjects[ptr] = jsObject;
    },
    bufferOnUnmaps: [],
    futures: [],
    futureInsert: (futureId, promise) => {
      WebGPU.Internals.futures[futureId] = new Promise(resolve => promise.finally(() => resolve(futureId)));
    }
  },
  getJsObject: ptr => {
    if (!ptr) return undefined;
    assert(ptr in WebGPU.Internals.jsObjects);
    return WebGPU.Internals.jsObjects[ptr];
  },

If I print the webgpu tracked js objects:

WebGPU State: {trackedObjects: 561, objectKeys: Array(561)}
stem-worker.js:1 The entire WebGPU object: {Int

Also, in the beginning, it seems many of the operations, layers, or kernels are not supported in WebGPU ep:

2025-04-13 15:27:18.223495 [I:onnxruntime:, inference_session.cc:594 TraceSessionOptions] Session Options {  execution_mode:0 execution_order:DEFAULT enable_profiling:0 optimized_model_filepath:"" enable_mem_pattern:1 enable_mem_reuse:1 enable_cpu_mem_arena:1 profile_file_prefix:onnxruntime_profile_ session_logid: session_log_severity_level:0 session_log_verbosity_level:0 max_num_graph_transformation_steps:10 graph_optimization_level:3 intra_op_param:OrtThreadPoolParams { thread_pool_size: 0 auto_set_affinity: 0 allow_spinning: 1 dynamic_block_base_: 0 stack_size: 0 affinity_str:  set_denormal_as_zero: 0 } inter_op_param:OrtThreadPoolParams { thread_pool_size: 0 auto_set_affinity: 0 allow_spinning: 1 dynamic_block_base_: 0 stack_size: 0 affinity_str:  set_denormal_as_zero: 0 } use_per_session_threads:1 thread_pool_allow_spinning:1 use_deterministic_compute:0 config_options: {  } }
demucs_onnx_webgpu.js:1382 2025-04-13 15:27:18.224890 [I:onnxruntime:, inference_session.cc:414 operator()] Flush-to-zero and denormal-as-zero are off
demucs_onnx_webgpu.js:1382 2025-04-13 15:27:18.225090 [I:onnxruntime:, inference_session.cc:422 ConstructorCommon] Creating and using per session threadpools since use_per_session_threads_ is true
demucs_onnx_webgpu.js:1382 2025-04-13 15:27:18.225415 [I:onnxruntime:, inference_session.cc:440 ConstructorCommon] Dynamic block base set to 0
demucs_onnx_webgpu.js:1382 2025-04-13 15:27:18.390075 [I:onnxruntime:, inference_session.cc:783 RegisterExecutionProvider] Having memory pattern enabled is not supported while using WebGpuExecutionProvider. So disabling it for this session since it uses WebGpuExecutionProvider.
demucs_onnx_webgpu.js:1382 2025-04-13 15:27:18.390535 [I:onnxruntime:, inference_session.cc:1810 Initialize] Initializing session.
demucs_onnx_webgpu.js:1382 2025-04-13 15:27:18.390755 [I:onnxruntime:, inference_session.cc:1847 Initialize] Adding default CPU execution provider.
:8788/demucs_onnx_webgpu.js:1382 2025-04-13 15:27:18.410705 [I:onnxruntime:, webgpu_execution_provider.cc:794 GetCapability] webgpu kernel not found in registries for Op type: InstanceNormalization node name: /encoder.0/dconv/layers.0/layers.0.1/InstanceNormalization
:8788/demucs_onnx_webgpu.js:1382 2025-04-13 15:27:18.411135 [I:onnxruntime:, webgpu_execution_provider.cc:794 GetCapability] webgpu kernel not found in registries for Op type: InstanceNormalization node name: /encoder.0/dconv/layers.0/layers.0.4/InstanceNormalization
:8788/demucs_onnx_webgpu.js:1382 2025-04-13 15:27:18.411595 [I:onnxruntime:, webgpu_execution_provider.cc:794 GetCapability] webgpu kernel not found in registries for Op type: InstanceNormalization node name: /encoder.0/dconv/layers.1/layers.1.1/InstanceNormalization
demucs_onnx_webgpu.js:1382 2025-04-13 15:27:18.411940 [I:onnxruntime:, webgpu_execution_provider.cc:794 GetCapability] webgpu kernel not found in registries for Op type: InstanceNormalization node name: /encoder.0/dconv/layers.1/layers.1.4/InstanceNormalization
demucs_onnx_webgpu.js:1382 2025-04-13 15:27:18.412635 [I:onnxruntime:, webgpu_execution_provider.cc:794 GetCapability] webgpu kernel not found in registries for Op type: InstanceNormalization node name: /encoder.1/dconv/layers.0/layers.0.1/InstanceNormalization
put_char @ demucs_onnx_webgpu.js:1382

[fs-eire]

could you share the reproduce step?

[sevagh]

I will try to distill a MWE.

Does it matter to you to have access to the ONNX format, or is ORT enough?

I could create a small repo with:

• ORT model
• required_types_and_operators.config
• wasm module and js (with compile steps) that give me the above error in chrome with webgpu

If you consent, I would make it a private repo and invite you to it

[fs-eire]

the ONNX format model is better for debugging.

If the error is in _wgpuDeviceCreateBindGroup, it's probably not even need the full model to reproduce the issue. So, if possible, please use public repo because it will help for tracking and reference in future. (maybe you can use a public model to reproduce the issue?)

If you still find it difficult to make a public sharable repo, you can make a private repo and invite me.