onnxruntime gpu package for Aarch64?

[brainchip-india]

Describe the issue

I have a Jetson Orin Nano, I have a simple onnx model (matmul) which can run on Jetson CPU (A78) seamlessly.

However, if I want to test the same application with GPU will it work with the same onnxruntime library?

Because I see a separate GPU package for x86 Link, I am not sure it is the same case for Aarch64?

To reproduce

.

Urgency

Very urgent, Once I get this work, I have a big NLP model to work on the GPU.

Platform

Linux

OS Version

Linux server 5.15.148-tegra #1 SMP PREEMPT Tue Jan 7 17:14:38 PST 2025 aarch64 aarch64 aarch64 GNU/Linux

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.20.1

ONNX Runtime API

C++

Architecture

ARM64

Execution Provider

CUDA

Execution Provider Library Version

No response

[brainchip-india]

I tried the command mentioned in this query #11226

./build.sh --config Release --update --build --parallel --build_wheel --build_shared_lib --use_tensorrt --cuda_home /usr/local/cuda --cudnn_home /usr/lib/aarch64-linux-gnu/ --tensorrt_home /usr/lib/aarch64-linux-gnu/

However, I am seeing the below errors, and it appears that the issue is with the CUDA version. Currently, the CUDA version is 12+.

[ 61%] Built target onnxruntime_perf_test
gmake[1]: *** [CMakeFiles/Makefile2:2226: CMakeFiles/onnxruntime_providers_cuda.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
Traceback (most recent call last):
File "/home/snanduri/onnxruntime/tools/ci_build/build.py", line 2998, in
sys.exit(main())
^^^^^^
File "/home/snanduri/onnxruntime/tools/ci_build/build.py", line 2888, in main
build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
File "/home/snanduri/onnxruntime/tools/ci_build/build.py", line 1739, in build_targets
run_subprocess(cmd_args, env=env)
File "/home/snanduri/onnxruntime/tools/ci_build/build.py", line 867, in run_subprocess
return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/snanduri/onnxruntime/tools/python/util/run.py", line 49, in run
completed_process = subprocess.run(
^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.12/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '--build', '/home/snanduri/onnxruntime/build/Linux/Release', '--config', 'Release', '--', '-j6']' returned non-zero exit status 2.

Any help would be appreciated.

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.