[QNN EP] Enable QnnGpu backend in QNN EP. #24435
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
/azp run Linux QNN CI Pipeline,Windows x64 QNN CI Pipeline,Windows ARM64 QNN CI Pipeline,Linux Android Emulator QNN CI Pipeline |
|
Azure Pipelines successfully started running 4 pipeline(s). |
HectorSVC
reviewed
Apr 15, 2025
Description Enable the GPU backend also for the onnxruntime QNN EP. Motivation and Context Why is this change required? What problem does it solve? It allows QNN EP to run on the GPU backend also. With this change many models can now run fully on QNN EP GPU backend, like resnet_50, google_vit_base_fp32, squeezenet1.0-7 etc. Also the onnxruntime node tests and versioned operator tests pass numbers for the GPU is comparable to the HTP now. Note: Currently QNN_LOG_LEVEL_DEBUG need to be enabled to run correctly.
ce75ea8 to
5cdbae7
Compare
|
/azp run Big Models,Linux Android Emulator QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CUDA CI Pipeline,Windows GPU DML CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows x64 QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI |
|
/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, MacOS CI Pipeline, ONNX Runtime Web CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline, Linux QNN CI Pipeline,Linux OpenVINO CI Pipeline |
|
Azure Pipelines successfully started running 3 pipeline(s). |
|
Azure Pipelines successfully started running 6 pipeline(s). |
|
/azp run Windows GPU Doc Gen CI Pipeline |
|
Commenter does not have sufficient privileges for PR 24435 in repo microsoft/onnxruntime |
|
@HectorSVC Windows GPU doc gen failure looks like a network issue. If it is a required pass, can you rerun please. In fact, all the pipeline failures looks unrelated to this change. |
adrianlizarraga
added a commit
that referenced
this pull request
Apr 21, 2025
### Description Excludes QnnGpu.dll from Windows x64 NuGet package because it is not available for that architecture. ### Motivation and Context Fix failure in QNN packaging pipeline: ```shell CreateNativePackage: Generating nuspec for the native Microsoft.ML.OnnxRuntime.QNN nuget package... python ..\tools\nuget\generate_nuspec_for_native_nuget.py --package_version 1.22.0-dev-20250421-0439-2abab8d --package_name Microsoft.ML.OnnxRuntime.QNN --target_architecture x64 --build_config RelWithDebInfo --native_build_path D:\a\_work\1\b\RelWithDebInfo\RelWithDebInfo --packages_path D:\a\_work\1\b\packages --ort_build_path D:\a\_work\1\b --sources_path D:\a\_work\1\s --commit_id 2abab8d --is_release_build False --execution_provider None --nuspec_name NativeNuget.nuspec 1 file(s) copied. 1 file(s) copied. nuspec_name: NativeNuget.nuspec Bundling native shared library artifacts into Microsoft.ML.OnnxRuntime nuget package... nuget pack NativeNuget.nuspec Attempting to build package from 'NativeNuget.nuspec'. ##[error]EXEC(0,0): Error NU5019: File not found: 'D:\a\_work\1\b\RelWithDebInfo\RelWithDebInfo\QnnGpu.dll'. EXEC : error NU5019: File not found: 'D:\a\_work\1\b\RelWithDebInfo\RelWithDebInfo\QnnGpu.dll'. [D:\a\_work\1\s\csharp\OnnxRuntime.CSharp.proj] ##[error]csharp\OnnxRuntime.CSharp.proj(109,5): Error MSB3073: The command "nuget pack NativeNuget.nuspec" exited with code 1. ``` Introduced by this PR: #24435
ashrit-ms
pushed a commit
that referenced
this pull request
Apr 24, 2025
### Description Enable the GPU backend also for the onnxruntime QNN EP. ### Motivation and Context Why is this change required? What problem does it solve? It allows QNN EP to run on the GPU backend also. With this change many models can now run fully on QNN EP GPU backend, like resnet_50, google_vit_base_fp32, squeezenet1.0-7 etc. Also the onnxruntime node tests and versioned operator tests pass numbers for the GPU is comparable to the HTP now. Note: Currently QNN_LOG_LEVEL_DEBUG need to be enabled to run correctly.
intbf
pushed a commit
to intbf/onnxruntime
that referenced
this pull request
Apr 25, 2025
### Description Excludes QnnGpu.dll from Windows x64 NuGet package because it is not available for that architecture. ### Motivation and Context Fix failure in QNN packaging pipeline: ```shell CreateNativePackage: Generating nuspec for the native Microsoft.ML.OnnxRuntime.QNN nuget package... python ..\tools\nuget\generate_nuspec_for_native_nuget.py --package_version 1.22.0-dev-20250421-0439-2abab8d --package_name Microsoft.ML.OnnxRuntime.QNN --target_architecture x64 --build_config RelWithDebInfo --native_build_path D:\a\_work\1\b\RelWithDebInfo\RelWithDebInfo --packages_path D:\a\_work\1\b\packages --ort_build_path D:\a\_work\1\b --sources_path D:\a\_work\1\s --commit_id 2abab8d --is_release_build False --execution_provider None --nuspec_name NativeNuget.nuspec 1 file(s) copied. 1 file(s) copied. nuspec_name: NativeNuget.nuspec Bundling native shared library artifacts into Microsoft.ML.OnnxRuntime nuget package... nuget pack NativeNuget.nuspec Attempting to build package from 'NativeNuget.nuspec'. ##[error]EXEC(0,0): Error NU5019: File not found: 'D:\a\_work\1\b\RelWithDebInfo\RelWithDebInfo\QnnGpu.dll'. EXEC : error NU5019: File not found: 'D:\a\_work\1\b\RelWithDebInfo\RelWithDebInfo\QnnGpu.dll'. [D:\a\_work\1\s\csharp\OnnxRuntime.CSharp.proj] ##[error]csharp\OnnxRuntime.CSharp.proj(109,5): Error MSB3073: The command "nuget pack NativeNuget.nuspec" exited with code 1. ``` Introduced by this PR: microsoft#24435 Signed-off-by: bfilipek <bartlomiej.filipek@intel.com>
vraspar
pushed a commit
that referenced
this pull request
Apr 28, 2025
### Description Excludes QnnGpu.dll from Windows x64 NuGet package because it is not available for that architecture. ### Motivation and Context Fix failure in QNN packaging pipeline: ```shell CreateNativePackage: Generating nuspec for the native Microsoft.ML.OnnxRuntime.QNN nuget package... python ..\tools\nuget\generate_nuspec_for_native_nuget.py --package_version 1.22.0-dev-20250421-0439-2abab8d --package_name Microsoft.ML.OnnxRuntime.QNN --target_architecture x64 --build_config RelWithDebInfo --native_build_path D:\a\_work\1\b\RelWithDebInfo\RelWithDebInfo --packages_path D:\a\_work\1\b\packages --ort_build_path D:\a\_work\1\b --sources_path D:\a\_work\1\s --commit_id 2abab8d --is_release_build False --execution_provider None --nuspec_name NativeNuget.nuspec 1 file(s) copied. 1 file(s) copied. nuspec_name: NativeNuget.nuspec Bundling native shared library artifacts into Microsoft.ML.OnnxRuntime nuget package... nuget pack NativeNuget.nuspec Attempting to build package from 'NativeNuget.nuspec'. ##[error]EXEC(0,0): Error NU5019: File not found: 'D:\a\_work\1\b\RelWithDebInfo\RelWithDebInfo\QnnGpu.dll'. EXEC : error NU5019: File not found: 'D:\a\_work\1\b\RelWithDebInfo\RelWithDebInfo\QnnGpu.dll'. [D:\a\_work\1\s\csharp\OnnxRuntime.CSharp.proj] ##[error]csharp\OnnxRuntime.CSharp.proj(109,5): Error MSB3073: The command "nuget pack NativeNuget.nuspec" exited with code 1. ``` Introduced by this PR: #24435
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Enable the GPU backend also for the onnxruntime QNN EP.
Motivation and Context
Why is this change required? What problem does it solve?
It allows QNN EP to run on the GPU backend also.
With this change many models can now run fully on QNN EP GPU backend, like resnet_50, google_vit_base_fp32, squeezenet1.0-7 etc. Also the onnxruntime node tests and versioned operator tests pass numbers for the GPU is comparable to the HTP now.
Note: Currently QNN_LOG_LEVEL_DEBUG need to be enabled to run correctly.