Skip to content

Allow EpContext models with input/output models completely in buffers #24463

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 18, 2025
+101 −15
Merged

Allow EpContext models with input/output models completely in buffers #24463

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
984d24b
Allow EpContext models with input/output models completely in buffers
adrianlizarraga Apr 18, 2025
bca3f75
Fix unit test check for specific error message
adrianlizarraga Apr 18, 2025
f12be7f
Remove unneeded declarations
adrianlizarraga Apr 18, 2025
6224f72
Allow embed mode disabled with all models in buffers
adrianlizarraga Apr 18, 2025
2207249
Free allocations in unit tests
adrianlizarraga Apr 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Remove unneeded declarations
  • Loading branch information
@adrianlizarraga
adrianlizarraga committed Apr 18, 2025
commit f12be7fc04ebb0b8535fb4c62e47615502aa34ab
11 changes: 4 additions & 7 deletions onnxruntime/test/providers/qnn/qnn_ep_context_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -392,23 +392,20 @@ TEST_F(QnnHTPBackendTests, CompileApi_FromSessionOptions_InputAndOutputModelsInB
CreateTestModel(BuildGraphWithQAndNonQ(false), 21, logging::Severity::kERROR, test_model);
std::string model_data = test_model.Serialize();

const ORTCHAR_T* output_model_file = ORT_TSTR("./qnn_context_binary_multi_partition_test.onnx");
std::filesystem::remove(output_model_file);

// Initialize session options with QNN EP
Ort::SessionOptions so;
ProviderOptions provider_options;
provider_options["backend_type"] = "htp";
provider_options["offload_graph_io_quantization"] = "0";
so.AppendExecutionProvider("QNN", provider_options);

// Create model compilation options from the session options.
Ort::ModelCompilationOptions compile_options(*ort_env, so);
compile_options.SetInputModelFromBuffer(reinterpret_cast<const void*>(model_data.data()), model_data.size());

Ort::AllocatorWithDefaultOptions allocator;
void* output_model_buffer = nullptr;
size_t output_model_buffer_size = 0;

// Create model compilation options from the session options.
Ort::ModelCompilationOptions compile_options(*ort_env, so);
compile_options.SetInputModelFromBuffer(reinterpret_cast<const void*>(model_data.data()), model_data.size());
compile_options.SetOutputModelBuffer(allocator, &output_model_buffer, &output_model_buffer_size);
compile_options.SetEpContextEmbedMode(true);

Expand Down
Loading