-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Fix layout transformer for FusedConv #24169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Has you done some parity test for such FusedConv conversion? My understanding is that FusedConv might have activation, but Conv does not support that. I noticed that there was NhwcFusedConv: onnxruntime/onnxruntime/core/optimizer/nhwc_transformer.cc Lines 102 to 103 in a8fb786
That could impact the layout transformer as well. |
Specifically, the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FusedConv is not in the list of GetCUDALayoutSensitiveOps for CUDA:
onnxruntime/onnxruntime/core/optimizer/layout_transformation/layout_transformation.cc
Lines 35 to 48 in 1ef3044
const std::unordered_set<std::string_view>& GetCUDALayoutSensitiveOps() { | |
static std::unordered_set<std::string_view> cuda_nhwc_ops = []() { | |
return std::unordered_set<std::string_view>{ | |
"BatchNormalization", | |
"Conv", | |
"ConvTranspose", | |
"GlobalMaxPool", | |
"MaxPool", | |
"GlobalAveragePool", | |
"AveragePool", | |
"GridSample", | |
"DepthToSpace", | |
"SpaceToDepth", | |
"LRN"}; |
So this change shall be fine for CUDA.
### Description Fix layout transformer for FusedConv. The current layout transformer will transform `FusedConv` (kMSDomain) into `FusedConv` (kMSInternalNHWCDomain) if the EP wants channels_last. However, kMSInternalNHWCDomain uses OpType `Conv` for both Conv and FusedConv, so `FusedConv` (kMSInternalNHWCDomain) is invalid (unregistered op). This PR fixes this and allows layout transformer change `FusedConv` (kMSDomain) into `Conv` (kMSInternalNHWCDomain). ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. -->
Description
Fix layout transformer for FusedConv.
The current layout transformer will transform
FusedConv
(kMSDomain) intoFusedConv
(kMSInternalNHWCDomain) if the EP wants channels_last.However, kMSInternalNHWCDomain uses OpType
Conv
for both Conv and FusedConv, soFusedConv
(kMSInternalNHWCDomain) is invalid (unregistered op).This PR fixes this and allows layout transformer change
FusedConv
(kMSDomain) intoConv
(kMSInternalNHWCDomain).Motivation and Context