[Build] Build incorrectly assumes that AVX-VNNI is a core part of AVX2

[stephenswat]

Describe the issue

PR #20687 introduced the AVXVNNI instruction set extension as a core AVX2 feature:

onnxruntime/cmake/onnxruntime_mlas.cmake

Line 656 in c18e06d

message(STATUS "Using -mavx2 -mfma -mavxvnni flags")

However, AVXVNNI is not a core feature of AVX2 at all. AVXVNNI is not actually supported until AMD Zen 5 (2024) while the actual AVX2 instruction set has been supported by AMD since 2015.

Urgency

This issue breaks compilation on virtually all AMD CPUs.

Target platform

AMD CPUs before Zen 5; Intel CPUs before Alder Lake

Build script

N/A.

Error / output

N/A.

Visual Studio Version

No response

GCC / Compiler Version

No response

[fs-eire]

@microsoft/onnxruntime-mlas could you please help to take a look?

[tianleiwu]

It basically means enable both AVX2 and AVX-VNNI in compiling when GCC > 11.

The code does check that whether CPU supports VNNI, so VNNI kernel will not be launched in un-supported CPU:

onnxruntime/onnxruntime/core/mlas/lib/platform.cpp

Lines 426 to 433 in c18e06d

	if ((Cpuid7_1[0] & 0x10) != 0) {
	this->GemmU8U8Dispatch = &MlasGemmU8S8DispatchAvx2;
	this->GemmU8S8Kernel = MlasGemmU8S8KernelAvxVnni;
	this->GemvU8S8Kernel = MlasGemvU8S8KernelAvxVnni;
	this->ConvSymU8S8Dispatch = &MlasConvSymDispatchAvxVnni;
	this->QNBitGemmDispatch = &MlasSQNBitGemmDispatchAvx2vnni;
	}

@stephenswat, Could you describe the detail build error or runtime error you encountered?

[paulgessinger]

@tianleiwu I observed this build error. On current main with this build command:

./build.sh \
    --config MinSizeRel \
    --cmake_extra_defines CMAKE_INSTALL_PREFIX=$PWD/install \
    --build_shared_lib \
    --build_dir build \
    --skip_tests \
    --parallel 1

I get this:

2025-04-18 11:34:50,688 build [INFO] - cmake --build build/MinSizeRel --config MinSizeRel
[  0%] Built target gen_onnx_proto
[  0%] Built target gen_onnx_data_proto
[  0%] Built target gen_onnx_operators_proto
[  1%] Built target onnx_proto
[  2%] Built target flatbuffers
[  5%] Built target flatc
[  6%] Built target onnxruntime_flatbuffers
[  8%] Built target onnxruntime_common
[  8%] Building CXX object CMakeFiles/onnxruntime_mlas.dir/tmp/test_build_onnx/onnxruntime/onnxruntime/core/mlas/lib/sqnbitgemm_kernel_avx2.cpp.o
/tmp/ccxgpl2N.s: Assembler messages:
/tmp/ccxgpl2N.s:6094: Error: unsupported instruction `vpdpbusds'

// Lots more with different line numbers.

/tmp/ccxgpl2N.s:9919: Error: unsupported instruction `vpdpbusds'
gmake[2]: *** [CMakeFiles/onnxruntime_mlas.dir/build.make:987: CMakeFiles/onnxruntime_mlas.dir/tmp/test_build_onnx/onnxruntime/onnxruntime/core/mlas/lib/sqnbitgemm_kernel_avx2.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:416: CMakeFiles/onnxruntime_mlas.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2

Full build log is here: https://termbin.com/st5bw

Seems to me like the assembler on AMD fails to compile these instructions.

[tianleiwu]

From the build log:

-- CMAKE_CXX_COMPILER_ID: GNU
-- CMAKE_CXX_COMPILER_VERSION: 11.5.0
-- Using -mavx2 -mfma -mavxvnni flags
CMakeFiles/onnxruntime_mlas.dir/tmp/test_build_onnx/onnxruntime/onnxruntime/core/mlas/lib/sqnbitgemm_kernel_avx2.cpp.o
/tmp/ccxgpl2N.s: Assembler messages:
/tmp/ccxgpl2N.s:6094: Error: unsupported instruction `vpdpbusds'

So -mavx2 -mfma -mavxvnni is used in compiling sqnbitgemm_kernel_avx2.cpp. It shall allow vpdpbusds AVX VNNI instruction which is supported in GCC 11.x.

Could you try upgrade GCC to 12 (our build pipeline use 12.x) or 13 (In my machine, GCC 13.2 has no such issue)?

[paulgessinger]

@tianleiwu It seems to be working for me on GCC13, although I can't say I understand where the differences comes from.