tree 6cff4431a3aa5145d1b933becb48b5e4b0e87f51
parent a67fb36f2bf9e866b10ab4503c0139e1cffec393
author Benoit Jacob <jacob.benoit.1@gmail.com> 1781683539 -0400
committer GitHub <noreply@github.com> 1781683539 +0200
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsFcBAABCAAQBQJqMlVTCRC1aQ7uu5UhlAAAYMUQAGTALLvd5i1Hlthm2N12+v8c
 1KyEVJW+W+Zq1Z5yq6gBiG8deW9krR/VU86WCSooNjW5RUijlrwpIkwrGta220PW
 N4X6AhmCCckIw+z4+FqnvD0mD9ZTgCZs2n/AeIjNzTySltTs7Q+IeJRvezHxqi+1
 IjE8g/9qt7VT1QHsZq6PcSHEAhksZgx5R/hlteRYX//MwLK3e78US8+PRQxtT4M3
 mR6b/80LH1GfeDOmj+zvSswx4UBbjaxnj0QwOFl3fKN5ZUgpCZK0Hh11730Qvs5C
 lfc43QCcPj9V8xo1/GOSuXM0q9Zhra/0tfazJg1irJ3EHKtiqJZJqvbPrm9hPssn
 oevPmSaUQXU+BLLkg7rLwTCYSW9Y714IqVS8L4HB1a+W36GnAE0O3h5SJW6qeXpx
 8HVkbxSH02orsEdzGf8ljCyTWrFBX0ocWlis9BJkxrDkfxxjcRuMwHN5FfYlDEsr
 yY3oJVnHMoxscqpK39s5a9NJ2U6XKF92secNT1OlA4YITKfh4GqDI3it3LZWfFzc
 y6O5UM7q7R8fYECJeJt9L0GGrwxeRU2ms+vuW5WMiArOYwEOEdYqhQziEZsfMiN7
 jag7F6l1snpA/J6cTJy19FywpCtNLMgIWqljF3XRTzUyYrhtqDZMFJNJxu8rdHZh
 v4E8W/+jW5xDS11fK29n
 =Gbxt
 -----END PGP SIGNATURE-----
 

[Codegen][CPU] SelectUKernels for inner_tiled + --iree-llvmcpu-enable-llvm-ukernels. (#24569)

Wires the new C-bitcode ukernel framework into the LLVMCPU kernel-config
pipeline. Three pieces:

* `LLVMCPUSelectUKernels.{h,cpp}` — when the target config has
`llvm_ukernels = "inner_tiled"`, `selectUKernel` matches
`iree_codegen.inner_tiled` ops carrying a
`#iree_cpu.data_tiled_mma_layout` and returns a `UKernelDescriptorAttr`
whose name is the corresponding `MMAIntrinsic` enum value lowercased
with the `iree_uk_mma_` prefix (e.g.
`MMA_X86_AVX512BF16_1x16x2_F32_BF16` →
`iree_uk_mma_x86_avx512bf16_1x16x2_f32_bf16`). One ukernel per
intrinsic; `intrinsics_{m,n,k}` flow as function arguments and
specialize via bitcode LTO at the call site. Filters out `None` and the
`MMA_GENERIC_*` family — neither has a corresponding ukernel.

* `--iree-llvmcpu-enable-llvm-ukernels=<csv>` CL flag — bound to a new
`LLVMTarget.llvmUkernels` field, serialized into the target config as
`llvm_ukernels = "<csv>"`. Empty (the default) means off; passing
`inner_tiled` opts dispatches into the new framework. The flag name
leaves room for a future `mlir_ukernels`-style additional category (and
for now `inner_tiled` is also the only value). New helper
`hasLlvmUkernel(config, category)` in Codegen/Utils — distinct from the
existing `hasUkernel` to keep the legacy mmt4d ukernels and the new
LLVM-bitcode ukernels on independent enable flags.

* `KernelDispatch.cpp::setRootConfig(InnerTiledOp)` — calls
`selectUKernel` at the start and, on a match, sets the descriptor via
`setUKernelDescriptor`. Tile sizes and translation_info are unchanged:
the ukernel substitutes for the *innermost* intrinsic execution only,
the surrounding tile sizes and pipeline still hold. Bitcode resolution
is deferred to the `#iree_cpu.ukernel_provider` during
`LowerBitcodeUKernelsPass`, matching the GPU split between "match"
(here) and "attach" (provider).

Lit test (`select_ukernel.mlir`) runs
`iree-llvmcpu-select-lowering-strategy` on two `inner_tiled` ops sharing
the same MMA intrinsic but differing in target config: the one with
`llvm_ukernels = "inner_tiled"` picks up the
`iree_uk_mma_x86_avx512bf16_1x16x2_f32_bf16` descriptor; the other does
not, locking in the "off by default" property.

Progress towards https://github.com/iree-org/iree/issues/24574.

Signed-off-by: Benoit Jacob <jacob.benoit.1@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>