tree 2b83d7b063ea61664c8bf627350e9e28efae94d5
parent c1ecdbbb81bfaa05376257367877553ae8aca6e2
author Zmicier Prybysh <zprybysh@baylibre.com> 1786465384 +0200
committer GitHub <noreply@github.com> 1786465384 +0000
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsFcBAABCAAQBQJqe0xoCRC1aQ7uu5UhlAAArJoQAIfponLK7iregkYWNUYXfAU3
 Hlv0IehvC7BCppLcfqKiZP8ORLA+Y+W+YmElBbh1IABJjIsoBPFN81nozC6ya11R
 tdqaJqXwXhKd/Hx0Lu0OVfnrx/u2yMFWuTmqz9KMUPLyqpThpB0xtM9dFhr9v01w
 Mep/IGBozdbMdMgocy7gqP7818Zy4tXDX78IoOPppnwoSLV9WA1v0PcNZd2dmYnf
 6x1Yk1tXwrwuhYewpmFkt+vHV7908VyPCEi/vTBsGAbuhbtIdmMd0nXoDoz+HxFy
 d+B44hi3WqZWneDcpYzHArR7D579jc3QzZymiavbnSXjiY9ngpKNAbvcGnofxJMX
 1Y87/rlHofplIrrvW/iUPG1E3giaUficqiB2wN60kTTl4c01spXHU23XqD7Cwe4m
 X8igp2nd73y+XgUal91DAVU3TA/YtIHYzCJnwQ/x52BwPWrN6IuEodQVsgTegupf
 XJ6IfHmoWcNEoaNQKuFrLpbKYo8Pom4WIRVG9uY8dnspInqwGJ9keMyE5byuBoXj
 D7iJutqqO5PbCLLLlbjGd5s7cNEZoT87NXlioREH0maqP6iqYzD01x9cG6Pwg79L
 Ayd6HB5MBWpote9iu3pf8OVTILXvLRZtK7OGo7mfXt3whpLIiBUB1FTzK7atg4Wm
 uxuUsb5ak3DR3yu4TZBt
 =N2Fx
 -----END PGP SIGNATURE-----
 

[LLVMCPU] Use native bf16 converts when the target supports them (#24759)

## Summary

The bf16 arithmetic is getting promoted to f32 in codegen, which is
correct for every CPU (there's no CPU with native bf16 arithmetic yet,
RISC-V `Zvfbfmin` provides conversions only). The conversions around
that promotion, however, are always expanded in software. That prevents
targets that do have bf16 <-> f32 conversion instructions from selecting
them, costing ~6 integer ops per narrow and 2 per widen on every bf16
load and store.

Introduce another LLVMCPULoweringPipeline option: it's set to true on
RISC-V targets with both `Zfbfmin` and `Zvfbfmin` extensions. Both are
required, since with only the vector extension scalar bf16 residue
lowers to `__truncsfbf2` libcalls. bf16 loads/stores and the promotion's
extf/truncf then select as `vfwcvtbf16/vfncvtbf16` (vector) and
`fcvt.s.bf16/fcvt.bf16.s` (scalar). This option is passed to
`ConvertUnsupportedFloatToIntBuffers` and `arith-expand` passes.

## Testing

Measured on a SpaceMiT K3 (X100 cores) (yay, i have access to one now).

Tested on kernels: 1M-element bf16 kernels, native vs current expansion,
outputs bit-identical in every case:

| kernel | native | expanded | speedup |
| :-- | --: | --: | --: |
| elementwise add | 0.800 ms | 1.45 ms | 1.81× |
| f32 to bf16 cast | 0.650 ms | 1.01 ms | 1.55× |
| sigmoid-weighted elementwise chain | 7.83 ms | 8.31 ms | 1.06× |
| bf16 to f32 cast | 0.494 ms | 0.538 ms | 1.09× |

Also tested on whole models: a bf16 `whisper-tiny-en` encoder-decoder
runs in **3.580 s vs 3.936 s (1.10×)** with bit-identical output against
PyTorch reference. A small CLIP-style bf16 model compiles to a 7.5%
smaller vmfb.

---------

Signed-off-by: Zmicier Prybysh <zprybysh@baylibre.com>