commit ce92024f0c16faa4c77cb35c144d4c65f156c3e2
author Zhuoran Yin <zhuoryin@amd.com> Thu Aug 28 16:57:21 2025 -0400
committer GitHub <noreply@github.com> Thu Aug 28 20:57:21 2025 +0000
tree bff79caf2e0e990232f56a076f738a57bcc3deef
parent 31404c6e0bbf746aa5a79a85a62088f56186b8a3

[Codegen][GPU] Adding new heuristics to take all dimensions into account when distributing tiles (#21803)

This motivation of this PR is to address the multi-dimension
distribution situation in convolution codegen.

A sample convolution config that wouldn't distribute properly looks like
the following:
> convbfp16 -n 16 -c 768 -H 48 -W 32 -k 2048 -y3 -x 3 -p 1 -q 1 -u 1 -v
1 -l 1 -j 1 -m conv -g 1 -F 1 -t 1 ​

There are 3 dimension in M: [16, 48, 32]. There's one dimension in N:
[256]. Since N's last dimension is much larger than M's last dimension,
the current algorithm will yield an extremely imbalanced tile that
allocate all subgroup and tiles to N dimension, causing a small memory
bound workgroup.

The new tile allocation algorithm can prevent the problem by considering
the entire aggregated M and N dimension together and find optimal
balanced tile for the full scope. Then it will attempt to allocate the
full allocated tile to each sub-dimension. This yields a much more
reasonably distributed tiles. With the new algorithm, it will improve
the performance of this convolution from 5000us -> 1500us, and 5% of
performance among all 478 convolutions.

I'm pushing this to review as I gather gemm and model perf. Likely since
this has little impact with M/N problems that have a single dimension,
the performance should stay flat. I'll be posting perf updates as
follow-up comments soon.

---------

Signed-off-by: jerryyin <zhuoryin@amd.com>