Round up sdxl golden dispatch sizes and mi250 times by 10%. (#17879)
See [this discussion on
Discord](https://discord.com/channels/689900678990135345/689957613152239638/1261059802214305913).
These are change detector tests with _some_ use, but exact checking is
too noisy and is causing significant churn across PRs and postsubmit CI
runs. We could change the test script to allow a certain error range,
but for now just increase the thresholds used in `<=` checks by about
10% (then drop a few digits to 0 so we don't give the appearance of high
precision).
ci-exactly: build_packages,regression_test
diff --git a/.github/workflows/pkgci_regression_test.yml b/.github/workflows/pkgci_regression_test.yml
index 4a75540..98a6e2b 100644
--- a/.github/workflows/pkgci_regression_test.yml
+++ b/.github/workflows/pkgci_regression_test.yml
@@ -307,21 +307,24 @@
--durations=0 \
--config-files=${SDXL_VAE_CONFIG_FILE_PATH}
+ # Note: mi250 benchmark times are more lenient than mi300 (allowing about
+ # 10% deviation from observed averages), since the mi250 runners we use
+ # are more unstable and we care most about peak performance on mi300.
- name: "Running SDXL rocm pipeline benchmark (mi250)"
if: contains(matrix.name, 'rocm_mi250_gfx90a')
run: |
source ${VENV_DIR}/bin/activate
pytest SHARK-TestSuite/iree_tests/benchmarks/sdxl/benchmark_sdxl_rocm.py \
- --goldentime-rocm-e2e-ms 1340.0 \
- --goldentime-rocm-unet-ms 342.0 \
- --goldentime-rocm-clip-ms 17.5 \
- --goldentime-rocm-vae-ms 300.0 \
+ --goldentime-rocm-e2e-ms 1450.0 \
+ --goldentime-rocm-unet-ms 370.0 \
+ --goldentime-rocm-clip-ms 18.5 \
+ --goldentime-rocm-vae-ms 315.0 \
--goldendispatch-rocm-unet 1714 \
--goldendispatch-rocm-clip 1569 \
--goldendispatch-rocm-vae 248 \
- --goldensize-rocm-unet-bytes 2073609 \
- --goldensize-rocm-clip-bytes 783848 \
- --goldensize-rocm-vae-bytes 764909 \
+ --goldensize-rocm-unet-bytes 2280000 \
+ --goldensize-rocm-clip-bytes 860000 \
+ --goldensize-rocm-vae-bytes 840000 \
--gpu-number 6 \
--rocm-chip gfx90a \
--log-cli-level=info \
@@ -341,9 +344,9 @@
--goldendispatch-rocm-unet 1714 \
--goldendispatch-rocm-clip 1569 \
--goldendispatch-rocm-vae 248 \
- --goldensize-rocm-unet-bytes 2065289 \
- --goldensize-rocm-clip-bytes 783848 \
- --goldensize-rocm-vae-bytes 765869 \
+ --goldensize-rocm-unet-bytes 2270000 \
+ --goldensize-rocm-clip-bytes 860000 \
+ --goldensize-rocm-vae-bytes 840000 \
--gpu-number 0 \
--rocm-chip gfx942 \
--log-cli-level=info \
