tree e162e7d5487773bfeec4acb5c1e9f3ac1acc8c3a
parent 61a1f2e1c7958915bc54bf2a0c477701d46a895c
author Scott Todd <scott.todd0@gmail.com> 1711487082 -0700
committer GitHub <noreply@github.com> 1711487082 -0700
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsFcBAABCAAQBQJmAzhqCRC1aQ7uu5UhlAAAJYcQAEsp5KzcVEBroXmLZA8+nGal
 s8xIs/YEVS3Fpz9WLnfuOYqNR2Slm/TH/NH+apw/xxiEQ63oA/SoTu3JhQoebVPd
 +X+HN2Myz+ec4Oj7iEy7WD84AZs+ryC8KsQv2Z+3LJ0zKNdoM1r1SvD1fK6eKBsf
 +bS3YjlEem4k96FOJClT3JeK91/z3p1aZjISY0MdZMAMHimJz98qxH5qi6M/1gmF
 BuWuSKVGzG9bdxeFU3RtXA67wEi0fiRD2KgCmQDhKBCvTwLVLHpt+1Wufc8sDH8O
 q74OroWI9jpg1fiH9Vnarn8qSFBO+FDKxD96S8k5DWKI6lNNN6YItjqdo43GrCGm
 SZenb23mFqVGlk/PyuwQ/87zp5J2gVW5AyiXtNfMmbURb2miIXnyJ0gVLtecaB5r
 HFOjjZqkczKnKe0tCVl8V71R3td4UdoZjIb917K2bdN3l1xYhSVcUq7r562pNIEL
 g+dG2KEQMKAFJNemH/NydJMBFFLlgHN+u8hVp4FsRePw+Afc1PuidM3xgmwQqd8i
 wo/J9ekhSTwLx4otJGn/dhDklIom54VtNYdH2BzxcYzq/ghdb9NvJsJsjL+ck4Uf
 ib5iPgOOJszF0eaHFLua1JCVGqOeUkbdjOMG1ow7z3WYIerCT1phxok8qJ4A2gAo
 3E3PSnZ6iWGl5TynF+1t
 =ku/X
 -----END PGP SIGNATURE-----
 

Start testing real weight models from external test suite. (#16801)

See https://github.com/nod-ai/SHARK-TestSuite/pull/103 for the changes
to the external test suite itself, which this pulls in.

The test suite now has cases like
[`pytorch/models/resnet50/`](https://github.com/nod-ai/SHARK-TestSuite/tree/main/iree_tests/pytorch/models/resnet50)
which include:
* `resnet50.mlirbc` (150 KB, stored in the repo using git LFS)
* `splats.irpa` (72 KB, stored in the repo using git LFS)
* `test_cases.json` with metadata pointing at `real_weights.irpa` (195
MB) and input/output `.bin` or `.npy` files
* flagfiles for splats (running but not checking results, since they are
0s) and real weights (running and checking results)

Starting with just resnet50 and opt-125M, but I have SDXL models (unet,
clip, vae) just about ready to go in too.

Now IREE
1. Runs the test cases in the `onnx/` folder using one config file (with
flags to use and XFAIL lists) and set of pytest options (max
parallelism)
2. Downloads the remote files from Azure for "real weight" PyTorch tests
3. Runs the test cases in the `pytorch/` folder using a different config
file (local-task instead of local-sync, empty XFAIL list) and set of
pytest options (no parallelism)

ci-exactly:
build_packages,regression_test_amdgpu_vulkan,regression_test_cpu