Initial gen_test_matrix commit. (#6126)
This uses a new test configuration file and corresponding generator program to produce a matrix of test runners for all possible combinations. These are then globbed in to the build system and run in bulk.
For CMake, we generate the tests at configure time. For internal bazel, we will just generate them when we import into the tree, since Bazel is too inflexible to do anything like this.
This gives us:
* A file that I can run for each permutation and will let me iterate on the test locally.
* XFAIL semantics: tests fail if they start passing but were marked XFAIL.
I re-worked the TF math directory for this and true'd it up to what our current support matrix is.
Landing as a POC for further iteration.
Co-authored-by: Scott Todd <scotttodd@google.com>
diff --git a/build_tools/cmake/iree_python.cmake b/build_tools/cmake/iree_python.cmake
index 355f5a3..5940e0a 100644
--- a/build_tools/cmake/iree_python.cmake
+++ b/build_tools/cmake/iree_python.cmake
@@ -242,6 +242,8 @@
# ARGS: Command line arguments to the Python source file.
# LABELS: Additional labels to apply to the test. The package path is added
# automatically.
+# GENERATED_IN_BINARY_DIR: If present, indicates that the srcs have been
+# in the CMAKE_CURRENT_BINARY_DIR.
function(iree_py_test)
if(NOT IREE_BUILD_TESTS)
return()
@@ -249,12 +251,18 @@
cmake_parse_arguments(
_RULE
- ""
+ "GENERATED_IN_BINARY_DIR"
"NAME;SRCS"
"ARGS;LABELS"
${ARGN}
)
+ # Switch between source and generated tests.
+ set(_SRC_DIR "${CMAKE_CURRENT_SOURCE_DIR}")
+ if(_RULE_GENERATED_IN_BINARY_DIR)
+ set(_SRC_DIR "${CMAKE_CURRENT_BINARY_DIR}")
+ endif()
+
iree_package_name(_PACKAGE_NAME)
set(_NAME "${_PACKAGE_NAME}_${_RULE_NAME}")
@@ -271,7 +279,7 @@
COMMAND
"${CMAKE_SOURCE_DIR}/build_tools/cmake/run_test.${IREE_HOST_SCRIPT_EXT}"
"${Python3_EXECUTABLE}"
- "${CMAKE_CURRENT_SOURCE_DIR}/${_RULE_SRCS}"
+ "${_SRC_DIR}/${_RULE_SRCS}"
${_RULE_ARGS}
INSTALLED_COMMAND
python
