Update with `softrvv` workflow
Softrvv workflow is very different from the prior, with much fewer
lines of code entailed when adding a new op to test-suite.
This also includes documentation around using the team's tracker/sign-up
spreadsheet for the different vector-op-unit-tests.
Change-Id: I4afc3d768080bcffd7b8484643ddf12cbbb3c12f
diff --git a/Create_Springbok_Vector_Unit_Test.md b/Create_Springbok_Vector_Unit_Test.md
index afc9aa5..e3fe647 100644
--- a/Create_Springbok_Vector_Unit_Test.md
+++ b/Create_Springbok_Vector_Unit_Test.md
@@ -2,14 +2,16 @@
* [Step 0: Repo Sync and Build Prequisties](#step-0:-repo-sync-and-build-prequisties)
* [Step 1: Select Instruction and Update Tracker](#step-1:-select-instruction-and-update-tracker)
-* [Step 2: Go to the vector testing directory, and select a template](#step-2:-go-to-the-vector-testing-directory,-and-select-a-template)
-* [Step 3: Create Demo Test and Macro Test](#step-3:-create-demo-test-and-macro-test)
-* [Step 4: Add Test to CMakeLists.txt](#step-4:-add-test-to-cmakelists.txt)
-* [Step 5: Build springbok](#step-5:-build-springbok)
-* [Step 6: Run Qemu Tests](#step-6:-run-qemu-tests)
-* [Step 7: Run Renode Tests](#step-7:-run-renode-tests)
-* [Step 8: Mark Tests as Passing](#step-8:-mark-tests-as-passing)
-* [Step 9: Disable demo test](#step-9:-disable-demo-test)
+* [Step 2: Create operation definition in `${ROOTDIR}/sw/vec/softrvv/include`](#step-2:-create-operation-definition-in-`${rootdir}/sw/vec/softrvv/include`)
+* [Step 4: Add the header to `softrvv.h`](#step-4:-add-the-header-to-`softrvv.h`)
+* [Step 5: Add main test entry within the CMakeLists.txt](#step-5:-add-main-test-entry-within-the-cmakelists.txt)
+* [Step 6: Create subtest](#step-6:-create-subtest)
+ * [Add brief softrvv test](#add-brief-softrvv-test)
+ * [Add this smaller test to a 2nd Cmake file](#add-this-smaller-test-to-a-2nd-cmake-file)
+* [Step 7: Build and Run Tests](#step-7:-build-and-run-tests)
+* [Step 8 (optional): view generated code](#step-8-(optional):-view-generated-code)
+* [Step 9: Mark Tests as Passing](#step-9:-mark-tests-as-passing)
+* [Step 10: Git Commit](#step-10:-git-commit)
## Step 0: Repo Sync and Build Prequisties
@@ -18,6 +20,7 @@
m renode_clean renode
m qemu_clean qemu
```
+
## Step 1: Select Instruction and Update Tracker
Go to the following spreadsheet and write or select one's name under the `DRI - Unit Test`.
@@ -30,121 +33,147 @@
Side Note: The following pivot table is very useful for quickly scanning through the instructions available:
[DRI Pivot Table](https://docs.google.com/spreadsheets/d/1MVh0eQjdKkPiBOCXysfuLWWkFiL3gWzqvk-xA8-1rNg/edit#gid=1165644093)
-## Step 2: Go to the vector testing directory, and select a template
+## Step 2: Create operation definition in `${ROOTDIR}/sw/vec/softrvv/include`
-The easiest way to get started will be to look at a test that should be similar to the test one is planning to write.
-
-These tests are located at:
+Add definition of the operation as a file `softrvv_<op>h.h` in the following folder, where `<op>` is your operation (e.g. `vadd`, `vor`, `vmerge`, etc...):
```sh
-cd ${ROOTDIR}/sw/vec/tests
+${ROOTDIR}/sw/vec/softrvv/include
```
-Then change the names in the template to match one's current vector instruction of interest.
-
-## Step 3: Create Demo Test and Macro Test
-
-
-There is a demo test written in C++, which serves as a sample of the testing strategy, a convenient way to develop before making a macro version, as well as a place to debug and step through need be.
-
-Demo test will look like the following:
+This test will contain the definition of the operation (e.g. `vadd` operation below):
```cpp
-TEST_F(VaddViTest, vadd_vi_demo) {
+template <typename T>
+void vadd_vi(T *dest, T *src1, T src2, int32_t avl) {
+ for (int32_t idx = 0; idx < avl; idx++) {
+ dest[idx] = src1[idx] + src2;
+ }
+}
+
```
-One possible workflow is to develop the demo test, then proceed through the following steps to make sure it builds and passes in Qemu(proceeding straight to the next step), and then developing the macro version and ensuring it builds/passes as well.
+See [softrvv_vadd.h](https://spacebeaker.googlesource.com/shodan/sw/vec/+/refs/heads/master/softrvv/include/softrvv_vadd.h) for a reference.
+## Step 4: Add the header to `softrvv.h`
-Note: To disable a demo test, prefix the test with `DISABLED_` as shown:
+Add the header file as an includes in the `softrvv.h`:
+
```cpp
-TEST_F(VaddViTest, DISABLED_vadd_vi_demo) {
+#include "softrvv_vadd.h"
```
-## Step 4: Add Test to CMakeLists.txt
+` softrvv.h ` is in the `${ROOTDIR}/sw/vec/softrvv/include` directory.
-Edit the CmakeLists.txt (path follows):
+
+## Step 5: Add main test entry within the CMakeLists.txt
+
+
+`cd` into the following folder, and edit the `CMakeLists.txt` file there
```sh
${ROOTDIR}/sw/vec/tests/CMakeLists.txt
```
-Include a new `vec_cc_test` at the bottom of the file:
+Add an entry there with the opcode formats relavant to the operation:
+
+```cmake
+vec_cc_generated_test(
+ NAME
+ vadd
+ OPFMT
+ OPIVV
+ OPIVX
+ OPIVI
+ LINKOPTS
+ -Xlinker --defsym=__itcm_length__=200K
+)
+```
+
+Note: this operation uses OPIVV, OPIVX, and OPIVI formats.
+
+A range of tests will be be autogenerated for each one of the OPFMT's added.
+
+The Mako templates for each OPFMT test can be found at `${ROOTDIR}/sw/vec/tests/templates`.
+
+Note on `__itcm_length__`: keeping this number low for each test helps our overall performance, most tests are good with 128K, some might need a little more memory (`ctest` will let you know if you need to add more here).
+
+## Step 6: Create subtest
+
+Please also add a small manual test the new softvv instruction.
+Steps delineated as follows:
+
+### Add brief softrvv test
+
+```sh
+cd ${ROOTDIR}/sw/vec/softrvv/tests
+```
+
+Then add a test-file named `softrvv_<op>_test.cpp`.
+
+See other files in the same folder as references.
+
+In this test-file you'll define a few simple test cases to run on the defined functions:
+
+See for example this VV test from [softrvv_vadd_test.cpp](https://spacebeaker.googlesource.com/shodan/sw/vec/+/refs/heads/master/softrvv/tests/softrvv_vadd_test.cpp):
+
+```cpp
+uint32_t src1[] = {1, 2, 3, 4, 5};
+uint32_t src2[] = {1, 2, 3, 4, 5};
+
+uint32_t ref_vv[] = {2, 4, 6, 8, 10};
+
+TEST_F(SoftRvvVaddTest, VV) {
+ softrvv::vadd_vv<uint32_t>(dest, src1, src2, AVL_CONST);
+ ASSERT_EQ(memcmp(dest, ref_vv, sizeof(dest)), 0);
+}
+```
+
+### Add this smaller test to a 2nd Cmake file
+
+Edit another CMakeLists.txt file (path below):
+
+```sh
+${ROOTDIR}/sw/vec/softrvv/tests/CMakeLists.txt
+```
+
+Add the subtest just created:
```cmake
vec_cc_test(
NAME
- vadd_test
+ softrvv_vadd
SRCS
- vadd_vi_test.cpp
- vadd_vx_test.cpp
- vadd_vv_test.cpp
+ softrvv_vadd_test.cpp
+ DEPS
+ softrvv
LINKOPTS
-Xlinker --defsym=__itcm_length__=128K
- TIMEOUT
- 20
-)
```
-Name should be the root type of the test (e.g. as above `vadd_test` which covers `vadd.vi`, `vadd.vx`, and `vadd.vv`).
+## Step 7: Build and Run Tests
-Sources to be included in the test go under the `SRCS` section.
+Tests must be run from the `${ROOTDIR}/out/springbok` directory.
-Note: Most tests will only need around 20 seconds and require less than 128K, but some tests may require more, we need to play this by ear.
+`cd ${ROOTDIR}/out/springbok`
-## Step 5: Build springbok
-
-Check and address any build errors:
+Next, using Regex in the quoted section, select qemu, renode, and/or other operations
+to include in a test run:
```sh
-m springbok
+m springbok && ctest --verbose -R ".*vadd.*" --gtest_color=yes
```
-## Step 6: Run Qemu Tests
+## Step 8 (optional): view generated code
-We're using Qemu as our baseline for out tests, unit tests should at minimum pass in Qemu.
-If a test passes in Qemu, and not in Renode (whose vector support is WIP), it may indicate a problem in Renode's vector implementation.
-
-`cd` into the following directory:
+Code for the main test was autogenerated, however by `cd`'ing into the following directory, one can inspect the generated code.
```sh
-${ROOTDIR}/out/springbok
-```
-Then use the following to start running through all of the tests from `CMakeLists.txt` file earlier:
-
-```sh
-ctest --verbose -R qemu --gtest_color=yes
+${ROOTDIR}/out/springbok/tests/generated/
```
-For convenience, one might include the building/testing in a single line:
-
-```sh
-cd ${OUT}/springbok && m springbok && ctest --verbose -R qemu --gtest_color=yes
-```
-
-Note: The `-R` flag filters tests which match a regular expression, in this case we're filtering on qemu tests, but more specific filters are possble:
-
-For example, if you had wanted only to test the `vadd_tests` in qemu:
-
-```sh
-cd ${OUT}/springbok && m springbok && ctest --verbose -R qemu_vadd --gtest_color=yes
-```
-
-## Step 7: Run Renode Tests
-
-Renode tests won't necessarily pass, many of these are still in progress on the Antmicro side, but please do log success/fail of the Renode tests on the tracker:
-
-```sh
-ctest --verbose -R renode --gtest_color=yes
-```
-Running a specific test can be done (to save time), example below for the `vadd_test` group:
-
-```sh
-ctest --verbose -R renode_vadd --gtest_color=yes
-```
-
-## Step 8: Mark Tests as Passing
+## Step 9: Mark Tests as Passing
Note: at minimum, one's test should be expected to pass in Qemu. Since Renode vec support is WIP, a unit test failing may indicate problem in the Renode implementation.
@@ -154,10 +183,11 @@
2. Mark whether the unit test passes in Renode
3. Mark the instruction "Springbok Unit Test" as "implemented"
-## Step 9: Disable demo test
+## Step 10: Git Commit
-When done testing, disable the demo test by prefixing the demo test name with "`DISABLED_`":
+At this point there are two commitable chunks:
-```cpp
-TEST_F(VaddViTest, DISABLED_vadd_vi_demo) {
-```
+1. operation definition, softrvv.h change, and CMakeLists.txt entry.
+2. the manual test and its CMakeLists.txt entry
+
+These may be added as either one CL, or two CL's separating these two contributions.
