[doc] document styling rules for chip-level test library code
This partially addresses #7976, specifically the task that states:
"Documentation will be added to the website to describe
[sw/device/lib/testing] subfoldering rules and naming conventions."
Signed-off-by: Timothy Trippel <ttrippel@google.com>
diff --git a/sw/device/lib/_index.md b/sw/device/lib/_index.md
index a1afd51..a81bb5a 100644
--- a/sw/device/lib/_index.md
+++ b/sw/device/lib/_index.md
@@ -15,14 +15,19 @@
[freestanding C library headers]({{< relref "sw/device/lib/base/freestanding/README.md" >}}).
The Base Libraries are simple libraries that can be used by any other OpenTitan device software libraries.
- [`sw/device/lib/dif`]({{< relref "sw/device/lib/dif/README.md" >}})
- contains the
- [Device Interface Functions]({{< relref "doc/rm/device_interface_functions.md" >}}).
+ contains the [Device Interface Functions]({{< relref "doc/rm/device_interface_functions.md" >}}).
- [`sw/device/lib/arch`]({{< relref "sw/device/lib/arch/README.md" >}})
contains the libraries to be used on specific device configurations (for
instance FPGA, Simulation, etc.).
- [`sw/device/lib/runtime`]({{< relref "sw/device/lib/runtime/README.md" >}})
contains general libraries for more advanced on-device functionality
(including logging, printing, and interacting with the RISC-V core).
+- [`sw/device/lib/testing`]({{< relref "sw/device/lib/testing/_index.md" >}})
+ contains test library code. Test library code is any (reusable) code that
+ could aid in the writing of smoke, IP integration, or system-level tests,
+ that is a layer of above the DIFs. In other words, this code may make use
+ of the DIFs to actuate the hardware, and chip-level test code may make use
+ of this code to actuate the DIFs.
## Documentation Index
diff --git a/sw/device/lib/testing/_index.md b/sw/device/lib/testing/_index.md
new file mode 100644
index 0000000..28a9bfd
--- /dev/null
+++ b/sw/device/lib/testing/_index.md
@@ -0,0 +1,29 @@
+---
+title: "Chip-Level Test Libraries"
+---
+
+This subtree provides libary code for writing chip-level tests.
+Test library code should be considered any (reusable) code that could aid in the writing of chip-level tests.
+
+## Test Types
+- **Chip-level Tests** - A collection of software level tests that run on OpenTitan hardware, whose main purpose is pre-silicon verification and post-silicon bringup.
+These tests consist of: smoke, IP integration, and system-level tests.
+These tests are all executed on-device using the [on-device test framework]({{< relref "sw/device/lib/testing/test_framework/index.md" >}}).
+While most of these tests are top-level agnostic, some are not.
+ - **Smoke Tests** - A software level test, written in C using DIFs, that performs a minimal set of operations on a given IP block to verify that it is alive and functioning.
+ - **IP Integration Tests** - A software level test, written in C, that exercises some specific functionality specific to a given IP and toplevel.
+ The Eargrey toplevel [test plan](https://docs.opentitan.org/hw/top_earlgrey/doc/dv/#testplan) describes these tests.
+ - **System-level Scenario Test** - A software level test, written in C, that mimics complex system level use case scenarios.
+ Such a test is designed to encompass multiple pieces of functionality tested by the IP integration tests.
+
+## Subfoldering Rules
+- [on-device test framework]({{< relref "sw/device/lib/testing/test_framework/index.md" >}})
+code will live in: **sw/device/lib/testing/test\_framework**.
+- Remaining test library code will **_not_** be subfoldered.
+
+## File Naming Conventions
+- Test libary code will be named: **{IP or functionality name}_testutils.{h,c}**
+
+## Documentation Index
+
+{{% sectionContent %}}
diff --git a/sw/device/lib/testing/test_framework/chip_level_test_infra.svg b/sw/device/lib/testing/test_framework/chip_level_test_infra.svg
new file mode 100644
index 0000000..d778dc2
--- /dev/null
+++ b/sw/device/lib/testing/test_framework/chip_level_test_infra.svg
@@ -0,0 +1,176 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg xmlns:xl="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="481 186 741 502" width="741" height="502">
+ <defs>
+ <font-face font-family="Helvetica Neue" font-size="16" panose-1="2 0 5 3 0 0 0 2 0 4" units-per-em="1000" underline-position="-100" underline-thickness="50" slope="0" x-height="517" cap-height="714" ascent="951.9958" descent="-212.99744" font-weight="400">
+ <font-face-src>
+ <font-face-name name="HelveticaNeue"/>
+ </font-face-src>
+ </font-face>
+ <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-1 -4 10 8" markerWidth="10" markerHeight="8" color="black">
+ <g>
+ <path d="M 8 0 L 0 -3 L 0 3 Z" fill="currentColor" stroke="currentColor" stroke-width="1"/>
+ </g>
+ </marker>
+ <marker orient="auto" overflow="visible" markerUnits="strokeWidth" id="FilledArrow_Marker_2" stroke-linejoin="miter" stroke-miterlimit="10" viewBox="-9 -4 10 8" markerWidth="10" markerHeight="8" color="black">
+ <g>
+ <path d="M -8 0 L 0 3 L 0 -3 Z" fill="currentColor" stroke="currentColor" stroke-width="1"/>
+ </g>
+ </marker>
+ <font-face font-family="Helvetica Neue" font-size="20" panose-1="2 0 5 3 0 0 0 2 0 4" units-per-em="1000" underline-position="-100" underline-thickness="50" slope="0" x-height="517" cap-height="714" ascent="951.9958" descent="-212.99744" font-weight="400">
+ <font-face-src>
+ <font-face-name name="HelveticaNeue"/>
+ </font-face-src>
+ </font-face>
+ <font-face font-family="Helvetica Neue" font-size="14" panose-1="2 0 5 3 0 0 0 2 0 4" units-per-em="1000" underline-position="-100" underline-thickness="50" slope="0" x-height="517" cap-height="714" ascent="951.9958" descent="-212.99744" font-weight="400">
+ <font-face-src>
+ <font-face-name name="HelveticaNeue"/>
+ </font-face-src>
+ </font-face>
+ </defs>
+ <metadata> Produced by OmniGraffle 7.18.5\n2021-08-30 23:10:50 +0000</metadata>
+ <g id="Canvas_1" stroke-dasharray="none" fill="none" stroke-opacity="1" fill-opacity="1" stroke="none">
+ <title>Canvas 1</title>
+ <rect fill="white" x="481" y="186" width="741" height="502"/>
+ <g id="Canvas_1_Layer_1">
+ <title>Layer 1</title>
+ <g id="Graphic_116">
+ <rect x="481.5" y="186.5" width="740" height="501" fill="white"/>
+ <rect x="481.5" y="186.5" width="740" height="501" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Graphic_75">
+ <rect x="497.0145" y="204" width="192.256" height="441" fill="white"/>
+ <rect x="497.0145" y="204" width="192.256" height="441" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Graphic_56">
+ <rect x="755.75" y="275" width="274" height="95.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Graphic_16">
+ <rect x="747.75" y="388.75" width="290" height="135.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Graphic_59">
+ <rect x="755.5" y="422" width="274" height="95.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Graphic_31">
+ <rect x="1069.5" y="425.4111" width="135.684" height="123.74388" stroke="#bb7fce" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Graphic_3">
+ <rect x="1079.059" y="494.7641" width="115.941" height="44.64088" fill="white"/>
+ <rect x="1079.059" y="494.7641" width="115.941" height="44.64088" stroke="#bc7fd1" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ <text transform="translate(1084.059 507.86056)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x="11.770501" y="15">test_main.c</tspan>
+ </text>
+ </g>
+ <g id="Graphic_15">
+ <rect x="747.5" y="242.5" width="290" height="135.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Graphic_17">
+ <rect x="747.75" y="537.543" width="290" height="95.5" fill="white"/>
+ <rect x="747.75" y="537.543" width="290" height="95.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Graphic_19">
+ <rect x="508.3925" y="388.75" width="169.5" height="244.293" fill="white"/>
+ <rect x="508.3925" y="388.75" width="169.5" height="244.293" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ <text transform="translate(513.3925 501.6725)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x="12.326" y="15">pytest (systemtest)</tspan>
+ </text>
+ </g>
+ <g id="Graphic_20">
+ <rect x="508.3925" y="242.5" width="169.5" height="135.5" fill="white"/>
+ <rect x="508.3925" y="242.5" width="169.5" height="135.5" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ <text transform="translate(513.3925 301.026)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x="47.438" y="15">dvsim.py</tspan>
+ </text>
+ </g>
+ <g id="Graphic_22">
+ <rect x="765.5" y="312.777" width="254.5" height="51.223" stroke="#bc7fce" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ <text transform="translate(770.5 329.1645)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x="65.362" y="15">Test Framework</tspan>
+ </text>
+ </g>
+ <g id="Graphic_23">
+ <text transform="translate(754.75 247.5)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x="30.208" y="15">DV Simulation (Synopsys VCS)</tspan>
+ </text>
+ </g>
+ <g id="Graphic_24">
+ <text transform="translate(825.854 389.638)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x="6252776e-19" y="15">Verilator Simulation</tspan>
+ </text>
+ </g>
+ <g id="Graphic_25">
+ <text transform="translate(871.718 542.543)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x="0" y="15">FPGA</tspan>
+ </text>
+ </g>
+ <g id="Graphic_32">
+ <rect x="765.5" y="460.82" width="254.5" height="51.223" stroke="#bc7fce" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ <text transform="translate(770.5 477.2075)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x="65.362" y="15">Test Framework</tspan>
+ </text>
+ </g>
+ <g id="Graphic_33">
+ <rect x="763.5" y="568.78176" width="254.5" height="51.223" stroke="#bc7fce" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ <text transform="translate(768.5 585.16926)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x="65.362" y="15">Test Framework</tspan>
+ </text>
+ </g>
+ <g id="Graphic_36">
+ <text transform="translate(1080.454 430.4111)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x="8171241e-19" y="15">Test Framework</tspan>
+ </text>
+ </g>
+ <g id="Line_44">
+ <line x1="737.6" y1="310.25" x2="687.7925" y2="310.25" marker-end="url(#FilledArrow_Marker)" marker-start="url(#FilledArrow_Marker_2)" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Line_45">
+ <line x1="687.7925" y1="453.8591" x2="737.85" y2="453.8591" marker-end="url(#FilledArrow_Marker)" marker-start="url(#FilledArrow_Marker_2)" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Graphic_57">
+ <text transform="translate(804.498 275.948)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x="6252776e-19" y="15">UVM Top-level Testbench</tspan>
+ </text>
+ </g>
+ <g id="Graphic_58">
+ <text transform="translate(793.098 425.948)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x="2557954e-19" y="15">Verilator Top-level Testbench</tspan>
+ </text>
+ </g>
+ <g id="Graphic_64">
+ <text transform="translate(856.102 209)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="20" font-weight="400" fill="black" x="7958079e-19" y="19">Device</tspan>
+ </text>
+ </g>
+ <g id="Line_68">
+ <line x1="687.7925" y1="596.22976" x2="737.85" y2="596.22976" marker-end="url(#FilledArrow_Marker)" marker-start="url(#FilledArrow_Marker_2)" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Line_71">
+ <line x1="1020" y1="461.48825" x2="1069.5" y2="425" stroke="#bb7fce" stroke-linecap="round" stroke-linejoin="round" stroke-dasharray="2.0,2.0" stroke-width="1"/>
+ </g>
+ <g id="Line_72">
+ <line x1="1020" y1="512.4021" x2="1069.5" y2="549.07477" stroke="#bb7fce" stroke-linecap="round" stroke-linejoin="round" stroke-dasharray="2.0,2.0" stroke-width="1"/>
+ </g>
+ <g id="Graphic_77">
+ <rect x="1079.059" y="453.8591" width="115.941" height="36.672678" fill="white"/>
+ <rect x="1079.059" y="453.8591" width="115.941" height="36.672678" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ <text transform="translate(1084.059 462.97146)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" font-weight="400" fill="black" x=".22650095" y="15">Chip-level Test</tspan>
+ </text>
+ </g>
+ <g id="Graphic_66">
+ <text transform="translate(572.0325 209)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="20" font-weight="400" fill="black" x="58264504e-20" y="19">Host</tspan>
+ </text>
+ </g>
+ <g id="Graphic_111">
+ <rect x="783.739" y="649.0238" width="23.184" height="21.016435" fill="#bc7fcf"/>
+ <rect x="783.739" y="649.0238" width="23.184" height="21.016435" stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Graphic_110">
+ <text transform="translate(811.279 651.336)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="14" font-weight="400" fill="black" x="9947598e-20" y="13">= On-Device Test Framework</tspan>
+ </text>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/sw/device/lib/testing/test_framework/index.md b/sw/device/lib/testing/test_framework/index.md
new file mode 100644
index 0000000..ff3c21f
--- /dev/null
+++ b/sw/device/lib/testing/test_framework/index.md
@@ -0,0 +1,26 @@
+---
+title: "On-Device Test Framework"
+---
+
+
+
+Currently, chip-level tests are executed across three targets (DV, Verilator, and FPGA), using host-side test initiation tools, and an on-device test framework, as shown in the figure above.
+On the _host_ side, two main tools are used to initiate tests on the device. For the DV simulation target, the [dvsim.py](https://cs.opensource.google/opentitan/opentitan/+/master:util/dvsim/) tool is used, while for Verilator and FPGA targets, the [systemtest (pytest)](https://cs.opensource.google/opentitan/opentitan/+/master:test/systemtest/) tool is used.
+Focusing on the _device_ side, for all three targets, the [on-device test framework](https://cs.opensource.google/opentitan/opentitan/+/master:sw/device/lib/testing/test_framework/test_main.c) is used to provide a uniform execution environment for chip-level tests.
+The [on-device test framework](https://cs.opensource.google/opentitan/opentitan/+/master:sw/device/lib/testing/test_framework/test_main.c) provides boilerplate setup code that configures the UART for communicating messages and test results back to the host.
+To write a chip-level test that uses this framework, one must create a new C file for the test with the following boilerplate code:
+
+```
+#include "sw/device/lib/testing/test_framework/test_main.h"
+#include "sw/device/lib/testing/check_.h" // if calls to CHECK() are made
+#include "sw/device/lib/runtime/log.h" // if calls to LOG_INFO() are made
+
+const test_config_t kTestConfig;
+
+bool test_main() {
+ // Test program entry point.
+ return true;
+}
+```
+
+Check out the [rv\_timer smoke test](https://cs.opensource.google/opentitan/opentitan/+/master:sw/device/tests/dif/dif_rv_timer_smoketest.c) for an example chip-level test that uses the current test framework.
