doc/rm/checklist.md - 3p/lowrisc/opentitan - Git at Google

 ---
 title: "Signoff Checklist"
 ---

 This document explains the recommended checklist items to review when transitioning from one [Hardware Stage]({{<relref "/doc/ug/hw_stages.md" >}}) to another, for both design and verification stages.
 It is expected that the items in each stage (D1, V1, etc) are completed.

 ## D1

 For a transition from D0 to D1, the following items are expected be completed.

 ### SPEC_COMPLETE

 Spec complete, feature set finalized.

 ### CSR_DEFINED

 CSRs defined and generated.

 ### CLKRST_CONNECTED

 Clock(s)/Reset(s) connected to all sub-modules.

 ### IP_TOP

 Unit `.sv` exists, meet comportability requirements.

 ### IP_INSTANCED

 Unit is instanced and binding in top level RTL. The unit must not break top
 level functionality such as propagating X through TL-UL interface.

 ### MEM_INSTANCED_80

 80% of expected memories instantiated, using behavioral RAMs.

 ### FUNC_IMPLEMENTED

 Main functional path implemented.

 ### ASSERT_KNOWN_ADDED

 All the outputs of the IP have `ASSERT_KNOWN` assertions.

 ### LINT_SETUP

 Lint run setup, compiles and runs. It is acceptable to have lint errors and
 warnings at this stage.

 ## D2

 ### NEW_FEATURES

 Any new features added since D1 are documented, reviewed with DV/SW/FPGA.
 Github Issue for RFC should be linked in `Notes` section.

 ### BLOCK_DIAGRAM

 Block diagrams updated.

 ### DOC_INTERFACE

 Documented non-registered block interfaces.

 ### MISSING_FUNC

 Documented the missing functionalities.

 ### FEATURE_FROZEN

 ### FEATURE_COMPLETE

 ### AREA_SANITY_CHECK

 Area sanity check completed either on FPGA or on Design Compiler.

 ### DEBUG_BUS

 Debug bus instantiated. DebugMux and DebugTraceMux Hooked up.

 ### PORT_FROZEN

 100% ports. Port is frozen.

 ### ARCHITECTURE_FROZEN

 100% architectural states exists (RAMs, CSRs, etc)

 ### REVIEW_TODO

 Review and sign off TODOs.

 ### STYLE_X

 Confirming to style guide regarding X usage. TODO: Related Github issue.

 ### STYLE_LINT_SETUP

 Style lint run setup.

 ### LINT_PASS

 Lint passes. Waiver reviewed.

 ### CDC_SETUP

 CDC run set up. No must fix errors, waiver file created.

 ### FPGA_TIMING

 FPGA synthesis timing meet (Fmax-10%) target or better

 ### CDC_SYNCMACRO

 CDC Sync flops use behavioral synchronization macros(`prim_flop_2sync`) not
 2flops.

 ## D3

 ### NEW_FEATURES_D3

 Any approved new features since D2 documented, and reviewed with DV/SW/FPGA

 ### TODO_COMPLETE

 Resolve all TODOs.

 ### LINT_COMPLETE

 Lint clean. Lint waiver file reviewed and signed off by tech steering committe.

 ### CDC_COMPLETE

 CDC clean. CDC waiver file reviewed and signed off by tech sterring committe.

 ### REVIEW_RTL

 Hold Design Review: Hold a RTL sanity check review by an independent designer.

 ### REVIEW_DELETED_FF

 Hold Design Review: Sign off deleted flops list (one last time).

 ### REVIEW_SW_CSR

 Review Design Change with SW: Review CSRs

 ### REVIEW_SW_FATAL_ERR

 Review Design Change with SW: Review Fatal Errors

 ### REVIEW_SW_CHANGE

 Review Design Change with SW: Review other SW visible changes

 ### REVIEW_SW_ERRATA

 Review Design Change with SW: Review known "Won't Fix" bugs and "Errata".

 ## V1

 For a transition from V0 to V1, the following items are expected be completed.

 ### DV_PLAN_DRAFT_COMPLETED

 - DV Plan document drafted, indicating the overall DV strategy, intent and the testbench environment details with diagrams, details on TB, UVCs, checkers, scoreboard, interfaces, assertions,
 coverage objects.
 - Details may be missing since most items are not expected to be fully developed at this stage.

 ### TESTPLAN_COMPLETED

 A fully completed Testplan written in Hjson, indicating the list of planned tests with descriptions indicating the goal of the test and optionally details on stimulus and the checking procedure.

 ### TB_TOP_CREATED

 - Top level testbench with DUT instantiated and following interfaces hooked up (as applicable): TileLink, clocks and resets, interrupts and major DUT interfaces.
 - All interfaces may not be hooked up at this point. Inputs for which interfaces have not yet been created may be tied off to 0.

 ### PRELIMINARY_ASSERTION_CHECKS_ADDED

 - All available interface assertion monitors hooked up (example: tlul_assert)

 ### TB_ENV_CREATED

 - UVM enviroment created with major interface agents and UVCs connected and instantiated.
 - TLM connections made from UVC monitors to the scoreboard

 ### RAL_MODEL_GEN_AUTOMATED

 RAL model is generated by using [regtool]({{< relref "/util/reggen/README.md" >}}) and instantiated in UVM environment.

 ### TB_GEN_AUTOMATED

 Full testbench automation completed if applicable. This may be required for verifying multiple flavors of parameterized DUT designs.

 ### SANITY_TEST_PASSING

 - Sanity test exercising a basic functionality of a major DUT datapath passing
 - What functionality to test and to what level may be governed by higher level (example: chip) integration requirements. These are to be captured when the Testplan is reviewed with the key stakeholders.

 ### CSR_MEM_TEST_SUITE_PASSING

 CSR test suite added for ALL interfaces that have access to system memory map (JTAG, TL, etc.):
 - HW reset test (test all resets)
 - CSR read/write
 - Bit Bash
 - Aliasing

 Memory test suite added for ALL interfaces that have access to system memory map (JTAG, TL, etc.) if the DUT has memories:
 - Mem walk

 Ensure all these tests verify back-2back access with zero delays, along with partial reads and partial writes.

 ### SANITY_REGRESSION_SETUP

 Sanity regression set up for code health. A small suite of tests identified for running the sanity regression on. If the testbench has more than one compile-time configuration, then a sanity test for each configuration should be ideally selected.

 ### NIGHTLY_REGRESSION_SETUP

 Nightly regression for running all tests with multiple random seeds (iterations) setup. Selecting the number of iterations for running each test is subjective - it depends on the failure rate and available compute resources. For starters, it is recommended to set iterations to 100 for each test. It may be trimmed down if the compute load is too high. As such, the goal should be to have the nightly regression finish overnight so that the results are available next morning for triage.

 ### ALT_TOOL_SETUP

 Verify that the sanity test passes cleanly (with no warnings) with one additional tool apart from the primary tool selected for signoff.

 ### COVERAGE_MODEL_ADDED

 - Structural coverage collection model checked in. This specifies what hierarchies and what types of coverage to collect. For example, pre-verified sub-mudules (including some `prim` components pre-verified thoroughly with FPV) can be black-boxed and only IO toggle coverage can be setup for those sub-modules for coverage collection.
 - Functional coverage shell object created - this may not contain coverpoints or covergroups yet, but it is primed for use in post-V2 test development.

 ### PRE_VERIFIED_SUB_MODULES_V1

 Sub-modules that are pre-verified with their own testbenches have already reached V1 or higher stage.

 ### DESIGN_SPEC_REVIEWED

 RTL (uArch) specification reviewed and signed off.

 ### DV_PLAN_TESTPLAN_REVIEWED

 DV Plan & Testplan reviewed with key stakeholders - designer, design lead, DV lead, architects, higher level (chip) design and DV leads.

 ### STD_TEST_CATEGORIES_PLANNED

 Following categories of post-V1 tests focused at in the Testplan review (as applicable):
 - Security/error
 - Power
 - Performance
 - Debug
 - Stress

 ### V2_CHECKLIST_SCOPED

 Reviewed V2 checklist to understand scope and estimate effort.

 ## V2

 ### DESIGN_DELTAS_CAPTURED

 It is possible for the design to have undergone some changes since the DV plan and Testplan was reviewed prior to V1 stage. Please ensure that those deltas have been captured adequately in the DV Plan and the Testplan documents.

 ### DV_PLAN_COMPLETED

 DV Plan is fully completed in terms of content.

 ### ALL_INTERFACES_EXERCISED

 All interfaces including sidebands have been connected and exercised.

 ### ALL_ASSERTION_CHECKS_ADDED

 All planned assertions have been written and enabled.

 ### TB_ENV_COMPLETED

 UVM environment fully developed with end-2-end checks in scoreboard enabled.

 ### ALL_TESTS_PASSING

 All tests in the Testplan written and passing with at least one random seed.

 ### FW_SIMULATED

 For chip-level, verified pieces of FW code (DIFs) in simulaton.

 ### NIGHTLY_REGRESSION_V2

 Nightly regression with multiple random seeds passing: 90%

 ### CODE_COVERAGE_V2

 Code coverage requirements: line, toggle, fsm (state & transition), branch, assertion: 90%

 ### FUNCTIONAL_COVERAGE_V2

 Functional coverage requirements: coverpoints: 100%, crosses: 75%

 ### NO_HIGH_PRIORITY_ISSUES_PENDING

 Ensure that all high priority (tagged P0 and P1) design bugs have been addressed and closed. If the bugs were found elsewhere, ensure that they are reproduced deterministically in DV (through additional tests or by tweaking existing tests as needed) and the fixes are adequately verified.

 ### ALL_LOW_PRIORITY_ISSUES_ROOT_CAUSED

 Ensure that all low priority (tagged P2 and P3) design bugs habe been root-caused. They may be deferred to post D2V2 for closure.

 ### PRE_VERIFIED_SUB_MODULES_V2

 Sub-modules that are pre-verified with their own testbenches have already reached V2 or higher stage.

 ### V3_CHECKLIST_SCOPED

 Reviewed V3 checklist to understand scope and estimate effort.

 ## V3

 ### DESIGN_DELTAS_CAPTURED_IF_ANY

 It is possible for the design to undergo changes even at this stage (when it is expected to be mature). Please ensure that those new deltas have been captured adequately in the DV Plan and the Testplan documents.

 ### ALL_TODOS_RESOLVED

 Ensure that the complete testbench code is free from TODOs.

 ### X_PROP_ANALYSIS_COMPLETED

 X Propagation Analysis complete

 ### NIGHTLY_REGRESSION_AT_100

 Nightly regression with multiple random seeds passing: 100% (with 1 week minimum soak time)

 ### CODE_COVERAGE_AT_100

 Code coverage requirements: line, toggle, fsm (state & transition), branch, assertion: 100%

 ### FUNCTIONAL_COVERAGE_AT_100

 Functional coverage requirements: coverpoints: 100%, crosses: 100%

 ### NO_ISSUES_PENDING

 Ensure that all design bugs have been addressed and closed.

 ### NO_TOOL_WARNINGS_THROWN

 Clean up all compile-time and run-time warnings thrown by the simulator.

 ### PRE_VERIFIED_SUB_MODULES_V3

 Sub-modules that are pre-verified with their own testbenches have already reached V3 stage.
	---
	title: "Signoff Checklist"
	---

	This document explains the recommended checklist items to review when transitioning from one [Hardware Stage]({{<relref "/doc/ug/hw_stages.md" >}}) to another, for both design and verification stages.
	It is expected that the items in each stage (D1, V1, etc) are completed.

	## D1

	For a transition from D0 to D1, the following items are expected be completed.

	### SPEC_COMPLETE

	Spec complete, feature set finalized.

	### CSR_DEFINED

	CSRs defined and generated.

	### CLKRST_CONNECTED

	Clock(s)/Reset(s) connected to all sub-modules.

	### IP_TOP

	Unit `.sv` exists, meet comportability requirements.

	### IP_INSTANCED

	Unit is instanced and binding in top level RTL. The unit must not break top
	level functionality such as propagating X through TL-UL interface.

	### MEM_INSTANCED_80

	80% of expected memories instantiated, using behavioral RAMs.

	### FUNC_IMPLEMENTED

	Main functional path implemented.

	### ASSERT_KNOWN_ADDED

	All the outputs of the IP have `ASSERT_KNOWN` assertions.

	### LINT_SETUP

	Lint run setup, compiles and runs. It is acceptable to have lint errors and
	warnings at this stage.

	## D2

	### NEW_FEATURES

	Any new features added since D1 are documented, reviewed with DV/SW/FPGA.
	Github Issue for RFC should be linked in `Notes` section.

	### BLOCK_DIAGRAM

	Block diagrams updated.

	### DOC_INTERFACE

	Documented non-registered block interfaces.

	### MISSING_FUNC

	Documented the missing functionalities.

	### FEATURE_FROZEN

	### FEATURE_COMPLETE

	### AREA_SANITY_CHECK

	Area sanity check completed either on FPGA or on Design Compiler.

	### DEBUG_BUS

	Debug bus instantiated. DebugMux and DebugTraceMux Hooked up.

	### PORT_FROZEN

	100% ports. Port is frozen.

	### ARCHITECTURE_FROZEN

	100% architectural states exists (RAMs, CSRs, etc)

	### REVIEW_TODO

	Review and sign off TODOs.

	### STYLE_X

	Confirming to style guide regarding X usage. TODO: Related Github issue.

	### STYLE_LINT_SETUP

	Style lint run setup.

	### LINT_PASS

	Lint passes. Waiver reviewed.

	### CDC_SETUP

	CDC run set up. No must fix errors, waiver file created.

	### FPGA_TIMING

	FPGA synthesis timing meet (Fmax-10%) target or better

	### CDC_SYNCMACRO

	CDC Sync flops use behavioral synchronization macros(`prim_flop_2sync`) not
	2flops.

	## D3

	### NEW_FEATURES_D3

	Any approved new features since D2 documented, and reviewed with DV/SW/FPGA

	### TODO_COMPLETE

	Resolve all TODOs.

	### LINT_COMPLETE

	Lint clean. Lint waiver file reviewed and signed off by tech steering committe.

	### CDC_COMPLETE

	CDC clean. CDC waiver file reviewed and signed off by tech sterring committe.

	### REVIEW_RTL

	Hold Design Review: Hold a RTL sanity check review by an independent designer.

	### REVIEW_DELETED_FF

	Hold Design Review: Sign off deleted flops list (one last time).

	### REVIEW_SW_CSR

	Review Design Change with SW: Review CSRs

	### REVIEW_SW_FATAL_ERR

	Review Design Change with SW: Review Fatal Errors

	### REVIEW_SW_CHANGE

	Review Design Change with SW: Review other SW visible changes

	### REVIEW_SW_ERRATA

	Review Design Change with SW: Review known "Won't Fix" bugs and "Errata".

	## V1

	For a transition from V0 to V1, the following items are expected be completed.

	### DV_PLAN_DRAFT_COMPLETED

	- DV Plan document drafted, indicating the overall DV strategy, intent and the testbench environment details with diagrams, details on TB, UVCs, checkers, scoreboard, interfaces, assertions,
	coverage objects.
	- Details may be missing since most items are not expected to be fully developed at this stage.

	### TESTPLAN_COMPLETED

	A fully completed Testplan written in Hjson, indicating the list of planned tests with descriptions indicating the goal of the test and optionally details on stimulus and the checking procedure.

	### TB_TOP_CREATED

	- Top level testbench with DUT instantiated and following interfaces hooked up (as applicable): TileLink, clocks and resets, interrupts and major DUT interfaces.
	- All interfaces may not be hooked up at this point. Inputs for which interfaces have not yet been created may be tied off to 0.

	### PRELIMINARY_ASSERTION_CHECKS_ADDED

	- All available interface assertion monitors hooked up (example: tlul_assert)

	### TB_ENV_CREATED

	- UVM enviroment created with major interface agents and UVCs connected and instantiated.
	- TLM connections made from UVC monitors to the scoreboard

	### RAL_MODEL_GEN_AUTOMATED

	RAL model is generated by using [regtool]({{< relref "/util/reggen/README.md" >}}) and instantiated in UVM environment.

	### TB_GEN_AUTOMATED

	Full testbench automation completed if applicable. This may be required for verifying multiple flavors of parameterized DUT designs.

	### SANITY_TEST_PASSING

	- Sanity test exercising a basic functionality of a major DUT datapath passing
	- What functionality to test and to what level may be governed by higher level (example: chip) integration requirements. These are to be captured when the Testplan is reviewed with the key stakeholders.

	### CSR_MEM_TEST_SUITE_PASSING

	CSR test suite added for ALL interfaces that have access to system memory map (JTAG, TL, etc.):
	- HW reset test (test all resets)
	- CSR read/write
	- Bit Bash
	- Aliasing

	Memory test suite added for ALL interfaces that have access to system memory map (JTAG, TL, etc.) if the DUT has memories:
	- Mem walk

	Ensure all these tests verify back-2back access with zero delays, along with partial reads and partial writes.

	### SANITY_REGRESSION_SETUP

	Sanity regression set up for code health. A small suite of tests identified for running the sanity regression on. If the testbench has more than one compile-time configuration, then a sanity test for each configuration should be ideally selected.

	### NIGHTLY_REGRESSION_SETUP

	Nightly regression for running all tests with multiple random seeds (iterations) setup. Selecting the number of iterations for running each test is subjective - it depends on the failure rate and available compute resources. For starters, it is recommended to set iterations to 100 for each test. It may be trimmed down if the compute load is too high. As such, the goal should be to have the nightly regression finish overnight so that the results are available next morning for triage.

	### ALT_TOOL_SETUP

	Verify that the sanity test passes cleanly (with no warnings) with one additional tool apart from the primary tool selected for signoff.

	### COVERAGE_MODEL_ADDED

	- Structural coverage collection model checked in. This specifies what hierarchies and what types of coverage to collect. For example, pre-verified sub-mudules (including some `prim` components pre-verified thoroughly with FPV) can be black-boxed and only IO toggle coverage can be setup for those sub-modules for coverage collection.
	- Functional coverage shell object created - this may not contain coverpoints or covergroups yet, but it is primed for use in post-V2 test development.

	### PRE_VERIFIED_SUB_MODULES_V1

	Sub-modules that are pre-verified with their own testbenches have already reached V1 or higher stage.

	### DESIGN_SPEC_REVIEWED

	RTL (uArch) specification reviewed and signed off.

	### DV_PLAN_TESTPLAN_REVIEWED

	DV Plan & Testplan reviewed with key stakeholders - designer, design lead, DV lead, architects, higher level (chip) design and DV leads.

	### STD_TEST_CATEGORIES_PLANNED

	Following categories of post-V1 tests focused at in the Testplan review (as applicable):
	- Security/error
	- Power
	- Performance
	- Debug
	- Stress

	### V2_CHECKLIST_SCOPED

	Reviewed V2 checklist to understand scope and estimate effort.

	## V2

	### DESIGN_DELTAS_CAPTURED

	It is possible for the design to have undergone some changes since the DV plan and Testplan was reviewed prior to V1 stage. Please ensure that those deltas have been captured adequately in the DV Plan and the Testplan documents.

	### DV_PLAN_COMPLETED

	DV Plan is fully completed in terms of content.

	### ALL_INTERFACES_EXERCISED

	All interfaces including sidebands have been connected and exercised.

	### ALL_ASSERTION_CHECKS_ADDED

	All planned assertions have been written and enabled.

	### TB_ENV_COMPLETED

	UVM environment fully developed with end-2-end checks in scoreboard enabled.

	### ALL_TESTS_PASSING

	All tests in the Testplan written and passing with at least one random seed.

	### FW_SIMULATED

	For chip-level, verified pieces of FW code (DIFs) in simulaton.

	### NIGHTLY_REGRESSION_V2

	Nightly regression with multiple random seeds passing: 90%

	### CODE_COVERAGE_V2

	Code coverage requirements: line, toggle, fsm (state & transition), branch, assertion: 90%

	### FUNCTIONAL_COVERAGE_V2

	Functional coverage requirements: coverpoints: 100%, crosses: 75%

	### NO_HIGH_PRIORITY_ISSUES_PENDING

	Ensure that all high priority (tagged P0 and P1) design bugs have been addressed and closed. If the bugs were found elsewhere, ensure that they are reproduced deterministically in DV (through additional tests or by tweaking existing tests as needed) and the fixes are adequately verified.

	### ALL_LOW_PRIORITY_ISSUES_ROOT_CAUSED

	Ensure that all low priority (tagged P2 and P3) design bugs habe been root-caused. They may be deferred to post D2V2 for closure.

	### PRE_VERIFIED_SUB_MODULES_V2

	Sub-modules that are pre-verified with their own testbenches have already reached V2 or higher stage.

	### V3_CHECKLIST_SCOPED

	Reviewed V3 checklist to understand scope and estimate effort.

	## V3

	### DESIGN_DELTAS_CAPTURED_IF_ANY

	It is possible for the design to undergo changes even at this stage (when it is expected to be mature). Please ensure that those new deltas have been captured adequately in the DV Plan and the Testplan documents.

	### ALL_TODOS_RESOLVED

	Ensure that the complete testbench code is free from TODOs.

	### X_PROP_ANALYSIS_COMPLETED

	X Propagation Analysis complete

	### NIGHTLY_REGRESSION_AT_100

	Nightly regression with multiple random seeds passing: 100% (with 1 week minimum soak time)

	### CODE_COVERAGE_AT_100

	Code coverage requirements: line, toggle, fsm (state & transition), branch, assertion: 100%

	### FUNCTIONAL_COVERAGE_AT_100

	Functional coverage requirements: coverpoints: 100%, crosses: 100%

	### NO_ISSUES_PENDING

	Ensure that all design bugs have been addressed and closed.

	### NO_TOOL_WARNINGS_THROWN

	Clean up all compile-time and run-time warnings thrown by the simulator.

	### PRE_VERIFIED_SUB_MODULES_V3

	Sub-modules that are pre-verified with their own testbenches have already reached V3 stage.