doc/rm/ref_manual_fpga.md

---
title: "FPGA Reference Manual"
---

This manual provides additional usage details about the FPGA.
Specifically, it provides instructions on SW development flows and testing procedures.
Refer to the [FPGA Setup]({{< relref "doc/getting_started/setup_fpga" >}}) guide for more information on initial setup.

## FPGA SW Development Flow

The FPGA is meant for both boot ROM and general software development.
The flow for each is different, as the boot ROM is meant to be fairly static while general software can change very frequently.

### Boot ROM development

The FPGA bitstream is built after compiling whatever code is sitting in `sw/device/lib/testing/test_rom`.
This binary is used to initialize internal FPGA memory and is part of the bitstream directly.

To update this content without rebuilding the FPGA, a flow is required to splice a new boot ROM binary into the bitstream.
First, you must splice the new content into an existing bitstream.
Then, you can flash the new bitstream onto the FPGA with `opentitantool`.

There are two ways to splice content into a bitstream.

1. Define a Bazel target (or use an existing one).
For example, see the `//hw/bitstream:rom` target defined in [hw/bitstream/BUILD](https://github.com/lowRISC/opentitan/blob/master/hw/bitstream/BUILD).

2. Use the [`splice_rom.sh`](https://github.com/lowRISC/opentitan/blob/master/util/fpga/splice_rom.sh) script.

    There are two prerequisites in order for this flow to work:

    * The boot ROM during the build process must be correctly inferred by the tool.
      * See [prim_rom](https://github.com/lowRISC/opentitan/blob/master/hw/ip/prim_generic/rtl/prim_generic_rom.sv) and [vivado_hook_opt_design_post.tcl](https://github.com/lowRISC/opentitan/blob/master/hw/top_earlgrey/util/vivado_hook_opt_design_post.tcl).
    * The MMI file outlining the physical boot ROM placement and mapping to FPGA block RAM primitives needs to be generated by the tool.
      * See [vivado_hook_write_bitstream_pre.tcl](https://github.com/lowRISC/opentitan/blob/master/hw/top_earlgrey/util/vivado_hook_write_bitstream_pre.tcl).

    With these steps in place, a script can be invoked to take a new binary and push its contents into an existing bitfile.
    For details, please see the [`splice_rom.sh`](https://github.com/lowRISC/opentitan/blob/master/util/fpga/splice_rom.sh) script.

    See example below:

    ```console
    $ cd $REPO_TOP
    $ ./util/fpga/splice_rom.sh
    $ bazel run //sw/host/opentitantool fpga load-bitstream build/lowrisc_systems_chip_earlgrey_cw310_0.1/synth-vivado/lowrisc_systems_chip_earlgrey_cw310_0.1.bit
    ```

    The script assumes that there is an existing bitfile `build/lowrisc_systems_chip_earlgrey_cw310_0.1/synth-vivado/lowrisc_systems_chip_earlgrey_cw310_0.1.bit` (this is created after following the steps in [FPGA Setup]({{< relref "doc/getting_started/setup_fpga" >}})).

    The script assumes that there is an existing boot ROM image under `build-bin/sw/device/lib/testing/test_rom` and then creates a new bitfile of the same name at the same location.
    The original input bitfile is moved to `build/lowrisc_systems_chip_earlgrey_cw310_0.1/synth-vivado/lowrisc_systems_chip_earlgrey_cw310_0.1.bit.orig`.

    `opentitantool` can then be used to directly flash the updated bitstream to the FPGA.

### General Software Development

After building, the FPGA bitstream contains only the boot ROM.
Using this boot ROM, the FPGA is able to load additional software to the emulated flash, such as software in the `sw/device/benchmark`, `sw/device/examples` and `sw/device/tests` directories.
To load additional software, `opentitantool` is required.

Also the binary you wish to load needs to be built first.
For the purpose of this demonstration, we will use `sw/device/examples/hello_world`, but it applies to any software image that is able to fit in the emulated flash space.
The example below builds the `hello_world` image and loads it onto the FPGA.

```console
$ cd ${REPO_TOP}
$ bazel run //sw/host/opentitantool fpga set-pll # This needs to be done only once.
$ bazel build //sw/device/examples/hello_world:hello_world_fpga_cw310_bin
$ bazel run //sw/host/opentitantool bootstrap $(ci/scripts/target-location.sh //sw/device/examples/hello_world:hello_world_fpga_cw310_bin)
```

Uart output:
```
I00000 test_rom.c:81] Version: earlgrey_silver_release_v5-5886-gde4cb1bb9, Build Date: 2022-06-13 09:17:56
I00001 test_rom.c:87] TestROM:6b2ca9a1
I00000 test_rom.c:81] Version: earlgrey_silver_release_v5-5886-gde4cb1bb9, Build Date: 2022-06-13 09:17:56
I00001 test_rom.c:87] TestROM:6b2ca9a1
I00002 test_rom.c:118] Test ROM complete, jumping to flash!
I00000 hello_world.c:66] Hello World!
I00001 hello_world.c:67] Built at: Jun 13 2022, 14:16:59
I00002 demos.c:18] Watch the LEDs!
I00003 hello_world.c:74] Try out the switches on the board
I00004 hello_world.c:75] or type anything into the console window.
I00005 hello_world.c:76] The LEDs show the ASCII code of the last character.
```

For more details on the exact operation of the loading flow and how the boot ROM processes incoming data, please refer to the [boot ROM readme](https://github.com/lowRISC/opentitan/tree/master/sw/device/lib/testing/test_rom).

## Bitstreams: Caching and Splicing

This section gives an overview of where bitstreams are generated, how they are uploaded to the GCP cache, and how Bazel reaches into the cache.

OpenTitan runs CI tasks on Azure Pipelines that build FPGA bitstreams.
A full bitstream build can take hours, so we cache the output artifacts in a GCS bucket.
These cached bitstreams can be downloaded and used as-is, or we can splice in freshly-compiled components, including the ROM and the OTP image.

### Building bitstreams on CI and uploading artifacts to GCS

The `chip_earlgrey_cw310` CI job builds the `//hw/bitstream/vivado:standard` target which will build a bitstream with the test ROM and RMA OTP image.
This target will also produce bitstreams with the ROM spliced in and the DEV OTP image spliced in.
The following files are produced as a result:

* `fpga_cw310_rom.bit` (ROM, RMA OTP image)
* `fpga_cw310_rom_otp_dev.bit` (ROM, DEV OTP image)
* `lowrisc_systems_chip_earlgrey_cw310_0.1.bit` (test ROM, RMA OTP image)
* `otp.mmi`
* `rom.mmi`

If CI is working on the `master` branch, it puts selected build artifacts into a tarball, which it then uploads to the GCS bucket. The latest tarball is available here: https://storage.googleapis.com/opentitan-bitstreams/master/bitstream-latest.tar.gz

### Exposing GCS-cached artifacts to Bazel

The `@bitstreams//` workspace contains autogenerated Bazel targets for the GCS-cached artifacts.
This magic happens in `rules/scripts/bitstreams_workspace.py`.
Under the hood, it fetches the latest tarball from the GCS bucket and constructs a BUILD file that defines one target per artifact.

One meta-level up, we have targets in `//hw/bitstream` that decide whether to use cached artifacts or to build them from scratch.
By default, these targets use cached artifacts by pulling in their corresponding `@bitstreams//` targets.

### Related work

* TODO Define the new naming scheme.
  https://github.com/lowRISC/opentitan/issues/13807

* TODO Splice a variety of ROM and OTP pairs.
  https://github.com/lowRISC/opentitan/issues/13603