RunningOpenTitanVectorSimulation.md

# OpenTitan RISCV Vector Simulation in Renode

Follow setups in this document to build and run the following examples:

1. [OpenTitan Hello World Basic Example](#opentitan-hello-world-basic-example)
2. [OpenTitan Hello World Vector Sim Examples](#opentitan-hello-world-vector-sim-example)

Keep an eye on the file below for new build targets as they become available:

```
$ROOTDIR/build/opentitan_sw.mk
```

## OpenTitan Hello World Basic Example

These instructions guide through setting up and running a test of OpenTitan with a bare-bones Renode
robot-test-script.

### Sync and set up environment

```
repo sync
source ./build/setup.sh
```

### Build tools

```
m tools
```

### Build OpenTitan helloworld Demo

```
m opentitan_sw_bootrom
```

```
m opentitan_sw_helloworld
```

### Start OpenTitan Renode Script

```
renode -e "i @sim/config/shodan_secure.resc; start"
```

### Run `shodan_bootrom.robot` Robot-Test-Script

```
cd "${ROOTDIR}/sim/tests"
./test.sh shodan_bootrom.robot
```

Note: since we build Renode from source, we are to use Renode's provided `test.sh` instead of
`renode-test` for running robot test scripts.

[More details in Renode's documentation](https://renode.readthedocs.io/en/latest/introduction/testing.html#running-the-robot-test-script)

### View Results of the Tests

Results of the robot-test-script will be found in the following files:

```
$ROOTDIR/out/host/renode/tests/report.html
$ROOTDIR/out/host/renode/tests/robot_output.xml
$ROOTDIR/out/host/renode/tests/log.html
```

The `report.html` will share the results of each test.

One can utilize `robot_output.xml` for scripts/programs which may want to parse
and act on results.

## OpenTitan Hello World Vector Sim Example

How to run the HelloVector example and how to modify or add a new vector OpCode for Renode simulation.

1. [Run OpenTitan HelloVector Renode Demo](#run-opentitan-hellovector-renode-demo)
2. [Modify Vector OpCode Behavior and Simulate](#modify-vector-opcode-behavior-and-simulate)
3. [Add New Vector OpCode and Simulate](#add-new-vector-opcode-and-simulate)

### Run OpenTitan HelloVector Renode Demo

#### Sync and set up environment

```
repo sync
source ./build/setup.sh
```

#### Build tools

```
m tools
```

#### Build OpenTitan HelloVector Demo

```
m shodan_test_sw_bootrom
m shodan_test_sw_hellovector
```

#### Start OpenTitan-HelloVector Renode Script

```
renode -e "i @sim/config/shodan_hellovector.resc; start"
```

### Modify Vector OpCode Behavior and Simulate

To modify the functionality of an opcode, we will be modifying the following files:

`$ROOTDIR/sim/renode/src/Infrastructure/src/Emulator/Cores/tlib/arch/riscv/translate.c`
`$ROOTDIR/sim/renode/src/Infrastructure/src/Emulator/Cores/tlib/arch/riscv/instmap.h`

#### Add new opcode in `instmap.h`'s enum

To add a new opcode, create a new entry in `instmap.h`:

```
enum {
    OPC_RISC_VADD_IVV           = OPC_RISC_V_ARITH_IVV | (0x0 << 26),
    OPC_RISC_VSUB_IVV           = OPC_RISC_V_ARITH_IVV | (0x2 << 26),
    OPC_RISC_VMINU_IVV          = OPC_RISC_V_ARITH_IVV | (0x4 << 26),
    YOUR_OPC_HERE               = OPC_RISC_V_ARITH_IVV | (0x8 << 26),
```

#### Add `switch...case` entry and definition into `translate.c`

Open `translate.c` and look for the following function (which will be around line 1450):

```
static void gen_v_arith_ivv(DisasContext *dc, uint32_t opc, int vd, int vs1, int vs2) {
  tlib_printf(LOG_LEVEL_ERROR, "V_ARITH_IVV");
```
In this function there will be a `switch case`.
Add an entry here for your new opcode, and fill out the behavior with tcg-lang.

```
switch (opc) {
    case YOUR_OPC_HERE:
        tlib_printf(LOG_LEVEL_ERROR, "OpCode Not Yet Defined");
        break;
```

[Link to TCG Documentation for Reference](https://wiki.qemu.org/Documentation/TCG)

#### Rebuild Renode 

```
m renode_clean
m renode
```

#### Add OpCode Test into `hello_vector.c`

Wrap the assembly within an `__asm__` block and add to `hello_vector.c`:

```
$ROOTDIR/hw/opentitan/sw_shodan/device/examples/hello_vector/hello_vector.c
```

#### Rebuild `shodan_test_sw_hellovector` target

If `hello_vector.c` was modified, run the following to rebuild the `shodan_test_sw_hellovector` target:

```
m shodan_test_sw_hellovector
```

#### Run Renode Script:

```
renode -e "i @sim/config/shodan_hellovector.resc; start"
```

#### Quicklinks to Relevant Files for OpCode Modification/Testing:

- `$ROOTDIR/build/shodan_test_sw.mk` -- build file for `hellovector` example
- `$ROOTDIR/sim/config/shodan_hellovector.resc` -- renode script which loads ELFs into memory
- `$ROOTDIR/hw/opentitan/sw_shodan/device/examples/hello_vector/hello_vector.c` -- add assembly here (in `__asm__` blocks) for simulation
- `$ROOTDIR/sim/renode/src/Infrastructure/src/Emulator/Cores/tlib/arch/riscv/translate.c` -- where to add or modify opcode behavior
- `$ROOTDIR/sim/renode/src/Infrastructure/src/Emulator/Cores/tlib/arch/riscv/instmap.h` -- where to add numerical opcode definitions


### Add New Vector OpCode and Simulate

To add a new vector opcode, we will be modifying the following toolchain files:
- `$ROOTDIR/toolchain/riscv-binutils/include/opcode/riscv-opc.h`
- `$ROOTDIR/toolchain/riscv-binutils/opcodes/riscv-opc.c`

And also these files for defining the behavior in simulation:
- `$ROOTDIR/sim/renode/src/Infrastructure/src/Emulator/Cores/tlib/arch/riscv/translate.c`
- `$ROOTDIR/sim/renode/src/Infrastructure/src/Emulator/Cores/tlib/arch/riscv/instmap.h`

#### Step 1: Obtain the RiscV OpCode Repo

Clone the [RiscV OpCode Support Repo](https://github.com/riscv/riscv-opcodes)


#### Step 2: Add Vector Opcode definition to `opcodes-rvv` file:

`cd` into the `riscv-opcodes` folder, add the instruction to the `opcodes-rvv` file:

Example for `vrgatherei16.vv`, add the following line to `opcodes-rvv`:
```
vrgatherei16.vv 31..26=0x0e vm vs2 vs1 14..12=0x0 vd 6..0=0x57
```

(Note the function6 opcode `0x0e` was determined in this case with the [RiscV Vector OpCode Table](https://github.com/riscv/riscv-v-spec/blob/master/inst-table.adoc));

#### Step 3: Utilize the contained python script `parse_opcodes` to generate `MASK` and `MATCH` codes for the new instruction

Use the `parse_opcodes` script within the `riscv-opcodes` repo to create `MATCH`
and `MASK` macros for the new operation.

Example:
```
python3 parse_opcodes opcodes-rvv > temp_opc.h
```

This will yield these lines within the output `temp_opc.h` file:
```
#define MATCH_VRGATHEREI16VV   0x38000057
#define MASK_VRGATHEREI16VV    0xfc00707f
```

#### Step 4: Add these two lines to the `riscv-opc.h` file located at:

Place the above two macros in a reasonable location within the `riscv-opc.h`
file.

This file is located:
- `$ROOTDIR/toolchain/riscv-binutils/include/opcode/riscv-opc.h`

#### Step 5: Add an entry to the RVV Opcode Struct in the `riscv-opc.c` file:

Take care in spelling in the assembly instruction name (in this example `vrgatherei16.vv`)
as well as spelling in the mask and match names (in this example `MATCH_VRGATHEREI16VV` and `MASK_VRGATHEREI16VV`):

```
{"vrgatherei16.vv",0, INSN_CLASS_V, "Vd,Vt,VsVm", MATCH_VRGATHEREI16VV, MASK_VRGATHEREI16VV, match_vd_neq_vs1_neq_vs2_neq_vm, 0},

```

#### Step 6: Rebuild Relevant RiscV Toolchain

Will need to rebuild the gnu-toolchain to implement the changes:

```
m clean
m tools
```

#### Step 7: Add new opcode in `instmap.h`'s enum

Now we need to define the behavior in the Renode simulator.

For Renode to interpret this new opcode, create a new entry in `instmap.h`:

Note to use the `function6` opc in hex:
```
    OPC_RISC_VXOR_IVV           = OPC_RISC_V_ARITH_IVV | (0xB << 26),
    OPC_RISC_VRGATHER_IVV       = OPC_RISC_V_ARITH_IVV | (0xC << 26),
    OPC_RISC_VRGATHEREI16_IVV   = OPC_RISC_V_ARITH_IVV | (0xE << 26),
```
See the [RiscV Vector OpCode Table](https://github.com/riscv/riscv-v-spec/blob/master/inst-table.adoc) for guidance.


#### Step 8: Define behavior within the `switch...case` within the `translate.c` file

For example, we can provide means for testing this function with 8 bit elements
of a 512 bit register length (i.e. SEW=8, VLEN=512, LMUL=1)

```
  case OPC_RISC_VRGATHEREI16_IVV:
      tlib_printf(LOG_LEVEL_ERROR, "VRGATHEREI16_IVV");
      for (int i = 0; i < 64; ++i) {
        vreg.e8[vd][i] = (vreg.e8[vs1][i] > VLMAX)? 0 : vreg.e8[vs2][vreg.e8[vs1][i]];
      }
      break;
```

`translate.c` file is located:
- `$ROOTDIR/sim/renode/src/Infrastructure/src/Emulator/Cores/tlib/arch/riscv/translate.c`

#### Step 9: Rebuild Renode

```
m renode_clean
m renode
```

Note: Care to look out for errors or warnings which may halt compilation (our make
will halt compilation if there are any unused values for example).


#### Step 10: Add opcode test to `hello_vector.c`

```
LOG_INFO("trying vrgatherei16");
__asm__ volatile("vrgatherei16.vv v0, v1, v2");
```

#### Step 11: Build OpenTitan HelloVector Demo

```
m shodan_test_sw_bootrom
m shodan_test_sw_hellovector
```

#### Step 12: Run Renode Script

```
renode -e "i @sim/config/shodan_hellovector.resc; start"
```

#### Quicklinks Specific for Creating/Defining new OpCodes:

- [RiscV OpCode Support Repo](https://github.com/riscv/riscv-opcodes) -- tools for adding new instructions and creating new opcodes

Toolchain Side:
- `$ROOTDIR/toolchain/riscv-binutils/include/opcode/riscv-opc.h` -- where to add new opcode definitions
- `$ROOTDIR/toolchain/riscv-binutils/opcodes/riscv-opc.c`-- gnu riscv-opcode struct definitions (also needs modification)

Renode Simulation Side:
- `$ROOTDIR/sim/renode/src/Infrastructure/src/Emulator/Cores/tlib/arch/riscv/translate.c` -- where to add or modify renode opcode sim behavior
- `$ROOTDIR/sim/renode/src/Infrastructure/src/Emulator/Cores/tlib/arch/riscv/instmap.h` -- where to add new opcodes for renode to simulate

## General References

- [RiscV v0.9 Vector Extension Draft](https://github.com/riscv/riscv-v-spec/releases/tag/0.9)
- [RiscV Vector OpCode Table](https://github.com/riscv/riscv-v-spec/blob/master/inst-table.adoc)