# What is this?

This is Project Shodan, a project to research the fusion of novel hardware and
software architectures to produce a low-power, ambient AI core. For more
information, see our
[internal site](https://sites.google.com/corp/google.com/cerebrahardware/projects/current-projects/shodan).

## Developing in this Codebase

We've stored our code in Gerrit, and like the Android developers before us, we
use `repo` to manage the projects in our Gerrit repositories.

To get started, first make sure you have a Git login for all our projects
by going to [googlesource.com/new-password](https://www.googlesource.com/new-password)
and pasting the provided script into a terminal.

Now you need to pull down a copy of the `repo`
tool from our public facing sites and add it to your path:

```bash
mkdir -p bin
export PATH=$PATH:$HOME/bin
curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo
```

Make sure you've initialized git with your name and email address, and have
configured it properly for fetching the sources:

```bash
git config --global user.name "Your Name"
git config --global user.email "you@example.com"
```

Once you've done this, you're actually ready to check out the sources. Make a
new directory where you'd like it to live, and initialize `repo` with the
current release branch.

```bash
repo init -u https://spacebeaker.googlesource.com/shodan/manifest -g default,internal --no-use-superproject
repo sync -j$(nproc)
```

## System Setup for Development

Development for shodan requires that the necessary tools and prerequisites be
installed.

To setup the build system:

```bash
source build/setup.sh
```

Install the prerequisites:

```bash
m prereqs
```

Install the tools used for development:

```bash
m tools
```

## Day-to-day Development Workflow

In general, working with repo is relatively simple:

  1. Create a working topic branch to make changes on with `repo start
     ${TOPICNAME}`
  2. Make your changes to the files. Add them with `git add -A` and commit with
     `git commit`.
  3. Upload your changes with `repo upload --re ${REVIEWER} --cc ${CC_LIST}`
  4. Go to the URL repo spits out to read and reply to comments.
  5. Eventually your reviewer will give you a +2 LGTM on your change. To submit,
     click the "SUBMIT" button on the change in the web interface.
  6. Run `repo sync -j$(nproc)` to update.
  7. Run `repo prune` to remove your topic branch.

For more information on how to use repo and git effectively, take a look at the
[official documentation](https://source.android.com/setup/create/coding-tasks).

### A Note on Code Review Policies

In the Shodan project, we follow a "sticky +2" policy.

What this means is that you *must* get a +2 Code Review from a peer, or a TL.
Gerrit will automatically remove any +2 Code Review scores you receive when you
upload a patchset. In this case, once a peer has given you a +2 Code Review, you
are free to re-+2 your change yourself if the changes are minor or otherwise
address comments in the CL.

If you change significant parts of the CL post-+2, please ask for additional
review from your peers in a reply. Note: two +1s does *not* count as a +2!

You may *not* self-+2 your own CLs, no matter the urgency or how simple the
change is! Resist the temptation! This policy is in place so that we can quickly
get through code review and get our code submitted without requiring lots of
round-trips with your reviewers.

Please don't abuse this policy and use good judgement: if you don't know who to
send a CL to, put one of the TLs on the review list, and they'll redirect it
appropriately, or review your change. If we abuse the policy by +2-ing before
peer review, this privilige will unfortunately have to be revoked, slowing down
the entire team in the process, and possibly getting yourself in trouble with
security/compliance.

### Who are the TLs?

At the time of this writing, the TLs are:

Cindy Liu <hcindyl@>
June Tate-Gans <jtgans@>
Bangfei Pan <pbf@>
Steve Xu <stevexu@>

Finally, if you can't get one of the above people, please put Kai Yick
<kingkai@> on the review line.

*Special note if you're listed above*: you still must get your +2 from someone
else -- we need the TLs to lead by example and demonstrate the appropriate
behavior.

### How to sync my local copy with latest?

```bash
repo sync -j$(nproc)
```

Then if you have any outstanding branches, a `repo rebase` will help.

### How to send code for review?

To upload a branch to gerrit for review, do this:

```bash
repo upload --re reviewer1,reviewer2 --cc email@host.com,email2@host2.com
```

Reviewers can be specified as usernames or full email addresses, likewise for
`--cc`.

Repo will then output a URL for you to visit that allows you to make comments
and abandon and merge the changes into the repository. To make changes during
the review process, make your changes to the files, then:

```bash
git add -A                                    # To add the files you've changed
git commit --amend                            # To update the previous change
repo upload -t --re ${REVIEWER} --cc ${CC_LIST} # To upload the change to Gerrit for review
```

## Repository Layout

Our layout is pretty simple:

### build/

Contains build scripts for the whole tree. This is effectively just an
orchestration layer to make building the whole shebang easier. Each subtree may
have its own build systems and have their own ways of building.

### cache/

The cached cross-compilation toolchain, including rust and RISC-V GCC/LLVM
toolchain.

### cicd/

Contains continuous integration scripts and tooling for Jenkins, our CI/CD tool.

### docs/

Lots of extra documentation (we hope) about how the repo is laid out, how the
build system works, code reviews, licensing, etc.

### hw/

Contains all of the source code and RTL required to build the Shodan
hardware, as well as simulate it in Verilator.

#### opentitan

Security core.

#### ibex

System management controller (SMC).

#### springbok

Vector core for ML acceleration.

### kata/

Operating system software for the SMC; including seL4 kernel & CAmkES framework,
and custom CAmkES components that support Shodan (or maybe KataOS) applications.

### manifest/

The repo manifest used to glue all the git repositories together.

### scripts/

Contains utility scripts to help automate a few things.

### sim/

Contains tools and src for simulators (Renode and Verilator) of the shodan system.

### sw/

Contains the source code of applications running in all shodan cores.

#### libtock-rs

**TODO**: add more details

#### multihart_boot_rom

Bootstrap for System Management Controller, Security Core, and Vector Core. This
is the first software to run after reset; it does low-level hardware setup and
starts TockOs (SC) and seL4 (SMC).

#### pigweed

pigweed frameworks. Currently it is used for vector core functional tests.

#### tock

The operating system running on the Security Core.

#### vec

Springbok BSP, as well as the RVV instruction functional tests.

#### vec_iree

Springbok IREE application. It builds IREE runtime applications for ML models
using IREE libraries and Spingbok BSP.

### toolchain/

Contains the src to build the RISCV QEMU emulator, and
[IREE](https://github/com/google/iree) toolchain for ML models.

## Build and Test ML Artifacts

The ML executable is built with [IREE](https://github.com/google/iree) workflow,
targeted to RISCV 32-bit bare-metal config.

To build the IREE targets:

```bash
m iree
```

The IREE compiler sits in `out/host/iree_compiler`, while the runtime library/example
sits in `out/springbok_iree`.
To run the toy example (four-element vector element-wise multiplication) for
testing:

```bash
sim_springbok out/springbok_iree/samples/simple_vec_mul/simple_int_vec_mul_embedded_sync
```

The output should be shown as:

```bash
21:27:11.0241 [INFO] cpu2: simprint: "INFO |simple_vec_mul finished successfully", 0 (0x0)
21:27:11.0248 [INFO] cpu2: simprint: "main returned: ", 0 (0x0)
```

(Enter `quit` or `q` to exit the Renode simulation)

## Running The Full Shodan System Simulation

The simulator used for Shodan is [Renode](https://renode.io/). The configuration
for the Shodan system is `sim/config/platforms/shodan.repl`, while and starting script
is in `sim/config/shodan.resc`

To run the full system simulation, build the default target:

```bash
m
```

After all the artifacts are bulit, the Renode simulation session automatically
starts, and you should see the secured core and SMC booted

```bash
10:40:14.6784 [INFO] uart1: [output] init_kernel()
10:40:14.6807 [INFO] uart1: [output] Init local IRQ
10:40:14.6831 [INFO] uart1: [output] Bootstrapping kernel
10:40:14.6842 [INFO] uart1: [output] Initialing PLIC...
10:40:14.7028 [INFO] uart0: [output] OpenTitan initialisation complete. Entering main loop
10:40:14.7066 [INFO] uart0: [output] Woo Tock!
10:40:14.7346 [INFO] uart0: [output] Hello Tock World
10:40:15.8820 [INFO] uart1: [output] Booting all finished, dropped to user space
```

At this point, you should be able to connect to the SMC debug console.

### Setting Up Debug Console Communication

Install socat to your machine

```bash
sudo apt install socat
```

In a new shell session, launch the debug console with

```bash
scripts/kshell.sh
```

You will see the kataOS prompt showing up when you hit `enter`. Use `ctrl-c` to leave the console.

### Connecting to Renode Console

Renode has its own console to control the emulation environment. You can connect to it by

```bash
telnet localhost 1234
```

For example, you can check the core status

```bash
(matcha) cpu1 IsHalted
False
```

or stop the whole emulation with

```bash
(matcha) quit
```

### Launch ML Job on the Vector Core

You can launch the ML workload execution by communicating through the debug console

```bash
KATA> test_mlexecute
```

and you should see

```bash
15:17:59.9864 [INFO] uart1: [output] KATA> test_mlexecute
...
15:18:02.6868 [INFO] cpu2: simprint: "main returned: ", 0 (0x0)

```

## More Information

For more available Shodan build targets, please see [Build target lists](./BuildTargetsExplained.md).

Also, [Information on how to use repo](https://go/repo)