Merge remote-tracking branch 'upstream/main' into update Bypass-Presubmit-Reason: no presubmit flows configured Change-Id: I2f44cf659a31e3740e78025fe420c3723c40c168
diff --git a/docs/GettingStarted.md b/docs/GettingStarted.md index d115e68..d97e2b0 100644 --- a/docs/GettingStarted.md +++ b/docs/GettingStarted.md
@@ -79,6 +79,7 @@ - The LLVM-based toolchain with CHERIoT support - The emulator generated from the Sail formal model of the CHERIoT ISA - The xmake build tool + - [Sonata only] u2futils to create images that Sonata's loader can boot Building LLVM is fairly simple, but requires a fast machine and several GiBs of disk space. Building the executable model requires a working ocaml installation. @@ -223,6 +224,29 @@ # apt install xmake ``` +### Installing u2futils + +If you are working with Sonata, you will need to convert the ELF files that the linker produces to [USB Flashing Format (UF2)](https://github.com/microsoft/uf2). +The firmware on the RPi 2040 on the Sonata board can then load these files onto the CHERIoT Ibex and run them. + +The [uf2utils](https://github.com/makerdiary/uf2utils) project provides a tool to generate this format, which can be installed from pip. +You will need to have Python 3 and pip installed. +You can then follow their instructions to install. +On *NIX systems, this is typically simply: + +``` +python3 -m pip install --pre -U git+https://github.com/makerdiary/uf2utils.git@main +``` + +Or the following on Windows: + +``` +py -3 -m pip install --pre -U git+https://github.com/makerdiary/uf2utils.git@main +``` + +The `xmake run` command for firmware using the `sonata` board will automatically invoke this correctly and provide the correct file. + + Running on the Arty A7 ---------------------- @@ -352,3 +376,99 @@ Test runner: Global(0x0) ... ``` + +Running on Sonata +----------------- + +The Sonata board from lowRISC is an FPGA prototyping platform designed specifically to work with the CHERIoT Ibex. +If you have installed a [release of their system firmware](https://github.com/lowRISC/sonata-system/releases) that is at least v0.2, everything should be ready to work out of the box. + +### Building, Copying, and Running the Firmware + +The steps to build firmware are the same as for any other CHERIoT hardware, this time specifying `sonata` as the board. +Let's build the test suite again: + +``` +$ cd tests +$ xmake config --sdk=/cheriot-tools/ --board=sonata +$ xmake +``` + +Note that /cheriot-tools is the location in the dev container. +This value may vary if you did not use the dev container and must be the directory containing a bin directory with your LLVM build in it. + +### Running the firmware + +You can usually run firmware on Sonata simply with `xmake run`: + +``` +$ xmake run +[100%]: build ok, spent 4.52s +Converted to uf2, output size: 258048, start address: 0x101000 +Wrote 258048 bytes to test-suite.uf2 +Firmware copied to /Volumes//SONATA/ +``` + +If the last line instead says something like the following, one extra step is needed: + +``` +Please copy {some path}/cheriot-rtos/tests/build/cheriot/cheriot/release/firmware.uf2 to the SONATA drive to load. +``` + +If your SONATA board is connected, you should have a filesystem mounted called `SONATA` as a USB mass storage device. +You can copy the file mentioned in this line to the `SONATA` drive to run it. +The script that `xmake run` invokes looks in common mount locations for macOS and some Linux distributions to try to find the SONATA device. + +If you are running in the dev container, you are in an isolated environment with no access to this filesystem. +On most systems, you can add this as an explicit mount by adding the following to the [`devcontainer.json`](https://github.com/microsoft/cheriot-rtos/blob/main/.devcontainer/devcontainer.json) file: + +```json + "mounts": [ + "source={/your/SONATA/mount/point},target=/mnt/SONATA,type=bind" + ] +``` + +Replacing `{/your/SONATA/mount/point}` with the mount point of your `SONATA` filesystem. +Your editor should prompt to rebuild the container at this point. + +If you invoked the dev container directly, rather than via an editor such as VS Code, then you will need to instead add the following to your `docker run` command: + +``` +--mount source={/your/SONATA/mount/point},target=/mnt/SONATA,type=bind +``` + +Again, replacing `{/your/SONATA/mount/point}` with the real mount point. + +If you are on Windows, this will not work because Docker on Windows runs containers in a WSL2 VM and needs to have host filesystems mounted explicitly (Docker on macOS does this automatically). +In theory, the following steps should work (run in the WSL2 terminal in the Docker WSL2 VM): + +``` +sudo mkdir /mnt/d +sudo mount -t drvfs D: /mnt/d/ +``` + +Assuming that `SONATA` is mounted as `D:` on Windows. +The above commands should then work. +Unfortunately, they do not. +The cause of this is not currently known but will hopefully be fixed in a newer version of the Sonata RPi2040 firmware. + +### Seeing UART output + +If you ran the above `xmake run` commands, you will have noted that there is no output from the test suite. +When you use a simulator, the UART output goes directly to the simulator's standard output. +In contrast, when you run on an FPGA the output will go to a USB serial device. + +You will need to connect a terminal emulator to the corresponding serial port. +The Sonata system exposes *four* serial devices, typically numbered 0-3. +The UART is device 2. +On Linux, this is usually `/dev/ttyUSB2` (if no other USB serial devices are connected). +On macOS, the device name includes the serial number of the device and so will be something like `/dev/tty.usbserial-LN28292` (the last digit is the serial port number). +On Windows, it will be a COM port, but unfortunately Windows is not always consistent about the order in which the ports appear. + +If you install a serial terminal (for example, `minicom` on *NIX or PuTTY on Windows), you can point it at the relevant serial port. + +The serial setup is the same as for the Arty A7 [above](#installing-and-running-the-firmware), aside from the different device name. + +Once a serial terminal is connected, running `xmake run` should show the output from the test suite. + +If you are unsure which of Sonata's serial ports is the correct one, you can connect your serial console to all four and see which produces output from the test suite.
diff --git a/sdk/boards/sonata.json b/sdk/boards/sonata.json index 7d8433a..ad5cc7e 100644 --- a/sdk/boards/sonata.json +++ b/sdk/boards/sonata.json
@@ -4,6 +4,10 @@ "start" : 0x30000000, "end" : 0x30004000 }, + "gpio" : { + "start" : 0x80000000, + "end" : 0x80000020 + }, "clint": { "start" : 0x80040000, "end" : 0x80050000 @@ -12,9 +16,29 @@ "start" : 0x80100000, "end" : 0x80100034 }, - "gpio" : { - "start" : 0x80000000, - "end" : 0x80000020 + "i2c0": { + "start" : 0x80200000, + "end" : 0x80200080 + }, + "i2c1": { + "start" : 0x80201000, + "end" : 0x80201080 + }, + "spi0": { + "start" : 0x80300000, + "end" : 0x80301000 + }, + "spi1": { + "start" : 0x80301000, + "end" : 0x80302000 + }, + "spi2": { + "start" : 0x80302000, + "end" : 0x80303000 + }, + "rgbled" : { + "start" : 0x80009000, + "end" : 0x80009020 }, "plic": { "start" : 0x88000000, @@ -50,6 +74,7 @@ "timer_hz" : 30000000, "tickrate_hz" : 100, "revoker" : "software", + "stack_high_water_mark" : true, "simulator" : "${sdk}/../scripts/run-sonata.sh", "simulation": false }
diff --git a/sdk/core/allocator/alloc.h b/sdk/core/allocator/alloc.h index 9d63a74..20ba2da 100644 --- a/sdk/core/allocator/alloc.h +++ b/sdk/core/allocator/alloc.h
@@ -1099,9 +1099,17 @@ /** * Tag type indicating that the requested allocation cannot succeed until + * some objects have been freed in the passed quota. + */ + struct AllocationFailureQuotaExceeded + { + }; + + /** + * Tag type indicating that the requested allocation cannot succeed until * some objects have been freed. */ - struct AllocationFailureDeallocationNeeded + struct AllocationFailureHeapFull { }; @@ -1110,7 +1118,8 @@ */ using AllocationResult = std::variant<AllocationFailurePermanent, AllocationFailureRevocationNeeded, - AllocationFailureDeallocationNeeded, + AllocationFailureQuotaExceeded, + AllocationFailureHeapFull, CHERI::Capability<void>>; /** @@ -1126,7 +1135,8 @@ * object. This allows it to be skipped when freeing all objects allocated * with a given quota. * - * @return User pointer if request can be satisfied, nullptr otherwise. + * @return User pointer if request can be satisfied, or a tag type + * representing the error otherwise. */ AllocationResult mspace_dispatch(size_t bytes, size_t "a, @@ -1161,7 +1171,7 @@ "quota is {})", alignSize, quota); - return AllocationFailureDeallocationNeeded{}; + return AllocationFailureQuotaExceeded{}; } CHERI::Capability<void> ret{mspace_memalign( alignSize, -CHERI::representable_alignment_mask(bytes))}; @@ -1180,7 +1190,7 @@ { return AllocationFailurePermanent{}; } - return AllocationFailureDeallocationNeeded{}; + return AllocationFailureHeapFull{}; } auto header = MChunkHeader::from_body(ret); @@ -1194,7 +1204,7 @@ header->size_get(), quota); mspace_free_internal(header); - return AllocationFailureDeallocationNeeded{}; + return AllocationFailureQuotaExceeded{}; } if constexpr (DEBUG_ALLOCATOR)
diff --git a/sdk/core/allocator/main.cc b/sdk/core/allocator/main.cc index 29a652b..75743ba 100644 --- a/sdk/core/allocator/main.cc +++ b/sdk/core/allocator/main.cc
@@ -240,7 +240,8 @@ LockGuard<decltype(lock)> &&g, PrivateAllocatorCapabilityState *capability, Timeout *timeout, - bool isSealedAllocation = false) + bool isSealedAllocation = false, + uint32_t flags = AllocateWaitAny) { check_gm(); @@ -254,8 +255,9 @@ { return std::get<Capability<void>>(ret); } - // If the timeout is 0, fail now. - if (!may_block(timeout)) + // If the call is non-blocking (`flags` is + // `AllocateWaitNone`, or `timeout` is 0), fail now. + if (flags == AllocateWaitNone || !may_block(timeout)) { return nullptr; } @@ -265,6 +267,13 @@ std::get_if<MState::AllocationFailureRevocationNeeded>(&ret); if (needsRevocation) { + if (!(flags & AllocateWaitRevocationNeeded)) + { + // The flags specify that we should not + // wait when revocation is needed. + return nullptr; + } + // If we are able to dequeue some objects from quarantine then // retry immediately, otherwise yield. // @@ -291,7 +300,7 @@ // Drop and reacquire the lock while yielding. // Sleep for a single tick. g.unlock(); - Timeout smallSleep{0}; + Timeout smallSleep{1}; thread_sleep(&smallSleep); if (!reacquire_lock(timeout, g, smallSleep.elapsed)) { @@ -303,9 +312,24 @@ } // If the heap is full, wait for someone to free an allocation and // then retry. - if (std::holds_alternative< - MState::AllocationFailureDeallocationNeeded>(ret)) + bool isHeapFullFailure = + std::holds_alternative<MState::MState::AllocationFailureHeapFull>( + ret); + bool isQuotaExceededFailure = + std::holds_alternative<MState::AllocationFailureQuotaExceeded>( + ret); + if (isHeapFullFailure || isQuotaExceededFailure) { + if ((isHeapFullFailure && !(flags & AllocateWaitHeapFull)) || + (isQuotaExceededFailure && + !(flags & AllocateWaitQuotaExceeded))) + { + // The flags specify that we should not + // wait when the heap is full and/or + // when the quota is exceeded. + return nullptr; + } + Debug::log("Not enough free space to handle {}-byte " "allocation, sleeping", bytes); @@ -832,11 +856,12 @@ } } -__cheriot_minimum_stack(0x1f0) void *heap_allocate(Timeout *timeout, +__cheriot_minimum_stack(0x200) void *heap_allocate(Timeout *timeout, SObj heapCapability, - size_t bytes) + size_t bytes, + uint32_t flags) { - STACK_CHECK(0x1f0); + STACK_CHECK(0x200); if (!check_timeout_pointer(timeout)) { return nullptr; @@ -853,7 +878,7 @@ return nullptr; } // Use the default memory space. - return malloc_internal(bytes, std::move(g), cap, timeout); + return malloc_internal(bytes, std::move(g), cap, timeout, false, flags); } __cheriot_minimum_stack(0x1b0) ssize_t @@ -955,12 +980,13 @@ return freed; } -__cheriot_minimum_stack(0x1f0) void *heap_allocate_array(Timeout *timeout, +__cheriot_minimum_stack(0x200) void *heap_allocate_array(Timeout *timeout, SObj heapCapability, size_t nElements, - size_t elemSize) + size_t elemSize, + uint32_t flags) { - STACK_CHECK(0x1f0); + STACK_CHECK(0x200); if (!check_timeout_pointer(timeout)) { return nullptr; @@ -982,7 +1008,7 @@ { return nullptr; } - return malloc_internal(req, std::move(g), cap, timeout); + return malloc_internal(req, std::move(g), cap, timeout, false, flags); } namespace @@ -1031,14 +1057,14 @@ __noinline allocate_sealed_unsealed(Timeout *timeout, SObj heapCapability, SealingKey key, - size_t sz, + size_t requestedSize, PermissionSet permissions) { - if (!check_pointer<PermissionSet{Permission::Load, Permission::Store}>( - timeout)) + if (!check_timeout_pointer(timeout)) { return {nullptr, nullptr}; } + if (!permissions.can_derive_from(key.permissions())) { Debug::log( @@ -1046,11 +1072,28 @@ return {nullptr, nullptr}; } - if (sz > 0xfe8 - ObjHdrSize) + // Round up the size to the next representable size. This ensures + // that, once we've added the header space, we have an allocation where + // both the object (from the end) and the header (with padding at the + // start) are representable. + size_t unsealedSize = CHERI::representable_length(requestedSize); + // Very large sizes may be rounded 'up' to zero. Don't allow this. + if (unsealedSize == 0) { - Debug::log("Cannot allocate sealed object of {} bytes, too large", - sz); - // TODO: Properly handle imprecision. + Debug::log("Requested size {} is not representable", requestedSize); + return {nullptr, nullptr}; + } + + // It shouldn't be possible to overflow the add due to the way the + // rounding works, but this is not guaranteed in future capability + // encodings, so we'll do a tiny bit of extra work here to avoid + // accidentally introducing a security vulnerability in a future + // encoding. + size_t sealedSize = unsealedSize + ObjHdrSize; + if (__builtin_add_overflow(ObjHdrSize, unsealedSize, &sealedSize)) + { + Debug::log("Requested size {} is too large to include header", + requestedSize); return {nullptr, nullptr}; } @@ -1061,19 +1104,32 @@ return {nullptr, nullptr}; } SealedAllocation obj{static_cast<SObj>(malloc_internal( - sz + ObjHdrSize, std::move(g), capability, timeout, true))}; + sealedSize, std::move(g), capability, timeout, true))}; if (obj == nullptr) { Debug::log("Underlying allocation failed for sealed object"); return {nullptr, nullptr}; } + obj.address() = obj.top() - sealedSize; + // Round down the base to the heap alignment size. + // This ensures that the header is aligned and gives the same alignment + // as a normal allocation. We will already be this aligned for most + // requested sizes. The allocator always aligns the top and bottom of + // any allocations on a `MallocAlignment` boundary, and also on a + // representable boundary (whichever is stricter). This extra + // alignment step ensures that this is also true for both the start of + // the header and the start of the unsealed object. If the + // representable-length rounding of the requested size increased the + // requested alignment beyond 8, this will be a no-op. + obj.align_down(MallocAlignment); obj->type = key.address(); auto sealed = obj; sealed.seal(SEALING_CAP()); obj.address() += ObjHdrSize; // Exclude the header. - obj.bounds() = obj.length() - ObjHdrSize; - Debug::log("Allocated sealed {}, unsealed {}", sealed, obj); + obj.bounds() = obj.top() - obj.address(); + Debug::Assert( + obj.is_valid(), "Unsealed object {} is not representable", obj); return {sealed, obj}; } } // namespace @@ -1101,14 +1157,14 @@ return nullptr; } -__cheriot_minimum_stack(0x250) SObj +__cheriot_minimum_stack(0x270) SObj token_sealed_unsealed_alloc(Timeout *timeout, SObj heapCapability, SKey key, size_t sz, void **unsealed) { - STACK_CHECK(0x250); + STACK_CHECK(0x270); if (!check_timeout_pointer(timeout)) { return INVALID_SOBJ; @@ -1147,23 +1203,18 @@ __noinline static SealedAllocation unseal_internal(SKey rawKey, SObj obj) { SealingKey key{rawKey}; + Capability unsealedInner = token_unseal<void>(key, obj); + if (!unsealedInner.is_valid()) + { + return nullptr; + } if (!key.permissions().contains(Permission::Unseal)) { return nullptr; } - auto unsealed = unseal_if_valid(obj); - if (!unsealed) - { - return nullptr; - } - if (unsealed->type != key.address()) - { - return nullptr; - } - - return unsealed; + return unseal_if_valid(obj); } __cheriot_minimum_stack(0x250) int token_obj_destroy(SObj heapCapability,
diff --git a/sdk/core/allocator/token.h b/sdk/core/allocator/token.h index 1c75670..e9ce8f7 100644 --- a/sdk/core/allocator/token.h +++ b/sdk/core/allocator/token.h
@@ -17,6 +17,29 @@ /// Opaque type. Sealing keys don't really point to anything. struct SKeyStruct; +/** + * The structure that represents a sealed object. + * + * The `data` field represents the unsealed object. There is some subtle + * complexity here: We must be able to precisely represent the unsealed object + * but we must *also* be able to precisely represent the entire sealed object. + * + * For larger objects, the alignment of the top and bottom are stricter and so + * we cannot simply place this header at the start of the allocation. Instead, + * we arrange the objects in the allocation as follows: + * + * ``` + * |Padding|Header|Unsealed Object| + * ^ + * Address of sealed capability is here. + * ``` + * + * This means that there is a small amount of padding at the start of the + * allocation, but the address of the capability points to the start of the + * header. For anything below 4072 bytes with the current encoding, this + * amount of padding is zero and then we gradually increase it as the object + * size increases. + */ struct SObjStruct { /// The sealing type for this object.
diff --git a/sdk/core/scheduler/main.cc b/sdk/core/scheduler/main.cc index 88442be..95ef0e2 100644 --- a/sdk/core/scheduler/main.cc +++ b/sdk/core/scheduler/main.cc
@@ -270,15 +270,20 @@ ExceptionGuard g{[=]() { sched_panic(mcause, mepc, mtval); }}; + bool tick = false; switch (mcause) { // Explicit yield call case MCAUSE_ECALL_MACHINE: - schedNeeded = true; + { + schedNeeded = true; + Thread *currentThread = Thread::current_get(); + tick = currentThread && currentThread->is_ready(); break; + } case MCAUSE_INTR | MCAUSE_MTIME: - Timer::do_interrupt(); schedNeeded = true; + tick = true; break; case MCAUSE_INTR | MCAUSE_MEXTERN: schedNeeded = false; @@ -293,6 +298,7 @@ std::tie(schedNeeded, std::ignore, std::ignore) = futex_wake(Capability{&word}.address()); }); + tick = schedNeeded; break; case MCAUSE_THREAD_EXIT: // Make the current thread non-runnable. @@ -305,13 +311,23 @@ // We cannot continue exiting this thread, make sure we will // pick a new one. schedNeeded = true; + tick = true; sealedTStack = nullptr; break; default: sched_panic(mcause, mepc, mtval); } + if (tick || !Thread::any_ready()) + { + Timer::expiretimers(); + } auto newContext = schedNeeded ? Thread::schedule(sealedTStack) : sealedTStack; +#if 0 + Debug::log("Thread: {}", + Thread::current_get() ? Thread::current_get()->id_get() : 0); +#endif + Timer::update(); if constexpr (Accounting) { @@ -351,7 +367,7 @@ if (shouldYield) { - Thread::yield_interrupt_enabled(); + yield(); } return ret; @@ -419,21 +435,24 @@ } __cheriot_minimum_stack(0x80) int __cheri_compartment("sched") - thread_sleep(Timeout *timeout) + thread_sleep(Timeout *timeout, uint32_t flags) { STACK_CHECK(0x80); if (!check_timeout_pointer(timeout)) { return -EINVAL; } - Thread::current_get()->suspend(timeout, nullptr, true); + // Debug::log("Thread {} sleeping for {} ticks", + // Thread::current_get()->id_get(), timeout->remaining); + Thread *current = Thread::current_get(); + current->suspend(timeout, nullptr, true, !(flags & ThreadSleepNoEarlyWake)); return 0; } __cheriot_minimum_stack(0xa0) int futex_timed_wait(Timeout *timeout, const uint32_t *address, uint32_t expected, - FutexWaitFlags flags) + uint32_t flags) { STACK_CHECK(0xa0); if (!check_timeout_pointer(timeout) || @@ -468,8 +487,10 @@ // If we try to block ourself, that's a mistake. if ((owningThread == currentThread) || (owningThread == nullptr)) { - Debug::log("futex_timed_wait: invalid owning thread {}", - owningThread); + Debug::log("futex_timed_wait: thread {} acquiring PI futex with " + "invalid owning thread {}", + currentThread->id_get(), + owningThreadID); return -EINVAL; } Debug::log("Thread {} boosting priority of {} for futex {}", @@ -550,7 +571,7 @@ if (shouldYield) { - Thread::yield_interrupt_enabled(); + yield(); } return woke;
diff --git a/sdk/core/scheduler/thread.h b/sdk/core/scheduler/thread.h index c23423a..4ed345b 100644 --- a/sdk/core/scheduler/thread.h +++ b/sdk/core/scheduler/thread.h
@@ -18,6 +18,8 @@ // thread structures. class MultiWaiterInternal; + uint64_t expiry_time_for_timeout(uint32_t timeout); + template<size_t NPrios> class ThreadImpl final : private utils::NoCopyNoMove { @@ -115,23 +117,18 @@ } /** - * When yielding inside the scheduler compartment, we almost always want - * to re-enable interrupts before ecall. If we don't, then a thread with - * interrupt enabled can just call a scheduler function with a long - * timeout, essentially gaining the ability to indefinitely block - * interrupts. Worse, if this is the only thread, then it blocks - * interrupts forever for the whole system. + * Returns true if any thread is ready to run. */ - static void yield_interrupt_enabled() + static bool any_ready() { - __asm volatile("ecall"); + return priorityMap != 0; } static uint32_t yield_timed() { uint64_t ticksAtStart = ticksSinceBoot; - yield_interrupt_enabled(); + yield(); uint64_t elapsed = ticksSinceBoot - ticksAtStart; if (elapsed > std::numeric_limits<uint32_t>::max()) @@ -152,11 +149,12 @@ */ bool suspend(Timeout *t, ThreadImpl **newSleepQueue, - bool yieldUnconditionally = false) + bool yieldUnconditionally = false, + bool yieldNotSleep = false) { if (t->remaining != 0) { - suspend(t->remaining, newSleepQueue); + suspend(t->remaining, newSleepQueue, yieldNotSleep); } if ((t->remaining != 0) || yieldUnconditionally) { @@ -188,6 +186,7 @@ OriginalPriority(priority), expiryTime(-1), state(ThreadState::Suspended), + isYielding(false), sleepQueue(nullptr), tStackPtr(tstack) { @@ -212,7 +211,7 @@ // We must be suspended. Debug::Assert(state == ThreadState::Suspended, "Waking thread that is in state {}, not suspended", - state); + static_cast<ThreadState>(state)); // First, remove self from the timer waiting list. timer_list_remove(&waitingList); if (sleepQueue != nullptr) @@ -233,11 +232,18 @@ schedule = true; } } + // If this is the same priority as the current thread, we may need + // to update the timer. + if (priority >= highestPriority) + { + schedule = true; + } if (reason == WakeReason::Timer || reason == WakeReason::Delete) { multiWaiter = nullptr; } list_insert(&priorityList[priority]); + isYielding = false; return schedule; } @@ -278,11 +284,14 @@ * waiting on a resource, add it to the list of that resource. No * matter what, it has to be added to the timer list. */ - void suspend(uint32_t waitTicks, ThreadImpl **newSleepQueue) + void suspend(uint32_t waitTicks, + ThreadImpl **newSleepQueue, + bool yieldNotSleep = false) { + isYielding = yieldNotSleep; Debug::Assert(state == ThreadState::Ready, "Suspending thread that is in state {}, not ready", - state); + static_cast<ThreadState>(state)); list_remove(&priorityList[priority]); state = ThreadState::Suspended; priority_map_remove(); @@ -291,8 +300,7 @@ list_insert(newSleepQueue); sleepQueue = newSleepQueue; } - expiryTime = - (waitTicks == UINT32_MAX ? -1 : ticksSinceBoot + waitTicks); + expiryTime = expiry_time_for_timeout(waitTicks); timer_list_insert(&waitingList); } @@ -407,7 +415,7 @@ Debug::Assert(state == ThreadState::Suspended, "Inserting thread into timer list that is in state " "{}, not suspended", - state); + static_cast<ThreadState>(state)); if (head == nullptr) { timerNext = timerPrev = *headPtr = this; @@ -511,6 +519,29 @@ return priority; } + bool is_ready() + { + return state == ThreadState::Ready; + } + + bool is_yielding() + { + return isYielding; + } + + /** + * Returns true if there are other runnable threads with the same + * priority as this thread. + */ + bool has_priority_peers() + { + Debug::Assert(state == ThreadState::Ready, + "Checking for peers on thread that is in state {}, " + "not ready", + static_cast<ThreadState>(state)); + return next != this; + } + ~ThreadImpl() { // We have static definition of threads. We only create threads in @@ -616,7 +647,13 @@ uint8_t priority; /// The original priority level for this thread. This never changes. const uint8_t OriginalPriority; - ThreadState state; + ThreadState state : 2; + /** + * If the thread is yielding, it may be scheduled before its timeout + * expires, as long as no other threads are runnable or sleeping with + * shorter timeouts. + */ + bool isYielding : 1; }; using Thread = ThreadImpl<ThreadPrioNum>;
diff --git a/sdk/core/scheduler/timer.h b/sdk/core/scheduler/timer.h index a86120b..45355b6 100644 --- a/sdk/core/scheduler/timer.h +++ b/sdk/core/scheduler/timer.h
@@ -26,36 +26,101 @@ IsTimer<TimerCore>, "Platform's timer implementation does not meet the required interface"); + /** + * Timer interface. Provides generic timer functionality to the scheduler, + * wrapping the platform's timer device. + */ class Timer final : private TimerCore { + inline static uint64_t lastTickTime = 0; + inline static uint64_t zeroTickTime = 0; + inline static uint32_t accumulatedTickError = 0; + public: + /** + * Perform any setup necessary for the timer device. + */ static void interrupt_setup() { static_assert(TIMERCYCLES_PER_TICK <= UINT32_MAX, "Cycles per tick can't be represented in 32 bits. " "Double check your platform config"); init(); - setnext(TIMERCYCLES_PER_TICK); + zeroTickTime = time(); } - static void do_interrupt() + /** + * Expose the timer device's method for returning the current time. + */ + using TimerCore::time; + + /** + * Update the timer to fire the next timeout for the thread at the + * front of the queue, or disable the timer if there are no threads + * blocked with a timeout and no threads with the same priority. + * + * The scheduler is a simple RTOS scheduler that does not allow any + * thread to run if a higher-priority thread is runnable. This means + * that we need a timer interrupt in one of two situations: + * + * - We have a thread of the same priority as the current thread and + * we are going to round-robin schedule it. + * - We have a thread of a higher priority than the current thread + * that is currently sleeping on a timeout and need it to preempt the + * current thread when its timeout expires. + * + * We currently over approximate the second condition by making the + * timer fire independent of the priority. If this is every changed, + * some care must be taken to ensure that dynamic priority propagation + * via priority-inheriting futexes behaves correctly. + * + * This should be called after scheduling has changed the list of + * waiting threads. + */ + static void update() { - ++Thread::ticksSinceBoot; - - expiretimers(); - setnext(TIMERCYCLES_PER_TICK); + auto *thread = Thread::current_get(); + bool waitingListIsEmpty = ((Thread::waitingList == nullptr) || + (Thread::waitingList->expiryTime == -1)); + bool threadHasNoPeers = + (thread == nullptr) || (!thread->has_priority_peers()); + if (waitingListIsEmpty && threadHasNoPeers) + { + clear(); + } + else + { + static constexpr uint64_t DistantFuture = + std::numeric_limits<uint64_t>::max(); + uint64_t nextTick = threadHasNoPeers + ? DistantFuture + : time() + TIMERCYCLES_PER_TICK; + uint64_t nextTimer = waitingListIsEmpty + ? DistantFuture + : Thread::waitingList->expiryTime; + setnext(std::min(nextTick, nextTimer)); + } } - private: + /** + * Wake any threads that were sleeping until a timeout before the + * current time. This also wakes yielded threads if there are no + * runnable threads. + * + * This should be called when a timer interrupt fires. + */ static void expiretimers() { + uint64_t now = time(); + Thread::ticksSinceBoot = + (now - zeroTickTime) / TIMERCYCLES_PER_TICK; if (Thread::waitingList == nullptr) { return; } for (Thread *iter = Thread::waitingList;;) { - if (iter->expiryTime <= Thread::ticksSinceBoot) + if (iter->expiryTime <= now) { Thread *iterNext = iter->timerNext; @@ -72,6 +137,39 @@ break; } } + // If there are not runnable threads, try to wake a yielded thread + if (!Thread::any_ready()) + { + // Look at the first thread. If it is not yielding, there may + // be another thread behind it that is, but that's fine. We + // don't want to encounter situations where (with a + // high-priority A and a low-priority B): + // + // 1. A yields for 5 ticks. + // 2. B starts and does a blocking operation (e.g. try_lock) + // with a 1-tick timeout. + // 3. A wakes up and prevents B from running even though we're + // still in its 5-tick yield period. + if (Thread *head = Thread::waitingList) + { + if (head->is_yielding()) + { + Debug::log("Woke thread {} {} cycles early", + head->id_get(), + int64_t(head->expiryTime) - now); + head->ready(Thread::WakeReason::Timer); + } + } + } } }; + + uint64_t expiry_time_for_timeout(uint32_t timeout) + { + if (timeout == -1) + { + return -1; + } + return Timer::time() + (timeout * TIMERCYCLES_PER_TICK); + } } // namespace
diff --git a/sdk/core/switcher/entry.S b/sdk/core/switcher/entry.S index 8836d08..716248d 100644 --- a/sdk/core/switcher/entry.S +++ b/sdk/core/switcher/entry.S
@@ -970,7 +970,13 @@ // Load the trusted stack pointer into a register that we will clobber in // the next instruction when we load the thread ID. cspecialr ca0, mtdc + cgettag a1, ca0 + // If this is a null pointer, don't try to dereference it and report that + // we are thread 0. This permits the debug code to work even from things + // that are not real threads. + beqz a1, .Lend clh a0, TrustedStack_offset_threadID(ca0) +.Lend: cret
diff --git a/sdk/core/token_library/token_unseal.S b/sdk/core/token_library/token_unseal.S index 2c9ca62..8391902 100644 --- a/sdk/core/token_library/token_unseal.S +++ b/sdk/core/token_library/token_unseal.S
@@ -80,9 +80,17 @@ bne t0, t1, .Lexit_failure /* Subset bounds to ->data */ + // Get the top into t1 cgetlen t1, ca0 + cgetbase t0, ca0 + add t1, t1, t0 + // Move the address to the start of the data cincoffset ca0, ca0, TokenSObj_offset_data - addi t1, t1, -TokenSObj_offset_data + // Subtract the address of the (to-be-returned-unsealed) data from the top to + // give the length. + sub t1, t1, a0 + // Set the new bounds, using an exact setting so that any errors in the + // allocator's alignment turn into an untagged capability here. csetboundsexact ca0, ca0, t1 /* And that's an unwrap. */
diff --git a/sdk/include/FreeRTOS-Compat/task.h b/sdk/include/FreeRTOS-Compat/task.h index 5aac587..398ff58 100644 --- a/sdk/include/FreeRTOS-Compat/task.h +++ b/sdk/include/FreeRTOS-Compat/task.h
@@ -55,7 +55,7 @@ static inline void vTaskDelay(const TickType_t xTicksToDelay) { struct Timeout timeout = {0, xTicksToDelay}; - thread_sleep(&timeout); + thread_sleep(&timeout, ThreadSleepNoEarlyWake); } /**
diff --git a/sdk/include/cdefs.h b/sdk/include/cdefs.h index 1c6eca3..ad3b9ae 100644 --- a/sdk/include/cdefs.h +++ b/sdk/include/cdefs.h
@@ -107,4 +107,12 @@ # define __clang_ignored_warning_pop() #endif +/** + * Define the symbol for the libcall that the compiler will expand the `strlen` + * builtin to. This builtin is used internally in libc++ (and possibly in + * other places) to avoid the namespace pollution from including `string.h` but + * is either constant folded in the front end or expanded to a libcall. + */ +unsigned __builtin_strlen(const char *str) __asm__("_Z6strlenPKc"); + #endif // _CDEFS_H_
diff --git a/sdk/include/cheri.hh b/sdk/include/cheri.hh index 59eb773..8e18db3 100644 --- a/sdk/include/cheri.hh +++ b/sdk/include/cheri.hh
@@ -990,6 +990,33 @@ { return ptr[index]; } + + /** + * Returns true if the capability is `align`-byte aligned, false + * otherwise. + */ + bool is_aligned(size_t align) + { + return __builtin_is_aligned(ptr, align); + } + + /** + * Aligns the capability down to the nearest `align`-byte boundary. + */ + Capability &align_down(size_t align) + { + ptr = __builtin_align_down(ptr, align); + return *this; + } + + /** + * Aligns the capability up to the nearest `align`-byte boundary. + */ + Capability &align_up(size_t align) + { + ptr = __builtin_align_up(ptr, align); + return *this; + } }; /**
diff --git a/sdk/include/ctype.h b/sdk/include/ctype.h index 1011ad5..5861b28 100644 --- a/sdk/include/ctype.h +++ b/sdk/include/ctype.h
@@ -3,6 +3,11 @@ #pragma once +static inline int isprint(int c) +{ + return c >= '\x20' && c <= '\x7e'; +} + static inline int isdigit(int c) { return c >= '0' && c <= '9';
diff --git a/sdk/include/debug.hh b/sdk/include/debug.hh index 21fb6b8..af0732e 100644 --- a/sdk/include/debug.hh +++ b/sdk/include/debug.hh
@@ -285,6 +285,19 @@ }; /** + * String view specialisation, use the C string handler. + */ +template<> +struct DebugFormatArgumentAdaptor<std::string> +{ + __always_inline static DebugFormatArgument construct(std::string &value) + { + return DebugFormatArgumentAdaptor<const char *>::construct( + value.c_str()); + } +}; + +/** * Enum specialisation, prints the enum as a string and then the numeric value. * * This specialisation uses the generic printing facility in the library call
diff --git a/sdk/include/ds/linked_list.h b/sdk/include/ds/linked_list.h index 2894bb2..5194d2f 100644 --- a/sdk/include/ds/linked_list.h +++ b/sdk/include/ds/linked_list.h
@@ -4,6 +4,8 @@ /** * @file A (circular) doubly linked list, abstracted over cons cell * representations. + * + * See `tests/list-test.cc` for additional information on how to use it. */ #pragma once @@ -356,7 +358,7 @@ /** * Search through all elements of a ring *except* `elem`. If `elem` is the * sentinel of a ring, then this is, as one expects, a `search` over all - * non-sentinel memebers of the ring. + * non-sentinel members of the ring. */ template<cell::HasCellOperations Cell, typename F> __always_inline bool search(Cell *elem, F f)
diff --git a/sdk/include/ds/pointer.h b/sdk/include/ds/pointer.h index de52755..598c726 100644 --- a/sdk/include/ds/pointer.h +++ b/sdk/include/ds/pointer.h
@@ -113,7 +113,7 @@ __always_inline Pointer<T> &operator=(Pointer const &p) { - ref = *p.ref; + ref = p.ref; return *this; }
diff --git a/sdk/include/futex.h b/sdk/include/futex.h index 7907e2a..482919a 100644 --- a/sdk/include/futex.h +++ b/sdk/include/futex.h
@@ -14,7 +14,7 @@ * are assumed to hold the thread ID of the thread that currently holds the * lock. */ - FutexPriorityInheritance}; + FutexPriorityInheritance = (1 << 0)}; /** * Compare the value at `address` to `expected` and, if they match, sleep the @@ -37,10 +37,10 @@ * - `-ETIMEOUT` if the timeout expires. */ [[cheri::interrupt_state(disabled)]] int __cheri_compartment("sched") - futex_timed_wait(Timeout *ticks, - const uint32_t *address, - uint32_t expected, - enum FutexWaitFlags flags __if_cxx(= FutexNone)); + futex_timed_wait(Timeout *ticks, + const uint32_t *address, + uint32_t expected, + uint32_t flags __if_cxx(= FutexNone)); /** * Compare the value at `address` to `expected` and, if they match, sleep the
diff --git a/sdk/include/locks.h b/sdk/include/locks.h index 1dbab8c..c33474a 100644 --- a/sdk/include/locks.h +++ b/sdk/include/locks.h
@@ -144,6 +144,33 @@ flaglock_upgrade_for_destruction(struct FlagLockState *lock); /** + * Return the thread ID of the owner of the lock. + * + * This is only available for priority inherited locks, as this is the only + * case where we store the thread ID of the owner. + * + * The return value is 0 if the lock is not owned or if called on a + * non-priority inherited flag lock. The return value is undefined if called on + * an uninitialized lock. + * + * This *will* race with succesful `lock` and `unlock` operations on other + * threads, and should thus not be used to check if the lock is owned. + * + * The main use case for this function is in the error handler to check whether + * or not the lock is owned by the thread on which the error handler was + * invoked. In this case we can call this function and compare the result with + * `thread_id_get` to know if the current thread owns the lock. + */ +__always_inline static inline uint16_t +flaglock_priority_inheriting_get_owner_thread_id(struct FlagLockState *lock) +{ + // The lock must be held at this point for the value to be stable so do + // a non-atomic read (simply &ing the lock word would result in a + // libcall for the atomic operation). + return ((*(uint32_t *)&(lock->lockWord)) & 0x0000ffff); +} + +/** * Try to acquire a recursive mutex. This is a priority-inheriting mutex that * can be acquired multiple times by the same thread. *
diff --git a/sdk/include/locks.hh b/sdk/include/locks.hh index 3f7808e..6819f84 100644 --- a/sdk/include/locks.hh +++ b/sdk/include/locks.hh
@@ -93,6 +93,19 @@ { flaglock_upgrade_for_destruction(&state); } + + /** + * Return the thread ID of the owner of the lock. + * + * This is only available for priority inherited locks, as this is the + * only case where we store the thread ID of the owner. See the + * documentation of `flaglock_priority_inheriting_get_owner_thread_id` + * for more information. + */ + __always_inline uint16_t get_owner_thread_id() requires(IsPriorityInherited) + { + return flaglock_priority_inheriting_get_owner_thread_id(&state); + } }; /**
diff --git a/sdk/include/microvium/microvium_port.h b/sdk/include/microvium/microvium_port.h index 1b2e4b1..c18188a 100644 --- a/sdk/include/microvium/microvium_port.h +++ b/sdk/include/microvium/microvium_port.h
@@ -295,11 +295,15 @@ * * The `context` passed to these macros is whatever value that the host passes * to `mvm_restore`. It can be any value that fits in a pointer. + * + * Similarly to `malloc` and `calloc`, this will only ever + * block to wait for the quarantine to be processed. */ #define MVM_CONTEXTUAL_MALLOC(size, context) \ ({ \ - Timeout t = {0, 0}; \ - void *ret = heap_allocate(&t, context, size); \ + Timeout t = {0, MALLOC_WAIT_TICKS}; \ + void *ret = \ + heap_allocate(&t, context, size, AllocateWaitRevocationNeeded); \ if (!__builtin_cheri_tag_get(ret)) \ { \ ret = NULL; \
diff --git a/sdk/include/platform/generic-riscv/platform-timer.hh b/sdk/include/platform/generic-riscv/platform-timer.hh index 377f323..7b0d1f6 100644 --- a/sdk/include/platform/generic-riscv/platform-timer.hh +++ b/sdk/include/platform/generic-riscv/platform-timer.hh
@@ -38,6 +38,23 @@ 2 * sizeof(uint32_t)); } + static uint64_t time() + { + // The timer is little endian, so the high 32 bits are after the low 32 + // bits. We can't do atomic 64-bit loads and so we have to read these + // separately. + volatile uint32_t *timerHigh = pmtimer + 1; + uint32_t timeLow, timeHigh; + + // Read the current time. Loop until the high 32 bits are stable. + do + { + timeHigh = *timerHigh; + timeLow = *pmtimer; + } while (timeHigh != *timerHigh); + return (uint64_t(timeHigh) << 32) | timeLow; + } + /** * Set the timer up for the next timer interrupt. We need to: * 1. read the current MTIME, @@ -51,28 +68,22 @@ * interrupt. Writing to any half is enough to clear the interrupt, * which is also why 2. is important. */ - static void setnext(uint32_t cycles) + static void setnext(uint64_t nextTime) { /// the high 32 bits of the 64-bit MTIME register volatile uint32_t *pmtimercmphigh = pmtimercmp + 1; - /// the low 32 bits - volatile uint32_t *pmtimerhigh = pmtimer + 1; uint32_t curmtimehigh, curmtime, curmtimenew; - // Read the current time. Loop until the high 32 bits are stable. - do - { - curmtimehigh = *pmtimerhigh; - curmtime = *pmtimer; - } while (curmtimehigh != *pmtimerhigh); - - // Add tick cycles to current time. Handle carry bit. - curmtimehigh += __builtin_add_overflow(curmtime, cycles, &curmtimenew); - // Write the new MTIMECMP value, at which the next interrupt fires. *pmtimercmphigh = -1; // Prevent spurious interrupts. - *pmtimercmp = curmtimenew; - *pmtimercmphigh = curmtimehigh; + *pmtimercmp = nextTime; + *pmtimercmphigh = nextTime >> 32; + } + + static void clear() + { + volatile uint32_t *pmtimercmphigh = pmtimercmp + 1; + *pmtimercmphigh = -1; // Prevent spurious interrupts. } private:
diff --git a/sdk/include/platform/sunburst/platform-i2c.hh b/sdk/include/platform/sunburst/platform-i2c.hh new file mode 100644 index 0000000..9628a43 --- /dev/null +++ b/sdk/include/platform/sunburst/platform-i2c.hh
@@ -0,0 +1,421 @@ +#pragma once +#include <cdefs.h> +#include <debug.hh> +#include <stdint.h> + +/** + * The interrupts of the OpenTitan's I2C block. + * + * Documentation source can be found at: + * https://github.com/lowRISC/opentitan/blob/9ddf276c64e2974ed8e528e8b2feb00b977861de/hw/ip/i2c/doc/interfaces.md + */ +enum class OpenTitanI2cInterrupt +{ + /** + * A host mode interrupt. This is asserted whilst the Format FIFO level is + * below the low threshold. This is a level status interrupt. + */ + FormatThreshold, + /** + * A host mode interrupt. This is asserted whilst the Receive FIFO level is + * above the high threshold. This is a level status interrupt. + */ + ReceiveThreshold, + /** + * A target mode interrupt. This is asserted whilst the Aquired FIFO level + * is above the high threshold. This is a level status interrupt. + */ + AcquiredThreshold, + /** + * A host mode interrupt. This is raised if the Receive FIFO has overflowed. + */ + ReceiveOverflow, + /** + * A host mode interrupt. This is raised if there is no ACK in response to + * an address or data. + */ + Nak, + /** + * A host mode interrupt. This is raised if the SCL line drops early (not + * supported without clock synchronization). + */ + SclInterference, + /** + * A host mode interrupt. This is raised if the SDA line goes low when host + * is trying to assert high. + */ + SdaInterference, + /** + * A host mode interrupt. This is raised if target stretches the clock + * beyond the allowed timeout period. + */ + StretchTimeout, + /** + * A host mode interrupt. This is raised if the target does not assert a + * constant value of SDA during transmission. + */ + SdaUnstable, + /** + * A host and target mode interrupt. In host mode, raised if the host issues + * a repeated START or terminates the transaction by issuing STOP. In target + * mode, raised if the external host issues a STOP or repeated START. + */ + CommandComplete, + /** + * A target mode interrupt. This is raised if the target is stretching + * clocks for a read command. This is a level status interrupt. + */ + TransmitStretch, + /** + * A target mode interrupt. This is asserted whilst the Transmit FIFO level + * is below the low threshold. This is a level status interrupt. + */ + TransmitThreshold, + /** + * A target mode interrupt. This is raised if the target is stretching + * clocks due to full Aquired FIFO or zero count in targetAckControl.NBYTES + * (if enabled). This is a level status interrupt. + */ + AcquiredFull, + /** + * A target mode interrupt. This is raised if STOP is received without a + * preceding NACK during an external host read. + */ + UnexpectedStop, + /** + * A target mode interrupt. This is raised if the host stops sending the + * clock during an ongoing transaction. + */ + HostTimeout, +}; + +static constexpr uint32_t interrupt_bit(const OpenTitanI2cInterrupt Interrupt) +{ + return 1 << static_cast<uint32_t>(Interrupt); +}; + +/** + * Driver for the OpenTitan's I2C block. + * + * Documentation source can be found at: + * https://github.com/lowRISC/opentitan/tree/9ddf276c64e2974ed8e528e8b2feb00b977861de/hw/ip/i2c + */ +struct OpenTitanI2c +{ + /// Interrupt State Register + uint32_t interruptState; + /// Interrupt Enable Register + uint32_t interruptEnable; + /// Interrupt Test Register + uint32_t interruptTest; + /// Alert Test Register (Unused in Sonata) + uint32_t alertTest; + /// I2C Control Register + uint32_t control; + /// I2C Live Status Register for Host and Target modes + uint32_t status; + /// I2C Read Data + uint32_t readData; + /// I2C Host Format Data + uint32_t formatData; + /// I2C FIFO control register + uint32_t fifoCtrl; + /// Host mode FIFO configuration + uint32_t hostFifoConfiguration; + /// Target mode FIFO configuration + uint32_t targetFifoConfiguration; + /// Host mode FIFO status register + uint32_t hostFifoStatus; + /// Target mode FIFO status register + uint32_t targetFifoStatus; + /// I2C Override Control Register + uint32_t override; + /// Oversampled Receive values + uint32_t values; + /** + * Detailed I2C Timings (directly corresponding to table 10 in the I2C + * Specification). + */ + uint32_t timing[5]; + /// I2C clock stretching timeout control. + uint32_t timeoutControl; + /// I2C target address and mask pairs + uint32_t targetId; + /// I2C target acquired data + uint32_t acquiredData; + /// I2C target transmit data + uint32_t transmitData; + /** + * I2C host clock generation timeout value (in units of input clock + * frequency). + */ + uint32_t hostTimeoutControl; + /// I2C target internal stretching timeout control. + uint32_t targetTimeoutControl; + /** + * Number of times the I2C target has NACK'ed a new transaction since the + * last read of this register. + */ + uint32_t targetNackCount; + /** + * Timeout in Host-Mode for an unhandled NACK before hardware automatically + * ends the transaction. + */ + uint32_t targetAckControl; + + /// Control Register Fields + enum [[clang::flag_enum]] : uint32_t{ + /// Enable Host I2C functionality + ControlEnableHost = 1 << 0, + /// Enable Target I2C functionality + ControlEnableTarget = 1 << 1, + /// Enable I2C line loopback test If line loopback is enabled, the + /// internal design sees ACQ and RX data as "1" + ControlLineLoopback = 1 << 2, + }; + + /// Status Register Fields + enum [[clang::flag_enum]] : uint32_t{ + /// Host mode Format FIFO is full + StatusFormatFull = 1 << 0, + /// Host mode Receive FIFO is full + StatusReceiveFull = 1 << 1, + /// Host mode Format FIFO is empty + StatusFormatEmpty = 1 << 2, + /// Host functionality is idle. No Host transaction is in progress + StatusHostIdle = 1 << 3, + /// Target functionality is idle. No Target transaction is in progress + StatusTargetIdle = 1 << 4, + /// Host mode Receive FIFO is empty + SmatusReceiveEmpty = 1 << 5, + /// Target mode Transmit FIFO is full + StatusTransmitFull = 1 << 6, + /// Target mode Receive FIFO is full + StatusAcquiredFull = 1 << 7, + /// Target mode Transmit FIFO is empty + StatusTransmitEmpty = 1 << 8, + /// Target mode Aquired FIFO is empty + StatusAcquiredEmpty = 1 << 9, + /** + * A Host-Mode active transaction has been ended by the + * HostNackHandlerTimeout mechanism. This bit is cleared when + * Control.EnableHost is set by software to start a new transaction. + */ + StatusHostDisabledNackTimeout = 1 << 10, + }; + + /// FormatData Register Fields + enum [[clang::flag_enum]] : uint32_t{ + /// Issue a START condition before transmitting BYTE. + FormatDataStart = 1 << 8, + /// Issue a STOP condition after this operation + FormatDataStop = 1 << 9, + /// Read BYTE bytes from I2C. (256 if BYTE==0) + FormatDataReadBytes = 1 << 10, + /** + * Do not NACK the last byte read, let the read + * operation continue + */ + FormatDataReadCount = 1 << 11, + /// Do not signal an exception if the current byte is not ACK’d + FormatDataNakOk = 1 << 12, + }; + + /// FifoControl Register Fields + enum [[clang::flag_enum]] : uint32_t{ + /// Receive fifo reset. Write 1 to the register resets it. Read returns 0 + FifoControlReceiveReset = 1 << 0, + /// Format fifo reset. Write 1 to the register resets it. Read returns 0 + FifoControlFormatReset = 1 << 1, + /// Aquired FIFO reset. Write 1 to the register resets it. Read returns 0 + FifoControlAcquiredReset = 1 << 7, + /// Transmit FIFO reset. Write 1 to the register resets it. Read returns 0 + FifoControlTransmitReset = 1 << 8, + }; + + /// Flag set when we're debugging this driver. + static constexpr bool DebugOpenTitanI2c = true; + + /// Helper for conditional debug logs and assertions. + using Debug = ConditionalDebug<DebugOpenTitanI2c, "OpenTitan I2C">; + + /** + * Performs a 32-bit integer unsigned division, rounding up. The bottom + * 16 bits of the result are then returned. + * + * As usual, a divisor of 0 is still Undefined Behavior. + */ + static uint16_t round_up_divide(uint32_t a, uint32_t b) + { + if (a == 0) + { + return 0; + } + const uint32_t Res = ((a - 1) / b) + 1; + Debug::Assert(Res <= UINT16_MAX, + "Division result too large to fit in uint16_t."); + return static_cast<uint16_t>(Res); + } + + /// Reset all of the fifos. + void reset_fifos() volatile + { + fifoCtrl = (FifoControlReceiveReset | FifoControlFormatReset | + FifoControlAcquiredReset | FifoControlTransmitReset); + } + + /// Configure the I2C block to be in host mode. + void host_mode_set() volatile + { + control = ControlEnableHost; + } + + /** + * Set the I2C timing parameters appropriately for the given bit rate. + * Distilled from: + * https://github.com/lowRISC/opentitan/blob/9ddf276c64e2974ed8e528e8b2feb00b977861de/hw/ip/i2c/doc/programmers_guide.md + */ + void speed_set(const uint32_t SpeedKhz) volatile + { + // We must round up the system clock frequency to lengthen intervals. + const uint16_t SystemClockKhz = round_up_divide(CPU_TIMER_HZ, 1000); + // We want to underestimate the clock period, to lengthen the timings. + const uint16_t ClockPeriod = (1000 * 1000) / SystemClockKhz; + + // Decide which bus mode this represents + uint32_t mode = (SpeedKhz > 100u) + (SpeedKhz > 400u); + + // Minimum fall time when V_DD is 3.3V + constexpr uint16_t MinimumFallTime = 20 * 3 / 5; + // Specification minimum timings (Table 10) in nanoseconds for each bus + // mode. + constexpr uint16_t MinimumTimeValues[5][2][3] = { + { + {4700u, 1300u, 150u}, // Low Period + {4000u, 600u, 260u}, // High Period + }, + { + // Fall time of SDA and SCL signals + {MinimumFallTime, MinimumFallTime, MinimumFallTime}, + // Rise time of SDA and SCL signals + {120, 120, 120}, + }, + { + {4700u, 600u, 260u}, // Hold time for a repeated start condition + {4000u, 600u, 260u}, // Set-up time for a repeated start condition + }, + { + {4000u, 1u, 1u}, // Data hold time + {500u, 100u, 50u}, // Data set-up time + }, + { + // Bus free time between a STOP and START condition + {4700u, 1300u, 500u}, + // Set-up time for a STOP condition + {4000u, 600u, 260u}, + }, + }; + for (uint32_t i = 0; i < 5; ++i) + { + timing[i] = + (round_up_divide(MinimumTimeValues[i][0][mode], ClockPeriod) + << 16) | + round_up_divide(MinimumTimeValues[i][1][mode], ClockPeriod); + } + } + + void blocking_write_byte(const uint32_t Fmt) volatile + { + while (0 != (StatusFormatFull & status)) {} + formatData = Fmt; + } + + /// Returns true when the format fifo is empty + [[nodiscard]] bool format_is_empty() volatile + { + return 0 != (StatusFormatEmpty & status); + } + + void blocking_write(const uint8_t Addr7, + const uint8_t data[], + const uint32_t NumBytes, + const bool SkipStop) volatile + { + if (NumBytes == 0) + { + return; + } + blocking_write_byte(FormatDataStart | (Addr7 << 1) | 0u); + for (uint32_t i = 0; i < NumBytes - 1; ++i) + { + blocking_write_byte(data[i]); + } + blocking_write_byte((SkipStop ? 0u : FormatDataStop) | + data[NumBytes - 1]); + } + + [[nodiscard]] bool blocking_read(const uint8_t Addr7, + uint8_t buf[], + const uint32_t NumBytes) volatile + { + for (uint32_t idx = 0; idx < NumBytes; idx += UINT8_MAX) + { + blocking_write_byte(FormatDataStart | (Addr7 << 1) | 1u); + while (!format_is_empty()) {} + if (interrupt_is_asserted(OpenTitanI2cInterrupt::Nak)) + { + interrupt_clear(OpenTitanI2cInterrupt::Nak); + return false; + } + uint32_t bytesRemaining = NumBytes - idx; + bool lastChunk = UINT8_MAX >= bytesRemaining; + uint8_t chunkSize = + lastChunk ? static_cast<uint8_t>(bytesRemaining) : UINT8_MAX; + + blocking_write_byte((lastChunk ? FormatDataStop : 0) | + FormatDataReadBytes | chunkSize); + while (!format_is_empty()) {} + + for (uint32_t chunkIdx = 0; chunkIdx < chunkSize; ++chunkIdx) + { + buf[idx + chunkIdx] = readData; + } + } + return true; + } + + /// Returns true if the given interrupt is asserted. + [[nodiscard]] bool + interrupt_is_asserted(OpenTitanI2cInterrupt interrupt) volatile + { + return 0 != (interruptState & interrupt_bit(interrupt)); + } + + /// Clears the given interrupt. + void interrupt_clear(OpenTitanI2cInterrupt interrupt) volatile + { + interruptState = interruptState & ~interrupt_bit(interrupt); + } + + /// Enables the given interrupt. + void interrupt_enable(OpenTitanI2cInterrupt interrupt) volatile + { + interruptEnable = interruptEnable | interrupt_bit(interrupt); + } + + /// Disables the given interrupt. + void interrupt_disable(OpenTitanI2cInterrupt interrupt) volatile + { + interruptEnable = interruptEnable & ~interrupt_bit(interrupt); + } + + /** + * Sets the thresholds for the format and receive fifos. + */ + void host_thresholds_set(uint16_t formatThreshold, + uint16_t receiveThreshold) volatile + { + hostFifoConfiguration = + (formatThreshold & 0xfff) << 16 | (receiveThreshold & 0xfff); + } +};
diff --git a/sdk/include/platform/sunburst/platform-rgbctrl.hh b/sdk/include/platform/sunburst/platform-rgbctrl.hh new file mode 100644 index 0000000..1e5a153 --- /dev/null +++ b/sdk/include/platform/sunburst/platform-rgbctrl.hh
@@ -0,0 +1,95 @@ +#pragma once +#include <cdefs.h> +#include <stdint.h> + +/** + * An enum representing each of the Sonata's RGB LEDs. + */ +enum class SonataRgbLed +{ + Led0 = 0, + Led1 = 1, +}; + +/** + * A driver for the Sonata's RGB LED Controller + */ +struct SonataRgbLedController +{ + /** + * Registers for setting the 8-bit red, green, and blue values + * for the two RGB Leds. + */ + uint32_t ledColors[2]; + /** + * Control Register. See `SonataRgbLedController::ControlFields` for the + * fields. + */ + uint32_t control; + /** + * Status Register See `SonataRgbLedController::StatusFields` for the + * fields. + */ + uint32_t status; + + /// Control Register Fields + enum [[clang::flag_enum]] ControlFields : uint32_t{ + /// Write 1 to set RGB LEDs to specified colours. + ControlSet = 1 << 0, + /** + * Write 1 to turn off RGB LEDs. + * Write to ControlSet to turn on again. + */ + ControlOff = 1 << 1, + }; + + /// Status Register Fields + enum [[clang::flag_enum]] StatusFields : uint32_t{ + /** + * When asserted controller is idle and new colours can be set, + * otherwise writes to regLed0, regLed1, and control are ignored. + */ + StatusIdle = 1 << 0, + }; + + /** + * Blocks until the controller is not busy. + * + * The controller can be busy when it is in the process of updating the + * LEDs. While busy, register writes will be ignored. + */ + void wait_for_idle() volatile + { + while ((status & StatusIdle) == 0) {} + } + + /** + * Set the desired Red, Green, and Blue value of an LED. To apply these + * changes, one needs to run `SonataRgbLedController::update()`. + */ + void + rgb(SonataRgbLed led, uint8_t red, uint8_t green, uint8_t blue) volatile + { + wait_for_idle(); + ledColors[static_cast<uint32_t>(led)] = + (static_cast<uint32_t>(blue) << 16) | + (static_cast<uint32_t>(green) << 8) | static_cast<uint32_t>(red); + } + + /// Update the colours of the LEDs. + void update() volatile + { + wait_for_idle(); + control = ControlSet; + } + + /// Switch all of the RGB LEDs off. + void off() volatile + { + wait_for_idle(); + control = ControlOff; + } +}; + +static_assert(sizeof(SonataRgbLedController) == 16, + "The SonataRgbLedController structure is the wrong size.");
diff --git a/sdk/include/platform/sunburst/platform-spi.hh b/sdk/include/platform/sunburst/platform-spi.hh new file mode 100644 index 0000000..fd8aa19 --- /dev/null +++ b/sdk/include/platform/sunburst/platform-spi.hh
@@ -0,0 +1,235 @@ +#pragma once +#include <cdefs.h> +#include <debug.hh> +#include <stdint.h> + +/** + * A Simple Driver for the Sonata's SPI. + * + * Documentation source can be found at: + * https://github.com/lowRISC/sonata-system/blob/1a59633d2515d4fe186a07d53e49ff95c18d9bbf/doc/ip/spi.md + * + * Rendered documentation is served from: + * https://lowrisc.org/sonata-system/doc/ip/spi.html + */ +struct SonataSpi +{ + /** + * The Sonata SPI block doesn't currently have support for interrupts. + * The following registers are reserved for future use. + */ + uint32_t interruptState; + uint32_t interruptEnable; + uint32_t interruptTest; + /** + * Configuration register. Controls how the SPI block transmits and + * receives data. This register can be modified only whilst the SPI block + * is idle. + */ + uint32_t configuration; + /** + * Controls the operation of the SPI block. This register can + * be modified only whilst the SPI block is idle. + */ + uint32_t control; + /// Status information about the SPI block + uint32_t status; + /** + * Writes to this begin an SPI operation. + * Writes are ignored when the SPI block is active. + */ + uint32_t start; + /** + * Data from the receive FIFO. When read the data is popped from the FIFO. + * If the FIFO is empty data read is undefined. + */ + uint32_t receiveFifo; + /** + * Bytes written here are pushed to the transmit FIFO. If the FIFO is full + * writes are ignored. + */ + uint32_t transmitFifo; + + /// Configuration Register Fields + enum : uint32_t + { + /** + * The length of a half period (i.e. positive edge to negative edge) of + * the SPI clock, measured in system clock cycles reduced by 1. For + * example, at a 50 MHz system clock, a value of 0 gives a 25 MHz SPI + * clock, a value of 1 gives a 12.5 MHz SPI clock, a value of 2 gives + * a 8.33 MHz SPI clock and so on. + */ + ConfigurationHalfClockPeriodMask = 0xffu << 0, + /* + * When set the most significant bit (MSB) is the first bit sent and + * received with each byte + */ + ConfigurationMSBFirst = 1u << 29, + /* + * The phase of the spi_clk signal. when clockphase is 0, data is + * sampled on the leading edge and changes on the trailing edge. The + * first data bit is immediately available before the first leading edge + * of the clock when transmission begins. When clockphase is 1, data is + * sampled on the trailing edge and change on the leading edge. + */ + ConfigurationClockPhase = 1u << 30, + /* + * The polarity of the spi_clk signal. When ClockPolarity is 0, clock is + * low when idle and the leading edge is positive. When ClkPolarity is + * 1, clock is high when idle and the leading edge is negative + */ + ConfigurationClockPolarity = 1u << 31, + }; + + /// Control Register Fields + enum : uint32_t + { + /// Write 1 to clear the transmit FIFO. + ControlTransmitClear = 1 << 0, + /// Write 1 to clear the receive FIFO. + ControlReceiveClear = 1 << 1, + /** + * When set bytes from the transmit FIFO are sent. When clear the state + * of the outgoing spi_cipo is undefined whilst the SPI clock is + * running. + */ + ControlTransmitEnable = 1 << 2, + /** + * When set incoming bits are written to the receive FIFO. When clear + * incoming bits are ignored. + */ + ControlReceiveEnable = 1 << 3, + /** + * The watermark level for the transmit FIFO, depending on the value + * the interrupt will trigger at different points + */ + ControlTransmitWatermarkMask = 0xf << 4, + /** + * The watermark level for the receive FIFO, depending on the value the + * interrupt will trigger at different points + */ + ControlReceiveWatermarkMask = 0xf << 8, + }; + + /// Status Register Fields + enum : uint32_t + { + /// Number of items in the transmit FIFO. + StatusTxFifoLevel = 0xffu << 0, + /// Number of items in the receive FIFO. + StatusRxFifoLevel = 0xffu << 8, + /** + * When set the transmit FIFO is full and any data written to it will + * be ignored. + */ + StatusTxFifoFull = 1u << 16, + /** + * When set the receive FIFO is empty and any data read from it will be + * undefined. + */ + StatusRxFifoEmpty = 1u << 17, + /// When set the SPI block is idle and can accept a new start command. + StatusIdle = 1u << 18, + }; + + /// Start Register Fields + enum : uint32_t + { + /// Number of bytes to receive/transmit in the SPI operation + StartByteCountMask = 0x7ffu, + }; + + /// Flag set when we're debugging this driver. + static constexpr bool DebugSonataSpi = false; + + /// Helper for conditional debug logs and assertions. + using Debug = ConditionalDebug<DebugSonataSpi, "Sonata SPI">; + + /** + * Initialises the SPI block + * + * @param ClockPolarity When false, the clock is low when idle and the + * leading edge is positive. When true, the opposite behaviour is + * set. + * @param ClockPhase When false, data is sampled on the leading edge and + * changes on the trailing edge. When true, the opposite behaviour is + * set. + * @param MsbFirst When true, the first bit of each byte sent is the most + * significant bit, as oppose to the least significant bit. + * @param HalfClockPeriod The length of a half period of the SPI clock, + * measured in system clock cycles reduced by 1. + */ + void init(const bool ClockPolarity, + const bool ClockPhase, + const bool MsbFirst, + const uint16_t HalfClockPeriod) volatile + { + configuration = (ClockPolarity ? ConfigurationClockPolarity : 0) | + (ClockPhase ? ConfigurationClockPhase : 0) | + (MsbFirst ? ConfigurationMSBFirst : 0) | + (HalfClockPeriod & ConfigurationHalfClockPeriodMask); + } + + /// Waits for the SPI device to become idle + void wait_idle() volatile + { + // Wait whilst IDLE field in STATUS is low + while ((status & StatusIdle) == 0) {} + } + + /** + * Sends `len` bytes from the given `data` buffer, + * where `len` is at most `0x7ff`. + */ + void blocking_write(const uint8_t data[], uint16_t len) volatile + { + Debug::Assert(len <= 0x7ff, + "You can't transfer more than 0x7ff bytes at a time."); + len &= StartByteCountMask; + + wait_idle(); + control = ControlTransmitEnable; + start = len; + + uint32_t transmitAvailable = 0; + for (uint32_t i = 0; i < len; ++i) + { + if (transmitAvailable == 0) + { + while (transmitAvailable < 64) + { + // Read number of bytes in TX FIFO to calculate space + // available for more bytes + transmitAvailable = 64 - (status & StatusTxFifoLevel); + } + } + transmitFifo = data[i]; + transmitAvailable--; + } + } + + /* + * Receives `len` bytes and puts them in the `data` buffer, + * where `len` is at most `0x7ff`. + * + * This method will block until the requested number of bytes + * has been seen. There is currently no timeout. + */ + void blocking_read(uint8_t data[], uint16_t len) volatile + { + Debug::Assert(len <= 0x7ff, + "You can't receive more than 0x7ff bytes at a time."); + len &= StartByteCountMask; + wait_idle(); + control = ControlReceiveEnable; + start = len; + + for (uint32_t i = 0; i < len; ++i) + { + // Wait for at least one byte to be available in the RX FIFO + while ((status & StatusRxFifoLevel) == 0) {} + data[i] = static_cast<uint8_t>(receiveFifo); + } + } +};
diff --git a/sdk/include/stdio.h b/sdk/include/stdio.h index fd6b605..9448b5c 100644 --- a/sdk/include/stdio.h +++ b/sdk/include/stdio.h
@@ -48,6 +48,7 @@ return ret; } +#ifdef stdout static inline int printf(const char *format, ...) { va_list ap; @@ -57,6 +58,7 @@ va_end(ap); return ret; } +#endif int __cheri_libcall snprintf(char *str, size_t size, const char *format, ...); int __cheri_libcall vsnprintf(const char *str,
diff --git a/sdk/include/stdlib.h b/sdk/include/stdlib.h index d60e4e4..6955d23 100644 --- a/sdk/include/stdlib.h +++ b/sdk/include/stdlib.h
@@ -85,6 +85,17 @@ */ #define MALLOC_CAPABILITY STATIC_SEALED_VALUE(__default_malloc_capability) +#ifndef MALLOC_WAIT_TICKS +/** + * Define how long a call to `malloc` and `calloc` can block to fulfil an + * allocation. Regardless of this value, `malloc` and `calloc` will only ever + * block to wait for the quarantine to be processed. This means that, even with + * a non-zero value of `MALLOC_WAIT_TICKS`, `malloc` would immediately return + * if the heap or the quota is exhausted. + */ +# define MALLOC_WAIT_TICKS 30 +#endif + __BEGIN_DECLS static inline void __dead2 panic() { @@ -95,15 +106,53 @@ } } +enum [[clang::flag_enum]] AllocateWaitFlags{ + /** + * Non-blocking mode. This is equivalent to passing a timeout with no time + * remaining. + */ + AllocateWaitNone = 0, + /** + * If there is enough memory in the quarantine to fulfil the allocation, wait + * for the revoker to free objects from the quarantine. + */ + AllocateWaitRevocationNeeded = (1 << 0), + /** + * If the quota of the passed heap capability is exceeded, wait for other + * threads to free allocations. + */ + AllocateWaitQuotaExceeded = (1 << 1), + /** + * If the heap memory is exhausted, wait for any other thread of the system + * to free allocations. + */ + AllocateWaitHeapFull = (1 << 2), + /** + * Block on any of the above reasons. This is the default behavior. + */ + AllocateWaitAny = (AllocateWaitRevocationNeeded | AllocateWaitQuotaExceeded | + AllocateWaitHeapFull), +}; + /** * Non-standard allocation API. Allocates `size` bytes. Blocking behaviour is - * controlled by the `timeout` parameter. + * controlled by the `flags` and the `timeout` parameters. * - * The non-blocking mode will return a successful allocation if one can be - * created immediately, or `nullptr` otherwise. - * The blocking versions of this may return `nullptr` if the timeout has expired - * or if the allocation cannot be satisfied under any circumstances (for example - * if `size` is larger than the total heap size). + * Specifically, the `flags` parameter defines on which conditions to wait, and + * the `timeout` parameter how long to wait. + * + * The non-blocking mode (`AllocateWaitNone`, or `timeout` with no time + * remaining) will return a successful allocation if one can be created + * immediately, or `nullptr` otherwise. + * + * The blocking modes may return `nullptr` if the condition to wait is not + * fulfiled, if the timeout has expired, or if the allocation cannot be + * satisfied under any circumstances (for example if `size` is larger than the + * total heap size). + * + * This means that calling this with `AllocateWaitAny` and `UnlimitedTimeout` + * will only ever return `nullptr` if the allocation cannot be satisfied under + * any circumstances. * * In both blocking and non-blocking cases, `-ENOTENOUGHSTACK` may be returned * if the stack is insufficiently large to safely run the function. This means @@ -115,24 +164,19 @@ void *__cheri_compartment("alloc") heap_allocate(Timeout *timeout, struct SObjStruct *heapCapability, - size_t size); + size_t size, + uint32_t flags __if_cxx(= AllocateWaitAny)); /** * Non-standard allocation API. Allocates `size` * `nmemb` bytes of memory, - * checking for arithmetic overflow. Blocking behaviour is controlled by the - * `timeout` parameter: + * checking for arithmetic overflow. Similarly to `heap_allocate`, blocking + * behaviour is controlled by the `flags` and the `timeout` parameters. * - * - 0 indicates that this call may not block. - * - The maximum value of the type indicates that this may block indefinitely. - * - Any other value indicates that this may block for, at most, that many - * ticks. - * - * The non-blocking mode will return a successful allocation if one can be - * created immediately, or `nullptr` otherwise. - * The blocking versions of this may return `nullptr` if the timeout has expired - * or if the allocation cannot be satisfied under any circumstances (for example - * if `nmemb` * `size` is larger than the total heap size, or if `nmemb` * - * `size` overflows). + * See `heap_allocate` for more information on the blocking behavior. One + * difference between this and `heap_allocate` is the definition of when the + * allocation cannot be satisfied under any circumstances, which is here if + * `nmemb` * `size` is larger than the total heap size, or if `nmemb` * `size` + * overflows. * * Similarly to `heap_allocate`, `-ENOTENOUGHSTACK` may be returned if the * stack is insufficiently large to run the function. See `heap_allocate`. @@ -143,7 +187,8 @@ heap_allocate_array(Timeout *timeout, struct SObjStruct *heapCapability, size_t nmemb, - size_t size); + size_t size, + uint32_t flags __if_cxx(= AllocateWaitAny)); /** * Add a claim to an allocation. The object will be counted against the quota @@ -254,8 +299,9 @@ #ifndef CHERIOT_NO_AMBIENT_MALLOC static inline void *malloc(size_t size) { - Timeout t = {0, 0}; - void *ptr = heap_allocate(&t, MALLOC_CAPABILITY, size); + Timeout t = {0, MALLOC_WAIT_TICKS}; + void *ptr = + heap_allocate(&t, MALLOC_CAPABILITY, size, AllocateWaitRevocationNeeded); if (!__builtin_cheri_tag_get(ptr)) { ptr = NULL; @@ -264,8 +310,9 @@ } static inline void *calloc(size_t nmemb, size_t size) { - Timeout t = {0, 0}; - void *ptr = heap_allocate_array(&t, MALLOC_CAPABILITY, nmemb, size); + Timeout t = {0, MALLOC_WAIT_TICKS}; + void *ptr = heap_allocate_array( + &t, MALLOC_CAPABILITY, nmemb, size, AllocateWaitRevocationNeeded); if (!__builtin_cheri_tag_get(ptr)) { ptr = NULL;
diff --git a/sdk/include/thread.h b/sdk/include/thread.h index 3c5430a..2f156ce 100644 --- a/sdk/include/thread.h +++ b/sdk/include/thread.h
@@ -23,6 +23,17 @@ [[cheri::interrupt_state(disabled)]] SystickReturn __cheri_compartment("sched") thread_systemtick_get(void); +enum ThreadSleepFlags : uint32_t +{ + /** + * Sleep for up to the specified timeout, but wake early if there are no + * other runnable threads. This allows a high-priority thread to yield for + * a fixed number of ticks for lower-priority threads to run, but does not + * prevent it from resuming early. + */ + ThreadSleepNoEarlyWake = 1 << 0, +}; + /** * Sleep for at most the specified timeout (see `timeout.h`). * @@ -34,9 +45,23 @@ * but reports the time spent sleeping. This requires a cross-domain call and * return in addition to the overheads of `yield` and so `yield` should be * preferred in contexts where the elapsed time is not required. + * + * The `flags` parameter is a bitwise OR of `ThreadSleepFlags`. + * + * A sleeping thread may be woken early if no other threads are runnable or + * have earlier timeouts. The thread with the earliest timeout will be woken + * first. This can cause a yielding thread to sleep when no other thread is + * runnable, but avoids a potential problem where a high-priority thread yields + * to allow a low-priority thread to make progress, but then the low-priority + * thread does a short sleep. In this case, the desired behaviour is not to + * wake the high-priority thread early, but to allow the low-priority thread to + * run for the full duration of the high-priority thread's yield. + * + * If you are using `thread_sleep` to elapse real time, pass + * `ThreadSleepNoEarlyWake` as the flags argument to prevent early wakeups. */ [[cheri::interrupt_state(disabled)]] int __cheri_compartment("sched") - thread_sleep(struct Timeout *timeout); + thread_sleep(struct Timeout *timeout, uint32_t flags __if_cxx(= 0)); /** * Return the thread ID of the current running thread. @@ -119,7 +144,7 @@ // In simulation builds, just yield once but don't bother trying to do // anything sensible with time. Timeout t = {0, 1}; - thread_sleep(&t); + thread_sleep(&t, 0); return milliseconds; #else static const uint32_t CyclesPerMillisecond = CPU_TIMER_HZ / 1'000; @@ -133,7 +158,7 @@ while ((end > current) && (end - current > MS_PER_TICK)) { Timeout t = {0, ((uint32_t)(end - current)) / CyclesPerTick}; - thread_sleep(&t); + thread_sleep(&t, ThreadSleepNoEarlyWake); current = rdcycle64(); } // Spin for the remaining time. @@ -142,7 +167,7 @@ current = rdcycle64(); } current = rdcycle64(); - return (current - start) * CyclesPerMillisecond; + return (current - start) / CyclesPerMillisecond; #endif }
diff --git a/sdk/lib/debug/debug.cc b/sdk/lib/debug/debug.cc index 0756164..2642d9f 100644 --- a/sdk/lib/debug/debug.cc +++ b/sdk/lib/debug/debug.cc
@@ -2,6 +2,7 @@ // SPDX-License-Identifier: MIT #include <debug.hh> +#include <thread.h> using namespace CHERI; @@ -335,7 +336,13 @@ DebugPrinter printer; printer.write("\x1b[35m"); printer.write(context); +#if 0 + printer.write(" [Thread "); + printer.write(thread_id_get()); + printer.write("]\033[0m: "); +#else printer.write("\033[0m: "); +#endif printer.format(format, messages, messageCount); printer.write("\n"); }
diff --git a/tests/allocator-test.cc b/tests/allocator-test.cc index c206f35..c64eaac 100644 --- a/tests/allocator-test.cc +++ b/tests/allocator-test.cc
@@ -1,6 +1,7 @@ // Copyright Microsoft and CHERIoT Contributors. // SPDX-License-Identifier: MIT // Use a large quota for this compartment. +#include "token.h" #define MALLOC_QUOTA 0x100000 #define TEST_NAME "Allocator" @@ -24,6 +25,9 @@ using namespace CHERI; #define SECOND_HEAP STATIC_SEALED_VALUE(secondHeap) +DECLARE_AND_DEFINE_ALLOCATOR_CAPABILITY(emptyHeap, 0); +#define EMPTY_HEAP STATIC_SEALED_VALUE(emptyHeap) + namespace { /** @@ -155,8 +159,50 @@ TEST(heap_allocate(&noWait, MALLOC_CAPABILITY, BigAllocSize) == nullptr, "Non-blocking heap allocation did not return failure with memory " "exhausted"); - debug_log("Trying a huge allocation"); + debug_log("Checking that the 'heap full' flag works"); Timeout forever{UnlimitedTimeout}; + TEST(heap_allocate(&forever, + MALLOC_CAPABILITY, + BigAllocSize, + AllocateWaitRevocationNeeded | + AllocateWaitQuotaExceeded) == nullptr, + "Blocking heap allocation with the heap full flag unset did not " + "return failure with memory " + "exhausted"); + Timeout thirtyticks{30}; + TEST(heap_allocate(&thirtyticks, + MALLOC_CAPABILITY, + BigAllocSize, + AllocateWaitHeapFull) == nullptr, + "Time-limited blocking allocation did not return failure with " + "memory exhausted"); + TEST(thirtyticks.remaining == 0, + "Allocation with heap full wait flag set did not wait on memory " + "exhausted"); + debug_log("Checking that the 'quota exhausted' flag works"); + TEST(heap_allocate(&forever, + EMPTY_HEAP, + BigAllocSize, + AllocateWaitRevocationNeeded) == nullptr, + "Blocking heap allocation with the quota exhausted flag unset did " + "not " + "return failure with memory " + "exhausted"); + thirtyticks = Timeout{30}; + TEST(heap_allocate(&thirtyticks, + EMPTY_HEAP, + BigAllocSize, + AllocateWaitQuotaExceeded) == nullptr, + "Time-limited blocking allocation did not return failure with " + "memory exhausted"); + TEST( + thirtyticks.remaining == 0, + "Allocation with quota exhausted wait flag set did not wait on quota " + "exhausted"); + // Note: we do not test the functioning of + // `AllocateWaitQuotaExceeded` as this would require to be able + // to manipulate the quarantine to be reliably done. + debug_log("Trying a huge allocation"); // nullptr check because we explicitly want to check for OOM TEST(heap_allocate(&forever, MALLOC_CAPABILITY, 1024 * 1024 * 1024) == nullptr, @@ -427,6 +473,49 @@ debug_log("Hazard pointer tests done"); } + void test_large_token(size_t tokenSize) + { + void *unsealedCapability; + auto sealingCapability = STATIC_SEALING_TYPE(sealingTest); + Capability sealedPointer = + token_sealed_unsealed_alloc(&noWait, + MALLOC_CAPABILITY, + sealingCapability, + tokenSize, + &unsealedCapability); + TEST(sealedPointer.is_valid(), + "Failed to allocate large sealed capability that requires padding " + "for the header"); + TEST(sealedPointer.is_sealed(), "Failed to allocate sealed capability"); + TEST(!Capability{unsealedCapability}.is_sealed(), + "Failed to allocate sealed capability"); + size_t unsealedLength = Capability{unsealedCapability}.length(); + TEST(unsealedLength >= tokenSize, + "Length of unsealed capability is not {}: {}", + tokenSize, + unsealedCapability); + TEST(sealedPointer.length() >= unsealedLength + 8, + "Length of unsealed capability is not the unsealed size plus the " + "header size: {}", + unsealedCapability); + TEST(sealedPointer.address() + 4 < + Capability{unsealedCapability}.address(), + "Header for the sealed capability ({}) is not before the start of " + "the unsealed capability ({})", + sealedPointer, + unsealedCapability); + Capability unsealedLarge = + token_unseal(sealingCapability, Sealed<void>{sealedPointer.get()}); + TEST(unsealedLarge == Capability{unsealedCapability}, + "Unsealing large capability gave a different capability to the " + "expected one ({} != {})", + unsealedLarge, + unsealedCapability); + int destroyed = token_obj_destroy( + MALLOC_CAPABILITY, sealingCapability, sealedPointer); + TEST(destroyed == 0, "Failed to destroy large sealed capability"); + } + /** * Test the sealing APIs. Tests that we can allocate and free sealed * objects, that we can't collect them with the wrong capabilities, and that @@ -439,6 +528,35 @@ */ __noinline void test_token() { + debug_log("Testing token allocation"); + size_t validSizes[] = { + 128, 0xfe9, 8193, 511, 511 << 1, 511 << 2, 511 << 3}; + for (size_t tokenSize : validSizes) + { + debug_log("Testing (expected valid) token size {}", tokenSize); + test_large_token(tokenSize); + } + size_t invalidSizes[] = {0, 0xffffffff}; + for (size_t tokenSize : invalidSizes) + { + debug_log("Testing (expected invalid) token size {}", tokenSize); + auto sealingCapability = STATIC_SEALING_TYPE(sealingTest); + void *unsealedCapability; + Capability sealedPointer = + token_sealed_unsealed_alloc(&noWait, + MALLOC_CAPABILITY, + sealingCapability, + tokenSize, + &unsealedCapability); + TEST(!sealedPointer.is_valid(), + "Allocated {} for invalid token size {}", + sealedPointer, + tokenSize); + TEST(!Capability{unsealedCapability}.is_valid(), + "Allocated {} for invalid token size {}", + unsealedCapability, + tokenSize); + } auto sealingCapability = STATIC_SEALING_TYPE(sealingTest); Timeout noWait{0}; void *unsealedCapability; @@ -447,14 +565,6 @@ heap_quota_remaining(SECOND_HEAP), SECOND_HEAP_QUOTA); Capability sealedPointer = token_sealed_unsealed_alloc( - &noWait, SECOND_HEAP, sealingCapability, 4096, &unsealedCapability); - // Note: If this test fails because the allocator can handle larger - // allocations, then this test can be removed. If it fails because the - // allocator is incorrectly using imprecise bounds, please fix it! - TEST(!sealedPointer.is_valid(), - "Successfully allocated capability that is too large for precise " - "bounds"); - sealedPointer = token_sealed_unsealed_alloc( &noWait, SECOND_HEAP, sealingCapability, 128, &unsealedCapability); TEST(sealedPointer.is_valid(), "Failed to allocate capability"); TEST(sealedPointer.is_sealed(), "Failed to allocate sealed capability");
diff --git a/tests/ccompile-freertos-test.c b/tests/ccompile-freertos-test.c new file mode 100644 index 0000000..8930b0b --- /dev/null +++ b/tests/ccompile-freertos-test.c
@@ -0,0 +1,12 @@ +#include <FreeRTOS-Compat/FreeRTOS.h> + +#if (CHERIOT_FREERTOS_SEMAPHORE + CHERIOT_FREERTOS_MUTEX + \ + CHERIOT_FREERTOS_RECURSIVE_MUTEX) == 1 +#if CHERIOT_FREERTOS_SEMAPHORE == 1 +_Static_assert(sizeof(StaticSemaphore_t) == sizeof(struct CountingSemaphoreState); +#elif CHERIOT_FREERTOS_MUTEX == 1 +_Static_assert(sizeof(StaticSemaphore_t) == sizeof(struct FlagLockState)); +#else +_Static_assert(sizeof(StaticSemaphore_t) == sizeof(struct RecursiveMutexState)); +#endif +#endif
diff --git a/tests/list-test.cc b/tests/list-test.cc new file mode 100644 index 0000000..7db652e --- /dev/null +++ b/tests/list-test.cc
@@ -0,0 +1,241 @@ +// Copyright Microsoft and CHERIoT Contributors. +// SPDX-License-Identifier: MIT + +#define TEST_NAME "List" +#include "tests.hh" +#include <ds/linked_list.h> + +using CHERI::Capability; + +namespace +{ + /** + * Example class we want to link into a doubly linked list. + * + * The class contains a single integer for the purposes of the test. + * + * `ds::linked_list` is an intrusive list: we embed the list node into + * the class we want to link. There are various implementations of the + * list nodes. Here we use the most simple one + * (`ds::linked_list::cell::Pointer`) which relies on two pointers + * `next` and `prev`. + */ + struct LinkedObject + { + using ObjectRing = ds::linked_list::cell::Pointer; + + int data; + /** + * List node: links objects into the doubly-linked list. + */ + ObjectRing ring __attribute__((__cheri_no_subobject_bounds__)) = {}; + /** + * Container-of for the above field. This is used to retrieve the + * corresponding object from a list element. + */ + __always_inline static struct LinkedObject *from_ring(ObjectRing *c) + { + return reinterpret_cast<struct LinkedObject *>( + reinterpret_cast<uintptr_t>(c) - + offsetof(struct LinkedObject, ring)); + } + }; +} // namespace + +/** + * `ds::linked_list`s are circular, doubly-linked collections. While they can + * stand on their own as rings of objects, it is sometimes convenient to create + * a designated 'sentinel' node that participates in the collection without + * being part of the collection: + * + * - a sentinel node provides pointers to the effective head and tail of the + * collection (the successor and predecessor of the sentinel, respectively) + * + * - a sentinel allows not having to special-case 'the collection is empty' in + * as many places as some other representations (that is, collections with + * sentinels need fewer NULL pointer checks) + * + * - a sentinel provides many handy functions to operate on the list + * + * Note: do not allocate the sentinel (or any list cell) on the stack, because + * it would lead some list nodes to hold a pointer to a stack value, i.e., to + * an invalid capability. This would manifest as a crash while using the list. + */ +ds::linked_list::Sentinel<LinkedObject::ObjectRing> objects = {}; + +void test_list() +{ + debug_log("Testing the list implementation."); + + // Number of elements we will add to the list in the test. Must be + // divisible by two. + static constexpr int NumberOfListElements = 30; + + auto heapAtStart = heap_quota_remaining(MALLOC_CAPABILITY); + + TEST(objects.is_empty(), "Newly created list is not empty"); + + // Create heap-allocated objects, and link them into the linked list. + for (int i = 0; i < NumberOfListElements; i++) + { + Timeout t{UnlimitedTimeout}; + LinkedObject *o = static_cast<LinkedObject *>( + heap_allocate(&t, MALLOC_CAPABILITY, sizeof(LinkedObject))); + TEST(Capability{o}.is_valid(), "Cannot allocate linked object"); + + // Use the object integer as an index. + o->data = i; + // The list node has not yet been initialized. + o->ring.cell_reset(); + + // Test that we can retrieve the object from the link node and + // that this results in a capability which is identical to what + // we got from the allocator. + TEST(Capability{o} == Capability{LinkedObject::from_ring(&(o->ring))}, + "The container of method does not return the right object"); + TEST(Capability{LinkedObject::from_ring(&(o->ring))}.is_valid(), + "Capability retrieved from `from_ring` is invalid"); + + // Add the new object to the list through the sentinel node. + objects.append(&(o->ring)); + } + + TEST(!objects.is_empty(), "The list is empty after adding objects"); + + // Test that the sentinel can be used to retrieve the first and last + // elements of the list as expected. + TEST(LinkedObject::from_ring(objects.last())->data == + NumberOfListElements - 1, + "Last element of the list is incorrect, expected {}, got {}", + NumberOfListElements - 1, + LinkedObject::from_ring(objects.last())->data); + TEST(objects.last()->cell_next() == &objects.sentinel, + "Last element in not followed by the sentinel"); + TEST(objects.last() == objects.sentinel.cell_prev(), + "Sentinel is not preceeded by the last element"); + + TEST(LinkedObject::from_ring(objects.first())->data == 0, + "First element of the list is incorrect, expected {}, got {}", + 0, + LinkedObject::from_ring(objects.last())->data); + TEST(objects.first()->cell_prev() == &objects.sentinel, + "First element in not preceeded by the sentinel"); + TEST(objects.first() == objects.sentinel.cell_next(), + "Sentinel is not followed by the first element"); + + // Test that we can go through the list by following `cell_next` + // pointers as expected. + int counter = 0; + // While at it, retrieve a pointer to the middle element which we will + // use to cleave the list later. + LinkedObject::ObjectRing *middle = nullptr; + // We reach the sentinel when we have gone through all elements of the + // list. + for (auto *cell = objects.first(); cell != &objects.sentinel; + cell = cell->cell_next()) + { + struct LinkedObject *o = LinkedObject::from_ring(cell); + TEST( + o->data == counter, + "Ordering of elements in the list is incorrect, expected {}, got {}", + o->data, + counter); + if (counter == NumberOfListElements / 2) + { + middle = cell; + } + counter++; + } + + TEST(middle != nullptr, "Could not find middle element of the list"); + + // Cut the list in the middle. `middle` is now a handle to the (valid) + // collection of objects [middle, last] that have become detached from + // the sentinel. + ds::linked_list::remove(middle, objects.last()); + + // This should leave us with a list of size `NumberOfListElements / 2`. + counter = 0; + for (auto *cell = objects.first(); cell != &objects.sentinel; + cell = cell->cell_next()) + { + counter++; + } + TEST(counter == NumberOfListElements / 2, + "Cleaving didn't leave a list with the right number of elements"); + + // Now remove (and free) a single element from the list. + TEST(LinkedObject::from_ring(objects.first())->data == 0, + "First element of the list is incorrect, expected {}, got {}", + 0, + LinkedObject::from_ring(objects.first())->data); + // We must keep a reference to the removed object to free it, as + // `remove` returns a pointer to the residual list (return value which + // we do not use here), not to the removed element. + LinkedObject::ObjectRing *removedCell = objects.first(); + ds::linked_list::remove(objects.first()); + heap_free(MALLOC_CAPABILITY, LinkedObject::from_ring(removedCell)); + TEST(LinkedObject::from_ring(objects.first())->data == 1, + "First element of the list is incorrect after removing the first " + "element, expected {}, got {}", + 1, + LinkedObject::from_ring(objects.first())->data); + TEST(objects.first()->cell_prev() == &objects.sentinel, + "First element in not preceeded by the sentinel after removing the " + "first object"); + + // We are done with the list, free it. + counter = 0; + LinkedObject::ObjectRing *cell = objects.first(); + while (cell != &objects.sentinel) + { + struct LinkedObject *o = LinkedObject::from_ring(cell); + cell = cell->cell_next(); + heap_free(MALLOC_CAPABILITY, o); + counter++; + } + + TEST(counter == (NumberOfListElements / 2) - 1, + "Incorrect number of elements freed, expected {}, got {}", + (NumberOfListElements / 2) - 1, + counter); + + // Now that the list is freed, reset the sentinel. + objects.reset(); + + TEST(objects.is_empty(), "Reset-ed list is not empty"); + + // We must also free the span of the list which we removed earlier. + // This time use the `::search` method to go through the collection. + ds::linked_list::search( + middle, [&counter](LinkedObject::ObjectRing *&cell) { + // `unsafe_remove` does not update the node pointers of the + // removed cell. This is great here because we will free the + // object anyways. We could also use `remove` here. + auto l = ds::linked_list::unsafe_remove(cell); + heap_free(MALLOC_CAPABILITY, LinkedObject::from_ring(cell)); + // `l` is the predecessor of `cell` in the residual ring, so + // this does exactly what we want when `::search` iterates. + cell = l; + counter++; + return false; + }); + // `::search` does not visit the element passed (`middle`) + heap_free(MALLOC_CAPABILITY, LinkedObject::from_ring(middle)); + counter++; + + TEST(counter == NumberOfListElements - 1, + "Incorrect number of elements freed, expected {}, got {}", + NumberOfListElements - 1, + counter); + + // Check that we didn't leak anything in the process + auto heapAtEnd = heap_quota_remaining(MALLOC_CAPABILITY); + TEST(heapAtStart == heapAtEnd, + "The list leaked {} bytes ({} vs. {})", + heapAtEnd - heapAtStart, + heapAtStart, + heapAtEnd); + + debug_log("Done testing the list."); +}
diff --git a/tests/locks-test.cc b/tests/locks-test.cc index ffd8da2..7b8db43 100644 --- a/tests/locks-test.cc +++ b/tests/locks-test.cc
@@ -59,7 +59,9 @@ "Trying to acquire lock spuriously succeeded"); if constexpr (!std::is_same_v<Lock, FlagLockPriorityInherited>) { +#ifndef SIMULATION TEST(t.elapsed >= 1, "Sleep slept for {} ticks", t.elapsed); +#endif } } @@ -200,6 +202,42 @@ "Unlocking unsets the destruction bit of flag lock"); } + /** + * Test that `get_owner_thread_id` returns the thread ID of the owner + * of the lock. + */ + void test_get_owner_thread_id(FlagLockPriorityInherited &lock) + { + debug_log("Testing that `get_owner_thread_id` works."); + + TEST(lock.get_owner_thread_id() == 0, + "`get_owner_thread_id` does not return 0 when called on an unheld " + "lock"); + + modified = false; + LockGuard g{lock}; + uint16_t ownerThreadId = thread_id_get(); + TEST(lock.get_owner_thread_id() == ownerThreadId, + "`get_owner_thread_id` does not return the thread ID of the lock " + "owner"); + + async([ownerThreadId, &lock]() { + TEST(thread_id_get() != ownerThreadId, + "Async has the same thread ID as the main thread"); + TEST(lock.get_owner_thread_id() == ownerThreadId, + "`get_owner_thread_id` does not return the thread ID of the " + "lock owner when called from a non-owning thread"); + modified = true; + }); + + sleep(1); + while (!modified) + { + debug_log("Other thread not finished, yielding"); + sleep(1); + } + } + void test_recursive_mutex() { static RecursiveMutexState recursiveMutex; @@ -363,6 +401,7 @@ test_lock(flagLock); test_lock(flagLockPriorityInherited); test_lock(ticketLock); + test_get_owner_thread_id(flagLockPriorityInherited); test_flaglock_unlock(); test_trylock(flagLock); test_trylock(flagLockPriorityInherited);
diff --git a/tests/misc-test.cc b/tests/misc-test.cc index 13e1adf..3cbab21 100644 --- a/tests/misc-test.cc +++ b/tests/misc-test.cc
@@ -3,6 +3,7 @@ #define TEST_NAME "Test misc APIs" #include "tests.hh" +#include <ds/pointer.h> #include <string.h> #include <timeout.h> @@ -85,8 +86,35 @@ "memchr must return NULL for zero-size pointers."); } +/** + * Test pointer utilities. + * + * Not comprehensive, would benefit from being expanded at some point. + */ +void check_pointer_utilities() +{ + debug_log("Test pointer utilities."); + + int integer = 42; + int *integerPointer = &integer; + ds::pointer::proxy::Pointer<int> pointer{integerPointer}; + + TEST((pointer == integerPointer) && (*pointer == 42), + "The pointer proxy does not return the value of its proxy."); + + int anotherInteger = -100; + int *anotherIntegerPointer = &anotherInteger; + ds::pointer::proxy::Pointer<int> anotherPointer{anotherIntegerPointer}; + + pointer = anotherPointer; + + TEST((pointer == anotherIntegerPointer) && (*pointer == -100), + "The pointer proxy `=` operator does not correctly set the pointer."); +} + void test_misc() { check_timeouts(); check_memchr(); + check_pointer_utilities(); }
diff --git a/tests/test-runner.cc b/tests/test-runner.cc index 8d99fdd..2227633 100644 --- a/tests/test-runner.cc +++ b/tests/test-runner.cc
@@ -4,8 +4,10 @@ #include "tests.hh" #include <compartment.h> #include <simulator.h> +#include <string> using namespace CHERI; +using namespace std::string_literals; namespace { @@ -104,8 +106,10 @@ 0x123456789012345ULL); const char *testString = "Hello, world! with some trailing characters"; // Make sure that we don't print the trailing characters - debug_log("Trying to print string: {}", std::string_view{testString, 13}); - + debug_log("Trying to print std::string_view: {}", + std::string_view{testString, 13}); + const std::string S = "I am a walrus"s; + debug_log("Trying to print std::string: {}", S); run_timed("All tests", []() { run_timed("Debug helpers (C++)", test_debug_cxx); run_timed("Debug helpers (C)", test_debug_c); @@ -122,6 +126,7 @@ run_timed("Queue", test_queue); run_timed("Futex", test_futex); run_timed("Locks", test_locks); + run_timed("List", test_list); run_timed("Event groups", test_eventgroup); run_timed("Multiwaiter", test_multiwaiter); run_timed("Allocator", test_allocator);
diff --git a/tests/tests.hh b/tests/tests.hh index bd63a3c..f5625bc 100644 --- a/tests/tests.hh +++ b/tests/tests.hh
@@ -12,6 +12,7 @@ __cheri_compartment("futex_test") void test_futex(); __cheri_compartment("queue_test") void test_queue(); __cheri_compartment("locks_test") void test_locks(); +__cheri_compartment("list_test") void test_list(); __cheri_compartment("crash_recovery_test") void test_crash_recovery(); __cheri_compartment("multiwaiter_test") void test_multiwaiter(); __cheri_compartment("stack_test") void test_stack();
diff --git a/tests/xmake.lua b/tests/xmake.lua index 2f9e5ed..267e791 100644 --- a/tests/xmake.lua +++ b/tests/xmake.lua
@@ -50,6 +50,8 @@ test("futex") -- Test locks built on top of the futex test("locks") +-- Test the generic linked list from ds/ +test("list") -- Test queues test("queue") -- Test minimal stdio implementation @@ -106,6 +108,7 @@ add_deps("futex_test") add_deps("queue_test") add_deps("locks_test") + add_deps("list_test") add_deps("static_sealing_test", "static_sealing_inner") add_deps("crash_recovery_test", "crash_recovery_inner", "crash_recovery_outer") add_deps("multiwaiter_test")