libtock_platform Design StoryNote: libtock_platform is part of the libtock 2.0 rewrite. It is partially implemented in this repository. A working version exists in @jrvanwhy's repository in the wip branch. It is being refined and merged upstream in pieces.
libtock_platform is a crate that will contain core abstractions that will be used by libtock_core‘s drivers. For example, it will contain abstractions for asynchronous APIs that are lighter weight (in terms of code size and RAM usage) than Rust’s futures.
This document serves as a justification for, and explanation of, the high-level design of libtock_platform. Instead of describing the various components and how they interact, it starts with a hello world application and extracts some of the functionality out into a reusable library. As we do so, we see that libtock_core's design considerations lead incrementally to a design for libtock_platform.
In order to keep things understandable, this document makes several simplifications. Error handling logic is omitted, although we still structure the code so as to allow it. We use unsafe in places where we should instead have an additional, safe, abstraction. We use a simplified form of the Tock 2.0 system calls, which are currently undergoing design revisions.
We start with the following example app. In order to show how libtock_platform will handle asynchronous callbacks, it is written in an asynchronous manner.
#![no_std] static GREETING: [u8; 7] = *b"Hello, "; static NOUN: [u8; 7] = *b"World!\n"; static mut DONE: bool = false; // Driver, command, allow, and subscription numbers. const DRIVER: usize = 1; const WRITE_DONE: usize = 1; const WRITE_BUFFER: usize = 1; const START_WRITE: usize = 1; fn main() { libtock_runtime::subscribe(DRIVER, WRITE_DONE, write_complete, 0); libtock_runtime::const_allow(DRIVER, WRITE_BUFFER, &GREETING); libtock_runtime::command(DRIVER, START_WRITE, GREETING.len(), 0); loop { libtock_runtime::yieldk(); } } extern "C" fn write_complete(bytes: usize, _: usize, _: usize, _data: usize) { // Detect if this write completion is a result of writing NOUN (the // second write). If so, return immediately to avoid an infinite loop. unsafe { if DONE { return; } DONE = true; } // At this point, we just finished writing GREETING and need to write NOUN. libtock_runtime::const_allow(DRIVER, WRITE_BUFFER, &NOUN); libtock_runtime::command(DRIVER, START_WRITE, NOUN.len(), 0); }
Strictly speaking, in this app, DONE could be tracked by passing it as the data argument to subscribe. However, most applications will need to track more than 32 bits of persistent state -- and use more than one callback -- so they will need to manage persistent state themselves. To make this app representative of most apps, we keep the state in userspace memory.
This example assumes that the libtock_runtime crate exposes system call implementations that look like the following (these are a simplification of the Tock 2.0 system calls):
pub fn subscribe(driver_num: usize, subscription_num: usize, callback: extern "C" fn(usize, usize, usize, usize), data: usize); pub fn const_allow(driver_num: usize, buffer_num: usize, buffer: &'static [u8]); pub fn command(driver_num: usize, command_num: usize, arg1: usize, arg2: usize); pub fn yieldk();
What do we already get “right” in this example? Except for the two intentional inefficiencies explained above (performing the write in two steps rather than 1 and maintaining the DONE state in process memory), it is very efficient. It has only 3 global variables: GREETING, NOUN, and DONE. It has almost zero bookkeeping overhead; it fairly directly makes 8 system calls (subscribe, const_allow, command, yieldk, const_allow, command, yieldk, yieldk).
What do we want to improve? It calls system calls directly from application code -- there should be a reusable console library that implements the system calls instead! The application shouldn't need to know what command number starts a write, it should just tell the console driver to do that for it.
Let‘s start taking some of the console-specific parts and moving them into a new crate. The first system call the app makes is to subscribe to the write done callback, so let’s add a function to a new libtock_console crate that sets a write callback using subscribe:
#![no_std] const DRIVER: usize = 1; const WRITE_DONE: usize = 1; fn set_write_callback(callback: fn(bytes: usize, data: usize), data: usize) { libtock_runtime::subscribe(DRIVER, WRITE_DONE, write_complete, data); // We need to store `callback` somewhere -- where to do so? } extern "C" fn write_complete(bytes: usize, _: usize, _: usize, data: usize) { // Hmm, how do we access the callback? }
You may notice that the code is not quite complete: set_write_callback takes a callback but doesn‘t store it anywhere. We don’t want to store it in a global variable because doing so would not be zero-cost: the original app didn‘t need to store a function pointer, and we want to do this refactoring without bloating the app. We could pass it through data, but what if the client code needs to use data itself? That pattern isn’t extensible: if there is another asynchronous layer about the console (e.g. a virtualization system), it won't have access to data to pass its callbacks through.
Instead, we can pass the callback through the type system. We need a new trait to represent the callback. This trait won‘t be specific to libtock_console, and we’ll later use it in unit tests -- which run on Linux, not Tock, so we'll put it in the libtock_platform crate:
pub trait FreeCallback<AsyncResponse> { fn call(response: AsyncResponse); }
We call this FreeCallback because it represents a free function as opposed to a method. (This forshadows MethodCallback, which we will need later)
The reason why we made this a shared generic trait rather than adding a libtock_console::Client trait as the Tock kernel does will be apparent later.
Using this trait, we can now write libtock_console::set_write_callback:
#![no_std] use libtock_platform::FreeCallback; const DRIVER: usize = 1; const WRITE_DONE: usize = 1; pub struct WriteCompleted { pub bytes: usize, pub data: usize, } pub fn set_write_callback<Callback: FreeCallback<WriteCompleted>>(data: usize) { libtock_runtime::subscribe(DRIVER, WRITE_DONE, write_complete::<Callback>, data); } extern "C" fn write_complete<Callback: FreeCallback<WriteCompleted>>( bytes: usize, _: usize, _: usize, data: usize) { Callback::call(WriteCompleted { bytes, data }); }
To finish libtock_console, we also need to add set_write_buffer (which calls allow) and start_write (which calls command), which are much simpler:
const WRITE_BUFFER: usize = 1; const START_WRITE: usize = 1; pub fn set_write_buffer(buffer: &'static [u8]) { libtock_runtime::const_allow(DRIVER, WRITE_BUFFER, buffer); } pub fn start_write(bytes: usize) { libtock_runtime::command(DRIVER, START_WRITE, bytes, 0); }
We can then make use of libtock_console in our app as follows:
#![no_std] static GREETING: [u8; 7] = *b"Hello, "; static NOUN: [u8; 7] = *b"World!\n"; static mut DONE: bool = false; fn main() { libtock_console::set_write_callback::<App>(0); libtock_console::set_write_buffer(&GREETING); libtock_console::start_write(GREETING.len()); loop { libtock_runtime::yieldk(); } } struct App; impl libtock_platform::FreeCallback<libtock_console::WriteCompleted> for App { fn call(_response: WriteCompleted) { unsafe { if DONE { return; } DONE = true; } libtock_console::set_write_buffer(&NOUN); libtock_console::start_write(NOUN.len()); } }
Now we have a reusable console library! However, we still don't have any unit tests.
A good unit test for the console driver would test not only the driver‘s operation with successful system calls but also also test the driver’s error-handling logic. That is difficult to do if we test using a real Tock kernel in an emulator -- the real kernel has the goal of avoiding system call errors! Instead of using a real Tock kernel, driver unit tests should use a “fake kernel” that simulates the kernel's functionality while allowing errors to be injected.
To keep this document reasonably short and understandable, we have omitted error handling, but we can still structure our unit tests in a manner that would allow a test to inject errors when error handling logic is added.
To allow the console driver to work with both a real kernel and a fake kernel, we add a Syscalls trait to libtock_platform. When compiled into a Tock process binary, Syscalls will be implemented by a zero-sized type, which avoids wasting non-volatile storage space or RAM area. To avoid passing around references to a Syscalls implementation, we can pass the Syscalls by value rather than by reference (i.e. take self rather than &self). For practical use, this requires the Syscalls implementations to be Copy.
pub trait Syscalls: Copy { fn subscribe(self, driver: usize, minor: usize, callback: extern "C" fn(usize, usize, usize, usize), data: usize); fn const_allow(self, major: usize, minor: usize, buffer: &'static [u8]); fn command(self, major: usize, miner: usize, arg1: usize, arg2: usize); fn yieldk(self); }
We then implement Syscalls using a real Tock kernel in libtock_runtime:
#[derive(Clone, Copy)] pub struct TockSyscalls; impl libtock_platform::Syscalls for TockSyscalls { /* Omitted implementation details */ }
We adapt libtock_console to use an app-provided Syscalls implementation rather than directly calling into libtock_runtime:
pub fn set_write_callback<S: Syscalls, Callback: FreeCallback<WriteCompleted>>( syscalls: S, data: usize) { syscalls.subscribe(1, 1, write_complete::<Callback>, data); } extern "C" fn write_complete<Callback: FreeCallback<WriteCompleted>>( bytes: usize, _: usize, _: usize, data: usize) { Callback::call(WriteCompleted { bytes, data }); } pub fn set_write_buffer<S: Syscalls>(syscalls: S, buffer: &'static [u8]) { syscalls.const_allow(1, 1, buffer); } pub fn start_write<S: Syscalls>(syscalls: S, bytes: usize) { syscalls.command(1, 1, bytes, 0); }
We'll create a new crate, libtock_unittest, which contains test utilities such as the fake Tock kernel. The fake kernel, unlike libtock_runtime::TockSyscalls, needs to maintain state, so it cannot be a zero-sized type. Instead of implementing Syscalls on the fake kernel directly, we implement it on a shared reference:
type RawCallback = extern "C" fn(usize, usize, usize, usize); pub struct FakeSyscalls { callback_pending: core::cell::Cell<Option<usize>>, output: core::cell::Cell<Vec<u8>>, write_buffer: core::cell::Cell<Option<&'static [u8]>>, write_callback: core::cell::Cell<Option<RawCallback>>, write_data: core::cell::Cell<usize>, } impl FakeSyscalls { pub fn new() -> Self { FakeSyscalls { callback_pending: Cell::new(None), output: Cell::new(Vec::new()), write_buffer: Cell::new(None), write_callback: Cell::new(None), write_data: Cell::new(0), } } pub fn read_buffer(&self) -> &'static [u8] { self.write_buffer.take().unwrap_or(&[]) } } impl libtock_platform::Syscalls for &FakeSyscalls { fn subscribe(self, driver: usize, minor: usize, callback: RawCallback, data: usize) { if driver == 1 && minor == 1 { self.write_callback.set(Some(callback)); self.write_data.set(data); } } fn const_allow(self, major: usize, minor: usize, buffer: &'static [u8]) { if major == 1 && minor == 1 { self.write_buffer.set(Some(buffer)); } } fn command(self, major: usize, minor: usize, arg1: usize, _arg2: usize) { if major != 1 || minor != 1 { return; } if let Some(buffer) = self.write_buffer.get() { let mut output = self.output.take(); let bytes = core::cmp::min(arg1, buffer.len()); output.extend_from_slice(&buffer[..bytes]); self.output.set(output); self.callback_pending.set(Some(bytes)); } } fn yieldk(self) { let bytes = match self.callback_pending.take() { Some(bytes) => bytes, None => return, }; if let Some(callback) = self.write_callback.get() { callback(bytes, 0, 0, self.write_data.get()); } } }
We‘re still not ready to add unit tests to libtock_console yet! libtock_console is asynchronous, which is difficult to work with in a unit test. libtock_core should provide synchronous APIs for apps that do not wish to be fully asynchronous, so lets go ahead and implement a synchronous API. To avoid re-implementing synchronous APIs for every driver, let’s make it work generically with all libtock_core drivers. This is where we benefit from making FreeCallback a generic trait rather than having a libtock_console::Client trait.
The synchronous adapter will need to store a copy of an AsyncResponse, so its callback cannot be a free function (it needs access to self). Therefore, we add the MethodCallback trait to libtock_platform:
pub trait MethodCallback<AsyncResponse> { fn call(&self, response: AsyncResponse); }
Using MethodCallback, we can now write SyncAdapter. We add SyncAdapter to a new crate, libtock_sync, as not all Tock apps will want it:
use libtock_platform::MethodCallback; pub struct SyncAdapter<AsyncResponse, Syscalls> { response: core::cell::Cell<Option<AsyncResponse>>, syscalls: Syscalls, } impl<AsyncResponse, Syscalls> SyncAdapter<AsyncResponse, Syscalls> { pub const fn new(syscalls: Syscalls) -> SyncAdapter<AsyncResponse, Syscalls> { SyncAdapter { response: core::cell::Cell::new(None), syscalls } } } impl<AsyncResponse, Syscalls: libtock_platform::Syscalls> SyncAdapter<AsyncResponse, Syscalls> { pub fn wait(&self) -> AsyncResponse { loop { match self.response.take() { Some(response) => return response, None => self.syscalls.yieldk(), } } } } impl<AsyncResponse, Syscalls: libtock_platform::Syscalls> MethodCallback<AsyncResponse> for SyncAdapter<AsyncResponse, Syscalls> { fn call(&self, response: AsyncResponse) { self.response.set(Some(response)); } }
Before we write the test itself, we should add one more utility to libtock_unittest. That utility is the test_component! macro, which creates a thread-local instance of a type and provides FreeCallback implementations for every MethodCallback implementation the type has:
#[macro_export] macro_rules! test_component { [$link:ident, $name:ident: $comp:ty = $init:expr] => { let $name = std::boxed::Box::leak(std::boxed::Box::new($init)) as &$comp; std::thread_local!(static GLOBAL: std::cell::Cell<Option<&'static $comp>> = std::cell::Cell::new(None)); GLOBAL.with(|g| g.set(Some($name))); struct $link; impl<T> libtock_platform::FreeCallback<T> for $link where $comp: libtock_platform::MethodCallback<T> { fn call(response: T) { GLOBAL.with(|g| g.get().unwrap()).call(response); } } }; }
We can finally add a unit test to libtock_console:
#[cfg(test)] mod tests { #[test] fn write() { extern crate std; use libtock_platform::MethodCallback; use libtock_sync::SyncAdapter; use libtock_unittest::FakeSyscalls; use std::boxed::Box; use std::thread_local; use super::{set_write_buffer, set_write_callback, start_write, WriteCompleted}; let syscalls: &_ = Box::leak(Box::new(FakeSyscalls::new())); libtock_unittest::test_component![SyncAdapterLink, sync_adapter: SyncAdapter<WriteCompleted, &'static FakeSyscalls> = SyncAdapter::new(syscalls)]; set_write_callback::<_, SyncAdapterLink>(syscalls, 1234); set_write_buffer(syscalls, b"Hello"); start_write(syscalls, 5); let response = sync_adapter.wait(); assert_eq!(response.bytes, 5); assert_eq!(response.data, 1234); assert_eq!(syscalls.read_buffer(), b"Hello"); } }
static_component!We added libtock_unittest::test_component! to make it easy to set up components in unit tests, but we have no equivalent for apps. Our app still uses unsafe to access its DONE variable. Instead, lets hide that unsafety behind a new macro. This macro is only sound in libtock_runtime's single-threaded environment, so we add it to libtock_runtime directly:
#[macro_export] macro_rules! static_component { [$link:ident, $name:ident: $comp:ty = $init:expr] => { static mut COMPONENT: $comp = $init; struct $link; impl<T> libtock_platform::FreeCallback<T> for $link where $comp: libtock_platform::MethodCallback<T> { fn call(response: T) { unsafe { &COMPONENT }.call(response); } } }; }
We can now use static_component! in our Hello World app to instantiate the App struct:
#![no_std] static GREETING: [u8; 7] = *b"Hello, "; static NOUN: [u8; 7] = *b"World!\n"; fn main() { libtock_console::set_write_callback::<_, AppLink>(0); libtock_console::set_write_buffer(&GREETING); libtock_console::start_write(GREETING.len()); loop { libtock_runtime::yieldk(); } } struct App { done: core::cell::Cell<bool> } impl App { pub const fn new() -> App { App { done: core::cell::Cell::new(false) } } } impl MethodCallback<WriteCompleted> for App { fn call(&self, _response: WriteCompleted) { if self.done.get() { return; } self.done.set(true); set_write_buffer(TockSyscalls, &NOUN ); start_write(TockSyscalls, NOUN.len() ); } } libtock_runtime::static_component![AppLink, APP: App = App::new()];
Now our app has no more unsafe!
We wrote a Hello World application that uses Tock's console system calls and asynchronous callbacks. We then extracted the console system call interface into a reusable library, creating the FreeCallback trait along the way.
In order to provide unit tests for the console library, we needed to create several new abstractions. We created Syscalls so that we can direct tho console driver's system calls to a fake kernel. We created the libtock_unittest crate which contains the fake kernel as well as a test_component! helper macro. We created SyncAdapter so that the unit test can be written in a synchronous manner -- although SyncAdapter is not testing-specific! We created MethodCallback because FreeCallback is not a powerful-enough abstraction on its own for SyncAdapter.
We ended up with six Rust crates:
libtock_console, which contains the console driver logic and unit test code.libtock_platform, which provides abstractions that can be shared with other drivers.libtock_runtime, which contains non-portable system call implementations.libtock_sync, which provides a synchronous interface using libtock_platform's traits.libtock_unittest, which provides a fake Tock kernel and other utilities.