Realtime concurrency primitives #

I am writing a desktop app for audio editing (You can read about that here). For that i need to communicate between the GUI and the audio thread. I can't use a regular mutex or channel, because the audio thread is a realtime thread. I wasn't able to find a lot of libraries for this, so i wrote my own: simple-left-right and rt-write-lock. You can find them on my tangled, codeberg or github. To get an understanding of the atomic orderings i would recommend Rust Atomics and Locks, which i also used to learn about atomics while writing the library.

What is realtime? #

The biggest difference between realtime code and other code is that the latency requirements are really aggressive. You often hear server devs talk about p99 latency, which means that 99% of the requests are faster than this time. In realtime you only care about p100 latency. Every single response has to be faster than some time or the code is wrong.

For desktop audio not reaching these latency goals means that the audio clicks or pops.

To reach these requirements the realtime thread isn't allowed to wait on a non-realtime thread. So in my case my audio thread can't wait on my GUI thread. This makes a mutex impossible to use. The audio thread also isn't allowed to communicate with the OS, because most OSes don't guarantee any response times. So allocating memory is not possible.

Requirements #

I need a way to commicate the current song state from the GUI to the audio thread. A song is a big object and changes are done very incremental.

I also need a way to commicate the current state of the audio playback to the GUI thread. Ideal there would be API like tokio::watch, because the GUI should always show the latest available state.

Channels (the almost solution) #

Realtime reading #

The GUI thread could encode changes done to the song and send them in a channel to the audio thread. The audio thread would then try to read from the channel and if there are changes update its own copy of the song.

This doesn't work, because parts of the song are stored in Vecs, so if the user creates more data the Vec needs to realloc on the audio thread. The update could done by sending a new Vec instead of updating the current one, so the audio thread only has to replace it, but then the old Vec would need to be freed on the audio thread. There are ways around this, but it starts to get inefficient and complicated.

Realtime writing #

The audio thread could try to write into a fixed size channel. If it's full the writer doesn't send the update to the GUI. This could work, but if the GUI thread is blocked for some time and then tries to catch up, it will be far behind the current state. It also doesn't allow reusing allocations inside of the sent state. This is not just an optimisation, because the audio thread can't allocate.

Implementation #

The idea behind both libraries is to have two copies of the data. This allows reading and writing at the same time by giving the reader and the writer access to different copies.

1pub struct Shared<T> {
2  state: AtomicU8,
3  value_1: UnsafeCell<T>,
4  value_2: UnsafeCell<T>,
5}

A data structure like this is used by both libraries. The state value stores a "ref count" (one or two) to free the memory when both reader and writer have been dropped, as well as lock states, which are different between the libraries.

Both libraries don't provide any methods that actually wait, like Mutex::lock() does, only try_lock methods. This is because the wait can't be implemented efficiently in the library. Mutex and similar data structures use the OS to sleep threads and allow other threads to wake them up later. But like i said previously the audio thread isn't allowed to communicate with the OS. So telling it to wake up another thread isn't possible and a spinlock becomes necessary. I don't provide a spin lock in these libraries because the way the library is used, makes a big difference in deciding how to spin.

For example in my own use of these libraries i use async sleep for spinning and the duration i sleep is depending on the audio settings, because this provide a good estimate on how long the audio thread will hold a lock.

simple-left-right #

simple-left-right is the library that allows realtime reads. The write thread could block. The API is actually similar to a channel that sends updates to the audio thread.

The write struct has access to the shared data structure. It stores the operations that were applied to one value in order to be able to apply them to the other value when a switch is done. It provides a method that tries to get a lock guard.

 1/// Should be implemented on structs that want to be shared with this library
 2pub trait Absorb<O> {
 3    /// has to be deterministic. Operations will be applied in the same order to both buffers
 4    fn absorb(&mut self, operation: O);
 5}
 6
 7pub struct Writer<T, O> {
 8    shared: NonNull<Shared<T>>,
 9    // sets which buffer the next write is applied to
10    // write_ptr doesn't need to be Atomics as it only changes, when the Writer itself swaps
11    write_ptr: Ptr,
12    // Operations to sync the two copies of T
13    op_buffer: VecDeque<O>,
14    // necessary to prevent `std::mem::forget` from causing UB
15    locked: bool,
16    // needed for drop_check
17    _own: PhantomData<Shared<T>>,
18}

If a write guard was acquired operations can be applied to the data.

The state atomic stores if there is both reader and writer or if only one exists. It also stores if and which value is being read, as well as the value that should be read next. I need have one compare exchange loop, but because only two threads have access to this atomic the contention should be very low.

rt-write-lock #

This library allows realtime writes, while the reading could block. If the reader is fast enough it works similar to a channel, but if the reader is slower it can replace already sent data by new data. This allows the reader to get updates quicker. The API isn't completely chanel-like, but actually provides a &mut T so allocations can be reused. This API is very similar to triple-buffer. The difference is that i only need two copies of the data, instead of three. To achieve this the reader could block, while in triple-buffer both sides are realtime safe.

Here the state is similar, but it can also disallow a read. This is done when the reader is behind and currently reading to force it to catch up the next time it renews its reaguard.

std::mem::forget #

Both libraries use LockGuards with a lifetime and a drop implementation as it is standard in rust. If such a guard is forgotten a regular mutex can't be locked anymore. Here it doesn't quite work that way. When locking the lockfree side (reader in simple-left-right and writer in rt-write-lock) the state doesn't need to be changed. So a later lock doesn't necessarily even notice this. Instead of trying to think of ways to make my atomic state changes more robust, i just decided to track this locally in each reader and writer and panic if a second read or write is attempted.

Conclusion #

I hope this was interesting to you! If you have any questions on tradeoffs or usecases you can either ask in one of the repos or write me on bsky. Writing simple-left-right was my first time really using unsafe rust and atomics. Learning from other libraries and the std was very helpful, so i hope someone can take something from the design of these libraries.