To expand on both @ppannuto's and @alevy's answer, there's a couple parts to the tock-registers implementation that can work (somewhat) independently. Some components are unsound, other are not, and some components assume MMIO, while others don't. Here's a (verbose) summary of these components:

• There is the register_bitfields! macro, which allows you to describe the layout of individual registers (independent of whether they are memory mapped or not):

  tock/chips/nrf52/src/uart.rs

  Lines 81 to 85 in 9af07b3

  	register_bitfields! [u32,
  	/// Start task
  	Task [
  	ENABLE OFFSET(0) NUMBITS(1)
  	],

  This macro then generates a bunch of modules and types that are used by the other interfaces of this library to describe operations on registers.

• There are the register types themselves, like ReadWrite, ReadOnly, or Aliased. These are designed to be instantiated as references over the actual underlying MMIO addresses.

  We also provide the register_structs! macro to generate a Rust struct that combines multiple of these reference types into a single Rust type, with the inner fields placed at offsets corresponding to each underlying MMIO register. Again, to use this struct, you will create a reference to it with an address identical to the base address of your MMIO peripheral:
  

  tock/chips/esp32-c3/src/rng.rs

  Lines 16 to 21 in 9af07b3

  	register_structs! {
  	pub RngRegister {
  	(0x0 => data: ReadOnly<u32>),
  	(0x4 => @END),
  	}
  	}

  This is also where the unsoundness lies within this library. Although these types use an UnsafeCell internally (and thus allow interior mutability), we must hold an immutable reference to use them. And Rust is always allowed to dereference any valid reference in scope. While this is fine for regular memory, in the case of MMIO, such reads can have potentially dangerous side effects.

• And then, there are the traits that the define the interface of readable, writeable, and readable + writeable registers respectively.

  These traits assume very little about the underlying registers that implement them. For instance, you can create custom register types implementing these traits that don't alias the undering MMIO register addresses, and instead internally store an address pointer to the actual register value. This would avoid the soundness issues that our included registers have, but would not be compatible with the register_structs! macro. They could also internally synchronize access to the underlying peripheral, or talk to port-mapped I/O instead.

  All the while, your custom types using these traits can still benefit from the rich interfaces that tock-registers provides for manipulating register contents, and the bitfield definitions of the register_bitfields! macro.

In order to satisfy Send and Sync bounds, I have to reason about concurrent writes, but the fact that the Writeable::set can be called even through a read handle on a RwLock makes this difficult. I'm wondering if I'm reasoning about safety wrong here. Is there some special property of mmap or UIO or write_volatile, or registers in general that make it safe to write through a shared reference?

This is a great question. In short, the reason why this is not an issue in practice is that the included tock_registers register types don't implement the Sync trait. This will make it impossible to share them across threads. And also, because they are designed to never be instantiated as owned types, but rather behind references that alias the underlying MMIO addresses, they can also not be moved between threads in practice. The RwLock doesn't change that, because of the constraints it places on its own implementation of the Sync trait (where T: Send + Sync). At least in this aspect, I do believe that the types we're providing are sound. 😄

You can, however, build your own custom types that are either owned (instead of designed to live behind references) and implement Send, and then wrap them in a Mutex (RwLock won't work because it needs Sync), or build register types that synchronize accesses to the underlying registers internally. I tried to illustrate this with the below example. Hope this clarifies things, let me know if there's any more questions.

use std::cell::Cell;
use std::sync::{Mutex, RwLock};
use tock_registers::interfaces::Writeable;
use tock_registers::registers::WriteOnly;

fn main() {
    // Pretend that a `Box<usize>` allocation is an MMIO register, and
    // transmute the register pointer into a `WriteOnly` register:
    let register_ptr: *mut usize = Box::leak(Box::new(0_usize)) as *mut usize;
    let rw_reg: &WriteOnly<usize> = unsafe { std::mem::transmute(register_ptr) };

    // We can write even though we only have a readable reference!  This is fine
    // though, as an `&WriteOnly` uses interior mutability:
    let rwlocked_rw_reg = RwLock::new(rw_reg);
    rwlocked_rw_reg.read().unwrap().set(1);

    // However, this fails to compile. The `WriteOnly` type does not implement
    // `Sync` to permit concurrent accesses, and so neither does `RwLock`:
    std::thread::spawn(|| { // doesn't compile!!!
	rwlocked_rw_reg.read().unwrap().set(2);
    });

    // Unfortunately, a Mutex doesn't help either, because you'd never have an
    // _owned_ copy of your register. The only thing that a Mutex will add
    // `Sync` to is the immutable reference stored within it, which itself
    // points to a type that is not `Sync` and uses interior mutability:
    let mutex_rw_reg = Mutex::new(rw_reg);
    mutex_rw_reg.lock().unwrap().set(1);
    std::thread::spawn(|| { // doesn't compile!!!
	mutex_rw_reg.lock().unwrap().set(2);
    });

    // What does work is if we create our own, owned register abstraction and
    // have it either internally synchronize accesses, ...
    struct InternallySyncReg(Mutex<usize>);
    impl Writeable for InternallySyncReg {
        type T = usize;
        type R = ();

        fn set(&self, value: usize) {
            *self.0.lock().unwrap() = value;
        }
    }
    let int_sync_reg = InternallySyncReg(Mutex::new(0));
    int_sync_reg.set(0);
    std::thread::spawn(move || {
        int_sync_reg.set(1);
    });

    // ... or use one that has internal mutability but is _owned_ (instead of
    // being behind a shared reference), and wrap that in a Mutex (RwLock won't
    // work, for the same reason as the first example):
    struct NonSyncReg(Cell<usize>);
    impl Writeable for NonSyncReg {
        type T = usize;
        type R = ();

        fn set(&self, value: usize) {
            self.0.set(value);
        }
    }
    let mutex_non_sync_reg = Mutex::new(NonSyncReg(Cell::new(0)));
    mutex_non_sync_reg.lock().unwrap().set(1);
    std::thread::spawn(move || {
        mutex_non_sync_reg.lock().unwrap().set(1);
    });
}

Okay, I've done a lot of thinking and experimenting with UIO devices, their registers, mmapping them, and thread safety, and I'm now confident that I've made a safe and sound implementation! Thanks all for the detailed information!

Closing the discussion for now since this seems to be resolved.