PCIem is a framework that creates virtual PCIe devices in the Linux kernel by leveraging a few novel techniques to populate synthetic cards as legitimate PCI devices to the host OS.

To brief what PCIem is: a framework for (Albeit not limited to) developing and testing PCIe device drivers without requiring actual hardware on the host.

PCIem and libfvio-user are two different solutions for different needs, there may be confusion when comparing both so herein the differences (See figure 1 for more).

The main one is that libvfio-user usually relies on a client (That implements the vfio-user protocol), usually QEMU (Through KVM, using VM exits) to expose the emulated PCIe device to the guest. You write your vfio server (With a callback mechanism usually) which then interacts with the client.

What PCIem does instead is, expose the device directly on the host; no KVM, no guests, no virtual machines, nothing. The device appears on the host's PCIe bus as if it was physically connected.

graph LR
    subgraph Kernel ["Host Linux Kernel"]
        direction TB

        RealDriver["Real PCIe Driver"]

        subgraph Framework ["PCIem Framework"]
            direction TB
            Config["PCI Config Space"]
            BARs["BARs"]
            IRQ["Interrupts"]
            DMA["DMA / IOMMU"]
        end

    end

    Interface(("/dev/pciem"))

    subgraph User ["Linux Userspace"]
        direction TB
        Shim["Device Emulation"]
    end

    Framework <==> Interface
    Interface <==> Shim

• BAR creation: Programmatically create and monitor BAR regions.
• Watchpoints: Access detection using CPU watchpoints to avoid polling for accesses.
• Interrupts support: Legacy/MSI/MSI-X interrupt injection.
• PCI capability framework: PCI capabilities system (Linked-list underneath).
• DMA system: IOMMU-aware DMA accesses from/to userspace device.
  • P2P support (Preliminar!): Peer-to-peer DMA between devices with whitelist-based access control.
• Userspace: Implement your PCIe devices entirely in userspace.

The following example basically shows an NVME controller with 1GB of storage attached to. User can freely format, mount, create and remove files from the memory.

The card is programmed entirely in QEMU (State machine for the card, basically), which does all the userspace initialization and command handling from the real driver running in the host.

Can run software-rendered DOOM (Submits finished frames with DMA to the card which QEMU displays) and also simple OpenGL 1.X games (On the screenshots, tyr-glquake and xash3d; thanks to a custom OpenGL state machine implemented entirely in QEMU that software-renders the command lists and updates the internal state accordingly).

Dual MIT/GPLv2 (pciem_framework.c and protopciem_driver.c due to using GPL symbols).

MIT (Rest).

• Blog post: https://cakehonolulu.github.io/introducing-pciem/
• Documentation: https://cakehonolulu.github.io/docs/pciem/
• Hackernews post: https://news.ycombinator.com/item?id=46689065
• PCI Express specification: https://pcisig.com/specifications