Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Pre-compile data as binary blobs #23

@overlookmotel

Description

@overlookmotel

Just to expand on discussion we had earlier...

In my opinion, it is never going to be possible to make this crate as fast to compile as we want it to be, no matter what we throw at it, without taking a different approach. It's just tons of data, so using a codegen to generate tons of code and then asking rust to parse and compile it all is always going to take a long time.

A possible solution could be to pre-compile it as binary data. Something like this:

At build time:

  • Convert JSON to HashMap/Vec/etc data structures.
  • Serialize those data structures with rkyv.
  • Save the serialized binary data to disk.

At runtime:

  • Load the binary data from disk.
  • Deserialize (which with rkyv is zero cost - just casting a pointer).
  • Bonus points: Don't even load the data - just mmap it from disk.

Background: rkyv's "special sauce" is relative pointers: https://rkyv.org/architecture/relative-pointers.html

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions