Add camomile-embedded package with compile-time data embedding#12
Open
toots wants to merge 1 commit into
Open
Conversation
3 tasks
75384f2 to
0c51ee0
Compare
Adds a new camomile-embedded opam package that embeds all Unicode data (database, charmaps, locales, mappings) directly into the compiled library using ocaml-crunch, eliminating any runtime filesystem dependency on installed data files. Key design points: - Database.Make functor with FileReader module type replaces the previous approach of a global mutable ref. The embedded reader is strictly contained in camomile_embedded.ml with no global state. - val get is added to Config.Type. Internal functors (Charmap.Configure, Unimap.Make, Unidata.Make) shadow Database with Database.Make(Config), picking up whatever reader the config provides. - The reader signature is (unit -> 'a) -> 'b so both the filesystem path (input_value) and the embedded path (Marshal.from_string) go through the same reader function, preserving type safety across all call sites including read_localedata which returns a different type. - The crunch rule passes a single root directory so keys include the subdirectory prefix (database/foo.mar, charmaps/bar.mar, etc.), matching what Filename.concat dir (fname ^ "." ^ suffix) produces in the embedded Config.get. locales/*.mar is also included. - The embedded test is scoped to the camomile-embedded package so it does not run under a plain camomile build.
Alizter
reviewed
Apr 19, 2026
| (write-file | ||
| %{target} | ||
| "\nlet prefix = \"%{env:CAMOMILE_PREFIX=/usr}/share/camomile\"\nlet datadir = Filename.concat prefix \"database\"\nlet localedir = Filename.concat prefix \"locales\"\nlet charmapdir = Filename.concat prefix \"charmaps\"\nlet unimapdir = Filename.concat prefix \"mappings\"\n"))) | ||
| "\nlet prefix = \"%{env:CAMOMILE_PREFIX=/usr}/share/camomile\"\nlet datadir = Filename.concat prefix \"database\"\nlet localedir = Filename.concat prefix \"locales\"\nlet charmapdir = Filename.concat prefix \"charmaps\"\nlet unimapdir = Filename.concat prefix \"mappings\"\nlet get = CamomileLib.Database.Filesystem.get\n"))) |
There was a problem hiding this comment.
Might be easier to write a small executable to do this rather than inline sources here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a new
camomile-embeddedopam package. It provides the same API ascamomilebut with all Unicode data files (database, charmaps, locales, mappings) embedded directly into the library at compile time usingocaml-crunch. Programs linking against it have no runtime filesystem dependency on data files.Changes
New
camomile-embeddedpackage (camomile-embedded.opam,src/embedded/,dune-project)A new library built from
camomile_embedded.ml+ a crunch-generatedembedded_data.ml. It defines aConfigthat reads data from the in-memory crunch table instead of the filesystem, then instantiatesCamomile.Make(Config).Database.Makefunctor (src/internal/database.ml,src/internal/database.mli)Replaces the single hardcoded filesystem
readwith aFileReadermodule type and aMakefunctor.Database.Filesystem(the default) reads from disk as before. The embedded config supplies its own reader backed by crunch. The reader signature is(unit -> 'a) -> 'bso callers that return a different type from their intermediate representation (e.g.read_localedata) continue to work correctly.val getinConfig.Type(src/config.mli,src/configBase.ml,src/configImpl.dune.ml,src/toolslib/camomileconfig.ml)Internal functors (
Charmap.Configure,Unimap.Make,Unidata.Make) instantiateDatabase.Make(Config)so they pick up whichever reader the config provides.Crunch rule (
src/dune)ocaml-crunchis invoked on a single root directory (the rule sandbox) rather than on each subdirectory separately, so keys include the subdirectory prefix (database/foo.mar,charmaps/bar.mar, etc.), matching the keys produced byConfig.get.locales/*.maris included alongside the other three directories.Test (
test/embedded/)Exercises UTF-8, general category, NFC normalisation, and charmap round-trip entirely from embedded data, scoped to the
camomile-embeddedpackage.