omop2meds

Rust-native OMOP CDM → MEDS converter.

Converts a local DuckDB OMOP database into a MEDS Parquet dataset with metadata, subject splits, and a conversion report. Supports direct DuckDB streaming or a partitioned local cache for parallel conversion on large cohorts.

Quick start

cargo build -p omop2meds --release

# Preflight
omop2meds inspect-duckdb --duckdb data/omop.duckdb --event-schema cdm --vocab-schema vocab

# Encrypted DuckDB files
omop2meds inspect-duckdb \
  --duckdb data/encrypted.duckdb \
  --duckdb-encryption-key-env OMOP2MEDS_DUCKDB_ENCRYPTION_KEY \
  --duckdb-attach-alias encrypted \
  --event-schema cdm \
  --vocab-schema vocab

# Convert (cached path — recommended for large databases)
omop2meds convert-duckdb \
  --duckdb data/omop.duckdb \
  --event-schema cdm \
  --vocab-schema vocab \
  --dataset-name my_cohort \
  --dataset-version 1.0 \
  --out output/MEDS_cohort \
  --cache-dir output/omop_cache \
  --cache-partitions 512 \
  --reuse-concept-index \
  --split-strategy temporal-last-visit \
  --temporal-tuning-start 2024-01-01 \
  --temporal-held-out-start 2025-01-01 \
  --threads 32 \
  --overwrite

omop2meds validate --meds output/MEDS_cohort --strict

For a campaign with an already frozen person split, stream the source DuckDB read-only and provide the authoritative Parquet manifest directly. The manifest must contain exactly one row per OMOP person, with person_id (or subject_id) and split; val/validation map to MEDS tuning, and test maps to held_out. Missing, extra, duplicate, null, or unknown entries fail closed.

omop2meds convert-duckdb \
  --duckdb /data/omop.duckdb \
  --event-schema main \
  --vocab-schema main \
  --table-prefix cdm_ \
  --subject-splits /run/locked_person_split.parquet \
  --dataset-name locked_campaign \
  --dataset-version 1 \
  --out /data/MEDS_locked_campaign \
  --threads 32

The prefixed-table direct-streaming path intentionally rejects --cache-dir until cache manifests include the same prefix identity.

For a two-step workflow (build cache, then convert), see docs/CACHE.md.

Output layout

MEDS_cohort/
  data/{train,tuning,held_out}/part-*.parquet
  metadata/{dataset.json,codes.parquet,subject_splits.parquet}
  conversion_report.json

MEDS version defaults to 0.3.3. Compression defaults to zstd.

Examples

Config	Use case
examples/config.duckdb-cache.toml	Production-style cached conversion
examples/config.duckdb.toml	Minimal cached demo
examples/config.synpuf.toml	CMS SynPUF (main schema, temporal split)

CLI commands

Command	Purpose
convert-duckdb	OMOP DuckDB → MEDS
cache-duckdb	Materialize OMOP tables to local cache only
inspect-duckdb	List required tables and row counts
inspect-cache	Inspect cached Parquet tables
validate	Validate MEDS output (--strict exits non-zero on failure)

Documentation

Doc	Purpose
docs/README.md	Documentation index
docs/ARCHITECTURE.md	Pipeline design and data flow
docs/CACHE.md	Cache build and reuse
docs/SCOPE.md	Supported and excluded features
docs/DEVELOPMENT.md	Local dev and smoke tests
docs/TROUBLESHOOTING.md	Common errors and fixes
docs/BENCHMARK.md	DuckDB vs cache benchmark script
CONTRIBUTING.md	CI checks and config precedence
COMPATIBILITY.md	Dependency pins and compatibility notes