[FEATURE] Add Taxonomy Explorer page and interactive visualization - #504
[FEATURE] Add Taxonomy Explorer page and interactive visualization#504Warren Ehrenfried (warrenehrenfried) wants to merge 23 commits into
Conversation
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter NATURAL_LANGUAGE |
🗺️ Overture Docs preview is live!
Note ♻️ This preview updates automatically with each push to this PR. |
Signed-off-by: Warren Ehrenfried <[email protected]>
|
Very nice visualization! 🙌 I was wondering: since you’ve already included the place counts and made it possible to resize the sunburst chart based on them, could we also show the corresponding percentage shares? For example:
|
|
Florian Nachtigall (@FlorianNachtigall) to respond to your first comment absolutely, those are great suggestions will do. To respond to your second comment I noticed this earlier this morning! I fixed this locally as there was a bug which was causing it to ignore the parent categories count and only looking at its subcategories. Good catch! |
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
|
I'm not sure the docs repo is the right place for this. I was chatting with John McCall (@lowlydba) and perhaps it could be a public labs.overture page that then gets linked to by the docs? |
|
Fantastic! |
|
Eric Godwin (@ericgodwin) yes Dana Bauer (@danabauer) mentioned this to me, happy for this to live elsewhere if that makes more sense. |
John McCall (lowlydba)
left a comment
There was a problem hiding this comment.
Nice work on the viz, and the fixes around it (redirect, sidebar exposing taxonomy.mdx, release ordering, the taxonomy.mdx rewrite) are solid on their own. Agree with Eric Godwin (@ericgodwin) that the explorer itself probably wants a different home. A few architectural points behind that, then some smaller things.
question: what does "canonical" mean here in practice? scripts/build-taxonomy.mjs reads a local checkout of tf-data-platform plus a hand-exported Athena counts CSV, and the output is committed. That's a manually synced snapshot with no provenance (source commit, checksum), and every release needs someone with pipeline access to re-run it and open a several-thousand-line PR. Who owns that per release, and where's the checklist?
suggestion: publish the taxonomy artifacts upstream with the release (S3 next to theme=places, or a public taxonomy repo with tagged releases), then have both the explorer and the download links point at those URLs. taxonomy.mdx already says the taxonomy is "published as a set of files with each release"; right now that's only true because the files were copied into static/. Once there's a real published source, dataUrl fetching makes sense and the docs repo stops being the distribution channel.
suggestion (non-blocking): the source of truth is already overture_taxonomy.json. I'd publish JSON (or Parquet for parity with the release) as the primary artifact and treat CSV as a derivative. A flat CSV with a " > "-joined path column is a lossy form for a 2,302-node tree. Related: taxonomy.json carries the counts but isn't downloadable, while the CSVs without counts are.
issue: _README.md and the script header point contributors at ../tf-data-platform/..., which is private. That's an internal runbook in a public repo and unusable by anyone outside.
issue: docs/guides/places/csv/2026-08-19-counts.csv (1,988 lines) isn't referenced anywhere on the branch. Also has the Athena _col0 header leaked in.
issue: taxonomy.mdx "at a glance" hardcodes 2,302 / 298 / 70,345,858 in prose while the generator already emits stats. Those will drift next release.
issue: taxonomy.mdx drops the "Provider Category Mappings" section (July 2026 BrightQuery/NAICS content) with no replacement. Seems outside the stated scope; was that intentional?
thought: on scope, this adds a 686-line d3 component, two runtime deps, and inlines ~2 MB of legacy CSV into one page via raw-loader. A standalone labs.overturemaps.org page that fetches from published URLs, linked or embedded from docs, keeps the docs build lean and lets the explorer iterate on its own cadence.
Suggested split:
- merge now: redirect, sidebar, ordering fix,
taxonomy.mdxprose (minus hardcoded stats and downloads table, provider-mappings section restored) - upstream: get taxonomy artifacts published with the release
- labs: explorer + viz fetching those URLs, docs links to it
Happy to help with the upstream issue or the labs scaffold.

Pull Request
Closes #492
Taxonomy Explorer
Rebuilds the Places taxonomy browser on the current canonical taxonomy and adds an interactive sunburst view of the full hierarchy.
The original idea and the first visualizations are from Ron Rice. How could we forget his presentation at the member summit in Florence! This is that concept brought into the docs site and regenerated from current data and with as much interactivity and features that I felt appropriate.
Intended to land for the September 2026 release, when the legacy categories property is retired and the taxonomy becomes the public reference for how Overture describes places.
What's here
Renamed from Taxonomy Browser to Taxonomy Explorer, with a redirect from the old URL.