Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[FEATURE] Add Taxonomy Explorer page and interactive visualization - #504

Draft
Warren Ehrenfried (warrenehrenfried) wants to merge 23 commits into
mainfrom
warrenehrenfried-taxonomy-browser-update
Draft

[FEATURE] Add Taxonomy Explorer page and interactive visualization#504
Warren Ehrenfried (warrenehrenfried) wants to merge 23 commits into
mainfrom
warrenehrenfried-taxonomy-browser-update

Conversation

@warrenehrenfried

Copy link
Copy Markdown
Contributor

Pull Request

Closes #492

Taxonomy Explorer

image

Rebuilds the Places taxonomy browser on the current canonical taxonomy and adds an interactive sunburst view of the full hierarchy.

The original idea and the first visualizations are from Ron Rice. How could we forget his presentation at the member summit in Florence! This is that concept brought into the docs site and regenerated from current data and with as much interactivity and features that I felt appropriate.

Intended to land for the September 2026 release, when the legacy categories property is retired and the taxonomy becomes the public reference for how Overture describes places.

What's here

  • Generated from the canonical source. scripts/build-taxonomy.mjs reads the taxonomy and basic-category files from the places pipeline and emits versioned artifacts. basic_category is derived the way the pipeline derives it, so the docs can't drift from the published data.
  • A sunburst view, alongside the existing tree. Colour follows the top-level groups and re-splits as you drill in; scroll to zoom, drag to pan.
  • Place counts from the release itself, queried from 2026-08-19.0 (70,345,858 places). Categories show both their own count and their roll-up.
  • Downloads — taxonomy, basic categories, and the legacy crosswalk as CSV
  • Fixes to what was there: opens on the newest release rather than the oldest, release labels carry their year, and the March entry's version tags were showing December's.
  • The taxonomy guide is now reachable from the sidebar, it was linked from every release note but wasn't navigable (bug)

Renamed from Taxonomy Browser to Taxonomy Explorer, with a redirect from the old URL.

Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

Super-linter summary

Language Validation result
JAVASCRIPT_ES Pass ✅
MARKDOWN Pass ✅
NATURAL_LANGUAGE Fail ❌
SQLFLUFF Pass ✅

Super-linter detected linting errors

For more information, see the GitHub Actions workflow run

Powered by Super-linter

NATURAL_LANGUAGE

/github/workspace/docs/guides/places/_README.md
  22:63  ✓ error  Incorrect term: “repo”, use “repository” instead  terminology
  34:60  ✓ error  Incorrect term: “repo”, use “repository” instead  terminology

✖ 2 problems (2 errors, 0 warnings, 0 infos)
✓ 2 fixable problems.
Try to run: $ textlint --fix [file]

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

🗺️ Overture Docs preview is live!

🌍 Preview site https://staging.overturemaps.org/docs/pr/504/index.html
🗂️ Schema ref main@39e4170
🕐 Updated Sep 04, 2026 20:37 UTC
📝 Commit 09e5dfe

Note

♻️ This preview updates automatically with each push to this PR.

@FlorianNachtigall

Copy link
Copy Markdown
Member

Very nice visualization! 🙌

I was wondering: since you’ve already included the place counts and made it possible to resize the sunburst chart based on them, could we also show the corresponding percentage shares?

For example:

  • In the tooltip, perhaps next to 2 categories beneath
  • In the panel on the right, after the total counts, e.g. Places at this category: 9,268 (X.X%) and Including subcategories: 463,997 (X.X%)

@FlorianNachtigall

Copy link
Copy Markdown
Member

By coincidence I noted that somehow the size of each slice of the sunburst chart is not proportional to the respective place count (when selecting "Size by place count").

For example, Cultural and Historic is the smallest slice with ~3.1M places whereas the second smallest slice Arts and Entertainment has ~1.5M places. I tried to figure out if it just sorts by a different count, but without success.

image

@warrenehrenfried

Warren Ehrenfried (warrenehrenfried) commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

Florian Nachtigall (@FlorianNachtigall) to respond to your first comment absolutely, those are great suggestions will do. To respond to your second comment I noticed this earlier this morning! I fixed this locally as there was a bug which was causing it to ignore the parent categories count and only looking at its subcategories. Good catch!

Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
Signed-off-by: Warren Ehrenfried <[email protected]>
@ericgodwin

Copy link
Copy Markdown
Contributor

I'm not sure the docs repo is the right place for this. I was chatting with John McCall (@lowlydba) and perhaps it could be a public labs.overture page that then gets linked to by the docs?

@FlorianNachtigall

Copy link
Copy Markdown
Member

Fantastic!

@warrenehrenfried

Copy link
Copy Markdown
Contributor Author

Eric Godwin (@ericgodwin) yes Dana Bauer (@danabauer) mentioned this to me, happy for this to live elsewhere if that makes more sense.

@lowlydba John McCall (lowlydba) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on the viz, and the fixes around it (redirect, sidebar exposing taxonomy.mdx, release ordering, the taxonomy.mdx rewrite) are solid on their own. Agree with Eric Godwin (@ericgodwin) that the explorer itself probably wants a different home. A few architectural points behind that, then some smaller things.

question: what does "canonical" mean here in practice? scripts/build-taxonomy.mjs reads a local checkout of tf-data-platform plus a hand-exported Athena counts CSV, and the output is committed. That's a manually synced snapshot with no provenance (source commit, checksum), and every release needs someone with pipeline access to re-run it and open a several-thousand-line PR. Who owns that per release, and where's the checklist?

suggestion: publish the taxonomy artifacts upstream with the release (S3 next to theme=places, or a public taxonomy repo with tagged releases), then have both the explorer and the download links point at those URLs. taxonomy.mdx already says the taxonomy is "published as a set of files with each release"; right now that's only true because the files were copied into static/. Once there's a real published source, dataUrl fetching makes sense and the docs repo stops being the distribution channel.

suggestion (non-blocking): the source of truth is already overture_taxonomy.json. I'd publish JSON (or Parquet for parity with the release) as the primary artifact and treat CSV as a derivative. A flat CSV with a " > "-joined path column is a lossy form for a 2,302-node tree. Related: taxonomy.json carries the counts but isn't downloadable, while the CSVs without counts are.

issue: _README.md and the script header point contributors at ../tf-data-platform/..., which is private. That's an internal runbook in a public repo and unusable by anyone outside.

issue: docs/guides/places/csv/2026-08-19-counts.csv (1,988 lines) isn't referenced anywhere on the branch. Also has the Athena _col0 header leaked in.

issue: taxonomy.mdx "at a glance" hardcodes 2,302 / 298 / 70,345,858 in prose while the generator already emits stats. Those will drift next release.

issue: taxonomy.mdx drops the "Provider Category Mappings" section (July 2026 BrightQuery/NAICS content) with no replacement. Seems outside the stated scope; was that intentional?

thought: on scope, this adds a 686-line d3 component, two runtime deps, and inlines ~2 MB of legacy CSV into one page via raw-loader. A standalone labs.overturemaps.org page that fetches from published URLs, linked or embedded from docs, keeps the docs build lean and lets the explorer iterate on its own cadence.

Suggested split:

  • merge now: redirect, sidebar, ordering fix, taxonomy.mdx prose (minus hardcoded stats and downloads table, provider-mappings section restored)
  • upstream: get taxonomy artifacts published with the release
  • labs: explorer + viz fetching those URLs, docs links to it

Happy to help with the upstream issue or the labs scaffold.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update taxonomy browser to reference canonical taxonomy and clear versioning for previous builds

4 participants