Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@RogerHYang
Copy link
Contributor

@RogerHYang RogerHYang commented Oct 22, 2025

Note

Updates tutorials to new Phoenix tracing/evals APIs, improves RAG and agent eval examples, adds an experiments-with-repetitions tutorial, and bumps MCP dependency.

  • Docs/Tutorials:
    • Migrate notebooks to new Phoenix tracing/evals APIs: register(auto_instrument), environment-based PHOENIX_COLLECTOR_ENDPOINT/PHOENIX_API_KEY, and px_client.spans.log_span_annotations_dataframe (replacing older annotations APIs).
    • Refactor evals to new LLM, create_classifier, and evaluate_dataframe/async_evaluate_dataframe patterns; enhance prompts and trace/span-level evaluations (RAG relevancy, hallucination, QA correctness).
    • Modernize OpenAI/agents examples and Pydantic evals; update UI/text; restructure LLM Ops tutorial with end-to-end tracing, span/trace evals, and experiments.
    • Add new tutorial: running experiments with repetitions.
  • Integrations:
    • Update logging calls in LangChain, Vertex AI, Weaviate, Bedrock, Cleanlab, Amazon Bedrock agents, MCP tracing examples.
  • Dependencies:
    • Bump mcp to 1.10.0 in MCP tutorial requirements.

Written by Cursor Bugbot for commit c1b2356. This will update automatically on new commits. Configure here.

mikeldking and others added 30 commits October 2, 2025 02:39
chore: sync Docs to main
* refactor: migrate to new span methods

* cleanup
* refactor multiselector data-fetching to use useQueryLoader

* refactor SelectedCompareExperiments data loading

* refactor compare page contents to use preloaded queries

* address PR comments

* add loading state
* feat: Add PromptMenu component and replace PromptComboBox usages with it
)

* feat(phoenix-client): session annotation functions

* changeset

* cleanup
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* wip

* wip

* wip

* bulk operation

* wip

* wip

* wip

* wip

* Update dataset_label_mutations.py

Co-authored-by: Xander Song <[email protected]>

* Update dataset_label_mutations.py

Co-authored-by: Xander Song <[email protected]>

* Update src/phoenix/server/api/mutations/dataset_label_mutations.py

Co-authored-by: Xander Song <[email protected]>

* wip

* wip

* wip

* wip

* wip

* wip

* Update src/phoenix/server/api/mutations/dataset_label_mutations.py

Co-authored-by: Xander Song <[email protected]>

* wip

---------

Co-authored-by: Xander Song <[email protected]>
* feat: Improve binding ergonomics

* Remove unnecessary cast
* Create toastQueue singleton for notifications

* Defer rendering of ToastRegion until after app root instantiation
* Remove retries for OpenAI models

* Use LLM for bedrock, vertex

* Ruff 🐶
* update classification evaluator doc string"

* update docstrings
* refactor: use autocomplete for annotation config

* cleanup

* cleanup

* cleanup
* feat(evals): add regex evaluator

* refactor(matches_regex): simplify score evaluation using bool(matches)

* Update packages/phoenix-evals/src/phoenix/evals/metrics/matches_regex.py

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* minor fixes

* implement explicit async evaluate to call sync regex directly without thread overhead

* fixing the docstring

---------

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
* Upgrade internationalized

* Fix type errors

* Upgrade react aria utils
…age (#9621)

* refactor example slideover pagination to support duplicate examples + add pagination to list page slideover

* add repetition number to list page table + slideover

* recalculate column sizes when column visibility changes

* load more when on the last row

* fix potential array access errors

* remove old relay types
* wip

* Update app/src/pages/datasets/DatasetsTable.tsx

Co-authored-by: Tony Powell <[email protected]>

* feature flag

---------

Co-authored-by: Tony Powell <[email protected]>
* wip

* prettier

* feature flag

* resolve comments

* new toast

* prettier

* wip

* use directive for deleting edges in dataset connection rather than updater

* Update app/src/pages/settings/datasets/DatasetLabelsTable.tsx

Co-authored-by: Xander Song <[email protected]>

* Update app/src/components/dataset/NewDatasetLabelDialog.tsx

Co-authored-by: Xander Song <[email protected]>

* Update app/src/pages/settings/datasets/SettingsDatasetsPage.tsx

Co-authored-by: Xander Song <[email protected]>

---------

Co-authored-by: Alexander Song <[email protected]>
* update experiment run output card scroll behavior

* ensure annotation alignment in slideover run cards

* rework annotation section layout

* refactor

* add stories

* fix annotations scroll

* fix placeholder item

* revert scrolling within cards

* refactor

* respsect bottom padding in runs panel

* address feedback

* rename stories
yfrigui2 and others added 15 commits October 20, 2025 11:18
…mpare slideover (#9872)

* add annotation selector + selected annotation values to slideover sidebar

* sort experiment output cards by selected annotation

* handle sorting across experiments in repetition-naive mode

* update stories

* handle long experiment names & annotation values

* missing run styling

* add button to toggle sort direction

* add toggle-all checkbox

* layout tweaks

* add sort by repetition

* update Select styles

* update sort button hover behavior

* refactor to use context

* styling

* remove comment

* use Menu instead of Select to enable deselect

* remove sort by repetition number

* secondary sort by repetition number

* handle menu overflow

* update sort by annotation menu placeholder
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 6.3.6 to 6.4.1.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/[email protected]/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 6.4.1
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 4.5.14 to 5.4.21.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v5.4.21/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v5.4.21/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 5.4.21
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* wip

* wip

* wip

* Update app/src/pages/dataset/datasetLoader.ts

Co-authored-by: Xander Song <[email protected]>

* use fetch query

* Update app/src/pages/dataset/DatasetPage.tsx

Co-authored-by: Tony Powell <[email protected]>

---------

Co-authored-by: Xander Song <[email protected]>
Co-authored-by: Tony Powell <[email protected]>
* enable collapsing repetitions in sidebar

* consistent card heights

* add ghost annotations to output cards with missing annotations
…#9953)

* filter annotation summaries in slideover to only those included in selected experiment runs

* handle annotation missing from example on list page

* add comment
…le (#9928)

* playground slideover

* remove comment

* fix

* clean

* clean

* clean

* clean

* fix

* use helper

* clean
Bumps [hono](https://github.com/honojs/hono) from 4.9.7 to 4.10.2.
- [Release notes](https://github.com/honojs/hono/releases)
- [Commits](honojs/hono@v4.9.7...v4.10.2)

---
updated-dependencies:
- dependency-name: hono
  dependency-version: 4.10.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* return scores/details as json in dataframe evals; only deep copy input mapping when binding to avoid issues copying LLM

* evaluate dataframe returns json score/details columns; annotation utility updated to match

* move LLM to top level import

* update doc strings with LLM as top level import

* move LLM to top level import

* update docstring

* set default progress bar for dataframe evals

* update docstrings

* update tests

* Update packages/phoenix-evals/src/phoenix/evals/evaluators.py

Co-authored-by: Xander Song <[email protected]>

* update relevance metric docstring

* add json utility function

* small fix

---------

Co-authored-by: Xander Song <[email protected]>
* refetch projects list after creation

* refetch API keys after creation
@RogerHYang RogerHYang requested review from a team as code owners October 22, 2025 22:52
@github-project-automation github-project-automation bot moved this to 📘 Todo in phoenix Oct 22, 2025
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Oct 22, 2025
)

Bumps [langchain-text-splitters](https://github.com/langchain-ai/langchain) from 0.3.5 to 0.3.9.
- [Release notes](https://github.com/langchain-ai/langchain/releases)
- [Commits](langchain-ai/langchain@langchain-text-splitters==0.3.5...langchain-text-splitters==0.3.9)

---
updated-dependencies:
- dependency-name: langchain-text-splitters
  dependency-version: 0.3.9
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#9891)

Bumps [llama-index](https://github.com/run-llama/llama_index) from 0.12.41 to 0.13.0.
- [Release notes](https://github.com/run-llama/llama_index/releases)
- [Changelog](https://github.com/run-llama/llama_index/blob/main/CHANGELOG.md)
- [Commits](run-llama/llama_index@v0.12.41...v0.13.0)

---
updated-dependencies:
- dependency-name: llama-index
  dependency-version: 0.13.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 5.4.20 to 5.4.21.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v5.4.21/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v5.4.21/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 5.4.21
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@axiomofjoy axiomofjoy merged commit 5a14ada into docs Oct 22, 2025
52 checks passed
@github-project-automation github-project-automation bot moved this from 📘 Todo to ✅ Done in phoenix Oct 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.