test: add MCP protocol smoke test for the stdio transport#134
Open
yagna-1 wants to merge 2 commits into
Open
Conversation
Adds two end-to-end tests that spawn `mcp-server-qdrant --transport stdio` as a real subprocess and exercise the JSON-RPC protocol: - `test_initialize_handshake_succeeds`: verifies the server completes the MCP `initialize` handshake and advertises the `tools` capability. - `test_documented_tools_are_listed`: verifies that the `qdrant-find` and `qdrant-store` tools (both documented in the README) are returned by `tools/list` and have valid object input schemas. Today the test suite covers settings, FastEmbed, and Qdrant integration in isolation, but never exercises the actual MCP entry point. A regression in `main.py`, a FastMCP API drift, or a `mcp.run()` wiring issue could ship without a single test failing. These tests close that gap with no new infra: they run under the existing `uv run pytest` command, take ~25 s on a cold start, use an in-memory Qdrant via `QDRANT_LOCAL_PATH`, and require no network or external services. Implementation uses `pytest-mcp-plugin`, a small pytest plugin that drives MCP servers over stdio/HTTP from inside the test runner. The plugin is added to `[tool.uv.dev-dependencies]`. Both tests `importorskip` it and skip cleanly if the binary isn't on PATH, so this file is safe in any environment. Co-authored-by: Cursor <[email protected]>
0.2.3 fixes a handshake bug where the client did not send notifications/initialized after initialize, which is required by the MCP spec and enforced by FastMCP-backed servers. mcp-server-qdrant happens to tolerate the omission (FastMCP 2.7.0 here is older), but the newer release is more correct and the smoke test depends on the fix to assert notifications/initialized appears in the wire trace. Co-authored-by: Cursor <[email protected]>
Author
|
Heads-up: If maintainers later want fuller coverage, v0.3.0 ships an opt-in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds two end-to-end tests in
tests/test_mcp_protocol.pythat spawnmcp-server-qdrant --transport stdioas a real subprocess and exercise the JSON-RPC protocol over its stdin/stdout:test_initialize_handshake_succeeds— server completes the MCPinitializehandshake and advertises thetoolscapability.test_documented_tools_are_listed—qdrant-findandqdrant-store(both documented in the README) are returned bytools/listwith valid object input schemas.Why
The current suite covers settings, FastEmbed, and Qdrant integration in isolation, but nothing actually drives
main.pyend-to-end. A regression in the entry script, FastMCP API drift, or a wiring change inmcp.run()could ship without a single test failing. These tests close that gap.I noticed this while reviewing Python MCP servers for conformance/issues/258 (Python/stdio support in Anthropic's official conformance suite). Once that lands, this file can grow into a proper conformance check; until then, the smoke test catches the obvious regressions for free.
How
tests/test_mcp_protocol.py(~70 lines, plus docstring).pytest-mcp-plugin>=0.2.1— a small pytest plugin that drives MCP servers over stdio/HTTP from inside the test runner.pytest.importorskip(\"mcp_test\")and apytestmarkskip ifmcp-server-qdrantisn't on PATH, so this file is safe in any environment, even before the dep is installed.No production code changes. No new CI workflow.
Verified locally
```text
$ uv sync
$ uv run pytest tests/ -q
.......................... [100%]
26 passed in 34.26s
```
That's 24 existing tests + 2 new, ~25 s of which is FastEmbed loading models on first call. The MCP server runs in-memory via `QDRANT_LOCAL_PATH` — no network, no external Qdrant required.
Notes
tests/, env viamonkeypatch, no asyncio (the plugin is sync; the existingasyncio_mode = "auto"is harmless).dependency-groups.dev, or split the dep change into its own commit if you'd prefer.Disclosure: I'm the author of
pytest-mcp-plugin(MIT, on PyPI). It's an Apache/MIT-compatible community library — not a vendored dependency. Repo: https://github.com/yagna-1/mcp-test.Made with Cursor