fix(email): triage fails fast when Lemonade is unreachable#1678
Merged
Conversation
POST /v1/email/triage hung ~30s when no Lemonade Server was running: the chat path's 900s scalar request timeout also governs the TCP connect, so an unreachable server blocked on the OS SYN timeout before erroring. Add a pre-flight /health probe with a short connect timeout in _build_llm_chat so 'server down' surfaces as a prompt, actionable HTTP 502 instead of a hang. Closes #1677
kovtcharov-amd
approved these changes
Jun 15, 2026
3 tasks
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Jun 16, 2026
## Why this matters Before: every PR touching `hub/agents/python/**` (or `setup.py`) kicked off `publish_agents.yml`'s `Discover Agent Packages` + `build` matrix, rebuilding **all** agent wheels just to twine-check them — on #1678, an email-only fix triggered the full agent-wheel matrix. And since PyPI publishing is paused (#1179), the workflow never actually published anything, so on PRs it was pure noise. After: the workflow is tag-driven — it fires on `v*` tags (plus `workflow_dispatch`), never on PRs. Broken wheels surface at release time instead of on every PR. Publishing **stays paused**: the `approve-publish`/`publish` jobs keep their `false &&` gates, so a `v*` tag builds + twine-checks the wheels but uploads nothing to PyPI. Re-enabling later is just dropping those gates — the tag trigger is now already in place. ## Test plan - [ ] Open/sync a PR touching `hub/agents/python/**` → `Discover Agent Packages` and `build` checks no longer appear. - [ ] `python -c "import yaml; yaml.safe_load(open('.github/workflows/publish_agents.yml'))"` parses clean (verified locally). - [ ] On a future `v*` tag, the build matrix runs and `approve-publish`/`publish` remain skipped (paused). Co-authored-by: Ovtcharov <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Before: with no Lemonade Server running,
POST /v1/email/triagehung ~30s (until the caller's HTTP timeout) before any error reached the client — "server down" looked like "server slow." After: it returns a prompt HTTP 502 within a couple seconds, naming that Lemonade isn't reachable and how to start it.Root cause: the triage chat path's request timeout is a 900s scalar (right for long generation), but a scalar
requeststimeout also caps the TCP connect — so an unreachable server blocked on the OS-level SYN timeout (~30s) before failing. The fix adds a fast pre-flight/healthprobe with a short connect timeout inEmailTriageService._build_llm_chat; a connection/timeout failure raisesLLMTriageError, which the route already maps to 502. Any HTTP response (even an error status) still counts as "reachable," so auth/model errors keep surfacing on the real chat call where their messages are specific. Fully contained to the email hub package — no change to the shared Lemonade client.Closes #1677
Test plan
python -m pytest hub/agents/python/email/tests/test_email_agent.pypasses (incl. newtest_triage_fails_fast_when_lemonade_unreachable)python -m pytest tests/unit/ -xpasses (5640 passed)python util/lint.py --allpassesgaia api) andPOST /v1/email/triagewith a valid single-message body — confirm a 502local LLM triage failed: Local Lemonade Server is not reachable …returns within ~1–3s instead of hanging ~30s.