Make non-blocking serve uvicorn thread a daemon with force_exit - #47371
Open
SebTardif wants to merge 1 commit into
Open
Make non-blocking serve uvicorn thread a daemon with force_exit#47371SebTardif wants to merge 1 commit into
SebTardif wants to merge 1 commit into
Conversation
kill_server joins the background uvicorn thread with a 2s timeout. When the thread is non-daemon and uvicorn does not exit in time, the Python process hangs forever on interpreter shutdown. Mark the thread daemon and set server.force_exit so process exit cannot be blocked by a stuck serve. Signed-off-by: Sebastien Tardif <[email protected]>
Contributor
|
Thank you for your contribution 🤗! CI Security Gate — automatic approval blockedThis PR was not automatically approved for CI because the security gate failed. Possible reasons:
See the workflow run for the exact violations. A maintainer can review and manually approve CI if a finding is a false positive. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Prevents the Python process from hanging when the non-blocking
transformers serveuvicorn thread does not exit withinkill_server()'s 2s join timeout.Problem
start_server()creates a non-daemon background thread for uvicorn (daemon=False).kill_server()setsserver.should_exit = Trueand joins withtimeout=2. If uvicorn does not leaveserve()within 2 seconds:joinreturns while the thread is still aliveThis path is used heavily by the serve test suite (
non_blocking=True+kill_server()intearDownClass).Fix
daemon=Trueso a stuck serve cannot block process exit after the join timeout.server.force_exit = Truein addition toshould_exitso uvicorn abandons a stuck graceful shutdown when possible.The default CLI path (
non_blocking=False) still runs uvicorn on the main thread viaserver.run()and is unchanged.Origin: Background thread introduced in #41487 (2025-10-16) with
daemon=False.Test:
Before submitting
Who can review?
@LysandreJik @SunMarc (serve)