fix: gentle error handling on server crash during chat response #614
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
23d0439 to
1b18054
Compare
Member
|
seems there are more changes than this PR aims for. |
Contributor
Author
This is intended, can we merge #613 first? |
Member
|
can you resolve the conflicts in #613? |
Member
|
#613 is in now but the merge conflicts here are a bit more complex. |
9b15323 to
f693b66
Compare
Contributor
Author
I resolved the conflicts and removed the refactoring. |
c467301 to
fa5fdfb
Compare
c583772 to
a819b57
Compare
3cd18dd to
5e26c14
Compare
The chat was abruptly failing with the following stack trace if the chat
is answering and the server crashes during that time:
```
lab chat
╭──────────────────────────────────────────────────────────────────── system ────────────────────────────────────────────────────────────────────╮
│ Welcome to Chat CLI w/ MERLINITE-7B-Q4_K_M (type /h for help) │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
>>> [S][default]
>>> hello [S][default]
╭───────────────────────────────────────────────────────────── merlinite-7b-Q4_K_M ──────────────────────────────────────────────────────────────╮
│ Hello! I'm here to │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 1.031 seconds ─╯
Traceback (most recent call last):
File "/home/leseb/cli/test_run_2Fv9yrYE/venv/lib64/python3.10/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions
yield
File "/home/leseb/cli/test_run_2Fv9yrYE/venv/lib64/python3.10/site-packages/httpx/_transports/default.py", line 113, in __iter__
for part in self._httpcore_stream:
File "/home/leseb/cli/test_run_2Fv9yrYE/venv/lib64/python3.10/site-packages/httpcore/_sync/connection_pool.py", line 367, in __iter__
raise exc from None
File "/home/leseb/cli/test_run_2Fv9yrYE/venv/lib64/python3.10/site-packages/httpcore/_sync/connection_pool.py", line 363, in __iter__
for part in self._stream:
File "/home/leseb/cli/test_run_2Fv9yrYE/venv/lib64/python3.10/site-packages/httpcore/_sync/http11.py", line 349, in __iter__
raise exc
File "/home/leseb/cli/test_run_2Fv9yrYE/venv/lib64/python3.10/site-packages/httpcore/_sync/http11.py", line 341, in __iter__
for chunk in self._connection._receive_response_body(**kwargs):
File "/home/leseb/cli/test_run_2Fv9yrYE/venv/lib64/python3.10/site-packages/httpcore/_sync/http11.py", line 210, in _receive_response_body
event = self._receive_event(timeout=timeout)
File "/home/leseb/cli/test_run_2Fv9yrYE/venv/lib64/python3.10/site-packages/httpcore/_sync/http11.py", line 220, in _receive_event
with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}):
File "/usr/lib64/python3.10/contextlib.py", line 153, in __exit__
self.gen.throw(typ, value, traceback)
File "/home/leseb/cli/test_run_2Fv9yrYE/venv/lib64/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)
```
Now we handle this more elagantly and print before exiting:
```
lab chat
╭──────────────────────────────────────────────────────────────────── system ────────────────────────────────────────────────────────────────────╮
│ Welcome to Chat CLI w/ MERLINITE-7B-Q4_K_M (type /h for help) │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
>>> hello [S][default]
Connection to the server was closed
Executing chat failed with: API issue found while executing chat: Connection to the server was closed
```
Signed-off-by: Sébastien Han <[email protected]>
xukai92
approved these changes
Mar 22, 2024
Member
xukai92
left a comment
There was a problem hiding this comment.
PR looks very neat now! thanks!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The chat was abruptly failing with the following stack trace if the chat is answering and the server crashes during that time:
Now we handle this more elagantly and print before exiting:
DO NOT MERGE - DEPENDS ON: #613