CPL-6982: Server Error Handling #36
Conversation
homcc/server/server.py
Outdated
| class ServerInitializationError(Exception): | ||
| """Indicates that an error occurred during server startup.""" | ||
|
|
||
|
|
There was a problem hiding this comment.
Since our error classes are mere indicators and never depend on other parts of our code and never imply any logic, do we want to move /client/errors.py to /common/errors.py and include ServerInitializationError there? Currently, I think the amount of error classes there is quite manageable but if we add a lot more in the future, centralizing them might have not been the best idea in the end. What are your thoughts about that?
There was a problem hiding this comment.
As discussed, I think we can move them to common. I'll do this in this PR.
homcc/server/main.py
Outdated
| server, server_thread = start_server(address=address, port=port, limit=limit, profiles=profiles) | ||
| except ServerInitializationError: | ||
| logger.error("Could not start homccd, terminating.") | ||
| sys.exit(1) |
There was a problem hiding this comment.
I might be too much a fan of named exit codes but since ServerInitializationError will only be raised in the case of an OSError we could exit with os.EX_OSERR (71) here.
homcc/server/server.py
Outdated
| try: | ||
| request.sendall(message.to_bytes()) | ||
| except ConnectionError as err: | ||
| logger.error("Connection error while trying to send data. %s", err) |
There was a problem hiding this comment.
I forgot if ConnectionError includes the message information/string itself.
If not I think we should also log the message that was tried to be sent here, at least for debug.
There was a problem hiding this comment.
ConnectionError only includes the client IP and used port on server side, I think.
Also note: this connection error only happens if there is an I/O issue, meaning that for example the client closed the socket or lost connection. So this error is not really dependent on the message content.
Nevertheless, I would add the type of message that was tried to be sent to the error log. What do you think about that? Should I also add another debug log with the message content?
There was a problem hiding this comment.
Yeah, I think doing both, including the MessageType for error and debug-logging the message content, would be good.
In the future I'd like to have a tool that goes over our logs and does some analysis to find potential issues, so I think it's nice to have as much (useful) info as possible.
| try: | ||
| root_temp_folder = self.root_temp_folder | ||
| except AttributeError: | ||
| return | ||
|
|
||
| root_temp_folder.cleanup() |
There was a problem hiding this comment.
Do we know the cause of this btw? I was wondering if it was basically a double free or bad initialization issue and if we should adapt our logic somehow instead of just ignoring the issue here?
There was a problem hiding this comment.
Yes, one cause is if the server gets closed immediately after starting it, namely before it can initialize completely and create the temporary directory. When it was never created, we also do not have to clean it up. Previously, we always assumed that it was created.
There was a problem hiding this comment.
I see, thank you! I think we should add an explicit comment regarding this behavior here then.
Other than that I am satisfied with this approach. 👍
homcc/server/main.py
Outdated
| from homcc.server.server import ( # pylint: disable=wrong-import-position | ||
| start_server, | ||
| stop_server, | ||
| ServerInitializationError, |
There was a problem hiding this comment.
Maybe just a missed change here, but personally, I prefer importing directly from the containing module:
| ServerInitializationError, | |
| from homcc.common.errors import ServerInitializationError # pylint: disable=wrong-import-position |
There was a problem hiding this comment.
Good catch, this was because it "kept working" after I moved the error to commons.
homcc/server/server.py
Outdated
| except ConnectionError as err: | ||
| logger.error("Connection error while trying to send '%s' message. %s", message.message_type, err) | ||
| logger.debug( | ||
| "The following message could not be sent due to a connection error: \n%s", message.get_json_str() |
There was a problem hiding this comment.
[Minor]
| "The following message could not be sent due to a connection error: \n%s", message.get_json_str() | |
| "The following message could not be sent due to a connection error:\n%s", message.get_json_str() |
This PR adds handling for some uncaught exceptions in the server.