Should catch InvalidParameterValue from server .notify calls #148
Description
Please confirm the following
- I agree to follow this project's code of conduct.
- I have checked the current issues for duplicates.
- I understand that dispatcherd is open source software provided for free and that I might not receive a timely response.
Bug Summary
While trying to test the demo again, I saw this:
DEBUG:dispatcherd.service.pool:Dispatching task (uuid=sch-212) to worker (id=1)
INFO:dispatcherd.service.main:Control action status returned message len=9891, sending back reply
ERROR:asyncio:Exception in callback CallbackHolder.done_callback(<Task finishe...ng too long')>)
handle: <Handle CallbackHolder.done_callback(<Task finishe...ng too long')>)>
Traceback (most recent call last):
File "/usr/lib/python3.12/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/dispatcherd/src/dispatcherd/service/asyncio_tasks.py", line 14, in done_callback
task.result()
File "/dispatcherd/src/dispatcherd/producers/brokered.py", line 51, in produce_forever
async for channel, payload in self.broker.aprocess_notify(connected_callback=self.connected_callback):
File "/dispatcherd/src/dispatcherd/brokers/pg_notify.py", line 227, in aprocess_notify
await self.apublish_message_from_cursor(cur, channel=reply_to, message=reply_message)
File "/dispatcherd/src/dispatcherd/brokers/pg_notify.py", line 232, in apublish_message_from_cursor
await cursor.execute(self.NOTIFY_QUERY_TEMPLATE, (channel, message))
File "/dispatcherd/venv/lib/python3.12/site-packages/psycopg/cursor_async.py", line 97, in execute
raise ex.with_traceback(None)
psycopg.errors.InvalidParameterValue: payload string too long
We should catch this exception and probably the broader DataError because it categorically means that the database connection is okay, but the message is bad.
The reason is because this causes the server to panic and drop the connection, which can have negative consequences.
INFO:dispatcherd.producers.brokered:recycle event received, restarting producer
INFO:dispatcherd.brokers.pg_notify:Closing asynchronous psycopg connection
In this specific exception case, this should not be done
Environment
all
Steps to reproduce
demo with many hacks
Actual results
recycling of connection/broker
Expected results
log exception and do not recycle broker
Additional information
No response