feat(coderd): add inbox notifications endpoints #16889

defelmnq · 2025-03-12T11:54:53Z

This PR is part of the inbox notifications topic, and rely on previous PRs merged - it adds :

Endpoints to :
- WS : watch new inbox notifications
- REST : list inbox notifications
- REST : update the read status of a notification

Also, this PR acts as a follow-up PR from previous work and :

fix DB query issues
fix DBMem logic to match DB

coderd/coderd.go

defelmnq · 2025-03-12T12:24:59Z

coderd/database/queries/notificationsinbox.sql

@@ -21,8 +21,8 @@ SELECT * FROM inbox_notifications WHERE
 -- param limit_opt: The limit of notifications to fetch. If the limit is not specified, it defaults to 25
 SELECT * FROM inbox_notifications WHERE
 	user_id = @user_id AND
-	template_id = ANY(@templates::UUID[]) AND


The previous query was not working as we want for empty values -
I've done a lot of tests and both template and targets logic were culprits here.

To clarify :

Templates

Either you can give an array of template_id as parameter and if the template_id of the entry match any of those, it will match.

Either you can give an empty (NULL) value and all the entries will match.

Targets

Either you can give an array of targets as parameter and if all the targets match, then the entry will match.

either you can give an empty (NULL) value and all the entries will match.

codersdk/inboxnotification.go

coderd/inboxnotifications_test.go

johnstcn · 2025-03-12T14:12:09Z

coderd/inboxnotifications.go

+	var targets []uuid.UUID
+	if targetsParam != "" {
+		splitTargets := strings.Split(targetsParam, ",")
+		for _, target := range splitTargets {
+			id, err := uuid.Parse(target)
+			if err != nil {
+				httpapi.Write(ctx, rw, http.StatusBadRequest, codersdk.Response{
+					Message: "Invalid target ID.",
+					Detail:  err.Error(),
+				})
+				return
+			}
+			targets = append(targets, id)
+		}
+	}
+
+	var templates []uuid.UUID
+	if templatesParam != "" {
+		splitTemplates := strings.Split(templatesParam, ",")
+		for _, template := range splitTemplates {
+			id, err := uuid.Parse(template)
+			if err != nil {
+				httpapi.Write(ctx, rw, http.StatusBadRequest, codersdk.Response{
+					Message: "Invalid template ID.",
+					Detail:  err.Error(),
+				})
+				return
+			}
+			templates = append(templates, id)
+		}
+	}
+
+	if readStatusParam != "" {
+		readOptions := []string{
+			string(database.InboxNotificationReadStatusRead),
+			string(database.InboxNotificationReadStatusUnread),
+			string(database.InboxNotificationReadStatusAll),
+		}
+
+		if !slices.Contains(readOptions, readStatusParam) {
+			httpapi.Write(ctx, rw, http.StatusBadRequest, codersdk.Response{
+				Message: "Invalid read status.",
+			})
+			return
+		}
+	}


Take a look at httpapi.NewQueryParamParser() as it should help you with this.

johnstcn · 2025-03-12T14:16:40Z

coderd/inboxnotifications.go

+			func(ctx context.Context, payload pubsub.InboxNotificationEvent, err error) {
+				if err != nil {
+					api.Logger.Error(ctx, "inbox notification event", slog.Error(err))
+					return
+				}
+
+				// filter out notifications that don't match the targets
+				if len(targets) > 0 {
+					for _, target := range targets {
+						if isFound := slices.Contains(payload.InboxNotification.Targets, target); !isFound {
+							return
+						}
+					}
+				}
+
+				// filter out notifications that don't match the templates
+				if len(templates) > 0 {
+					if isFound := slices.Contains(templates, payload.InboxNotification.TemplateID); !isFound {
+						return
+					}
+				}
+
+				// filter out notifications that don't match the read status
+				if readStatusParam != "" {
+					if readStatusParam == string(database.InboxNotificationReadStatusRead) {
+						if payload.InboxNotification.ReadAt == nil {
+							return
+						}
+					} else if readStatusParam == string(database.InboxNotificationReadStatusUnread) {
+						if payload.InboxNotification.ReadAt != nil {
+							return
+						}
+					}
+				}
+
+				notificationCh <- payload.InboxNotification
+			},


review: Could this be extracted to its own function? This would probably enhance readability and allow independently unit-testing this filtering functionality.

I have isolated - on both the websocket and list endpoints - the query parameters parsing. Indeed clean up the endpoints and isolate this part , thanks !

johnstcn · 2025-03-12T14:18:50Z

coderd/inboxnotifications.go

+					}
+				}
+
+				notificationCh <- payload.InboxNotification


What if notificationCh already has an item in it?

So as of now, if there's already an item it will 'block' trying to push the second one, until the first one has been pushed into the ws. It will just delay the new messages but not fail per itself.

We could increase the size of the chan to a bigger value - and am up to any proposal I just would like to avoid using a huge value just for the matter of 'having a big value'.

Never say never my friend. It may happen that due to bug in a different components, the subscriber collapses, and this will hang forever.

something like this can save oncall

select { case notificationCh <- payload.InboxNotification: default: }

fixed ✅
I also increased the channel size just a bit, just to have a buffer and avoid skipping notifications too fast.

johnstcn · 2025-03-12T14:19:49Z

coderd/inboxnotifications.go

+			if err := encoder.Encode(codersdk.GetInboxNotificationResponse{
+				Notification: notif,
+				UnreadCount:  int(unreadCount),
+			}); err != nil {
+				api.Logger.Error(ctx, "encode notification", slog.Error(err))
+				return
+			}


Do we need to set some kind of write deadline here to ensure we either write the response to the websocket in a reasonable amount of time, or fail and log the error?

I agree with you here. I checked a bit more the code and feel like the best option would be to change the Encoder function to accept a context.Context - we could then use deadline, cancel, etc...

Here I'd have to handle it manually in the endpoints which feels a bit less clean.

Would you agree if I create a follow-up ticket - not related to notifs but more this function ?

johnstcn · 2025-03-12T14:20:31Z

coderd/inboxnotifications.go

+	httpapi.Write(ctx, rw, http.StatusOK, codersdk.ListInboxNotificationsResponse{
+		Notifications: func() []codersdk.InboxNotification {
+			notificationsList := make([]codersdk.InboxNotification, 0, len(notifs))
+			for _, notification := range notifs {
+				notificationsList = append(notificationsList, codersdk.InboxNotification{
+					ID:         notification.ID,
+					UserID:     notification.UserID,
+					TemplateID: notification.TemplateID,
+					Targets:    notification.Targets,
+					Title:      notification.Title,
+					Content:    notification.Content,
+					Icon:       notification.Icon,
+					Actions: func() []codersdk.InboxNotificationAction {
+						var actionsList []codersdk.InboxNotificationAction
+						err := json.Unmarshal([]byte(notification.Actions), &actionsList)
+						if err != nil {
+							api.Logger.Error(ctx, "unmarshal inbox notification actions", slog.Error(err))
+						}
+						return actionsList
+					}(),
+					ReadAt: func() *time.Time {
+						if !notification.ReadAt.Valid {
+							return nil
+						}
+						return &notification.ReadAt.Time
+					}(),
+					CreatedAt: notification.CreatedAt,
+				})
+			}
+			return notificationsList
+		}(),
+		UnreadCount: int(unreadCount),
+	})


I think this could use some kind of convertNotificationsResponse helper function.

Makes sense, I tried first to put one in codersdk but we have some linter rules to avoid any coderd/database import into it.

I created one in the coderd package, next to the handlers - please tell me if that's good for you - makes it way more readable anyway.

johnstcn · 2025-03-12T14:21:23Z

coderd/inboxnotifications.go

+	parsedNotifID, err := uuid.Parse(notifID)
+	if err != nil {
+		api.Logger.Error(ctx, "failed to parse notification uuid", slog.Error(err))
+		httpapi.Write(ctx, rw, http.StatusInternalServerError, codersdk.Response{


This is probably more of a 4xx error (bad request?)

Yeah you're right ✅
Changed to 400 Bad Request to align with the other endpoints.

johnstcn · 2025-03-12T14:22:25Z

coderd/inboxnotifications.go

+		ReadAt: func() sql.NullTime {
+			if body.IsRead {
+				return sql.NullTime{
+					Time:  dbtime.Now(),
+					Valid: true,
+				}
+			}
+
+			return sql.NullTime{}


Does this allow you to mark a notification as 'unread'?

Yes, for now as a product requirement we want to be able to set a notification as 'unread'. It can be used if :

You missclicked and set a notification as read.

You want to keep some kind of reminder for a notification that you are interested by - so you set it as unread.

coderd/inboxnotifications_test.go

johnstcn · 2025-03-12T14:25:25Z

coderd/inboxnotifications_test.go

+		for i := range 40 {
+			_, err = api.Database.InsertInboxNotification(notifierCtx, database.InsertInboxNotificationParams{
+				ID:         uuid.New(),
+				UserID:     firstUser.UserID,
+				TemplateID: notifications.TemplateWorkspaceOutOfMemory,
+				Title:      fmt.Sprintf("Notification %d", i),
+				Actions:    json.RawMessage("[]"),
+				Content:    fmt.Sprintf("Content of the notif %d", i),
+				CreatedAt:  dbtime.Now(),
+			})
+			require.NoError(t, err)
+		}


This could be a good candidate for a dbgen helper function.

good catch - even more that the function was already there for other testing.. :')

I migrated all tests to it.

coderd/database/dbmem/dbmem.go

dannykopping

Generally fine with the changes, but the lack of testing around the meat of this PR (the websocket aspect) is why I'm withholding my ✅ for now.

coderd/database/queries/notificationsinbox.sql

coderd/inboxnotifications.go

dannykopping · 2025-03-13T05:05:24Z

coderd/inboxnotifications_test.go

@@ -0,0 +1,288 @@
+package coderd_test


The coverage of inboxnotifications.go is only about 41%. The two major testing blindspots right now are:

error conditions in general

all of watchInboxNotifications

coderd/inboxnotifications.go

dannykopping · 2025-03-13T05:12:12Z

coderd/inboxnotifications.go

+					return
+				}
+
+				// filter out notifications that don't match the targets


Curious why we need this here when the SQL query allows a list of targets to match on?

+1

unless this is intended, then please drop a comment on why this is mandatory.

This is indeed intended - I added a comment to clarify the situation but also here :

The inbox notifications received on the HandleInboxNotificationEvent are all the notifications pushed into the inbox table, without any filtering logic.

This is why here - for each websocket connection and based on the filters set as query params, we filters the events we want to process or not.

coderd/inboxnotifications.go

dannykopping · 2025-03-13T05:21:57Z

coderd/pubsub/inboxnotification.go

@@ -0,0 +1,44 @@
+package pubsub


Howcome you're creating this package inside coderd and not coderd/notifications?

I may have know the answer. It looks awkward but it seems ~consistent with coderd/wspubsub. You have 2 options:

Rename wspubsub to pubsub, and park everything there.

Just add leave this code in coderd/notifications

This is indeed what Marcin said. As we already had a wspubsub handling pubsub but only for workspace - I was willing to create a whole pubsub package on the which we could migrate the events handler.

Both solutions make sense to me, and both require anyway to move wspubsub to stay consistent. I tried to do it in this PR but it was generating a lot of new files changes and LoC so I preferred keep it for later.

If we agree on one or the other solution , I can create the follow-up PR for workspaces later.

coderd/pubsub/inboxnotification.go

coderd/inboxnotifications_test.go

mtojek

The first review round is done on my end. We're quite not there yet, but the direction is fine :)

Please remember to link to the Github issue 👍

coderd/database/dbmem/dbmem.go

coderd/database/queries/notificationsinbox.sql

mtojek · 2025-03-13T09:19:56Z

coderd/inboxnotifications.go

+)
+
+// convertInboxNotificationParameters parses and validates the common parameters used in get and list endpoints for inbox notifications
+func convertInboxNotificationParameters(ctx context.Context, logger slog.Logger, targetsParam string, templatesParam string, readStatusParam string) ([]uuid.UUID, []uuid.UUID, string, error) {


It is a general comment - if we want to use inbox just for notifications, we should adjust filenames and function names to indicate the relationship: notificationsinbox.go or convertNotificationsInboxParameters.

codersdk/inboxnotification.go

coderd/inboxnotifications.go

mtojek · 2025-03-13T09:30:45Z

coderd/inboxnotifications.go

+					return
+				}
+
+				// filter out notifications that don't match the targets


+1

unless this is intended, then please drop a comment on why this is mandatory.

mtojek · 2025-03-13T09:32:35Z

coderd/inboxnotifications.go

+					}
+				}
+
+				notificationCh <- payload.InboxNotification


Never say never my friend. It may happen that due to bug in a different components, the subscriber collapses, and this will hang forever.

something like this can save oncall

select { case notificationCh <- payload.InboxNotification: default: }

coderd/notifications/manager.go

mtojek · 2025-03-13T09:36:26Z

coderd/pubsub/inboxnotification.go

@@ -0,0 +1,44 @@
+package pubsub


I may have know the answer. It looks awkward but it seems ~consistent with coderd/wspubsub. You have 2 options:

Rename wspubsub to pubsub, and park everything there.

Just add leave this code in coderd/notifications

mtojek · 2025-03-13T09:41:47Z

codersdk/inboxnotification.go

+func (c *Client) UpdateInboxNotificationReadStatus(ctx context.Context, notifID uuid.UUID, req UpdateInboxNotificationReadStatusRequest) (UpdateInboxNotificationReadStatusResponse, error) {
+	res, err := c.Request(
+		ctx, http.MethodPut,
+		fmt.Sprintf("/api/v2/notifications/inbox/%v/read-status", notifID),


%v in the URL suggests, that it should be possible to access a single notification via REST endpoint /api/v2/notifications/inbox/%v which is not true.

Alternatively, we can change this to /api/v2/notifications/inbox/update: { "notification_ids": [ ... ], "action": "MARK_READ" }.

PS. Ignore this if it is too late to change the API.

defelmnq

Thanks everyone for the first review - I tried to answer everything but please do not hesitate to ping me on a topic if you feel like I missed it.

Globally, the biggest changes are :

Testing coverage on the handlers.
Improving parameters validation on the handlers using existing functions.
Cleaning up some code, adding comments and smaller points...

defelmnq · 2025-03-13T12:50:09Z

coderd/pubsub/inboxnotification.go

@@ -0,0 +1,44 @@
+package pubsub


This is indeed what Marcin said. As we already had a wspubsub handling pubsub but only for workspace - I was willing to create a whole pubsub package on the which we could migrate the events handler.

Both solutions make sense to me, and both require anyway to move wspubsub to stay consistent. I tried to do it in this PR but it was generating a lot of new files changes and LoC so I preferred keep it for later.

If we agree on one or the other solution , I can create the follow-up PR for workspaces later.

coderd/notifications/manager.go

defelmnq · 2025-03-13T13:08:00Z

coderd/inboxnotifications.go

+					return
+				}
+
+				// filter out notifications that don't match the targets


This is indeed intended - I added a comment to clarify the situation but also here :

The inbox notifications received on the HandleInboxNotificationEvent are all the notifications pushed into the inbox table, without any filtering logic.

This is why here - for each websocket connection and based on the filters set as query params, we filters the events we want to process or not.

coderd/inboxnotifications.go

codersdk/inboxnotification.go

defelmnq · 2025-03-13T13:42:53Z

coderd/inboxnotifications.go

+					}
+				}
+
+				notificationCh <- payload.InboxNotification


fixed ✅
I also increased the channel size just a bit, just to have a buffer and avoid skipping notifications too fast.

coderd/inboxnotifications_test.go

coderd/inboxnotifications.go

defelmnq · 2025-03-17T02:16:14Z

coderd/inboxnotifications.go

+// @Param read_status query string false "Filter notifications by read status. Possible values: read, unread, all"
+// @Success 200 {object} codersdk.GetInboxNotificationResponse
+// @Router /notifications/inbox/watch [get]
+func (api *API) watchInboxNotifications(rw http.ResponseWriter, r *http.Request) {


Added some tests to cover most of the endpoint. ✅

dannykopping

Thanks for the improvements @defelmnq 👍
A few things left but nothing major.

dannykopping · 2025-03-17T05:48:24Z

coderd/inboxnotifications.go

+				select {
+				case notificationCh <- payload.InboxNotification:
+				default:
+					api.Logger.Error(ctx, "unable to push notification in channel")


Whenever you log a message, I want you to think to yourself: "If I were an operator, how would I know what happened here, why I should care, and what to do next". This doesn't really satisfy any of those 3 clauses.

coderd/inboxnotifications.go

coderd/inboxnotifications_test.go

mtojek

Looks good 👍 once you're confident with other comments, feel free to merge it!

mtojek · 2025-03-17T11:47:46Z

coderd/inboxnotifications_test.go

+		err = json.Unmarshal(message, &notif)
+		require.NoError(t, err)
+
+		require.Equal(t, 1, notif.UnreadCount)


Is there a chance that notifs.UnreadCount will accidentally return 2 unread notifications? I haven't analyzed the code too deeply, but we have dealt with similar flakiness around insights tests (expected 1, returned 2).

Funny that you raised that point as it was the case at first - I changed the logic to make sure we do not depend on potential latency and create flakiness - Here it should be all good.

add endpoints for inbox notifications

3c6512b

github-actions bot assigned defelmnq Mar 12, 2025

defelmnq commented Mar 12, 2025

View reviewed changes

coderd/coderd.go Show resolved Hide resolved

defelmnq commented Mar 12, 2025

View reviewed changes

codersdk/inboxnotification.go Outdated Show resolved Hide resolved

cleanup comments and errors

c8ccc60

BrunoQuaresma reviewed Mar 12, 2025

View reviewed changes

codersdk/inboxnotification.go Show resolved Hide resolved

skip tests on windows

18b694b

defelmnq marked this pull request as ready for review March 12, 2025 14:05

defelmnq requested review from DanielleMaywood, BrunoQuaresma, dannykopping, johnstcn and mtojek March 12, 2025 14:06

johnstcn reviewed Mar 12, 2025

View reviewed changes

DanielleMaywood reviewed Mar 12, 2025

View reviewed changes

coderd/database/dbmem/dbmem.go Show resolved Hide resolved

work on pr comments - cleanup endpoints and isolate logic

0e8ac4c

dannykopping reviewed Mar 13, 2025

View reviewed changes

mtojek reviewed Mar 13, 2025

View reviewed changes

defelmnq added 8 commits March 13, 2025 14:10

work on PR comments

d72d1f2

fix parameters validation

796bcd0

improve parameters validation

75c310d

fmt and lint

07ab7c4

websocket testing wip

cb41d1a

improve tests

6ff4c7e

make fmt and lint

1ebc7f4

make fmt and lint

736a2d7

defelmnq commented Mar 17, 2025

View reviewed changes

remove windows from watch handler tests

c28002e

dannykopping approved these changes Mar 17, 2025

View reviewed changes

mtojek approved these changes Mar 17, 2025

View reviewed changes

BrunoQuaresma mentioned this pull request Mar 17, 2025

Coder Inbox Web UI coder/internal#336

Closed

defelmnq added 5 commits March 17, 2025 22:10

fix comments

2637d86

fix tests

4d0a561

fix ci lint

1f18868

fix ci lint

4f01a86

fix ci lint

cf6af1b

defelmnq merged commit 3ae55bb into main Mar 17, 2025
33 of 35 checks passed

defelmnq deleted the notif-inbox/internal-335 branch March 17, 2025 23:02

github-actions bot locked and limited conversation to collaborators Mar 17, 2025

feat(coderd): add inbox notifications endpoints #16889

feat(coderd): add inbox notifications endpoints #16889

Conversation

defelmnq commented Mar 12, 2025

defelmnq Mar 12, 2025 • edited Loading

Choose a reason for hiding this comment

Templates

Targets

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dannykopping left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dannykopping Mar 13, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtojek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

defelmnq left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dannykopping left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtojek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

defelmnq Mar 12, 2025 •

edited

Loading

dannykopping Mar 13, 2025 •

edited

Loading