Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix: throttle prebuild notifications #18483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

evgeniy-scherbina
Copy link
Contributor

@evgeniy-scherbina evgeniy-scherbina commented Jun 20, 2025

Closes coder/internal#674

Description:

  • Throttle notifications to template admins: send no more than one notification per 7 days.
  • Throttle notifications to template version author: send no more than one notification per 1 day.

Note:
It's done per template:

CREATE TABLE template_prebuild_notification_cooldowns (
	template_id UUID NOT NULL,
	notification_type TEXT NOT NULL,
	last_notification_sent TIMESTAMPTZ NOT NULL,
	PRIMARY KEY (template_id, notification_type)
);

We can have multiple broken templates - in this case Template Admins will get one notification per template per 7 days.
I think it shouldn't be more than few notifications per week.

If it's still too noisy - we can consider making cooldowns global instead of template-specific. It should be enough to remove template_id field.

@evgeniy-scherbina evgeniy-scherbina force-pushed the yevhenii/throttle-prebuild-notifications branch from 92b5e63 to 272c338 Compare June 20, 2025 20:06
@evgeniy-scherbina evgeniy-scherbina changed the title fix: throttle-prebuild-notifications fix: throttle prebuild notifications Jun 20, 2025
@evgeniy-scherbina evgeniy-scherbina force-pushed the yevhenii/throttle-prebuild-notifications branch from ae7135c to bd04330 Compare June 23, 2025 19:52
@evgeniy-scherbina evgeniy-scherbina force-pushed the yevhenii/throttle-prebuild-notifications branch from bd04330 to 362911e Compare June 23, 2025 20:16
@evgeniy-scherbina evgeniy-scherbina marked this pull request as ready for review June 23, 2025 22:34
@dannykopping
Copy link
Contributor

I'm really not a fan of this approach. It feels like we're just digging ourselves a deeper hole of technical debt instead of addressing the core issue.

We should move towards a daily or weekly summary email, like we have done for other build failures, and put logging and observability in place for operators to notice when problems occur with lower latency & greater resolution.

@github-actions github-actions bot locked and limited conversation to collaborators Jun 24, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reduce Notification Noise for Prebuild Failures
2 participants