Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: implement scheduling mechanism for prebuilds #18126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 46 commits into from
Jun 19, 2025
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
47cb6fc
feat: implement autoscaling mechanism for prebuilds
evgeniy-scherbina May 29, 2025
0f4d521
fix: precision issue with is-within-range function
evgeniy-scherbina Jun 13, 2025
b957eb6
test: fix is-within-range test
evgeniy-scherbina Jun 13, 2025
de920b8
test: minor fixes in is-within-range test
evgeniy-scherbina Jun 13, 2025
5d31b43
test: fix tests failed due to is-within-range function
evgeniy-scherbina Jun 13, 2025
b3130a8
fix: migration numbers
evgeniy-scherbina Jun 13, 2025
0565fb7
feat: define functions for validation of schedules
evgeniy-scherbina Jun 15, 2025
99d76e5
Merge remote-tracking branch 'origin/main' into yevhenii/prebuilds-au…
evgeniy-scherbina Jun 15, 2025
bdfcd9e
regenerate protofiles
evgeniy-scherbina Jun 15, 2025
ea7766f
test: improve test coverage for schedules-overlap function
evgeniy-scherbina Jun 15, 2025
1e78317
test: improve test coverage for validate-schedules function
evgeniy-scherbina Jun 16, 2025
5d99ef1
refactor: add logger to preset-snapshot
evgeniy-scherbina Jun 16, 2025
158b92e
refactor: fallback to default on error in calc-desired-instances
evgeniy-scherbina Jun 16, 2025
396d080
refactor: change signature of calc-desired-instances
evgeniy-scherbina Jun 16, 2025
12dba6c
fix: improve schedule validation
evgeniy-scherbina Jun 16, 2025
3fa98b2
fix: bug related to DOM and DOW interpretation
evgeniy-scherbina Jun 16, 2025
8a063ff
fix: add schedules validation
evgeniy-scherbina Jun 16, 2025
12d3d88
fix: use TimeRange function instead of Weekly
evgeniy-scherbina Jun 16, 2025
3facaf2
fix: linter
evgeniy-scherbina Jun 17, 2025
6ce7ff2
fix: migration numbers
evgeniy-scherbina Jun 17, 2025
178676f
Merge remote-tracking branch 'origin/main' into yevhenii/prebuilds-au…
evgeniy-scherbina Jun 17, 2025
368fb36
deps: update version of tf-provider-coder
evgeniy-scherbina Jun 17, 2025
44b67ae
refactor: use ValidateSchedules from tf-provider-coder
evgeniy-scherbina Jun 17, 2025
df202c2
fix: CR's fixes
evgeniy-scherbina Jun 17, 2025
affee62
Update coderd/database/dbgen/dbgen.go
evgeniy-scherbina Jun 17, 2025
909d950
refactor: rename instances to desired_instances for consistency
evgeniy-scherbina Jun 17, 2025
e08f8d2
fix: make gen
evgeniy-scherbina Jun 17, 2025
4da1e64
fix: migrations test
evgeniy-scherbina Jun 17, 2025
4d2557c
fix: optimize get-preset-prebuild-schedules query
evgeniy-scherbina Jun 17, 2025
faf9ec7
refactor: rename DB method
evgeniy-scherbina Jun 17, 2025
821eda7
refactor: rename autoscaling to scheduling
evgeniy-scherbina Jun 18, 2025
7617042
refactor: rename autoscaling to scheduling (migration files)
evgeniy-scherbina Jun 18, 2025
6c2350f
refactor: minor refactor after renaming
evgeniy-scherbina Jun 18, 2025
cacdb1f
deps: update version of tf-provider-coder
evgeniy-scherbina Jun 18, 2025
8d2c08e
gen: make gen
evgeniy-scherbina Jun 18, 2025
5600054
docs: document api changes in proto/version.go
evgeniy-scherbina Jun 18, 2025
2215a9e
test: extend convert-resources test with prebuilds.scheduling
evgeniy-scherbina Jun 18, 2025
69ab8e1
fix: make gen
evgeniy-scherbina Jun 19, 2025
4a29496
refactor: make fmt
evgeniy-scherbina Jun 19, 2025
8edb051
deps: update version of tf-provider-coder
evgeniy-scherbina Jun 19, 2025
954bf66
Merge remote-tracking branch 'origin/main' into yevhenii/prebuilds-au…
evgeniy-scherbina Jun 19, 2025
2d945cf
fix: migration numbers
evgeniy-scherbina Jun 19, 2025
533a2b5
refactor: fix formatting
evgeniy-scherbina Jun 19, 2025
2b7119a
fix: set scheduling_timezone to empty string by default
evgeniy-scherbina Jun 19, 2025
96caa18
refactor: improve logging
evgeniy-scherbina Jun 19, 2025
83872d6
refactor: improve logging
evgeniy-scherbina Jun 19, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions coderd/database/dbauthz/dbauthz.go
Original file line number Diff line number Diff line change
Expand Up @@ -2282,6 +2282,13 @@ func (q *querier) GetPresetParametersByTemplateVersionID(ctx context.Context, ar
return q.db.GetPresetParametersByTemplateVersionID(ctx, args)
}

func (q *querier) GetPresetPrebuildSchedules(ctx context.Context) ([]database.TemplateVersionPresetPrebuildSchedule, error) {
if err := q.authorizeContext(ctx, policy.ActionRead, rbac.ResourceTemplate.All()); err != nil {
return nil, err
}
return q.db.GetPresetPrebuildSchedules(ctx)
}

func (q *querier) GetPresetsAtFailureLimit(ctx context.Context, hardLimit int64) ([]database.GetPresetsAtFailureLimitRow, error) {
// GetPresetsAtFailureLimit returns a list of template version presets that have reached the hard failure limit.
// Request the same authorization permissions as GetPresetsBackoff, since the methods are similar.
Expand Down Expand Up @@ -3633,6 +3640,15 @@ func (q *querier) InsertPresetParameters(ctx context.Context, arg database.Inser
return q.db.InsertPresetParameters(ctx, arg)
}

func (q *querier) InsertPresetPrebuildSchedule(ctx context.Context, arg database.InsertPresetPrebuildScheduleParams) (database.TemplateVersionPresetPrebuildSchedule, error) {
err := q.authorizeContext(ctx, policy.ActionUpdate, rbac.ResourceTemplate)
if err != nil {
return database.TemplateVersionPresetPrebuildSchedule{}, err
}

return q.db.InsertPresetPrebuildSchedule(ctx, arg)
}

func (q *querier) InsertProvisionerJob(ctx context.Context, arg database.InsertProvisionerJobParams) (database.ProvisionerJob, error) {
// TODO: Remove this once we have a proper rbac check for provisioner jobs.
// Details in https://github.com/coder/coder/issues/16160
Expand Down
29 changes: 29 additions & 0 deletions coderd/database/dbauthz/dbauthz_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -979,6 +979,29 @@ func (s *MethodTestSuite) TestOrganization() {
}
check.Args(insertPresetParametersParams).Asserts(rbac.ResourceTemplate, policy.ActionUpdate)
}))
s.Run("InsertPresetPrebuildSchedule", s.Subtest(func(db database.Store, check *expects) {
org := dbgen.Organization(s.T(), db, database.Organization{})
user := dbgen.User(s.T(), db, database.User{})
template := dbgen.Template(s.T(), db, database.Template{
CreatedBy: user.ID,
OrganizationID: org.ID,
})
templateVersion := dbgen.TemplateVersion(s.T(), db, database.TemplateVersion{
TemplateID: uuid.NullUUID{UUID: template.ID, Valid: true},
OrganizationID: org.ID,
CreatedBy: user.ID,
})
preset := dbgen.Preset(s.T(), db, database.InsertPresetParams{
TemplateVersionID: templateVersion.ID,
Name: "test",
})
arg := database.InsertPresetPrebuildScheduleParams{
PresetID: preset.ID,
}
check.Args(arg).
Asserts(rbac.ResourceTemplate, policy.ActionUpdate).
ErrorsWithInMemDB(dbmem.ErrUnimplemented)
}))
s.Run("DeleteOrganizationMember", s.Subtest(func(db database.Store, check *expects) {
o := dbgen.Organization(s.T(), db, database.Organization{})
u := dbgen.User(s.T(), db, database.User{})
Expand Down Expand Up @@ -4913,6 +4936,12 @@ func (s *MethodTestSuite) TestPrebuilds() {
Asserts(template.RBACObject(), policy.ActionRead).
Returns(insertedParameters)
}))
s.Run("GetPresetPrebuildSchedules", s.Subtest(func(db database.Store, check *expects) {
check.Args().
Asserts(rbac.ResourceTemplate.All(), policy.ActionRead).
Returns([]database.TemplateVersionPresetPrebuildSchedule{}).
ErrorsWithInMemDB(dbmem.ErrUnimplemented)
}))
s.Run("GetPresetsByTemplateVersionID", s.Subtest(func(db database.Store, check *expects) {
ctx := context.Background()
org := dbgen.Organization(s.T(), db, database.Organization{})
Expand Down
1 change: 1 addition & 0 deletions coderd/database/dbfake/dbfake.go
Original file line number Diff line number Diff line change
Expand Up @@ -395,6 +395,7 @@ func (t TemplateVersionBuilder) Do() TemplateVersionResponse {
CreatedAt: version.CreatedAt,
DesiredInstances: preset.DesiredInstances,
InvalidateAfterSecs: preset.InvalidateAfterSecs,
AutoscalingTimezone: preset.AutoscalingTimezone,
})
}

Expand Down
11 changes: 11 additions & 0 deletions coderd/database/dbgen/dbgen.go
Original file line number Diff line number Diff line change
Expand Up @@ -1259,11 +1259,22 @@ func Preset(t testing.TB, db database.Store, seed database.InsertPresetParams) d
CreatedAt: takeFirst(seed.CreatedAt, dbtime.Now()),
DesiredInstances: seed.DesiredInstances,
InvalidateAfterSecs: seed.InvalidateAfterSecs,
AutoscalingTimezone: seed.AutoscalingTimezone,
})
require.NoError(t, err, "insert preset")
return preset
}

func PresetPrebuildSchedule(t testing.TB, db database.Store, seed database.InsertPresetPrebuildScheduleParams) database.TemplateVersionPresetPrebuildSchedule {
schedule, err := db.InsertPresetPrebuildSchedule(genCtx, database.InsertPresetPrebuildScheduleParams{
PresetID: seed.PresetID,
CronExpression: seed.CronExpression,
Instances: seed.Instances,
})
require.NoError(t, err, "insert preset")
return schedule
}

func PresetParameter(t testing.TB, db database.Store, seed database.InsertPresetParametersParams) []database.TemplateVersionPresetParameter {
parameters, err := db.InsertPresetParameters(genCtx, database.InsertPresetParametersParams{
TemplateVersionPresetID: takeFirst(seed.TemplateVersionPresetID, uuid.New()),
Expand Down
13 changes: 13 additions & 0 deletions coderd/database/dbmem/dbmem.go
Original file line number Diff line number Diff line change
Expand Up @@ -4369,6 +4369,10 @@ func (q *FakeQuerier) GetPresetParametersByTemplateVersionID(_ context.Context,
return parameters, nil
}

func (q *FakeQuerier) GetPresetPrebuildSchedules(ctx context.Context) ([]database.TemplateVersionPresetPrebuildSchedule, error) {
return nil, ErrUnimplemented
}

func (q *FakeQuerier) GetPresetsAtFailureLimit(ctx context.Context, hardLimit int64) ([]database.GetPresetsAtFailureLimitRow, error) {
return nil, ErrUnimplemented
}
Expand Down Expand Up @@ -9164,6 +9168,15 @@ func (q *FakeQuerier) InsertPresetParameters(_ context.Context, arg database.Ins
return presetParameters, nil
}

func (q *FakeQuerier) InsertPresetPrebuildSchedule(ctx context.Context, arg database.InsertPresetPrebuildScheduleParams) (database.TemplateVersionPresetPrebuildSchedule, error) {
err := validateDatabaseType(arg)
if err != nil {
return database.TemplateVersionPresetPrebuildSchedule{}, err
}

return database.TemplateVersionPresetPrebuildSchedule{}, ErrUnimplemented
}

func (q *FakeQuerier) InsertProvisionerJob(_ context.Context, arg database.InsertProvisionerJobParams) (database.ProvisionerJob, error) {
if err := validateDatabaseType(arg); err != nil {
return database.ProvisionerJob{}, err
Expand Down
14 changes: 14 additions & 0 deletions coderd/database/dbmetrics/querymetrics.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

30 changes: 30 additions & 0 deletions coderd/database/dbmock/dbmock.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 15 additions & 1 deletion coderd/database/dump.sql

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions coderd/database/foreign_key_constraint.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
-- Drop the autoscaling schedules table
DROP TABLE template_version_preset_prebuild_schedules;

-- Remove autoscaling_timezone column from template_version_presets table
ALTER TABLE template_version_presets
DROP COLUMN autoscaling_timezone;
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
-- Add autoscaling_timezone column to template_version_presets table
ALTER TABLE template_version_presets
ADD COLUMN autoscaling_timezone TEXT DEFAULT 'UTC' NOT NULL;

-- Add table for autoscaling schedules
CREATE TABLE template_version_preset_prebuild_schedules (
id UUID PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
preset_id UUID NOT NULL,
cron_expression TEXT NOT NULL,
instances INTEGER NOT NULL,
FOREIGN KEY (preset_id) REFERENCES template_version_presets (id) ON DELETE CASCADE
);
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
INSERT INTO
template_version_preset_prebuild_schedules (
id,
preset_id,
cron_expression,
instances
)
VALUES (
'e387cac1-9bf1-4fb6-8a34-db8cfb750dd0',
'28b42cc0-c4fe-4907-a0fe-e4d20f1e9bfe',
'* 8-18 * * 1-5',
1
);
8 changes: 8 additions & 0 deletions coderd/database/models.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions coderd/database/querier.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading