feat: support postgresql connection via individual flags #15379

joobisb · 2024-11-05T13:42:15Z

Currently CODER_PG_CONNECTION_URL is the only variable through which we will be able to provide the postgres credentials.

With this PR, we will support additional envs to configure PostgreSQL connections, namely

CODER_PG_HOST
CODER_PG_PORT 
CODER_PG_USERNAME
CODER_PG_PASSWORD
CODER_PG_DATABASE
CODER_PG_OPTIONS

mafredri

Thanks for the PR! Left a few suggestions and things to consider.

mafredri · 2024-11-05T14:09:54Z

cli/server.go

@@ -395,6 +395,22 @@ func (r *RootCmd) Server(newAPI func(context.Context, *coderd.Options) (*coderd.

 			config := r.createConfig()

+			if vals.PostgresURL == "" && vals.PostgresHost != "" && vals.PostgresUsername != "" && vals.PostgresPassword != "" && vals.PostgresDatabase != "" {


We should probably return an error if both connection URL and any other setting is provided. There's no good decision to make on which to prefer over the other so the user has to decide (hence the error) or we may introduce surprising behavior.

mafredri · 2024-11-05T14:10:39Z

cli/server.go

+				if err != nil {
+					return err
+				}
+				cliui.Infof(inv.Stdout, "Created PostgreSQL URL from provided flags")


Let's remove this log, the user doesn't need to know about inner workings of how the options are handled.

mafredri · 2024-11-05T14:11:44Z

cli/testdata/coder_server_--help.golden

+          provided via other flags, PostgreSQL binaries will be downloaded from
+          Maven (https://repo1.maven.org/maven2) and store all data in the
+          config root. Access the built-in database with "coder server
+          postgres-builtin-url".


We should also document that this can't be used together with the other options.

mafredri · 2024-11-05T14:14:42Z

cli/server.go

@@ -395,6 +395,22 @@ func (r *RootCmd) Server(newAPI func(context.Context, *coderd.Options) (*coderd.

 			config := r.createConfig()

+			if vals.PostgresURL == "" && vals.PostgresHost != "" && vals.PostgresUsername != "" && vals.PostgresPassword != "" && vals.PostgresDatabase != "" {
+				pgURL := fmt.Sprintf("postgres://%s:%s@%s:%s/%s?%s",


Consider what happens if only username is given, or no database, etc. I'm not sure if Set below will error under what circumstances, but is the error message helpful in the user figuring out what's missing? Additional validation needed?

@mafredri
if either of them is not set, the existing behaviour will continue, we will only form the URL if PostgresHost PostgresUsername PostgresPassword and PostgresDatabase are present

If I don't realize I have to specify all flags, I will see the following confusing error:

❯ CODER_PG_HOST=mypgip ./coder server [...] error: Failed to start built-in PostgreSQL. Optionally, specify an external deployment with `--postgres-url`: could not start postgres using /home/coder/.config/coderv2/postgres/bin/bin/pg_ctl start -w -D /home/coder/.config/coderv2/postgres/data -o -p 41307:

I tried to use individual pg flag(s) but am directed towards pg url. We should handle this case as well. (I.e. it is a configuration error but there's no hint.)

Also, thinking about this fmt.Sprintf some more, I think we should leave the encoding to url.URL just to be safe. This also entails changing PostgresOptions from serpent.String to serpent.StringArray, otherwise we expose an implementation detail that options need to be URL encoded (and joined by &).

So something like this (didn't test):

vals.PostgresURL.Set((&url.URL{ Scheme: "postgres", User: url.UserPassword(vals.PostgresUsername.String(), vals.PostgresPassword.String()), Host: vals.PostgresHost.String() + ":" + vals.PostgresPort.String(), Path: "/" + vals.PostgresDatabase.String(), RawQuery: strings.Join(vals.PostgresOptions.GetSlice(), "&"), }).String())

joobisb · 2024-11-06T08:50:56Z

@mafredri I've addressed the comments, please do have a look

mafredri

Thanks for amending the previous feedback, just a couple more and I think this is good to go 👍🏻

mafredri · 2024-11-06T10:57:19Z

cli/server.go

+			if vals.PostgresURL != "" && (vals.PostgresHost != "" || vals.PostgresUsername != "" ||
+				vals.PostgresPassword != "" || vals.PostgresDatabase != "") {
+				return xerrors.New("cannot specify both --postgres-url and individual postgres connection flags " +
+					"(--postgres-host, --postgres-username, etc). Please use only one connection method")


Suggested change

"(--postgres-host, --postgres-username, etc). Please use only one connection method")

"(--postgres-host, --postgres-username, etc), please specify only one connection method")

Nit: Consistent terminology.

mafredri · 2024-11-06T11:13:08Z

cli/server.go

@@ -395,6 +395,22 @@ func (r *RootCmd) Server(newAPI func(context.Context, *coderd.Options) (*coderd.

 			config := r.createConfig()

+			if vals.PostgresURL == "" && vals.PostgresHost != "" && vals.PostgresUsername != "" && vals.PostgresPassword != "" && vals.PostgresDatabase != "" {
+				pgURL := fmt.Sprintf("postgres://%s:%s@%s:%s/%s?%s",


If I don't realize I have to specify all flags, I will see the following confusing error:

❯ CODER_PG_HOST=mypgip ./coder server [...] error: Failed to start built-in PostgreSQL. Optionally, specify an external deployment with `--postgres-url`: could not start postgres using /home/coder/.config/coderv2/postgres/bin/bin/pg_ctl start -w -D /home/coder/.config/coderv2/postgres/data -o -p 41307:

I tried to use individual pg flag(s) but am directed towards pg url. We should handle this case as well. (I.e. it is a configuration error but there's no hint.)

Also, thinking about this fmt.Sprintf some more, I think we should leave the encoding to url.URL just to be safe. This also entails changing PostgresOptions from serpent.String to serpent.StringArray, otherwise we expose an implementation detail that options need to be URL encoded (and joined by &).

So something like this (didn't test):

vals.PostgresURL.Set((&url.URL{ Scheme: "postgres", User: url.UserPassword(vals.PostgresUsername.String(), vals.PostgresPassword.String()), Host: vals.PostgresHost.String() + ":" + vals.PostgresPort.String(), Path: "/" + vals.PostgresDatabase.String(), RawQuery: strings.Join(vals.PostgresOptions.GetSlice(), "&"), }).String())

joobisb · 2024-11-06T19:06:10Z

Thanks for amending the previous feedback, just a couple more and I think this is good to go 👍🏻

@mafredri I've updated the PR

mafredri

I just have one minor nit remaining but otherwise this is good to go. Thanks a lot for working on this @joobisb and sorry for the wait.

mafredri · 2024-11-14T13:58:58Z

cli/server.go

+				return xerrors.Errorf("validate postgres configuration: %w", err)
+			}
+
+			if vals.PostgresURL == "" && vals.PostgresHost != "" && vals.PostgresUsername != "" && vals.PostgresPassword != "" && vals.PostgresDatabase != "" {


Could we instead return a boolean from validatePostgresFlags that tells us to use flags? We basically have the same logic in two places and there's a risk of one of them going stale if something changes in the future.

@mafredri I've addressed this

mafredri · 2024-11-15T11:42:39Z

Thanks for your contribution @joobisb 🎉! This is now ready to go 🚀 (as soon as final CI checks have completed).

joobisb · 2024-11-15T20:06:17Z

@matifali could you please help with this as well as e2e is failing here as well

ammario

I see the problem in the original issue but I don’t think it warrants more server flags. Users can create the connection URL via interpolation in bash, etc.

We should have a very high bar for adding server flags in general and an even higher bar for multiple ways to configure the same thing.

mafredri · 2024-11-20T14:19:31Z

@joobisb Thank you very much for the effort you put into this and I want to apologize that we didn't catch the bad product fit earlier. I hope this doesn't discourage you and want you to know it was a pleasure working with you on this, I look forward to more such opportunities in the future.

joobisb · 2024-11-22T11:42:43Z

@joobisb Thank you very much for the effort you put into this and I want to apologize that we didn't catch the bad product fit earlier. I hope this doesn't discourage you and want you to know it was a pleasure working with you on this, I look forward to more such opportunities in the future.

@mafredri Thank-you for your kind words, and no worries. Things won't always go as planned. I also look forward to working with you again.

feat: support postgresql connection via individual flags

9bc7b28

cdr-bot bot added the community Pull Requests and issues created by the community. label Nov 5, 2024

github-actions bot assigned joobisb Nov 5, 2024

fix: updated golden file

5deb4da

mafredri reviewed Nov 5, 2024

View reviewed changes

joobisb added 2 commits November 6, 2024 14:11

fix: addressed review comments

31b3b9b

updated docs

8574592

mafredri reviewed Nov 6, 2024

View reviewed changes

fix: appropriate error message

bfa3cb2

matifali requested a review from mafredri November 12, 2024 09:44

make gen

f50963b

mafredri approved these changes Nov 14, 2024

View reviewed changes

joobisb and others added 3 commits November 15, 2024 16:21

return bool from validatePostgresFlags

7e8573c

Merge branch 'main' into issue#15264

b3c396b

Merge branch 'main' into issue#15264

a15c797

mafredri enabled auto-merge (squash) November 15, 2024 11:40

ammario requested changes Nov 16, 2024

View reviewed changes

mafredri closed this Nov 20, 2024

auto-merge was automatically disabled November 20, 2024 14:19
Pull request was closed

joobisb deleted the issue#15264 branch November 22, 2024 13:30

matifali mentioned this pull request May 7, 2025

Provide Postgresql database settings not only via connection string #15264

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support postgresql connection via individual flags #15379

feat: support postgresql connection via individual flags #15379

joobisb commented Nov 5, 2024

mafredri left a comment

mafredri Nov 5, 2024

mafredri Nov 5, 2024

mafredri Nov 5, 2024

mafredri Nov 5, 2024

joobisb Nov 6, 2024

mafredri Nov 6, 2024 •

edited

Loading

joobisb commented Nov 6, 2024

mafredri left a comment

mafredri Nov 6, 2024

mafredri Nov 6, 2024 •

edited

Loading

joobisb commented Nov 6, 2024

mafredri left a comment

mafredri Nov 14, 2024

joobisb Nov 15, 2024

mafredri commented Nov 15, 2024

joobisb commented Nov 15, 2024

ammario left a comment

mafredri commented Nov 20, 2024

joobisb commented Nov 22, 2024

		@@ -395,6 +395,22 @@ func (r RootCmd) Server(newAPI func(context.Context, coderd.Options) (*coderd.

		config := r.createConfig()

		if vals.PostgresURL == "" && vals.PostgresHost != "" && vals.PostgresUsername != "" && vals.PostgresPassword != "" && vals.PostgresDatabase != "" {

	"(--postgres-host, --postgres-username, etc). Please use only one connection method")
	"(--postgres-host, --postgres-username, etc), please specify only one connection method")

feat: support postgresql connection via individual flags #15379

feat: support postgresql connection via individual flags #15379

Conversation

joobisb commented Nov 5, 2024

mafredri left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mafredri Nov 6, 2024 • edited Loading

Choose a reason for hiding this comment

joobisb commented Nov 6, 2024

mafredri left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mafredri Nov 6, 2024 • edited Loading

Choose a reason for hiding this comment

joobisb commented Nov 6, 2024

mafredri left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mafredri commented Nov 15, 2024

joobisb commented Nov 15, 2024

ammario left a comment

Choose a reason for hiding this comment

mafredri commented Nov 20, 2024

joobisb commented Nov 22, 2024

mafredri Nov 6, 2024 •

edited

Loading

mafredri Nov 6, 2024 •

edited

Loading