Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: support postgresql connection via individual flags #15379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from
Closed

feat: support postgresql connection via individual flags #15379

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
9bc7b28
feat: support postgresql connection via individual flags
joobisb Nov 5, 2024
5deb4da
fix: updated golden file
joobisb Nov 5, 2024
31b3b9b
fix: addressed review comments
joobisb Nov 6, 2024
8574592
updated docs
joobisb Nov 6, 2024
bfa3cb2
fix: appropriate error message
joobisb Nov 6, 2024
f50963b
make gen
joobisb Nov 13, 2024
7e8573c
return bool from validatePostgresFlags
joobisb Nov 15, 2024
b3c396b
Merge branch 'main' into issue#15264
mafredri Nov 15, 2024
a15c797
Merge branch 'main' into issue#15264
mafredri Nov 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
feat: support postgresql connection via individual flags
  • Loading branch information
@joobisb
joobisb committed Nov 5, 2024
commit 9bc7b28a0734c512a633d71e03cdf24c96759834
16 changes: 16 additions & 0 deletions cli/server.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -395,6 +395,22 @@ func (r *RootCmd) Server(newAPI func(context.Context, *coderd.Options) (*coderd.

config := r.createConfig()

if vals.PostgresURL == "" && vals.PostgresHost != "" && vals.PostgresUsername != "" && vals.PostgresPassword != "" && vals.PostgresDatabase != "" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably return an error if both connection URL and any other setting is provided. There's no good decision to make on which to prefer over the other so the user has to decide (hence the error) or we may introduce surprising behavior.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we instead return a boolean from validatePostgresFlags that tells us to use flags? We basically have the same logic in two places and there's a risk of one of them going stale if something changes in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mafredri I've addressed this

pgURL := fmt.Sprintf("postgres://%s:%s@%s:%s/%s?%s",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider what happens if only username is given, or no database, etc. I'm not sure if Set below will error under what circumstances, but is the error message helpful in the user figuring out what's missing? Additional validation needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mafredri
if either of them is not set, the existing behaviour will continue, we will only form the URL if PostgresHost PostgresUsername PostgresPassword and PostgresDatabase are present

Copy link
Member

@mafredri mafredri Nov 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I don't realize I have to specify all flags, I will see the following confusing error:

❯ CODER_PG_HOST=mypgip ./coder server
[...]
error: Failed to start built-in PostgreSQL. Optionally, specify an external deployment with `--postgres-url`: could not start postgres using /home/coder/.config/coderv2/postgres/bin/bin/pg_ctl start -w -D /home/coder/.config/coderv2/postgres/data -o -p 41307:

I tried to use individual pg flag(s) but am directed towards pg url. We should handle this case as well. (I.e. it is a configuration error but there's no hint.)

Also, thinking about this fmt.Sprintf some more, I think we should leave the encoding to url.URL just to be safe. This also entails changing PostgresOptions from serpent.String to serpent.StringArray, otherwise we expose an implementation detail that options need to be URL encoded (and joined by &).

So something like this (didn't test):

vals.PostgresURL.Set((&url.URL{
	Scheme:   "postgres",
	User:     url.UserPassword(vals.PostgresUsername.String(), vals.PostgresPassword.String()),
	Host:     vals.PostgresHost.String() + ":" + vals.PostgresPort.String(),
	Path:     "/" + vals.PostgresDatabase.String(),
	RawQuery: strings.Join(vals.PostgresOptions.GetSlice(), "&"),
}).String())

vals.PostgresUsername.String(),
vals.PostgresPassword.String(),
vals.PostgresHost.String(),
vals.PostgresPort.String(),
vals.PostgresDatabase.String(),
vals.PostgresOptions.String(),
)
err = vals.PostgresURL.Set(pgURL)
if err != nil {
return err
}
cliui.Infof(inv.Stdout, "Created PostgreSQL URL from provided flags")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this log, the user doesn't need to know about inner workings of how the options are handled.

}

builtinPostgres := false
// Only use built-in if PostgreSQL URL isn't specified!
if !vals.InMemoryDatabase && vals.PostgresURL == "" {
Expand Down
25 changes: 25 additions & 0 deletions cli/server_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,31 @@ func TestReadGitAuthProvidersFromEnv(t *testing.T) {
func TestServer(t *testing.T) {
t.Parallel()

t.Run("PostgresURLFromFlags", func(t *testing.T) {
t.Parallel()

inv, cfg := clitest.New(t,
"server",
"--http-address", ":0",
"--access-url", "https://foobarbaz.mydomain",
"--cache-dir", t.TempDir(),
"--postgres-host", "localhost",
"--postgres-port", "5432",
"--postgres-username", "coder",
"--postgres-password", "password",
"--postgres-database", "coder",
"--postgres-options", "sslmode=disable",
)

pty := ptytest.New(t).Attach(inv)

clitest.Start(t, inv)

// Just wait for startup
_ = waitAccessURL(t, cfg)

pty.ExpectMatch("Created PostgreSQL URL from provided flags")
})
t.Run("BuiltinPostgres", func(t *testing.T) {
t.Parallel()
if testing.Short() {
Expand Down
27 changes: 23 additions & 4 deletions cli/testdata/coder_server_--help.golden
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,29 @@ OPTIONS:
Type of auth to use when connecting to postgres.

--postgres-url string, $CODER_PG_CONNECTION_URL
URL of a PostgreSQL database. If empty, PostgreSQL binaries will be
downloaded from Maven (https://repo1.maven.org/maven2) and store all
data in the config root. Access the built-in database with "coder
server postgres-builtin-url".
URL of a PostgreSQL database. If empty and credentials are not
provided via other flags, PostgreSQL binaries will be downloaded from
Maven (https://repo1.maven.org/maven2) and store all data in the
config root. Access the built-in database with "coder server
postgres-builtin-url".
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also document that this can't be used together with the other options.


--postgres-database string, $CODER_PG_DATABASE
PostgreSQL database name.

--postgres-host string, $CODER_PG_HOST
PostgreSQL server hostname.

--postgres-options string, $CODER_PG_OPTIONS
PostgreSQL connection options (e.g. 'sslmode=require').

--postgres-password string, $CODER_PG_PASSWORD
PostgreSQL password.

--postgres-port string, $CODER_PG_PORT (default: 5432)
PostgreSQL server port.

--postgres-username string, $CODER_PG_USERNAME
PostgreSQL username.

--ssh-keygen-algorithm string, $CODER_SSH_KEYGEN_ALGORITHM (default: ed25519)
The algorithm to use for generating ssh keys. Accepted values are
Expand Down
2 changes: 2 additions & 0 deletions coderd/deployment_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ func TestDeploymentValues(t *testing.T) {
cfg.OIDC.AuthURLParams.Set(`{"foo":"bar"}`)
cfg.OIDC.EmailField.Set("some_random_field_you_never_expected")
cfg.PostgresURL.Set(hi)
cfg.PostgresPassword.Set(hi)
cfg.SCIMAPIKey.Set(hi)
cfg.ExternalTokenEncryptionKeys.Set("the_random_key_we_never_expected,an_other_key_we_never_unexpected")
cfg.Provisioner.DaemonPSK = "provisionersftw"
Expand All @@ -45,6 +46,7 @@ func TestDeploymentValues(t *testing.T) {
require.Empty(t, scrubbed.Values.OAuth2.Github.ClientSecret.Value())
require.Empty(t, scrubbed.Values.OIDC.ClientSecret.Value())
require.Empty(t, scrubbed.Values.PostgresURL.Value())
require.Empty(t, scrubbed.Values.PostgresPassword.Value())
require.Empty(t, scrubbed.Values.SCIMAPIKey.Value())
require.Empty(t, scrubbed.Values.ExternalTokenEncryptionKeys.Value())
require.Empty(t, scrubbed.Values.Provisioner.DaemonPSK.Value())
Expand Down
57 changes: 56 additions & 1 deletion codersdk/deployment.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,12 @@ type DeploymentValues struct {
InMemoryDatabase serpent.Bool `json:"in_memory_database,omitempty" typescript:",notnull"`
PostgresURL serpent.String `json:"pg_connection_url,omitempty" typescript:",notnull"`
PostgresAuth string `json:"pg_auth,omitempty" typescript:",notnull"`
PostgresHost serpent.String `json:"pg_host,omitempty" typescript:",notnull"`
PostgresPort serpent.String `json:"pg_port,omitempty" typescript:",notnull"`
PostgresUsername serpent.String `json:"pg_username,omitempty" typescript:",notnull"`
PostgresPassword serpent.String `json:"pg_password,omitempty" typescript:",notnull"`
PostgresDatabase serpent.String `json:"pg_database,omitempty" typescript:",notnull"`
PostgresOptions serpent.String `json:"pg_options,omitempty" typescript:",notnull"`
OAuth2 OAuth2Config `json:"oauth2,omitempty" typescript:",notnull"`
OIDC OIDCConfig `json:"oidc,omitempty" typescript:",notnull"`
Telemetry TelemetryConfig `json:"telemetry,omitempty" typescript:",notnull"`
Expand Down Expand Up @@ -2251,7 +2257,7 @@ when required by your organization's security policy.`,
},
{
Name: "Postgres Connection URL",
Description: "URL of a PostgreSQL database. If empty, PostgreSQL binaries will be downloaded from Maven (https://repo1.maven.org/maven2) and store all data in the config root. Access the built-in database with \"coder server postgres-builtin-url\".",
Description: "URL of a PostgreSQL database. If empty and credentials are not provided via other flags, PostgreSQL binaries will be downloaded from Maven (https://repo1.maven.org/maven2) and store all data in the config root. Access the built-in database with \"coder server postgres-builtin-url\".",
Flag: "postgres-url",
Env: "CODER_PG_CONNECTION_URL",
Annotations: serpent.Annotations{}.Mark(annotationSecretKey, "true"),
Expand All @@ -2266,6 +2272,55 @@ when required by your organization's security policy.`,
Value: serpent.EnumOf(&c.PostgresAuth, PostgresAuthDrivers...),
YAML: "pgAuth",
},
{
Name: "Postgres Host",
Description: "PostgreSQL server hostname.",
Flag: "postgres-host",
Env: "CODER_PG_HOST",
Value: &c.PostgresHost,
YAML: "pgHost",
},
{
Name: "Postgres Port",
Description: "PostgreSQL server port.",
Flag: "postgres-port",
Env: "CODER_PG_PORT",
Default: "5432",
Value: &c.PostgresPort,
YAML: "pgPort",
},
{
Name: "Postgres Username",
Description: "PostgreSQL username.",
Flag: "postgres-username",
Env: "CODER_PG_USERNAME",
Value: &c.PostgresUsername,
YAML: "pgUsername",
},
{
Name: "Postgres Password",
Description: "PostgreSQL password.",
Flag: "postgres-password",
Env: "CODER_PG_PASSWORD",
Annotations: serpent.Annotations{}.Mark(annotationSecretKey, "true"),
Value: &c.PostgresPassword,
},
{
Name: "Postgres Database",
Description: "PostgreSQL database name.",
Flag: "postgres-database",
Env: "CODER_PG_DATABASE",
Value: &c.PostgresDatabase,
YAML: "pgDatabase",
},
{
Name: "Postgres Options",
Description: "PostgreSQL connection options (e.g. 'sslmode=require').",
Flag: "postgres-options",
Env: "CODER_PG_OPTIONS",
Value: &c.PostgresOptions,
YAML: "pgOptions",
},
{
Name: "Secure Auth Cookie",
Description: "Controls if the 'Secure' property is set on browser session cookies.",
Expand Down
3 changes: 3 additions & 0 deletions codersdk/deployment_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@ func TestDeploymentValues_HighlyConfigurable(t *testing.T) {
"Postgres Connection URL": {
yaml: true,
},
"Postgres Password": {
yaml: true,
},
"SCIM API Key": {
yaml: true,
},
Expand Down
57 changes: 56 additions & 1 deletion docs/reference/cli/server.md
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 23 additions & 4 deletions enterprise/cli/testdata/coder_server_--help.golden
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,29 @@ OPTIONS:
Type of auth to use when connecting to postgres.

--postgres-url string, $CODER_PG_CONNECTION_URL
URL of a PostgreSQL database. If empty, PostgreSQL binaries will be
downloaded from Maven (https://repo1.maven.org/maven2) and store all
data in the config root. Access the built-in database with "coder
server postgres-builtin-url".
URL of a PostgreSQL database. If empty and credentials are not
provided via other flags, PostgreSQL binaries will be downloaded from
Maven (https://repo1.maven.org/maven2) and store all data in the
config root. Access the built-in database with "coder server
postgres-builtin-url".

--postgres-database string, $CODER_PG_DATABASE
PostgreSQL database name.

--postgres-host string, $CODER_PG_HOST
PostgreSQL server hostname.

--postgres-options string, $CODER_PG_OPTIONS
PostgreSQL connection options (e.g. 'sslmode=require').

--postgres-password string, $CODER_PG_PASSWORD
PostgreSQL password.

--postgres-port string, $CODER_PG_PORT (default: 5432)
PostgreSQL server port.

--postgres-username string, $CODER_PG_USERNAME
PostgreSQL username.

--ssh-keygen-algorithm string, $CODER_SSH_KEYGEN_ALGORITHM (default: ed25519)
The algorithm to use for generating ssh keys. Accepted values are
Expand Down
6 changes: 6 additions & 0 deletions site/src/api/typesGenerated.ts
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading