Add LoggingConfiguration struct for providing log path to pgbackrest … #4263

dsessler7 · 2025-09-03T18:30:07Z

…processes.

Add ability to send pgbackrest logs to additional volume for repo host processes.

Checklist:

Have you added an explanation of what your changes do and why you'd like them to be included?
Have you updated or added documentation for the change, as applicable?
Have you tested your changes on all related environments with successful results, as applicable?
- Have you added automated tests?

Type of Changes:

What is the current behavior (link to any open issues here)?

What is the new behavior (if this is a feature change)?

Breaking change (fix or feature that would cause existing functionality to change)

Other Information:

dsessler7 · 2025-09-03T18:31:16Z

pkg/apis/postgres-operator.crunchydata.com/v1beta1/pgbackrest_types.go

+	// Logging configuration for pgbackrest processes running in postgres instance pods.
+	// +optional
+	Log *LoggingConfiguration `json:"log,omitempty"`


I am open to other ideas for the placement of this LoggingConfiguration.

I do wonder if there would be an even better place to put it, but not sure where, so: Seems reasonable to me for now.

Hmm... Theoretically users could have multiple InstanceSets that have different additional volumes mounted, which would mean we'd need to allow different log paths per InstanceSet... So maybe the LoggingConfiguration should go on the PostgresInstanceSetSpec struct? 🤔

But we mount the same pgbackrest config to every postgres pod, right?

I was thinking that we had a different pgbackrest_instance.conf file per InstanceSet, but it seems you are correct... Only one pgbackrest_instance.conf regardless of the number of InstanceSets... Do we move the Volumes.Additional out of the PostgresInstanceSetSpec and into the PostgresClusterSpec and mount them into all instances so the log paths can be consistent amongst instances? 🤔

But then we'd be mounting the same PVC to all the different InstanceSet pods and having them all write to the same locations. (Which actually is the situation now if you have a single instanceSet with multiple replicas, so... it's already a fraught situation.)

Since we currently only allow people to bring their own volumes (rather than set up a template), people have to be a little careful about the pods-volumes connections. The safest scenario would probably be something like

multiple instance sets

each with only 1 replica

each with the same name of additional volumes but different PVC claims

That way, the config remains the same without any worry about collisions/overwriting between processes (where does each pod log? to the PVC that only that pod is connected)

In other words, I think this is (for now) going to be a lot of socialization and documentation.

dsessler7 · 2025-09-03T18:34:43Z

pkg/apis/postgres-operator.crunchydata.com/v1/pgbackrest_types.go

+)
+
+// PGBackRestArchive defines a pgBackRest archive configuration
+// +kubebuilder:validation:XValidation:rule=`!has(self.repoHost) || !has(self.repoHost.log) || !has(self.repoHost.log.path) || self.repoHost.volumes.additional.exists(x, self.repoHost.log.path.startsWith("/volumes/"+x.name))`,message=`log path is restricted to an existing additional volume`


Right now I just have it set to validate that the repoHost.log.path isn't set or if it is, it's set to an additional volume configured in repoHost... Not sure if we want to allow other paths or not... Thoughts?

I believe the repohost (as it now exists) will always have a /tmp directory and a /pgbackrest/repoN directory. Is that right and if so, would we want to allow logging to those? (Though actually: if we let logging into /pgbackrest/repoN, what subdirectories would we allow? Probably /pgbackrest/repoN/logs/pgbackrest only?)

Oh, but /pgbackrest/repoN/log (or logs?) is our default, so... I suppose we could allow explicit setting of our default just to let people be explicit.

I believe they all (instance for sidecar, repo host, and backup job) have /tmp...

Could enforce using /tmp/pgbackrest subdir if logging to /tmp

pkg/apis/postgres-operator.crunchydata.com/v1beta1/pgbackrest_types.go

pkg/apis/postgres-operator.crunchydata.com/v1/pgbackrest_types.go

internal/pgbackrest/config.go

benjaminjb · 2025-09-12T13:20:13Z

pkg/apis/postgres-operator.crunchydata.com/v1beta1/shared_types.go

+
+// LoggingConfiguration provides logging configuration for various components
+type LoggingConfiguration struct {
+	// +kubebuilder:validation:MaxLength=4096


Why 4096 as maxlength?

I figured any limit was better than no limit and I read somewhere that 4096 is usually the character limit for a directory in linux. Can't imagine anyone ever getting close to this.

Let's start smaller and go larger if anyone hits it. 🤔 256? 300?

📝 These are multiplied by 4x during cost estimation because UTF-8 characters can be up to 4 bytes.

benjaminjb · 2025-09-12T15:03:47Z

internal/pgbackrest/config.go

+			// If the user has set a log path in the spec, use it. Otherwise,
+			// default to /pgbackrest/repo#/log where pgBackRest will log to the
+			// first configured repo volume when commands are run on the pgBackRest
+			// repo host. With our previous check in RepoHostVolumeDefined(),
+			// we've already validated that at least one defined repo has a volume.
+			if repoHost != nil && repoHost.Log != nil && repoHost.Log.Path != "" {
+				global.Set("log-path", repoHost.Log.Path)
+			} else {
+				global.Set("log-path", fmt.Sprintf(naming.PGBackRestRepoLogPath, repo.Name))
+			}


The logic here seems similar to generateRepoHostLogPath, though I suppose it might be a little awkward to reuse that func (which loops through repos) here (in a loop of repos). I suppose you could simplify by pulling out the logic from ll 784-790 and reusing it -- but I wonder if there might be a way to keep this legible while reducing even more duplication. I mean: maybe we can pull this out of the repo loop?

will push a refactor commit in a few

pkg/apis/postgres-operator.crunchydata.com/v1/pgbackrest_types.go

benjaminjb

LGTM, but still have that question about annotation & additional volumes

pkg/apis/postgres-operator.crunchydata.com/v1/pgbackrest_types.go

internal/pgbackrest/config.go

…processes. Add ability to send pgbackrest logs to additional volume for repo host processes.

…ditional volume.

…add tests.

Add maxlength to log path and optional tag to volumes.

…n signature.

Add validation tests for log path and additional volumes.

…l volume when attempting to log pgbackrest sidecar to /volumes. Add test cases.

…Path function.

cbandy · 2025-09-16T15:07:21Z

pkg/apis/postgres-operator.crunchydata.com/v1/postgrescluster_types.go

+// TODO: Also check the above path against volume names: `i.?volumes.additional.hasValue() && i.volumes.additional.exists(a, v.startsWith("/volumes/" + a.name))`
+// https://github.com/kubernetes-sigs/controller-tools/pull/1270#issuecomment-3272211184


📝 I see this matches the TODO I left a few lines up. I was able to make that change in #4290 by setting max-items on instance sets. 🤔 It probably requires #4284, too.

dsessler7 commented Sep 3, 2025

View reviewed changes

dsessler7 requested review from benjaminjb and cbandy September 3, 2025 18:35

benjaminjb reviewed Sep 3, 2025

View reviewed changes

pkg/apis/postgres-operator.crunchydata.com/v1beta1/pgbackrest_types.go Show resolved Hide resolved

benjaminjb reviewed Sep 3, 2025

View reviewed changes

pkg/apis/postgres-operator.crunchydata.com/v1/pgbackrest_types.go Show resolved Hide resolved

benjaminjb reviewed Sep 3, 2025

View reviewed changes

internal/pgbackrest/config.go Outdated Show resolved Hide resolved

dsessler7 force-pushed the custom-pvcs-pgbackrest-logs branch 4 times, most recently from acabe60 to 1312df1 Compare September 5, 2025 21:27

dsessler7 marked this pull request as ready for review September 9, 2025 21:36

dsessler7 force-pushed the custom-pvcs-pgbackrest-logs branch from 69d7ce8 to a8dceef Compare September 10, 2025 00:25

dsessler7 requested a review from benjaminjb September 10, 2025 16:39

cbandy mentioned this pull request Sep 11, 2025

Separate postgrescluster.Reconciler client concerns #4256

Open

10 tasks

benjaminjb reviewed Sep 12, 2025

View reviewed changes

dsessler7 requested a review from benjaminjb September 12, 2025 20:16

benjaminjb reviewed Sep 15, 2025

View reviewed changes

cbandy reviewed Sep 15, 2025

View reviewed changes

pkg/apis/postgres-operator.crunchydata.com/v1/pgbackrest_types.go Outdated Show resolved Hide resolved

cbandy approved these changes Sep 15, 2025

View reviewed changes

internal/pgbackrest/config.go Outdated Show resolved Hide resolved

dsessler7 force-pushed the custom-pvcs-pgbackrest-logs branch from c382d94 to 9bb7e57 Compare September 15, 2025 20:21

dsessler7 added 9 commits September 15, 2025 13:25

Add LoggingConfiguration struct for providing log path to pgbackrest …

0b25484

…processes. Add ability to send pgbackrest logs to additional volume for repo host processes.

Refactor sending repo host pgbackrest logs to additional volume.

4689d83

Another small refactor for sending pgbackrest logs in Repo Host to ad…

f778519

…ditional volume.

Allow pgbackrest in cloud backup jobs to log to an additional volume.

cc9e4f9

Adjust function that provides the log path for cloud backup jobs and …

2762696

…add tests.

Allow pgbackrest in instance pod to log to an additional volumes.

fcdfdfe

Add maxlength to log path and optional tag to volumes.

Refactor generating and setting pgbackrest log path for repo host.

ccc5b79

Remove unneeded repohost from CreatePGBackRestConfigMapIntent functio…

0393f39

…n signature.

Add validation to prevent setting pgbackrest log-path via global.

06c0bde

Add validation tests for log path and additional volumes.

dsessler7 added 2 commits September 15, 2025 13:25

Add validation rule that ensures that all instances have an additiona…

e6beb45

…l volume when attempting to log pgbackrest sidecar to /volumes. Add test cases.

Reword pgbackrest log.path validation rules.

1cbac34

dsessler7 force-pushed the custom-pvcs-pgbackrest-logs branch from 9bb7e57 to 1cbac34 Compare September 15, 2025 20:26

Adjust max length for log path. Small refactor of generateRepoHostLog…

f5261ca

…Path function.

cbandy reviewed Sep 16, 2025

View reviewed changes

		// TODO: Also check the above path against volume names: `i.?volumes.additional.hasValue() && i.volumes.additional.exists(a, v.startsWith("/volumes/" + a.name))`
		// https://github.com/kubernetes-sigs/controller-tools/pull/1270#issuecomment-3272211184

Add LoggingConfiguration struct for providing log path to pgbackrest … #4263

Are you sure you want to change the base?

Add LoggingConfiguration struct for providing log path to pgbackrest … #4263

Conversation

dsessler7 commented Sep 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

benjaminjb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!