Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[#1119] Add test cases for tablespaces#1139

Open
sksingh2005 wants to merge 1 commit into
pgmoneta:mainfrom
sksingh2005:test/tablespaces
Open

[#1119] Add test cases for tablespaces#1139
sksingh2005 wants to merge 1 commit into
pgmoneta:mainfrom
sksingh2005:test/tablespaces

Conversation

@sksingh2005
Copy link
Copy Markdown
Collaborator

No description provided.

@sksingh2005 sksingh2005 self-assigned this May 8, 2026
@sksingh2005 sksingh2005 added the test test cases and infrastructure label May 8, 2026
@sksingh2005 sksingh2005 marked this pull request as draft May 8, 2026 17:53
@jesperpedersen
Copy link
Copy Markdown
Member

@ashu3103 Please, help here

@ashu3103
Copy link
Copy Markdown
Collaborator

ashu3103 commented May 9, 2026

@ashu3103 Please, help here

Sure, on it

@sksingh2005 sksingh2005 force-pushed the test/tablespaces branch 3 times, most recently from 941210f to 9aaf4f2 Compare May 19, 2026 23:04
@sksingh2005 sksingh2005 marked this pull request as ready for review May 19, 2026 23:04
@sksingh2005 sksingh2005 requested a review from Jubilee101 May 19, 2026 23:04
@jesperpedersen
Copy link
Copy Markdown
Member

@ashu3103 Can you go through this ?

Copy link
Copy Markdown
Member

@Jubilee101 Jubilee101 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work! Very thorough and looks like it indeed caught some issues. Left some questions. My only concern is that some fixes seems a bit ad hoc.

Comment thread src/libpgmoneta/aes.c Outdated
char path[1024];

if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0)
if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0 || strcmp(entry->d_name, "pg_tblspc") == 0)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? pg_tblspc is a dir with nothing but symlinks the previous check on DT_LNK should work

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You’re right. Hard-coding pg_tblspc was not needed. The real issue is avoiding symlink traversal during decrypt. I will update this to skip DT_LNK entries directly, so we no longer need the pg_tblspc name check.

Comment thread src/libpgmoneta/aes.c

while ((entry = readdir(dir)) != NULL)
{
if (entry->d_type == DT_DIR || entry->d_type == DT_LNK)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why removing DT_LNK?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh sorry, removing DT_LNK was too broad. I will change it to keep symlink handling explicit, but skip symlink traversal before recursing. I think that will then prevents duplicate decrypt work through pg_tblspc/<oid>.

Comment thread src/libpgmoneta/archive.c
if (pgmoneta_compare_string(tup->data[1], tblspc->path))
{
tblspc->oid = atoi(tup->data[0]);
pgmoneta_snprintf(manifest_prefix, sizeof(manifest_prefix), "pg_tblspc/%d", tblspc->oid);
Copy link
Copy Markdown
Member

@Jubilee101 Jubilee101 May 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it because the untar process isn't reading the file path from pg_tblspc dir correclty and causing manifest verification failure? Can you giev an example?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is for manifest verification. Tablespace tar entries are relative, for example:
PG_17_202406281/16385/16390

But the backup manifest stores the same file as:

pg_tblspc/<oid>/PG_17_202406281/16385/16390

So without prefixing pg_tblspc/<oid>, checksum/size verification uses the wrong key.

Comment thread src/libpgmoneta/archive.c
}

// get the copy out response
while (msg == NULL || msg->kind != 'H')
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, did test fail because of this? Nice catch. Something is wrong with this, this looks like a regression issue.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the PG14 failure matched this path. In PG14 multiple tar streams. The old code freed/reset msg after one stream, and the next stream could hit bad state. The log showed Connection reset by peer, then the next test failed because the server stayed active.

Comment thread src/libpgmoneta/archive.c Outdated
goto error;
}

if (msg == NULL)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit too ad hoc. I think we can reuse the same msg throughout the entire process. So at the end of out most iteration we should just do memset(msg, 0, ...) instead of pgmoneta_free_message(msg);. That'll save us some malloc.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I will update this to reuse the same msg and just call:

memset(msg, 0, sizeof(struct message));

at the end of each tar iteration instead of freeing/reallocating it.

Comment thread test/check.sh
fi
}

make_tablespace_cleanup_writable() {
Copy link
Copy Markdown
Member

@Jubilee101 Jubilee101 May 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this have to be run time? Can we put this into Dockerfile? And should we?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Dockerfile alone won’t help here because /conf is bind-mounted from $PGCONF_DIRECTORY, so image-time permissions are hidden by the host mount. This cleanup handles stale host-side tablespace dirs from previous runs, where files may be owned by the container postgres user and block rm -Rf "$PGMONETA_ROOT_DIR". We can keep creation in run-postgresql, but cleanup still needs to happen in check.sh. What are your suggestions ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it's better to move the test cases to test_backup and test_restore, respectively.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s a valid option. I kept them in test_tablespaces.c because the tests share tablespace setup, symlink checks, and cleanup, and it lets us run only this coverage with -m tablespaces. I can move backup checks to test_backup.c and restore checks to test_restore.c if you want . Let me know :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test test cases and infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants