[#1119] Add test cases for tablespaces#1139
Conversation
7937982 to
6648e17
Compare
|
@ashu3103 Please, help here |
Sure, on it |
941210f to
9aaf4f2
Compare
9aaf4f2 to
87449a0
Compare
|
@ashu3103 Can you go through this ? |
Jubilee101
left a comment
There was a problem hiding this comment.
Amazing work! Very thorough and looks like it indeed caught some issues. Left some questions. My only concern is that some fixes seems a bit ad hoc.
| char path[1024]; | ||
|
|
||
| if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0) | ||
| if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0 || strcmp(entry->d_name, "pg_tblspc") == 0) |
There was a problem hiding this comment.
Why? pg_tblspc is a dir with nothing but symlinks the previous check on DT_LNK should work
There was a problem hiding this comment.
You’re right. Hard-coding pg_tblspc was not needed. The real issue is avoiding symlink traversal during decrypt. I will update this to skip DT_LNK entries directly, so we no longer need the pg_tblspc name check.
|
|
||
| while ((entry = readdir(dir)) != NULL) | ||
| { | ||
| if (entry->d_type == DT_DIR || entry->d_type == DT_LNK) |
There was a problem hiding this comment.
Ohh sorry, removing DT_LNK was too broad. I will change it to keep symlink handling explicit, but skip symlink traversal before recursing. I think that will then prevents duplicate decrypt work through pg_tblspc/<oid>.
| if (pgmoneta_compare_string(tup->data[1], tblspc->path)) | ||
| { | ||
| tblspc->oid = atoi(tup->data[0]); | ||
| pgmoneta_snprintf(manifest_prefix, sizeof(manifest_prefix), "pg_tblspc/%d", tblspc->oid); |
There was a problem hiding this comment.
Is it because the untar process isn't reading the file path from pg_tblspc dir correclty and causing manifest verification failure? Can you giev an example?
There was a problem hiding this comment.
Yes, this is for manifest verification. Tablespace tar entries are relative, for example:
PG_17_202406281/16385/16390
But the backup manifest stores the same file as:
pg_tblspc/<oid>/PG_17_202406281/16385/16390
So without prefixing pg_tblspc/<oid>, checksum/size verification uses the wrong key.
| } | ||
|
|
||
| // get the copy out response | ||
| while (msg == NULL || msg->kind != 'H') |
There was a problem hiding this comment.
Hmm, did test fail because of this? Nice catch. Something is wrong with this, this looks like a regression issue.
There was a problem hiding this comment.
Yes, the PG14 failure matched this path. In PG14 multiple tar streams. The old code freed/reset msg after one stream, and the next stream could hit bad state. The log showed Connection reset by peer, then the next test failed because the server stayed active.
| goto error; | ||
| } | ||
|
|
||
| if (msg == NULL) |
There was a problem hiding this comment.
This is a bit too ad hoc. I think we can reuse the same msg throughout the entire process. So at the end of out most iteration we should just do memset(msg, 0, ...) instead of pgmoneta_free_message(msg);. That'll save us some malloc.
There was a problem hiding this comment.
Agreed. I will update this to reuse the same msg and just call:
memset(msg, 0, sizeof(struct message));
at the end of each tar iteration instead of freeing/reallocating it.
| fi | ||
| } | ||
|
|
||
| make_tablespace_cleanup_writable() { |
There was a problem hiding this comment.
Does this have to be run time? Can we put this into Dockerfile? And should we?
There was a problem hiding this comment.
I think Dockerfile alone won’t help here because /conf is bind-mounted from $PGCONF_DIRECTORY, so image-time permissions are hidden by the host mount. This cleanup handles stale host-side tablespace dirs from previous runs, where files may be owned by the container postgres user and block rm -Rf "$PGMONETA_ROOT_DIR". We can keep creation in run-postgresql, but cleanup still needs to happen in check.sh. What are your suggestions ?
There was a problem hiding this comment.
I wonder if it's better to move the test cases to test_backup and test_restore, respectively.
There was a problem hiding this comment.
That’s a valid option. I kept them in test_tablespaces.c because the tests share tablespace setup, symlink checks, and cleanup, and it lets us run only this coverage with -m tablespaces. I can move backup checks to test_backup.c and restore checks to test_restore.c if you want . Let me know :)
87449a0 to
845c474
Compare
No description provided.