Multiple table_disk tables with s3_plain_rewritable disk with shared endpoint corrupt each other's data #98512
Description
Problem
table_disk assumes that the entire disk is designated for that table. This leads to data corruption when multiple tables use table_disk with the same underlying disk, as one table pollutes the other.
Repro
Add to server config:
<s3_plain_rewritable>
<type>s3_plain_rewritable</type>
<endpoint>http://localhost:11111/plain-rewritable/</endpoint>
</s3_plain_rewritable>
Run :
CREATE TABLE t1 (a Int64, b Int64) ENGINE = MergeTree settings table_disk = 1, disk = disk(type = cache, path = '/tmp/filesystem_caches/stateful_1', max_size = '4G', disk = 's3_plain_rewritable');
CREATE TABLE t2 (a Int64, b Int64) ENGINE = MergeTree settings table_disk = 1, disk = disk(type = cache, path = '/tmp/filesystem_caches/stateful_2', max_size = '4G', disk = 's3_plain_rewritable');
INSERT INTO t1 SELECT * FROM generateRandom() LIMIT 100000;
INSERT INTO t2 SELECT * FROM generateRandom() LIMIT 200000;
SELECT * FROM t1;
DB::Exception: Cannot read all data in MergeTreeReaderCompact. Rows read: 8192. Rows expected: 10000: (while reading column b): (while reading from part all_1_1_0/ in table default.t1 (aadd0f6a-b23b-438f-9cdd-d0a77235fc5f) located on disk s3_plain_rewritable of type s3, from mark 0 with max_rows_to_read = 10000, offset = 0): While reading part all_1_1_0: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_READ_ALL_DATA)
SELECT * FROM t2;
┌────────────────────a─┬────────────────────b─┐
1. │ 2141105054873339885 │ 2228051567476793245 │
2. │ -4163394820885054867 │ 8737670515907451729 │
3. │ 3768795027477847959 │ 2947923418608131607 │
...