Hard-coding of client.scanner.io.tmpdir as the temp dir of SQL Gateway #1845
Description
Search before asking
- I searched in the issues and found nothing similar.
Fluss version
0.7.0 (latest release)
Please describe the bug 🐞
In the class FlinkTableFactory there is an hard-coding of the client.scanner.io.tmpdir (ConfigOptions.CLIENT_SCANNER_IO_TMP_DIR) with the flink io tmp dir:
// pass flink io tmp dir to fluss client.
flussConfig.setString(
ConfigOptions.CLIENT_SCANNER_IO_TMP_DIR,
new File(flinkConfig.get(CoreOptions.TMP_DIRS), "/fluss").getAbsolutePath());
As a consequence, when the Flink cluster (or SQL Gateway) and the client run on different hosts, the client attempts to access a directory that may not exist (or to which it lacks privileges) on its own host, since the path references the Flink host’s temp directory.
Caused by: java.io.IOException: Failed to create directory <flink_java_io_tmp_dir>/fluss/<snapshot-id> for storing kv snapshot files.
Solution
A possible solution to this problem would be to remove the lines
as the ConfigOptions.CLIENT_SCANNER_IO_TMP_DIR already defaults to the local java.tmp.dir
public static final ConfigOption<String> CLIENT_SCANNER_IO_TMP_DIR =
key("client.scanner.io.tmpdir")
.stringType()
.defaultValue(System.getProperty("java.io.tmpdir") + "/fluss")
.withDescription(
"Local directory that is used by client for"
+ " storing the data files (like kv snapshot, log segment files) to read temporarily");
Are you willing to submit a PR?
- I'm willing to submit a PR!