-
Notifications
You must be signed in to change notification settings - Fork 63
Closed
Description
Getting below error while querying parquet stored on GCS location
INTERNAL Error: Attempted to dereference unique_ptr that is NULL!
Stack Trace:
/tmp/libduckdb_java4521970872809828390.so(+0x469a76) [0x7f1431669a76]
/tmp/libduckdb_java4521970872809828390.so(+0x469b24) [0x7f1431669b24]
/tmp/libduckdb_java4521970872809828390.so(+0x46c511) [0x7f143166c511]
/tmp/libduckdb_java4521970872809828390.so(+0x38fb05) [0x7f143158fb05]
/tmp/libduckdb_java4521970872809828390.so(+0x1a06b66) [0x7f1432c06b66]
/tmp/libduckdb_java4521970872809828390.so(+0x1a06876) [0x7f1432c06876]
/tmp/libduckdb_java4521970872809828390.so(+0x1a18f92) [0x7f1432c18f92]
/tmp/libduckdb_java4521970872809828390.so(+0x1a195e0) [0x7f1432c195e0]
/tmp/libduckdb_java4521970872809828390.so(+0x1a69fa7) [0x7f1432c69fa7]
/tmp/libduckdb_java4521970872809828390.so(+0xa5a63c) [0x7f1431c5a63c]
/tmp/libduckdb_java4521970872809828390.so(+0x102886d) [0x7f143222886d]
/tmp/libduckdb_java4521970872809828390.so(+0x1032f88) [0x7f1432232f88]
/tmp/libduckdb_java4521970872809828390.so(+0x1033342) [0x7f1432233342]
/tmp/libduckdb_java4521970872809828390.so(+0x102a376) [0x7f143222a376]
/tmp/libduckdb_java4521970872809828390.so(+0x1032cc4) [0x7f1432232cc4]
/tmp/libduckdb_java4521970872809828390.so(+0xeba730) [0x7f14320ba730]
/tmp/libduckdb_java4521970872809828390.so(+0xeba96c) [0x7f14320ba96c]
/tmp/libduckdb_java4521970872809828390.so(+0xebab16) [0x7f14320bab16]
/tmp/libduckdb_java4521970872809828390.so(+0xebd220) [0x7f14320bd220]
/tmp/libduckdb_java4521970872809828390.so(+0x3e0ff2) [0x7f14315e0ff2]
/tmp/libduckdb_java4521970872809828390.so(Java_org_duckdb_DuckDBNative_duckdb_1jdbc_1execute+0x11) [0x7f14315e5ec1]
[0x841ff1053c]
This error signals an assertion failure within DuckDB. This usually occurs due to unexpected conditions or errors in the program's logic.
For more information, see https://duckdb.org/docs/stable/dev/internal_errors.
Here is code where we are creating connection
@Bean(name = "duckDbConnection")
public DuckDBConnection duckDbConnections() throws SQLException, IOException {
String dbFilePath = "/opt/app/db";
String jdbcUrl = "jdbc:duckdb:" + dbFilePath;
File dbFile = new File(dbFilePath);
dbFile.deleteOnExit();
String connectionInitSql =
"SET threads=4; " +
"SET memory_limit='2GB';";
log.info("label=DUCK_DB_IN_MEM_INIT_SQL, sql={}", connectionInitSql);
String keyId = parquetReaderConfig.getKeyId();
String secret = parquetReaderConfig.getSecret();
connectionInitSql += "CREATE OR REPLACE SECRET (TYPE gcs, KEY_ID '" + keyId + "',SECRET '" + secret + "');";
Properties connProperties = new Properties();
connProperties.setProperty("duckdb.read_only", "true");
if (!dbFile.exists()) {
try (Connection conn = DriverManager.getConnection(jdbcUrl)) {
} catch (SQLException e) {
log.error("Error while creating db file");
throw new RuntimeException(e);
}
}
DuckDBConnection conn = (DuckDBConnection) DriverManager.getConnection(jdbcUrl, connProperties);
conn.createStatement().execute(connectionInitSql);
return conn;
}
And here is the code where we are using above created connection
String selectQuery1 = "SELECT * FROM parquet_scan(['gs://<bucket-name>/historical/destinationlocation=1/*.parquet','gs://<bucket-name>/current/destinationlocation=1/*.parquet']) WHERE replgrpnbr = '1000000'";
try (Connection conn = duckDbConnection.duplicate(); Statement stmt = conn.createStatement()) {
try (ResultSet rs = stmt.executeQuery(selectQuery1)) {
while (rs.next()) {
// utilise rs to prepare response object
}
}
} catch (SQLException e) {
throw new SQLException("Failed to retrieve fulfillment plan due to SQL error", lastException);
}
Below is dependency we are using in our project
org.duckdb
duckdb_jdbc
1.3.0.0
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels