Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Unable to alter iceberg table when location has trailing slash #14552

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
billoliver42 opened this issue Oct 10, 2022 · 4 comments
Closed

Unable to alter iceberg table when location has trailing slash #14552

billoliver42 opened this issue Oct 10, 2022 · 4 comments
Labels
bug Something isn't working

Comments

@billoliver42
Copy link

billoliver42 commented Oct 10, 2022

Description

When I create a table using the iceberg connector and set the table's location to a bucket (without a subfolder), Trino throws a metadata location mismatch error (see below).

Setup

Trino with the Iceberg connector using Minio as storage layer. Hive standalone metadata server using Postgres backend.

Error

SQL Error [65536]: Query failed (#20221010_171704_00057_2juwr): Metadata location [s3a://datalake/metadata/00000-17fce192-87bb-4d14-8627-3508fbf1e4fd.metadata.json] is not same as table metadata location [s3a://datalake//metadata/00000-17fce192-87bb-4d14-8627-3508fbf1e4fd.metadata.json] for billtest.test1

It appears that the only difference between the two paths is an extra forward slash character before 'metadata'.
s3a://datalake/metadata
vs
s3a://datalake//metadata

Steps to recreate

Step 1: Create table using iceberg connector

CREATE TABLE iceberg.billtest.test1
(
    id BIGINT,
    name VARCHAR
)
WITH (
     format = 'PARQUET',
     location = 's3a://datalake/'
)

Step 2: alter table to add new column

ALTER TABLE iceberg.billtest.test1 ADD COLUMN IF NOT EXISTS score DOUBLE

After step two, the above error is returned.

Workaround

The workaround is when creating the table, use a folder underneath the bucket, like this:

CREATE TABLE iceberg.billtest.test1
(
    id BIGINT,
    name VARCHAR
)
WITH (
     format = 'PARQUET',
     location = 's3a://datalake/test1'
)
@findepi findepi added the bug Something isn't working label Oct 11, 2022
@findepi
Copy link
Member

findepi commented Oct 11, 2022

@ebyhr's #14389 may help here, but that PR is supposed to deal with tables that got corrupted by versions 393 and 394.

Also #13984 may be related.

cc @ebyhr @alexjo2144

@ebyhr
Copy link
Member

ebyhr commented Oct 11, 2022

@billoliver42 I can't reproduce the issue in upstream master. What Trino version are you using?

@billoliver42
Copy link
Author

@ebyhr trino 372. We can try an updated version if it's been resolved recently?

@ebyhr ebyhr changed the title Unable to alter iceberg table if created in top-level bucket Unable to alter iceberg table when location has trailing slash Oct 13, 2022
@ebyhr
Copy link
Member

ebyhr commented Oct 13, 2022

Yes, I would recommend upgrading to upcoming 400. By the way, I believe the actual issue is trailing slash rather than under root bucket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

No branches or pull requests

3 participants