Thanks to visit codestin.com
Credit goes to github.com

Skip to content

create-table-mapping issue if the mapping size exceeds 10Mbs #4675

@qili86

Description

@qili86

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

customer has ~ 100K tables in their external HMS, when running databricks labs ucx create-table-mapping , it throws the size limit error:
ERROR [d.l.ucx.create-table-mapping] BadRequest: File size imported is (16413218 bytes), exceeded max size (10485760 bytes)
the detailed process is:
qi.li@X9NWF957DL ~ % databricks labs ucx create-table-mapping -p intuit-sg-e2e
15:18:02 ERROR [d.l.ucx.create-table-mapping] BadRequest: Inline byte limit exceeded. Statements executed with disposition=INLINE can have a result size of at most 26214400 bytes. Please execute the statement with disposition=EXTERNAL_LINKS if you want to download the full result.
2. then in the config.yml file, I changed the query_statement_disposition: INLINE to query_statement_disposition: EXTERNAL_LINKS
but when I rerun the cmd
3. qi.li@X9NWF957DL ~ % databricks labs ucx create-table-mapping -p intuit-sg-e2e
15:18:55 ERROR [d.l.ucx.create-table-mapping] BadRequest: File size imported is (16413218 bytes), exceeded max size (10485760 bytes)

Expected Behavior

No response

Steps To Reproduce

No response

Cloud

AWS

Operating System

macOS

Version

latest via Databricks CLI

Relevant log output

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions