-
Notifications
You must be signed in to change notification settings - Fork 99
Description
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
customer has ~ 100K tables in their external HMS, when running databricks labs ucx create-table-mapping , it throws the size limit error:
ERROR [d.l.ucx.create-table-mapping] BadRequest: File size imported is (16413218 bytes), exceeded max size (10485760 bytes)
the detailed process is:
qi.li@X9NWF957DL ~ % databricks labs ucx create-table-mapping -p intuit-sg-e2e
15:18:02 ERROR [d.l.ucx.create-table-mapping] BadRequest: Inline byte limit exceeded. Statements executed with disposition=INLINE can have a result size of at most 26214400 bytes. Please execute the statement with disposition=EXTERNAL_LINKS if you want to download the full result.
2. then in the config.yml file, I changed the query_statement_disposition: INLINE to query_statement_disposition: EXTERNAL_LINKS
but when I rerun the cmd
3. qi.li@X9NWF957DL ~ % databricks labs ucx create-table-mapping -p intuit-sg-e2e
15:18:55 ERROR [d.l.ucx.create-table-mapping] BadRequest: File size imported is (16413218 bytes), exceeded max size (10485760 bytes)
Expected Behavior
No response
Steps To Reproduce
No response
Cloud
AWS
Operating System
macOS
Version
latest via Databricks CLI
Relevant log output
Metadata
Metadata
Assignees
Labels
Type
Projects
Status