Thanks to visit codestin.com
Credit goes to github.com

Skip to content

branch-3.0: [fix](iceberg) fix insert iceberg transform partition table problems and add some cases #54557#54722

Closed
github-actions[bot] wants to merge 1 commit into
branch-3.0from
auto-pick-54557-branch-3.0
Closed

branch-3.0: [fix](iceberg) fix insert iceberg transform partition table problems and add some cases #54557#54722
github-actions[bot] wants to merge 1 commit into
branch-3.0from
auto-pick-54557-branch-3.0

Conversation

@github-actions

@github-actions github-actions Bot commented Aug 13, 2025

Copy link
Copy Markdown
Contributor

Cherry-picked from #54557

@suxiaogang223
[fix](iceberg) fix insert iceberg transform partition table problems …
e169721 
…and add some cases (#54557)

### What problem does this PR solve?

Problem Summary:
1. Fixed the problem with inserting into Iceberg decimal bucket
partition table due to memory issues
2. Fixed the exception caused by division by zero when selecting
count(*) from an empty iceberg partition table
3. Add cases to cover the logic of Iceberg transform partition table
https://iceberg.apache.org/spec/#partition-transforms
@github-actions github-actions Bot requested a review from dataroaring as a code owner August 13, 2025 23:52
@hello-stephen

hello-stephen commented Aug 13, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Aug 13, 2025
@hello-stephen

hello-stephen commented Aug 13, 2025

Copy link
Copy Markdown
Contributor

run buildall

@doris-robot

doris-robot commented Aug 14, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 39678 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e169721e17e9c63e7c5a0df36b1145af567cb2a6, data reload: false

------ Round 1 ----------------------------------
q1	17623	6889	6681	6681
q2	2069	179	162	162
q3	10671	1124	1164	1124
q4	10502	739	690	690
q5	7741	2860	2723	2723
q6	209	132	133	132
q7	954	619	606	606
q8	9335	1914	2006	1914
q9	6666	6397	6402	6397
q10	7029	2272	2287	2272
q11	470	260	251	251
q12	386	208	205	205
q13	17807	2975	2994	2975
q14	236	213	211	211
q15	527	461	445	445
q16	486	376	373	373
q17	972	590	561	561
q18	7293	6651	6753	6651
q19	1432	1113	1051	1051
q20	476	196	207	196
q21	3907	3069	3240	3069
q22	1098	1030	989	989
Total cold run time: 107889 ms
Total hot run time: 39678 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6610	6537	6523	6523
q2	329	226	229	226
q3	2856	2897	2909	2897
q4	2018	1855	1847	1847
q5	5653	5686	5721	5686
q6	211	126	131	126
q7	2261	1820	1814	1814
q8	3344	3496	3560	3496
q9	8778	8906	8838	8838
q10	3569	3510	3511	3510
q11	587	500	496	496
q12	784	591	599	591
q13	8082	3166	3134	3134
q14	318	262	286	262
q15	514	477	471	471
q16	492	435	446	435
q17	1823	1625	1657	1625
q18	8231	7712	7808	7712
q19	1687	1443	1636	1443
q20	2092	1920	1921	1920
q21	5320	5070	5040	5040
q22	1180	1084	1110	1084
Total cold run time: 66739 ms
Total hot run time: 59176 ms

@doris-robot

doris-robot commented Aug 14, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 196583 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e169721e17e9c63e7c5a0df36b1145af567cb2a6, data reload: false

query1	1263	990	899	899
query2	6318	1885	1884	1884
query3	10939	4594	4556	4556
query4	32849	23380	23504	23380
query5	3780	464	437	437
query6	276	177	179	177
query7	4003	305	315	305
query8	282	227	219	219
query9	9443	2556	2556	2556
query10	478	258	255	255
query11	17938	15627	15302	15302
query12	170	100	104	100
query13	1551	414	420	414
query14	8964	6713	7192	6713
query15	254	179	187	179
query16	8073	482	527	482
query17	1598	599	618	599
query18	2167	318	306	306
query19	304	170	168	168
query20	128	118	118	118
query21	217	118	113	113
query22	4869	4353	4352	4352
query23	35360	34244	34115	34115
query24	11304	2893	2870	2870
query25	679	414	429	414
query26	1343	169	175	169
query27	3002	340	346	340
query28	7597	2185	2175	2175
query29	881	442	438	438
query30	270	157	162	157
query31	1034	823	827	823
query32	93	53	51	51
query33	764	306	289	289
query34	959	514	530	514
query35	877	722	729	722
query36	1102	932	944	932
query37	131	63	67	63
query38	4125	3993	4014	3993
query39	1548	1436	1466	1436
query40	208	102	109	102
query41	48	52	46	46
query42	114	97	101	97
query43	531	495	469	469
query44	1296	812	816	812
query45	195	175	164	164
query46	1179	728	739	728
query47	2039	1906	1951	1906
query48	479	385	389	385
query49	952	438	395	395
query50	838	423	421	421
query51	7530	7351	7242	7242
query52	100	91	88	88
query53	254	185	181	181
query54	1293	491	496	491
query55	80	81	78	78
query56	275	240	238	238
query57	1351	1194	1213	1194
query58	223	219	225	219
query59	3152	3109	3026	3026
query60	287	264	263	263
query61	115	112	117	112
query62	886	689	684	684
query63	223	189	197	189
query64	5131	711	669	669
query65	3387	3311	3283	3283
query66	1546	306	304	304
query67	15950	15451	15484	15451
query68	4795	557	565	557
query69	456	267	255	255
query70	1192	1071	1117	1071
query71	337	249	257	249
query72	6257	4073	4044	4044
query73	764	340	341	340
query74	10327	9107	9230	9107
query75	3361	2641	2662	2641
query76	2611	1096	969	969
query77	445	278	284	278
query78	10558	9532	9559	9532
query79	2543	625	617	617
query80	1105	412	426	412
query81	549	220	218	218
query82	883	100	85	85
query83	242	143	148	143
query84	232	79	77	77
query85	1365	311	286	286
query86	455	300	296	296
query87	4435	4309	4177	4177
query88	4412	2337	2339	2337
query89	424	287	293	287
query90	1960	182	180	180
query91	179	145	152	145
query92	69	50	48	48
query93	2363	543	544	543
query94	867	314	298	298
query95	355	251	248	248
query96	620	272	274	272
query97	3299	3153	3140	3140
query98	218	201	203	201
query99	1490	1293	1278	1278
Total cold run time: 305763 ms
Total hot run time: 196583 ms

@doris-robot

doris-robot commented Aug 14, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 29.64 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e169721e17e9c63e7c5a0df36b1145af567cb2a6, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.02
query3	0.23	0.07	0.07
query4	1.63	0.11	0.10
query5	0.51	0.52	0.49
query6	1.13	0.74	0.73
query7	0.03	0.02	0.02
query8	0.04	0.03	0.04
query9	0.56	0.50	0.51
query10	0.56	0.56	0.55
query11	0.14	0.11	0.10
query12	0.14	0.11	0.10
query13	0.61	0.59	0.60
query14	0.78	0.78	0.78
query15	0.84	0.82	0.81
query16	0.38	0.41	0.38
query17	1.03	1.01	1.01
query18	0.23	0.22	0.23
query19	1.90	1.78	1.78
query20	0.01	0.00	0.01
query21	15.42	0.56	0.58
query22	2.21	1.88	1.89
query23	16.79	0.97	0.80
query24	3.37	0.48	1.68
query25	0.20	0.07	0.11
query26	0.43	0.14	0.14
query27	0.06	0.03	0.03
query28	10.36	0.52	0.46
query29	12.63	3.22	3.21
query30	0.25	0.07	0.06
query31	2.87	0.38	0.39
query32	3.24	0.47	0.46
query33	3.03	3.04	3.04
query34	17.17	4.58	4.51
query35	4.56	4.56	4.49
query36	0.66	0.48	0.47
query37	0.09	0.06	0.05
query38	0.04	0.03	0.04
query39	0.04	0.02	0.03
query40	0.16	0.13	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.57 s
Total hot run time: 29.64 s

@hello-stephen

hello-stephen commented Aug 14, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 100.00% (6/6) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 42.06% (11233/26707)
Line Coverage 32.57% (96089/295027)
Region Coverage 30.51% (55172/180835)
Branch Coverage 26.83% (27292/101740)

dataroaring
dataroaring approved these changes Aug 18, 2025
View reviewed changes

@dataroaring dataroaring left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Aug 18, 2025
@github-actions

github-actions Bot commented Aug 18, 2025

Copy link
Copy Markdown
Contributor Author

PR approved by at least one committer and no changes requested.

@github-actions

github-actions Bot commented Aug 18, 2025

Copy link
Copy Markdown
Contributor Author

PR approved by anyone and no changes requested.

@github-actions

github-actions Bot commented Feb 15, 2026

Copy link
Copy Markdown
Contributor Author

We're closing this PR because it hasn't been updated in a while.
This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!

@github-actions github-actions Bot added the Stale label Feb 15, 2026
@github-actions github-actions Bot closed this Feb 25, 2026
@morningman morningman deleted the auto-pick-54557-branch-3.0 branch July 8, 2026 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed Stale

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hello-stephen @doris-robot @dataroaring @suxiaogang223