Skip to content

branch-3.0: [fix](nereids) fix generate column need add cast in stream load #49167 #50029

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 22, 2025
Merged

branch-3.0: [fix](nereids) fix generate column need add cast in stream load #49167 #50029

merged 1 commit into from
Apr 22, 2025

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #49167

@feiniaofeiafei
[fix](nereids) fix generate column need add cast in stream load (#49167)
30b64f4 
### What problem does this PR solve?

Related PR: #35284

Problem Summary:
Generate column need add cast(casting slot to its own type) in stream
load, because when loading data(stream load and other load), the slots
reading from files are string type. So we need to cast it to its own
type to avoid error.
@github-actions github-actions bot requested a review from dataroaring as a code owner April 14, 2025 09:07
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Apr 14, 2025
@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39910 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 30b64f4ecf636c74665e4e58dc269da043a56ca3, data reload: false

------ Round 1 ----------------------------------
q1	17589	6692	6589	6589
q2	2066	172	166	166
q3	10598	1060	1125	1060
q4	10574	799	721	721
q5	7716	2864	2820	2820
q6	218	134	133	133
q7	958	611	598	598
q8	9349	1942	2040	1942
q9	6580	6358	6404	6358
q10	7054	2299	2309	2299
q11	461	265	259	259
q12	401	220	215	215
q13	17793	2963	2994	2963
q14	238	217	223	217
q15	516	468	455	455
q16	650	592	587	587
q17	958	538	550	538
q18	7080	6706	6628	6628
q19	1406	999	1032	999
q20	464	221	207	207
q21	4003	3275	3162	3162
q22	1113	994	996	994
Total cold run time: 107785 ms
Total hot run time: 39910 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6607	6561	6596	6561
q2	328	231	229	229
q3	2853	2823	2874	2823
q4	2176	1843	1837	1837
q5	5882	5850	5836	5836
q6	214	131	129	129
q7	2204	1869	1844	1844
q8	3403	3603	3615	3603
q9	8865	8943	8908	8908
q10	3581	3541	3516	3516
q11	613	500	499	499
q12	814	629	611	611
q13	9608	3186	3142	3142
q14	302	268	285	268
q15	517	470	464	464
q16	690	631	665	631
q17	1821	1616	1577	1577
q18	8161	7648	7636	7636
q19	1685	1408	1551	1408
q20	2063	1810	1867	1810
q21	5417	5402	5262	5262
q22	1130	1081	1050	1050
Total cold run time: 68934 ms
Total hot run time: 59644 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196550 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 30b64f4ecf636c74665e4e58dc269da043a56ca3, data reload: false

query1	1272	906	918	906
query2	6237	1970	1904	1904
query3	10802	4411	4201	4201
query4	61467	29094	23783	23783
query5	5326	462	460	460
query6	434	183	166	166
query7	5434	315	302	302
query8	294	208	213	208
query9	8368	2579	2556	2556
query10	450	261	248	248
query11	17549	15313	15776	15313
query12	160	99	100	99
query13	1409	443	446	443
query14	9902	7264	6488	6488
query15	197	184	178	178
query16	7132	474	505	474
query17	1180	603	623	603
query18	1844	313	299	299
query19	207	148	148	148
query20	119	104	109	104
query21	204	100	101	100
query22	4566	4494	4365	4365
query23	34860	34001	34791	34001
query24	6116	2842	2906	2842
query25	524	412	396	396
query26	677	168	159	159
query27	2063	339	350	339
query28	4461	2459	2464	2459
query29	703	451	432	432
query30	241	164	170	164
query31	1007	824	849	824
query32	71	56	60	56
query33	424	299	315	299
query34	908	511	523	511
query35	844	745	735	735
query36	1116	955	938	938
query37	119	78	67	67
query38	4021	4027	3918	3918
query39	1729	1477	1458	1458
query40	219	102	103	102
query41	51	53	51	51
query42	117	106	102	102
query43	519	496	492	492
query44	1190	807	802	802
query45	184	168	173	168
query46	1137	732	741	732
query47	2015	1915	1956	1915
query48	499	399	388	388
query49	790	412	434	412
query50	879	419	420	419
query51	7280	7298	7329	7298
query52	105	93	91	91
query53	261	185	185	185
query54	575	479	484	479
query55	85	81	82	81
query56	270	270	268	268
query57	1252	1142	1102	1102
query58	223	211	217	211
query59	3075	2846	2984	2846
query60	277	255	255	255
query61	137	107	107	107
query62	757	684	674	674
query63	211	183	184	183
query64	1363	667	661	661
query65	3282	3225	3199	3199
query66	722	300	297	297
query67	16053	15742	15506	15506
query68	4288	581	566	566
query69	442	261	276	261
query70	1175	1092	1086	1086
query71	340	259	255	255
query72	6119	4001	4022	4001
query73	745	344	358	344
query74	10191	8997	9315	8997
query75	3344	2656	2691	2656
query76	2078	1109	1016	1016
query77	535	271	260	260
query78	10516	9863	9729	9729
query79	1854	594	585	585
query80	1327	414	427	414
query81	516	243	251	243
query82	1054	92	85	85
query83	155	147	147	147
query84	288	81	82	81
query85	1050	304	296	296
query86	349	302	299	299
query87	4479	4302	4251	4251
query88	3901	2376	2347	2347
query89	406	288	283	283
query90	2026	183	184	183
query91	179	151	150	150
query92	66	49	51	49
query93	2475	568	557	557
query94	809	292	285	285
query95	361	254	252	252
query96	601	279	286	279
query97	3320	3136	3197	3136
query98	210	214	207	207
query99	1597	1278	1307	1278
Total cold run time: 314761 ms
Total hot run time: 196550 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.35 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 30b64f4ecf636c74665e4e58dc269da043a56ca3, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.02	0.03
query3	0.23	0.06	0.07
query4	1.62	0.10	0.10
query5	0.53	0.51	0.53
query6	1.16	0.74	0.74
query7	0.02	0.01	0.02
query8	0.04	0.04	0.03
query9	0.56	0.50	0.51
query10	0.55	0.56	0.55
query11	0.15	0.10	0.10
query12	0.14	0.12	0.13
query13	0.61	0.60	0.60
query14	2.75	2.75	2.86
query15	0.90	0.84	0.84
query16	0.38	0.38	0.38
query17	1.07	1.05	1.04
query18	0.24	0.23	0.22
query19	1.93	1.84	1.97
query20	0.01	0.01	0.01
query21	15.35	0.61	0.57
query22	2.99	2.77	2.44
query23	16.95	0.98	0.80
query24	2.98	0.51	1.60
query25	0.15	0.08	0.11
query26	0.55	0.15	0.14
query27	0.07	0.05	0.03
query28	10.53	0.49	0.46
query29	12.58	3.30	3.28
query30	0.24	0.06	0.05
query31	2.87	0.40	0.39
query32	3.25	0.46	0.46
query33	3.05	2.99	3.05
query34	17.23	4.56	4.44
query35	4.51	4.48	4.57
query36	0.69	0.48	0.50
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.07	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.45 s
Total hot run time: 32.35 s

dataroaring
dataroaring approved these changes Apr 22, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor Author

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 22, 2025
@github-actions GitHub Actions
Copy link
Contributor Author

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit ae32003 into branch-3.0 Apr 22, 2025
23 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-49167-branch-3.0 branch April 22, 2025 03:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@hello-stephen @doris-robot @dataroaring @feiniaofeiafei