Replaced std::sort with AVX512-bitonic-sort when creating index #1325
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This change is to add avx512 sort during create Index. Before this change std::sort is used during creating index. Avx512 sort is intruduced by paper "A Novel Hybrid Quicksort Algorithm Vectorized using AVX-512 on Intel Skylake".
Why are the changes needed?
After adopting the change. creating index can speed up from 20% ~ 47% according to different index type. The performance workload is TPCH. The table and creating index steps is as following SQL:
Create database tpch30;
Use tpch30;
create tablegroup if not exists tpch_tg_lineitem_order_group binding true partition by key 1 partitions 48;
create tablegroup if not exists tpch_tg_lineitem_order_group binding true partition by key 1 partitions 48;
drop table if exists lineitem;
create table lineitem (
l_orderkey bigint not null,
l_partkey bigint not null,
l_suppkey bigint not null,
l_linenumber bigint not null,
l_quantity bigint not null,
l_extendedprice bigint not null,
l_discount bigint not null,
l_tax bigint not null,
l_returnflag char(1) default null,
l_linestatus char(1) default null,
l_shipdate date not null,
l_commitdate date default null,
l_receiptdate date default null,
l_shipinstruct char(25) default null,
l_shipmode char(10) default null,
l_comment varchar(44) default null,
primary key(l_orderkey, l_linenumber))
tablegroup = tpch_tg_lineitem_order_group
partition by key (l_orderkey) partitions 48;
load data infile '/mnt/nvme5/liuzhuan/tpch_30G/lineitem.tbl' into table lineitem fields terminated by '|';
create index I_L_ORDERKEY on lineitem(l_orderkey) local;
create index I_L_LINENUMBER on lineitem(l_linenumber) local;
create index I_L_DISCOUNT_QUANTITY on lineitem(l_discount, l_quantity) local;
drop index I_L_ORDERKEY on lineitem;
drop index I_L_LINENUMBER on lineitem;
drop index I_L_DISCOUNT_QUANTITY on lineitem;
The performance is test on Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz and the results is (unit is second):
Will break the compatibility? How if so?
Because the AVX512 is only available on Intel CPU and even since Intel Skylake CPU can support AVX512. Two conditions are added to avoid breaking the compability:
So these two switcheds decide if the AVX512 sort is running.
Does this PR introduce any user-facing change?
No
Dependencies
This PR needs the 3rd party avx512 bitonic sort https://github.com/intel/x86-simd-sort. Since this 3rd party is just C/C++ head file lib, we add the needed header file in deps/libs/x86-simd-sort directory
How was this patch tested?
The patch currently does not run the unit test.
Checklist
Intruduction for this change
The attached PDF summary the changes for this path
OSD_AVX512_Eanbling_README.MD at main · epeshared_OSD_AVX512_Eanbling.pdf