Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Replaced std::sort with AVX512-bitonic-sort when creating index #1325

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: 3.1
Choose a base branch
from

Conversation

xtangxtang
Copy link
Contributor

@xtangxtang xtangxtang commented Mar 21, 2023

What changes were proposed in this pull request?

This change is to add avx512 sort during create Index. Before this change std::sort is used during creating index. Avx512 sort is intruduced by paper "A Novel Hybrid Quicksort Algorithm Vectorized using AVX-512 on Intel Skylake".

Why are the changes needed?

After adopting the change. creating index can speed up from 20% ~ 47% according to different index type. The performance workload is TPCH. The table and creating index steps is as following SQL:

Create database tpch30;
Use tpch30;

create tablegroup if not exists tpch_tg_lineitem_order_group binding true partition by key 1 partitions 48;
create tablegroup if not exists tpch_tg_lineitem_order_group binding true partition by key 1 partitions 48;

drop table if exists lineitem;
create table lineitem (
l_orderkey bigint not null,
l_partkey bigint not null,
l_suppkey bigint not null,
l_linenumber bigint not null,
l_quantity bigint not null,
l_extendedprice bigint not null,
l_discount bigint not null,
l_tax bigint not null,
l_returnflag char(1) default null,
l_linestatus char(1) default null,
l_shipdate date not null,
l_commitdate date default null,
l_receiptdate date default null,
l_shipinstruct char(25) default null,
l_shipmode char(10) default null,
l_comment varchar(44) default null,
primary key(l_orderkey, l_linenumber))
tablegroup = tpch_tg_lineitem_order_group
partition by key (l_orderkey) partitions 48;

load data infile '/mnt/nvme5/liuzhuan/tpch_30G/lineitem.tbl' into table lineitem fields terminated by '|';

create index I_L_ORDERKEY on lineitem(l_orderkey) local;
create index I_L_LINENUMBER on lineitem(l_linenumber) local;
create index I_L_DISCOUNT_QUANTITY on lineitem(l_discount, l_quantity) local;

drop index I_L_ORDERKEY on lineitem;
drop index I_L_LINENUMBER on lineitem;
drop index I_L_DISCOUNT_QUANTITY on lineitem;

The performance is test on Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz and the results is (unit is second):

image

Will break the compatibility? How if so?

Because the AVX512 is only available on Intel CPU and even since Intel Skylake CPU can support AVX512. Two conditions are added to avoid breaking the compability:

  • ENABLE_AVX512F is a new enviroment in build.sh. The OceanBase will compile the AVX512 code in case ENABLE_AVX512F = ON
  • __builtin_cpu_supports("avx512f"):is added in src/storage/ob_parallel_external_sort.h. It will runtime check if current CPU surpport AVX512.

So these two switcheds decide if the AVX512 sort is running.

image

Does this PR introduce any user-facing change?

No

Dependencies

This PR needs the 3rd party avx512 bitonic sort https://github.com/intel/x86-simd-sort. Since this 3rd party is just C/C++ head file lib, we add the needed header file in deps/libs/x86-simd-sort directory

How was this patch tested?

The patch currently does not run the unit test.

Checklist

  • [X ] I've run the tests to see all new and existing tests pass.
  • [X ] If this Pull Request resolves an issue, I linked to the issue in the text above.
  • [X ] I've informed the technical writer about the documentation change if necessary.

Intruduction for this change

The attached PDF summary the changes for this path
OSD_AVX512_Eanbling_README.MD at main · epeshared_OSD_AVX512_Eanbling.pdf

@xtangxtang xtangxtang changed the title Replaced std::sort with AVX512-bitonic-sort when creating index in oc… Replaced std::sort with AVX512-bitonic-sort when creating index Mar 21, 2023
@hnwyllmm hnwyllmm self-requested a review March 21, 2023 08:16
@CLAassistant
Copy link

CLAassistant commented Mar 21, 2023

CLA assistant check
All committers have signed the CLA.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants