-
Notifications
You must be signed in to change notification settings - Fork 7.9k
Repair subtree2 #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| void BaseDaemon::onInterruptSignals(int signal_id) | ||
| { | ||
| is_cancelled = true; | ||
| LOG_INFO(&logger(), "Received termination signal(" << strsignal(signal_id) << ")"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Нет пробела.
Отображается так:
Received termination signal(Interrupt)
|
Потеряна функциональность быстрого убиения по повторному нажатию Ctrl+C (очень важно). |
|
А такая функциональность была? в демонах только печаталось сообщение "Received second termination signal" |
|
а если вызвать два раза рестарт, то тоже девятка? вообщем стоит обсудить |
Это довольно забавно - такая функциональность была в clickhouse-server, но не было в других демонах Метрики. И сейчас мне её не хватает.
Даже не знаю, как лучше. Можно так же - это проще и понятнее, а можно отличить сигналы INT и TERM. |
|
Засчет чего она достигалась. я не могу найти код, который бы это делал |
|
За счёт того, что уже не ждём сигнал, и для него нет обработчика. |
…ngCloud' CLICKHOUSE-1 Support distributed ddl for qingcloud ClickHouse#1 See merge request !1
CI: https://s3.amazonaws.com/clickhouse-test-reports/75848/479ca9b0a40fa32074501bea7d9eb7d369f3ddd4/integration_tests__tsan__2_6_.html <details> <summary>Race</summary> ``` E Exception: Sanitizer assert found for instance ================== E WARNING: ThreadSanitizer: data race (pid=8) E Write of size 8 at 0x726000005c78 by thread T585 (mutexes: write M0, write M1): E #0 std::__1::enable_if<is_move_constructible<DB::SnapshotFileInfo*>::value && is_move_assignable<DB::SnapshotFileInfo*>::value, void>::type std::__1::swap[abi:ne180100]<DB::SnapshotFileInfo*>(DB::SnapshotFileInfo*&, DB::SnapshotFileInfo*&) build_docker/./contrib/llvm-project/libcxx/include/__utility/swap.h:44:7 (clickhouse+0x1dad0fb6) (BuildId: eb398fb9d76a60d418ba7222d10f730fae2208dc) E ClickHouse#1 std::__1::shared_ptr<DB::SnapshotFileInfo>::swap[abi:ne180100](std::__1::shared_ptr<DB::SnapshotFileInfo>&) build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:694:5 (clickhouse+0x1dad0fb6) E ClickHouse#2 std::__1::shared_ptr<DB::SnapshotFileInfo>::operator=[abi:ne180100](std::__1::shared_ptr<DB::SnapshotFileInfo>&&) build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:663:32 (clickhouse+0x1dad0fb6) E ClickHouse#3 DB::KeeperStateMachine<DB::KeeperStorage<DB::SnapshotableHashTable<DB::KeeperMemNode>>>::save_logical_snp_obj(nuraft::snapshot&, unsigned long&, nuraft::buffer&, bool, bool) build_docker/./src/Coordination/KeeperStateMachine.cpp:846:30 (clickhouse+0x1dad0fb6) E Previous read of size 8 at 0x726000005c78 by thread T572: E #0 std::__1::shared_ptr<DB::SnapshotFileInfo>::shared_ptr[abi:ne180100](std::__1::shared_ptr<DB::SnapshotFileInfo> const&) build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:565:82 (clickhouse+0x1daea9b2) (BuildId: eb398fb9d76a60d418ba7222d10f730fae2208dc) E ClickHouse#1 DB::KeeperStateMachine<DB::KeeperStorage<DB::SnapshotableHashTable<DB::KeeperMemNode>>>::create_snapshot(nuraft::snapshot&, std::__1::function<void (bool&, std::__1::shared_ptr<std::exception>&)>&)::'lambda'(std::__1::variant<std::__1::shared_ptr<DB::KeeperStorageSnapshot<DB::KeeperStorage<DB::SnapshotableHashTable<DB::KeeperMemNode>>>>, std::__1::shared_ptr<DB::KeeperStorageSnapshot<DB::KeeperStorage<DB::RocksDBContainer<DB::KeeperRocksNode>>>>>&&, bool)::operator()(std::__1::variant<std::__1::shared_ptr<DB::KeeperStorageSnapshot<DB::KeeperStorage<DB::SnapshotableHashTable<DB::KeeperMemNode>>>>, std::__1::shared_ptr<DB::KeeperStorageSnapshot<DB::KeeperStorage<DB::RocksDBContainer<DB::KeeperRocksNode>>>>>&&, bool) const build_docker/./src/Coordination/KeeperStateMachine.cpp:803:22 (clickhouse+0x1daea9b2) ``` </details>
CI: https://s3.amazonaws.com/clickhouse-test-reports/75848/479ca9b0a40fa32074501bea7d9eb7d369f3ddd4/integration_tests__tsan__2_6_.html <details> <summary>Race</summary> ``` E Exception: Sanitizer assert found for instance ================== E WARNING: ThreadSanitizer: data race (pid=8) E Write of size 8 at 0x726000005c78 by thread T585 (mutexes: write M0, write M1): E #0 std::__1::enable_if<is_move_constructible<DB::SnapshotFileInfo*>::value && is_move_assignable<DB::SnapshotFileInfo*>::value, void>::type std::__1::swap[abi:ne180100]<DB::SnapshotFileInfo*>(DB::SnapshotFileInfo*&, DB::SnapshotFileInfo*&) build_docker/./contrib/llvm-project/libcxx/include/__utility/swap.h:44:7 (clickhouse+0x1dad0fb6) (BuildId: eb398fb9d76a60d418ba7222d10f730fae2208dc) E ClickHouse#1 std::__1::shared_ptr<DB::SnapshotFileInfo>::swap[abi:ne180100](std::__1::shared_ptr<DB::SnapshotFileInfo>&) build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:694:5 (clickhouse+0x1dad0fb6) E ClickHouse#2 std::__1::shared_ptr<DB::SnapshotFileInfo>::operator=[abi:ne180100](std::__1::shared_ptr<DB::SnapshotFileInfo>&&) build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:663:32 (clickhouse+0x1dad0fb6) E ClickHouse#3 DB::KeeperStateMachine<DB::KeeperStorage<DB::SnapshotableHashTable<DB::KeeperMemNode>>>::save_logical_snp_obj(nuraft::snapshot&, unsigned long&, nuraft::buffer&, bool, bool) build_docker/./src/Coordination/KeeperStateMachine.cpp:846:30 (clickhouse+0x1dad0fb6) E Previous read of size 8 at 0x726000005c78 by thread T572: E #0 std::__1::shared_ptr<DB::SnapshotFileInfo>::shared_ptr[abi:ne180100](std::__1::shared_ptr<DB::SnapshotFileInfo> const&) build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:565:82 (clickhouse+0x1daea9b2) (BuildId: eb398fb9d76a60d418ba7222d10f730fae2208dc) E ClickHouse#1 DB::KeeperStateMachine<DB::KeeperStorage<DB::SnapshotableHashTable<DB::KeeperMemNode>>>::create_snapshot(nuraft::snapshot&, std::__1::function<void (bool&, std::__1::shared_ptr<std::exception>&)>&)::'lambda'(std::__1::variant<std::__1::shared_ptr<DB::KeeperStorageSnapshot<DB::KeeperStorage<DB::SnapshotableHashTable<DB::KeeperMemNode>>>>, std::__1::shared_ptr<DB::KeeperStorageSnapshot<DB::KeeperStorage<DB::RocksDBContainer<DB::KeeperRocksNode>>>>>&&, bool)::operator()(std::__1::variant<std::__1::shared_ptr<DB::KeeperStorageSnapshot<DB::KeeperStorage<DB::SnapshotableHashTable<DB::KeeperMemNode>>>>, std::__1::shared_ptr<DB::KeeperStorageSnapshot<DB::KeeperStorage<DB::RocksDBContainer<DB::KeeperRocksNode>>>>>&&, bool) const build_docker/./src/Coordination/KeeperStateMachine.cpp:803:22 (clickhouse+0x1daea9b2) ``` </details> v2: guard latest_snapshot_meta and latest_snapshot_buf as well (thanks to @davenger)
CI found [1] data-race:
WARNING: ThreadSanitizer: data race (pid=2462)
Write of size 8 at 0x7250003046d8 by thread T638:
#0 std::__1::unique_ptr<ThreadFromGlobalPoolImpl<true, true>, std::__1::default_delete<ThreadFromGlobalPoolImpl<true, true>>>::reset[abi:ne190107](ThreadFromGlobalPoolImpl<true, true>*) build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:290:20 (clickhouse+0x1b7cb690) (BuildId: 4a160c0aae7301fdf3b5d5fc2cec3707e5318bfa)
ClickHouse#1 std::__1::unique_ptr<ThreadFromGlobalPoolImpl<true, true>, std::__1::default_delete<ThreadFromGlobalPoolImpl<true, true>>>::operator=[abi:ne190107](std::__1::unique_ptr<ThreadFromGlobalPoolImpl<true, true>, std::__1::default_delete<ThreadFromGlobalPoolImpl<true, true>>>&&) build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:232:5 (clickhouse+0x1b7cb690)
ClickHouse#2 DB::ObjectStorageQueueMetadata::startup() build_docker/./src/Storages/ObjectStorageQueue/ObjectStorageQueueMetadata.cpp:173:32 (clickhouse+0x1b7cb690)
ClickHouse#3 DB::StorageObjectStorageQueue::startup() build_docker/./src/Storages/ObjectStorageQueue/StorageObjectStorageQueue.cpp:269:25 (clickhouse+0x1b89f8d6) (BuildId: 4a160c0aae7301fdf3b5d5fc2cec3707e5318bfa)
Previous read of size 8 at 0x7250003046d8 by thread T659:
#0 std::__1::unique_ptr<ThreadFromGlobalPoolImpl<true, true>, std::__1::default_delete<ThreadFromGlobalPoolImpl<true, true>>>::operator bool[abi:ne190107]() const build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:279:19 (clickhouse+0x1b7cb4b8) (BuildId: 4a160c0aae7301fdf3b5d5fc2cec3707e5318bfa)
ClickHouse#1 DB::ObjectStorageQueueMetadata::startup() build_docker/./src/Storages/ObjectStorageQueue/ObjectStorageQueueMetadata.cpp:172:10 (clickhouse+0x1b7cb4b8)
ClickHouse#2 DB::StorageObjectStorageQueue::startup() build_docker/./src/Storages/ObjectStorageQueue/StorageObjectStorageQueue.cpp:269:25 (clickhouse+0x1b89f8d6) (BuildId: 4a160c0aae7301fdf3b5d5fc2cec3707e5318bfa)
[1]: https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=77841&sha=42886968f1af877d77394c74cf07e0bf9dcbda45&name_0=PR&name_1=Integration+tests+%28tsan%2C+6%2F6%29
The reason is pretty obvious, the tables in test_registry shares the
same metadata object and on server restart may try to initialize it
twice, that leads to a data-race above.
MSan reports:
Uninitialized value was stored to memory at
#0 0xabcb3f7bdd40 in ReservoirSamplerDeterministic<char8_t, (ReservoirSamplerDeterministicOnEmpty)1>::write(DB::WriteBuffer&) const .build-msan/./src/AggregateFunctions/ReservoirSamplerDeterministic.h:198:18
ClickHouse#1 0xabcb3f7bdd40 in DB::(anonymous namespace)::QuantileReservoirSamplerDeterministic<char8_t>::serialize(DB::WriteBuffer&) const .build-msan/./src/AggregateFunctions/AggregateFunctionQuantileDeterministic.cpp:57:14
ClickHouse#2 0xabcb3f7bdd40 in DB::AggregateFunctionQuantile<char8_t, DB::(anonymous namespace)::QuantileReservoirSamplerDeterministic<char8_t>, DB::NameQuantileDeterministic, true, double, false, false>::serialize(char const*, DB::WriteBuffer&, std::__1::optional<unsigned long>) const .build-msan/./src/AggregateFunctions/AggregateFunctionQuantile.h:242:57
ClickHouse#3 0xabcb4ae6e754 in DB::serializeToString(std::__1::shared_ptr<DB::IAggregateFunction const> const&, DB::IColumn const&, unsigned long, unsigned long) .build-msan/./src/DataTypes/Serializations/SerializationAggregateFunction.cpp:102:15
ClickHouse#4 0xabcb4ae6e47c in DB::SerializationAggregateFunction::serializeText(DB::IColumn const&, unsigned long, DB::WriteBuffer&, DB::FormatSettings const&) const .build-msan/./src/DataTypes/Serializations/SerializationAggregateFunction.cpp:132:17
Uninitialized value was stored to memory at
#0 0xabcb3f7bdd40 in ReservoirSamplerDeterministic<char8_t, (ReservoirSamplerDeterministicOnEmpty)1>::write(DB::WriteBuffer&) const .build-msan/./src/AggregateFunctions/ReservoirSamplerDeterministic.h:198:18
ClickHouse#1 0xabcb3f7bdd40 in DB::(anonymous namespace)::QuantileReservoirSamplerDeterministic<char8_t>::serialize(DB::WriteBuffer&) const .build-msan/./src/AggregateFunctions/AggregateFunctionQuantileDeterministic.cpp:57:14
ClickHouse#2 0xabcb3f7bdd40 in DB::AggregateFunctionQuantile<char8_t, DB::(anonymous namespace)::QuantileReservoirSamplerDeterministic<char8_t>, DB::NameQuantileDeterministic, true, double, false, false>::serialize(char const*, DB::WriteBuffer&, std::__1::optional<unsigned long>) const .build-msan/./src/AggregateFunctions/AggregateFunctionQuantile.h:242:57
ClickHouse#3 0xabcb4ae6e754 in DB::serializeToString(std::__1::shared_ptr<DB::IAggregateFunction const> const&, DB::IColumn const&, unsigned long, unsigned long) .build-msan/./src/DataTypes/Serializations/SerializationAggregateFunction.cpp:102:15
ClickHouse#4 0xabcb4ae6e47c in DB::SerializationAggregateFunction::serializeText(DB::IColumn const&, unsigned long, DB::WriteBuffer&, DB::FormatSettings const&) const .build-msan/./src/DataTypes/Serializations/SerializationAggregateFunction.cpp:132:17
The problem is the assigment of the whole pair.
Fixes: ClickHouse#80862
Complete report: https://pastila.clickhouse.com/?0005f20c/c8363f3b9a4205781d88c2f3fe4a0841#19owb1ROpa4VkIUDulFBpw==
Found on CI [1]:
<details>
<summary>stacktrace</summary>
WARNING: ThreadSanitizer: data race (pid=328014)
Write of size 8 at 0x729800008140 by main thread:
0 __tsan_memcpy <null> (clickhouse+0x8b061be) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
1 std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::operator=(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./contrib/llvm-project/libcxx/include/string:2658:22 (clickhouse+0x191548c3) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
2 DB::SettingFieldTimezone::operator=(DB::SettingFieldTimezone const&) ci/tmp/build/./src/Core/SettingsFields.h:468:8 (clickhouse+0x191548c3)
3 DB::SettingsTraits::Data::operator=(DB::SettingsTraits::Data const&) ci/tmp/build/./src/Core/Settings.cpp:6957:1 (clickhouse+0x191548c3)
4 DB::BaseSettings<DB::SettingsTraits>::operator=(DB::BaseSettings<DB::SettingsTraits> const&) ci/tmp/build/./src/Core/BaseSettings.h:109:60 (clickhouse+0x18f98aec) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
5 DB::SettingsImpl::operator=(DB::SettingsImpl const&) ci/tmp/build/./src/Core/Settings.cpp:6963:8 (clickhouse+0x18f98aec)
6 DB::Settings::operator=(DB::Settings const&) ci/tmp/build/./src/Core/Settings.cpp:7170:11 (clickhouse+0x18f98aec)
7 DB::Context::setSettings(DB::Settings const&) ci/tmp/build/./src/Interpreters/Context.cpp:2618:15 (clickhouse+0x1a6aac4a) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
8 DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long)::$_0::operator()() const ci/tmp/build/./src/Client/ClientBase.cpp:2180:9 (clickhouse+0x1db11597) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
9 BasicScopeGuard<DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long)::$_0>::invoke() ci/tmp/build/./base/base/../base/scope_guard.h:101:9 (clickhouse+0x1db11597)
10 BasicScopeGuard<DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long)::$_0>::~BasicScopeGuard() ci/tmp/build/./base/base/../base/scope_guard.h:50:26 (clickhouse+0x1db11597)
11 DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long) ci/tmp/build/./src/Client/ClientBase.cpp:2245:5 (clickhouse+0x1db0418a) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
12 DB::ClientBase::executeMultiQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./src/Client/ClientBase.cpp:2618:21 (clickhouse+0x1db12b5e) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
13 DB::ClientBase::processQueryText(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./src/Client/ClientBase.cpp:2812:12 (clickhouse+0x1db142a6) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
14 DB::ClientBase::runInteractive() ci/tmp/build/./src/Client/ClientBase.cpp:3367:18 (clickhouse+0x1db1d57b) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
15 DB::Client::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&) ci/tmp/build/./programs/client/Client.cpp:399:9 (clickhouse+0x128ba9c1) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
16 non-virtual thunk to DB::Client::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&) ci/tmp/build/./programs/client/Client.cpp (clickhouse+0x128bb0a0) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
17 Poco::Util::Application::run() ci/tmp/build/./base/poco/Util/src/Application.cpp:315:8 (clickhouse+0x23017cbe) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
18 mainEntryClickHouseClient(int, char**) ci/tmp/build/./programs/client/Client.cpp:1139:23 (clickhouse+0x128c8ea6) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
19 main ci/tmp/build/./programs/main.cpp:323:21 (clickhouse+0x8b8ea04) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
Previous read of size 8 at 0x729800008140 by thread T2:
#0 __tsan_memcpy <null> (clickhouse+0x8b061be) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
ClickHouse#1 std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::operator=(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./contrib/llvm-project/libcxx/include/string:2658:22 (clickhouse+0x1259a1c7) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
ClickHouse#2 DateLUT::instance() ci/tmp/build/./src/Common/DateLUT.cpp:173:35 (clickhouse+0x1259a1c7)
ClickHouse#3 DateLUT::instance(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./src/Common/DateLUT.h:29:20 (clickhouse+0x192bbab2) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
ClickHouse#4 TimezoneMixin::TimezoneMixin(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./src/DataTypes/TimezoneMixin.h:16:21 (clickhouse+0x192bbab2)
ClickHouse#5 DB::DataTypeDateTime::DataTypeDateTime(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./src/DataTypes/DataTypeDateTime.cpp:13:7 (clickhouse+0x192bb645) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
6 std::__1::shared_ptr<DB::DataTypeDateTime> std::__1::allocate_shared[abi:ne190107]<DB::DataTypeDateTime, std::__1::allocator<DB::DataTypeDateTime>, 0>(std::__1::allocator<DB::DataTypeDateTime> const&) FunctionsExternalDictionaries.cpp (clickhouse+0x8ddfc9b) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
7 std::__1::shared_ptr<DB::DataTypeDateTime> std::__1::make_shared[abi:ne190107]<DB::DataTypeDateTime, 0>() ci/tmp/build/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:851:10 (clickhouse+0x193bbb51) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
8 DB::create(std::__1::shared_ptr<DB::IAST> const&) ci/tmp/build/./src/DataTypes/registerDataTypeDateTime.cpp:66:16 (clickhouse+0x193bbb51)
9 decltype(std::declval<std::__1::shared_ptr<DB::IDataType const> (*&)(std::__1::shared_ptr<DB::IAST> const&)>()(std::declval<std::__1::shared_ptr<DB::IAST> const&>())) std::__1::__invoke[abi:ne190107]<std::__1::shared_ptr<DB::IDataType const> (*&)(std::__1::shared_ptr<DB::IAST> const&), std::__1::shared_ptr<DB::IAST> const&>(std::__1::shared_ptr<DB::IDataType const> (*&)(std::__1::shared_ptr<DB::IAST> const&), std::__1::shared_ptr<DB::IAST> const&) ci/tmp/build/./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149:25 (clickhouse+0x192b5ea3) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
10 std::__1::shared_ptr<DB::IDataType const> std::__1::__invoke_void_return_wrapper<std::__1::shared_ptr<DB::IDataType const>, false>::__call[abi:ne190107]<std::__1::shared_ptr<DB::IDataType const> (*&)(std::__1::shared_ptr<DB::IAST> const&), std::__1::shared_ptr<DB::IAST> const&>(std::__1::shared_ptr<DB::IDataType const> (*&)(std::__1::shared_ptr<DB::IAST> const&), std::__1::shared_ptr<DB::IAST> const&) ci/tmp/build/./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:216:12 (clickhouse+0x192b5ea3)
11 std::__1::__function::__default_alloc_func<std::__1::shared_ptr<DB::IDataType const> (*)(std::__1::shared_ptr<DB::IAST> const&), std::__1::shared_ptr<DB::IDataType const> (std::__1::shared_ptr<DB::IAST> const&)>::operator()[abi:ne190107](std::__1::shared_ptr<DB::IAST> const&) ci/tmp/build/./contrib/llvm-project/libcxx/include/__functional/function.h:210:12 (clickhouse+0x192b5ea3)
12 std::__1::shared_ptr<DB::IDataType const> std::__1::__function::__policy_invoker<std::__1::shared_ptr<DB::IDataType const> (std::__1::shared_ptr<DB::IAST> const&)>::__call_impl[abi:ne190107]<std::__1::__function::__default_alloc_func<std::__1::shared_ptr<DB::IDataType const> (*)(std::__1::shared_ptr<DB::IAST> const&), std::__1::shared_ptr<DB::IDataType const> (std::__1::shared_ptr<DB::IAST> const&)>>(std::__1::__function::__policy_storage const*, std::__1::shared_ptr<DB::IAST> const&) ci/tmp/build/./contrib/llvm-project/libcxx/include/__functional/function.h:610:12 (clickhouse+0x192b5ea3)
13 std::__1::__function::__policy_func<std::__1::shared_ptr<DB::IDataType const> (std::__1::shared_ptr<DB::IAST> const&)>::operator()[abi:ne190107](std::__1::shared_ptr<DB::IAST> const&) const ci/tmp/build/./contrib/llvm-project/libcxx/include/__functional/function.h:716:12 (clickhouse+0x192cfcc6) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
14 std::__1::function<std::__1::shared_ptr<DB::IDataType const> (std::__1::shared_ptr<DB::IAST> const&)>::operator()(std::__1::shared_ptr<DB::IAST> const&) const ci/tmp/build/./contrib/llvm-project/libcxx/include/__functional/function.h:989:10 (clickhouse+0x192cfcc6)
15 std::__1::shared_ptr<DB::IDataType const> DB::DataTypeFactory::getImpl<false>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::IAST> const&) const ci/tmp/build/./src/DataTypes/DataTypeFactory.cpp:144:21 (clickhouse+0x192cfcc6)
16 std::__1::shared_ptr<DB::IDataType const> DB::DataTypeFactory::getImpl<false>(std::__1::shared_ptr<DB::IAST> const&) const ci/tmp/build/./src/DataTypes/DataTypeFactory.cpp:91:16 (clickhouse+0x192cf5f3) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
17 std::__1::shared_ptr<DB::IDataType const> DB::DataTypeFactory::getImpl<false>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const ci/tmp/build/./src/DataTypes/DataTypeFactory.cpp:73:12 (clickhouse+0x192cf18d) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
18 DB::DataTypeFactory::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const ci/tmp/build/./src/DataTypes/DataTypeFactory.cpp:35:12 (clickhouse+0x192ccfa0) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
19 DB::NativeReader::read() ci/tmp/build/./src/Formats/NativeReader.cpp:188:45 (clickhouse+0x1dde11c2) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
20 DB::Connection::receiveDataImpl(DB::NativeReader&) ci/tmp/build/./src/Client/Connection.cpp:1396:24 (clickhouse+0x1db4c3c9) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
21 DB::Connection::receiveProfileEvents() ci/tmp/build/./src/Client/Connection.cpp:1408:12 (clickhouse+0x1db4b471) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
22 DB::Connection::receivePacket() ci/tmp/build/./src/Client/Connection.cpp:1345:29 (clickhouse+0x1db4b471)
23 DB::Suggest::fetch(DB::IServerConnection&, DB::ConnectionTimeouts const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, DB::ClientInfo const&) ci/tmp/build/./src/Client/Suggest.cpp:170:36 (clickhouse+0x1dbbfce0) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
24 void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()::operator()() const ci/tmp/build/./src/Client/Suggest.cpp:103:17 (clickhouse+0x1dbc236e) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
25 decltype(std::declval<void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()>()()) std::__1::__invoke[abi:ne190107]<void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()>(void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()&&) ci/tmp/build/./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149:25 (clickhouse+0x1dbc2038) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
26 void std::__1::__thread_execute[abi:ne190107]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()>&, std::__1::__tuple_indices<...>) ci/tmp/build/./contrib/llvm-project/libcxx/include/__thread/thread.h:192:3 (clickhouse+0x1dbc2038)
27 void* std::__1::__thread_proxy[abi:ne190107]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()>>(void*) ci/tmp/build/./contrib/llvm-project/libcxx/include/__thread/thread.h:201:3 (clickhouse+0x1dbc2038)
</details>
What is interesting that it has been pops up only after ClickHouse#81759
[1]: https://s3.amazonaws.com/clickhouse-test-reports/PRs/82168/8ab5ff7cd742626a263fd7a262d4ff86aa28880d//stateless_tests_amd_tsan_s3_storage_3_3/job.log
…path)
This time I tested it with sanitizers:
$ ../tests/clickhouse-test shebang
Using queries from '/src/ch/clickhouse/tests/queries' directory
Connecting to ClickHouse server... OK
Connected to server 25.7.1.1 @ 9042a93 master
Found 1 parallel tests and 0 sequential tests
Running about 1 stateless tests (Process-3).
02203_shebang: [ FAIL ] 0.33 sec.
Reason: having stderror:
Path: /src/ch/clickhouse/tests/queries/0_stateless/test_ggjjtc06/clickhouse-351von_4.10890
==================
WARNING: ThreadSanitizer: data race (pid=10890)
Write of size 4 at 0x5555569e9718 by thread T2:
#0 thread2_func <null> (test-tsan+0xe7f68)
Previous write of size 4 at 0x5555569e9718 by thread T1:
#0 thread1_func <null> (test-tsan+0xe7f28)
Location is global 'shared_var' of size 4 at 0x5555569e9718 (test-tsan+0x1495718)
Thread T2 (tid=10893, running) created by main thread at:
#0 pthread_create /src/llvm/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1051:3 (test-tsan+0x640da)
ClickHouse#1 main <null> (test-tsan+0xe7fc5)
Thread T1 (tid=10892, finished) created by main thread at:
#0 pthread_create /src/llvm/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1051:3 (test-tsan+0x640da)
ClickHouse#1 main <null> (test-tsan+0xe7fae)
SUMMARY: ThreadSanitizer: data race (/tmp/test-tsan+0xe7f68) in thread2_func
==================
ThreadSanitizer: reported 1 warnings
stdout:
1
Database: test_ggjjtc06
Having 1 errors! 0 tests passed. 0 tests skipped. 0.34 s elapsed (Process-3).
All tests have finished.
And "Fatal" logs:
$ ../tests/clickhouse-test shebang
Using queries from '/src/ch/clickhouse/tests/queries' directory
Connecting to ClickHouse server... OK
Connected to server 25.7.1.1 @ 9042a93 master
Found 1 parallel tests and 0 sequential tests
Running about 1 stateless tests (Process-3).
02203_shebang: [ FAIL ] 0.33 sec.
Reason: having stderror:
Path: /src/ch/clickhouse/tests/queries/0_stateless/test_grqdqc9f/clickhouse-p8a4uip4
foo
stdout:
1
Having 1 errors! 0 tests passed. 0 tests skipped. 0.34 s elapsed (Process-3).
All tests have finished.
Fixes: ClickHouse#82234
…path)
This time I tested it with sanitizers:
$ ../tests/clickhouse-test shebang
Using queries from '/src/ch/clickhouse/tests/queries' directory
Connecting to ClickHouse server... OK
Connected to server 25.7.1.1 @ 9042a93 master
Found 1 parallel tests and 0 sequential tests
Running about 1 stateless tests (Process-3).
02203_shebang: [ FAIL ] 0.33 sec.
Reason: having stderror:
Path: /src/ch/clickhouse/tests/queries/0_stateless/test_ggjjtc06/clickhouse-351von_4.10890
==================
WARNING: ThreadSanitizer: data race (pid=10890)
Write of size 4 at 0x5555569e9718 by thread T2:
#0 thread2_func <null> (test-tsan+0xe7f68)
Previous write of size 4 at 0x5555569e9718 by thread T1:
#0 thread1_func <null> (test-tsan+0xe7f28)
Location is global 'shared_var' of size 4 at 0x5555569e9718 (test-tsan+0x1495718)
Thread T2 (tid=10893, running) created by main thread at:
#0 pthread_create /src/llvm/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1051:3 (test-tsan+0x640da)
ClickHouse#1 main <null> (test-tsan+0xe7fc5)
Thread T1 (tid=10892, finished) created by main thread at:
#0 pthread_create /src/llvm/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1051:3 (test-tsan+0x640da)
ClickHouse#1 main <null> (test-tsan+0xe7fae)
SUMMARY: ThreadSanitizer: data race (/tmp/test-tsan+0xe7f68) in thread2_func
==================
ThreadSanitizer: reported 1 warnings
stdout:
1
Database: test_ggjjtc06
Having 1 errors! 0 tests passed. 0 tests skipped. 0.34 s elapsed (Process-3).
All tests have finished.
And "Fatal" logs:
$ ../tests/clickhouse-test shebang
Using queries from '/src/ch/clickhouse/tests/queries' directory
Connecting to ClickHouse server... OK
Connected to server 25.7.1.1 @ 9042a93 master
Found 1 parallel tests and 0 sequential tests
Running about 1 stateless tests (Process-3).
02203_shebang: [ FAIL ] 0.33 sec.
Reason: having stderror:
Path: /src/ch/clickhouse/tests/queries/0_stateless/test_grqdqc9f/clickhouse-p8a4uip4
foo
stdout:
1
Having 1 errors! 0 tests passed. 0 tests skipped. 0.34 s elapsed (Process-3).
All tests have finished.
Fixes: ClickHouse#82234
Otherwise it may lead to data-race:
WARNING: ThreadSanitizer: data race (pid=26127)
Write of size 6 at 0x7298000099e8 by main thread:
0 __tsan_memcpy <null> (clickhouse-82444-tsan+0x8be71be) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
1 DB::SettingsTraits::Data::operator=(DB::SettingsTraits::Data const&) ci/tmp/build/./src/Core/Settings.cpp:6978:1 (clickhouse-82444-tsan+0x192ae4f9) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
2 DB::BaseSettings<DB::SettingsTraits>::operator=(DB::BaseSettings<DB::SettingsTraits> const&) ci/tmp/build/./src/Core/BaseSettings.h:109:60 (clickhouse-82444-tsan+0x190f2bcc) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
3 DB::SettingsImpl::operator=(DB::SettingsImpl const&) ci/tmp/build/./src/Core/Settings.cpp:6984:8 (clickhouse-82444-tsan+0x190f2bcc)
4 DB::Settings::operator=(DB::Settings const&) ci/tmp/build/./src/Core/Settings.cpp:7191:11 (clickhouse-82444-tsan+0x190f2bcc)
5 DB::Context::setSettings(DB::Settings const&) ci/tmp/build/./src/Interpreters/Context.cpp:2635:15 (clickhouse-82444-tsan+0x1aa94c0a) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
6 DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long)::$_0::operator()() const ci/tmp/build/./src/Client/ClientBase.cpp:2184:9 (clickhouse-82444-tsan+0x1df0f4d7) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
7 BasicScopeGuard<DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long)::$_0>::invoke() ci/tmp/build/./base/base/../base/scope_guard.h:101:9 (clickhouse-82444-tsan+0x1df0f4d7)
8 BasicScopeGuard<DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long)::$_0>::~BasicScopeGuard() ci/tmp/build/./base/base/../base/scope_guard.h:50:26 (clickhouse-82444-tsan+0x1df0f4d7)
9 DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long) ci/tmp/build/./src/Client/ClientBase.cpp:2249:5 (clickhouse-82444-tsan+0x1df021ca) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
Previous read of size 1 at 0x7298000099e8 by thread T4:
#0 DB::SettingFieldNumber<bool>::operator bool() const ci/tmp/build/./src/Core/SettingsFields.h:40:36 (clickhouse-82444-tsan+0x1b114e5f) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
ClickHouse#1 DB::ThreadStatus::finalizePerformanceCounters() ci/tmp/build/./src/Interpreters/ThreadStatusExt.cpp:547:17 (clickhouse-82444-tsan+0x1b114e5f)
ClickHouse#2 DB::ThreadStatus::detachFromGroup() ci/tmp/build/./src/Interpreters/ThreadStatusExt.cpp:375:5 (clickhouse-82444-tsan+0x1b113aec) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
ClickHouse#3 DB::CurrentThread::detachFromGroupIfNotDetached() ci/tmp/build/./src/Interpreters/ThreadStatusExt.cpp:716:21 (clickhouse-82444-tsan+0x1b1119b3) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
ClickHouse#4 DB::ThreadGroupSwitcher::~ThreadGroupSwitcher() ci/tmp/build/./src/Interpreters/ThreadStatusExt.cpp:261:9 (clickhouse-82444-tsan+0x1b1119b3)
ClickHouse#5 DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>, std::__1::function<void ()>>::operator()(std::__1::function<void ()>&&, Priority, std::__1::optional<unsigned long>)::'lambda'()::operator()() ci/tmp/build/./src/Common/threadPoolCallbackRunner.h:179:9 (clickhouse-82444-tsan+0x14beafde) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
Thread T4 'ThreadPool' (tid=26133, running) created by thread T3 at:
...
12 DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>, std::__1::function<void ()>>::operator()(std::__1::function<void ()>&&, Priority, std::__1::optional<unsigned long>) ci/tmp/build/./src/Common/threadPoolCallbackRunner.h:188:22 (clickhouse-82444-tsan+0x14be522d) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
13 DB::ParallelParsingInputFormat::scheduleParserThreadForUnitWithNumber(unsigned long) ci/tmp/build/./src/Processors/Formats/Impl/ParallelParsingInputFormat.h:287:9 (clickhouse-82444-tsan+0x1e495627) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
14 DB::ParallelParsingInputFormat::segmentatorThreadFunction(std::__1::shared_ptr<DB::ThreadGroup>) ci/tmp/build/./src/Processors/Formats/Impl/ParallelParsingInputFormat.cpp:45:13 (clickhouse-82444-tsan+0x1e495627)
Refs: https://pastila.nl/?0001fdef/9a58e9d59c32d45100a481de26dccf68#T4iNrMFUnu4F2hWST5wLdQ==
==13==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 112 byte(s) in 1 object(s) allocated from:
#0 0x562de572644f in malloc (/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/unit_tests_dbms+0xb7c744f) (BuildId: ece9c88fdfe6a6e98d1a0ed8ab535e80c258c855)
ClickHouse#1 0x562e12f3f8d7 in get_testing_kernel_expression (/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/unit_tests_dbms+0x38fe08d7) (BuildId: ece9c88fdfe6a6e98d1a0ed8ab535e80c258c855)
CI: https://s3.amazonaws.com/clickhouse-test-reports/PRs/84084/a27e4fcb365a35641c75fc886c90dafc8418b812//unit_tests_asan/job.log
Add promql parser #1: Add contrib and build scripts
Fix backup path detection in backupview
…y names When ClickHouse encounters a distributed table directory with a name that: - starts with the pattern `shardN_all_replicas` (e.g., `shard3_all_replicas_bkp`) - but doesn't end with exactly `_all_replicas` It would crash with a segmentation fault due to out-of-bounds array access. Trace example: ``` * thread ClickHouse#1, name = 'clickhouse-serv', stop reason = signal SIGSEGV * frame #0: 0x000000001194481b clickhouse-server`DB::Cluster::Address::Address(DB::Cluster::Address const&) + 27 frame ClickHouse#1: 0x0000000011944550 clickhouse-server`void std::__1::vector<DB::Cluster::Address, std::__1::allocator<DB::Cluster::Address> >::__push_back_slow_path<DB::Cluster::Address const&>(DB::Cluster::Address const&&&) + 208 frame ClickHouse#2: 0x00000000129a6c35 clickhouse-server`DB::StorageDistributed::parseAddresses(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const + 3349 frame ClickHouse#3: 0x00000000129a589b clickhouse-server`DB::StorageDistributed::getDirectoryQueue(std::__1::shared_ptr<DB::IDisk> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 3675 frame ClickHouse#4: 0x00000000129a3f22 clickhouse-server`DB::StorageDistributed::initializeDirectoryQueuesForDisk(std::__1::shared_ptr<DB::IDisk> const&) + 2082 frame ClickHouse#5: 0x000000000f681c48 clickhouse-server`DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true> >, std::__1::function<void ()> >::operator()(std::__1::function<void ()>&&, Priority)::'lambda'()::operator()() + 136 frame ClickHouse#6: 0x000000000f68286c clickhouse-server`std::__1::packaged_task<void ()>::operator()() + 44 frame ClickHouse#7: 0x000000000d83335b clickhouse-server`ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true> >::ThreadFromThreadPool::worker() + 2395 frame ClickHouse#8: 0x000000000d839e23 clickhouse-server`void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false, true>::ThreadFromGlobalPoolImpl<void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true> >::ThreadFromThreadPool::*)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true> >::ThreadFromThre adPool*>(void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true> >::ThreadFromThreadPool::*&&)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true> >::ThreadFromThreadPool*&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) + 99 frame ClickHouse#9: 0x000000000d830602 clickhouse-server`ThreadPoolImpl<std::__1::thread>::ThreadFromThreadPool::worker() + 2242 frame ClickHouse#10: 0x000000000d8379ba clickhouse-server`_ZNSt3__114__thread_proxyB6v15007INS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEMN14ThreadPoolImplINS_6threadEE20ThreadFromThreadPoolEFvvEPSA_EEEEEPvSF_ + 90 ```
Docs: move aggregate functions docs to source (#1)
Edited:
In 2016, we tried using git subtree instead of git submodules, but quickly found it difficult to use.
While git submodules is one of the worst examples of developer experience, subtree is even worse.