Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Extra include for std::mt19937_64 #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
smagellan wants to merge 2 commits into from
Closed

Extra include for std::mt19937_64 #5

smagellan wants to merge 2 commits into from

Conversation

@smagellan
Copy link

@smagellan smagellan commented Jun 15, 2016

Hi. I was not able to build ClickHouse under debian testing (g++-6). Extra include should fix it.

@smagellan
Copy link
Author

smagellan commented Jun 15, 2016 •
edited
Loading

here is detailed error info.

[ 56%] Building CXX object dbms/CMakeFiles/dbms.dir/src/Dictionaries/ComplexKeyCacheDictionary.cpp.o
In file included from /tmp/clickhouse/ClickHouse/dbms/src/Dictionaries/ComplexKeyCacheDictionary.cpp:1:0:
/tmp/clickhouse/ClickHouse/dbms/include/DB/Dictionaries/ComplexKeyCacheDictionary.h:264:15: error: ‘mt19937_64’ in namespace ‘std’ does not name a type
mutable std::mt19937_64 rnd_engine;
^~~~~~~~~~
/tmp/clickhouse/ClickHouse/dbms/src/Dictionaries/ComplexKeyCacheDictionary.cpp: In constructor ‘DB::ComplexKeyCacheDictionary::ComplexKeyCacheDictionary(const string&, const DB::DictionaryStructure&, DB::DictionarySourcePtr, DB::DictionaryLifetime, std::size_t)’:
/tmp/clickhouse/ClickHouse/dbms/src/Dictionaries/ComplexKeyCacheDictionary.cpp:48:40: error: class ‘DB::ComplexKeyCacheDictionary’ does not have any field named ‘rnd_engine’
size{round_up_to_power_of_two(size)}, rnd_engine{getSeed()}
^~~~~~~~~~
/tmp/clickhouse/ClickHouse/dbms/src/Dictionaries/ComplexKeyCacheDictionary.cpp: In member function ‘void DB::ComplexKeyCacheDictionary::update(const ConstColumnPlainPtrs&, const DB::PODArray&, const std::vector&, PresentKeyHandler&&, AbsentKeyHandler&&) const’:
/tmp/clickhouse/ClickHouse/dbms/src/Dictionaries/ComplexKeyCacheDictionary.cpp:624:92: error: ‘rnd_engine’ was not declared in this scope
cell.setExpiresAt(std::chrono::system_clock::now() + std::chrono::seconds{distribution(rnd_engine)});
^~~~~~~~~~
/tmp/clickhouse/ClickHouse/dbms/src/Dictionaries/ComplexKeyCacheDictionary.cpp:671:91: error: ‘rnd_engine’ was not declared in this scope
cell.setExpiresAt(std::chrono::system_clock::now() + std::chrono::seconds{distribution(rnd_engine)});
^~~~~~~~~~
/tmp/clickhouse/ClickHouse/dbms/src/Dictionaries/ComplexKeyCacheDictionary.cpp: In instantiation of ‘void DB::ComplexKeyCacheDictionary::update(const ConstColumnPlainPtrs&, const DB::PODArray&, const std::vector&, PresentKeyHandler&&, AbsentKeyHandler&&) const [with PresentKeyHandler = DB::ComplexKeyCacheDictionary::has(const ConstColumnPlainPtrs&, const DataTypes&, DB::PaddedPODArray&) const::<lambda(auto:2, auto:3)>; AbsentKeyHandler = DB::ComplexKeyCacheDictionary::has(const ConstColumnPlainPtrs&, const DataTypes&, DB::PaddedPODArray&) const::<lambda(auto:4, auto:5)>; DB::ConstColumnPlainPtrs = std::vector<const DB::IColumn*>]’:
/tmp/clickhouse/ClickHouse/dbms/src/Dictionaries/ComplexKeyCacheDictionary.cpp:247:3: required from here
/tmp/clickhouse/ClickHouse/dbms/src/Dictionaries/ComplexKeyCacheDictionary.cpp:624:56: error: no matching function for call to ‘std::chrono::duration::duration()’

@alexey-milovidov
Copy link
Member

alexey-milovidov commented Jun 15, 2016

Thank you!
But fix was already committed just now: ffb1672

@Nopius Nopius mentioned this pull request Dec 12, 2016
Closed
@jarwin123 jarwin123 mentioned this pull request Feb 15, 2017
Closed
@slw slw mentioned this pull request Nov 9, 2017
Closed
@aavdonkin aavdonkin mentioned this pull request Dec 14, 2017
Merged
alexey-milovidov pushed a commit that referenced this pull request Feb 9, 2019
@nikvas0
Merge pull request #5 from nikvas0/nikvas0/minmax
0536711 
Nikvas0/minmax
azat added a commit to azat/ClickHouse that referenced this pull request Feb 22, 2019
@azat
detach thread before erasing it from the FreeThreadPool
ec322cf 
Otherwise std::thread will call std::terminate() from the dtor:

  ...
  ClickHouse#4  0x000000000ac829c1 in std::terminate() ()
  ClickHouse#5  0x0000000006f1c87f in std::thread::~thread (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/7/thread:135
  ...
  ClickHouse#10 ThreadPoolImpl<std::thread>::worker (this=0xf370680 <ext::singleton<GlobalThreadPool>::instance()::instance>, thread_it=...) at ../dbms/src/Common/ThreadPool.cpp:191

Refs: f6b9b06 ("Attempt to implemnt global thread pool ClickHouse#4018")
azat added a commit to azat/ClickHouse that referenced this pull request May 19, 2019
@azat
Fix SimpleAggregateFunction for String longer MAX_SMALL_STRING_SIZE
4e8370b 
SimpleAggregateFunction do not pass arena to the
add_function -> getAddressOfAddFunction(), hence next crash happens:
  (gdb) bt
  #0  DB::Arena::alloc (size=64, this=0x0) at ../dbms/src/Common/Arena.h:124
  ClickHouse#1  DB::SingleValueDataString::changeImpl (this=0x7f97424a27d8, value=..., arena=0x0) at ../dbms/src/AggregateFunctions/AggregateFunctionMinMaxAny.h:274
  ClickHouse#2  0x0000000005ea5319 in DB::AggregateFunctionNullUnary<true>::add (arena=<optimized out>, row_num=<optimized out>, columns=<optimized out>, place=<optimized out>, this=<optimized out>) at ../dbms/src/AggregateFunctions/AggregateFunctionNull.h:43
  ClickHouse#3  DB::IAggregateFunctionHelper<DB::AggregateFunctionNullUnary<true> >::addFree (that=<optimized out>, place=<optimized out>, columns=<optimized out>, row_num=<optimized out>, arena=<optimized out>) at ../dbms/src/AggregateFunctions/IAggregateFunction.h:131
  ClickHouse#4  0x000000000679772f in DB::AggregatingSortedBlockInputStream::addRow (this=this@entry=0x7f982de19c00, cursor=...) at ../dbms/src/Common/AlignedBuffer.h:31
  ClickHouse#5  0x0000000006797faa in DB::AggregatingSortedBlockInputStream::merge (this=this@entry=0x7f982de19c00, merged_columns=..., queue=...) at ../dbms/src/DataStreams/AggregatingSortedBlockInputStream.cpp:140
  ClickHouse#6  0x0000000006798979 in DB::AggregatingSortedBlockInputStream::readImpl (this=0x7f982de19c00) at ../dbms/src/DataStreams/AggregatingSortedBlockInputStream.cpp:78
  ClickHouse#7  0x000000000622db55 in DB::IBlockInputStream::read (this=0x7f982de19c00) at ../dbms/src/DataStreams/IBlockInputStream.cpp:56
  ClickHouse#8  0x0000000006613bee in DB::MergeTreeDataMergerMutator::mergePartsToTemporaryPart (this=this@entry=0x7f97ec65e1a0, future_part=..., merge_entry=..., time_of_merge=<optimized out>, disk_reservation=<optimized out>, deduplicate=<optimized out>) at /usr/include/c++/8/bits/shared_ptr_base.h:1018
  ClickHouse#9  0x000000000658f7a4 in DB::StorageReplicatedMergeTree::tryExecuteMerge (this=0x7f97ec65b810, entry=...) at /usr/include/c++/8/bits/unique_ptr.h:342
  ClickHouse#10 0x00000000065940ab in DB::StorageReplicatedMergeTree::executeLogEntry (this=0x7f97ec65b810, entry=...) at ../dbms/src/Storages/StorageReplicatedMergeTree.cpp:910
  <snip>

  (gdb) f 1
  (gdb) p MAX_SMALL_STRING_SIZE
  $1 = 48
  (gdb) p capacity
  $2 = 64
  (gdb) p value
  $3 = {data = 0x7f97242fcbd0 "HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH", size = 61}

v2: avoid leaking of allocated by Arena memory on the intermediate step

Fixes: 8f8d2c0 ("Merge pull request ClickHouse#4629 from bgranvea/simple_aggregate_function")
@alexey-milovidov alexey-milovidov mentioned this pull request Jul 23, 2019
Merged
@nrv9999 nrv9999 mentioned this pull request Aug 25, 2019
Closed
@vzakaznikov vzakaznikov mentioned this pull request Aug 25, 2019
Closed
vitlibar referenced this pull request in vitlibar/ClickHouse Sep 16, 2019
Temp commit #5
cf254cb
akuzm added a commit that referenced this pull request Dec 12, 2019
@akuzm
Update lz4 to see if it fixes an msan failure.
51c1a12 
The failure goes like this:

https://clickhouse-test-reports.s3.yandex.net/7946/1f69a7a015468099b7395ce3b96fb638b9830e0b/functional_stateless_tests_(memory)/stderr.log

```
==199==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x2009fc34 in LZ4_count /build/obj-x86_64-linux-gnu/../contrib/lz4/lib/lz4.c:541:13
    #1 0x2009fc34 in LZ4_compress_generic /build/obj-x86_64-linux-gnu/../contrib/lz4/lib/lz4.c:999
    #2 0x2009fc34 in LZ4_compress_fast_extState /build/obj-x86_64-linux-gnu/../contrib/lz4/lib/lz4.c:1140
    #3 0x200ac663 in LZ4_compress_fast /build/obj-x86_64-linux-gnu/../contrib/lz4/lib/lz4.c:1211:14
    #4 0x200ac663 in LZ4_compress_default /build/obj-x86_64-linux-gnu/../contrib/lz4/lib/lz4.c:1222
    #5 0x1b76d2e8 in DB::CompressionCodecLZ4::doCompressData(char const*, unsigned int, char*) const /build/obj-x86_64-linux-gnu/../dbms/src/Compression/CompressionCodecLZ4.cpp:45:12
```
@mybluecrab mybluecrab mentioned this pull request Jan 14, 2020
Closed
alexey-milovidov pushed a commit that referenced this pull request Jan 23, 2020
@millb
Merge pull request #5 from ClickHouse/master
b598ce9 
refresh #5
@XLPE XLPE mentioned this pull request Mar 12, 2020
Closed
@jch0629 jch0629 mentioned this pull request Jul 14, 2020
Closed
azat added a commit to azat/ClickHouse that referenced this pull request Sep 1, 2020
@azat
Fix QueryPlan lifetime (for EXPLAIN PIPELINE graph=1) for queries wit…
d04cda0 
…h nested interpreter

Example of such queries are distributed queries, which creates local
InterpreterSelectQuery, which will have it's own QueryPlan but returns
Pipes that has that IQueryPlanStep attached.

After EXPLAIN PIPELINE graph=1 tries to use them, and will get SIGSEGV.

- TSAN:

<details>

```
==2782113==ERROR: AddressSanitizer: heap-use-after-free on address 0x6120000223c0 at pc 0x00002b8f3f3e bp 0x7fff18cfbff0 sp 0x7fff18cfbfe8
READ of size 8 at 0x6120000223c0 thread T22 (TCPHandler)
    #0 0x2b8f3f3d in DB::printPipelineCompact(std::__1::vector<std::__1::shared_ptr<DB::IProcessor>, std::__1::allocator<std::__1::shared_ptr<DB::IProcessor> > > const&, DB::WriteBuffer&, bool) /build/obj-x86_64-linux-gnu/../src/Processors/printPipeline.cpp:116:53
    ClickHouse#1 0x29ee698c in DB::InterpreterExplainQuery::executeImpl() /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterExplainQuery.cpp:275:17
    ClickHouse#2 0x29ee2e40 in DB::InterpreterExplainQuery::execute() /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterExplainQuery.cpp:73:14
    ClickHouse#3 0x2a7b44a2 in DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) /build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:389:28
    ClickHouse#4 0x2a7b1cb3 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) /build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:675:30
    ClickHouse#5 0x2b7993b2 in DB::TCPHandler::runImpl() /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:253:24
    ClickHouse#6 0x2b7b649a in DB::TCPHandler::run() /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1217:9
    ClickHouse#7 0x31d9c57e in Poco::Net::TCPServerConnection::start() /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3
    ClickHouse#8 0x31d9d281 in Poco::Net::TCPServerDispatcher::run() /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:114:20
    ClickHouse#9 0x3206b5d5 in Poco::PooledThread::run() /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199:14
    ClickHouse#10 0x320657ad in Poco::ThreadImpl::runnableEntry(void*) /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345:27
    ClickHouse#11 0x7ffff7f853e8 in start_thread (/usr/lib/libpthread.so.0+0x93e8)
    ClickHouse#12 0x7ffff7ea2292 in clone (/usr/lib/libc.so.6+0x100292)

0x6120000223c0 is located 0 bytes inside of 272-byte region [0x6120000223c0,0x6120000224d0)
freed by thread T22 (TCPHandler) here:
    #0 0x122f3b62 in operator delete(void*, unsigned long) (/src/ch/tmp/master-20200831/clickhouse+0x122f3b62)
    ClickHouse#1 0x2bd9e9fa in std::__1::default_delete<DB::IQueryPlanStep>::operator()(DB::IQueryPlanStep*) const /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2363:5
    ClickHouse#2 0x2bd9e9fa in std::__1::unique_ptr<DB::IQueryPlanStep, std::__1::default_delete<DB::IQueryPlanStep> >::reset(DB::IQueryPlanStep*) /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2618:7
    ClickHouse#3 0x2bd9e9fa in std::__1::unique_ptr<DB::IQueryPlanStep, std::__1::default_delete<DB::IQueryPlanStep> >::~unique_ptr() /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2572:19
    ClickHouse#4 0x2bd9e9fa in DB::QueryPlan::Node::~Node() /build/obj-x86_64-linux-gnu/../src/Processors/QueryPlan/QueryPlan.h:66:12
    ClickHouse#5 0x2bd9e9fa in void std::__1::allocator_traits<std::__1::allocator<std::__1::__list_node<DB::QueryPlan::Node, void*> > >::__destroy<DB::QueryPlan::Node>(std::__1::integral_constant<bool, false>, std::__1::allocator<std::__1::__list_node<DB::QueryPlan::Node, void*> >&,
 DB::QueryPlan::Node*) /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1798:23
    ClickHouse#6 0x2bd9e9fa in void std::__1::allocator_traits<std::__1::allocator<std::__1::__list_node<DB::QueryPlan::Node, void*> > >::destroy<DB::QueryPlan::Node>(std::__1::allocator<std::__1::__list_node<DB::QueryPlan::Node, void*> >&, DB::QueryPlan::Node*) /build/obj-x86_64-lin
ux-gnu/../contrib/libcxx/include/memory:1630:14
    ClickHouse#7 0x2bd9e9fa in std::__1::__list_imp<DB::QueryPlan::Node, std::__1::allocator<DB::QueryPlan::Node> >::clear() /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/list:762:13
    ClickHouse#8 0x29fece08 in DB::InterpreterSelectQuery::execute() /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:492:1
    ClickHouse#9 0x2abf7484 in DB::ClusterProxy::(anonymous namespace)::createLocalStream(std::__1::shared_ptr<DB::IAST> const&, DB::Block const&, DB::Context const&, DB::QueryProcessingStage::Enum) /build/obj-x86_64-linux-gnu/../src/Interpreters/ClusterProxy/SelectStreamFactory.cpp:
78:33
    ClickHouse#10 0x2abea85d in DB::ClusterProxy::SelectStreamFactory::createForShard(DB::Cluster::ShardInfo const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::shar
ed_ptr<DB::Throttler> const&, DB::SelectQueryInfo const&, std::__1::vector<DB::Pipe, std::__1::allocator<DB::Pipe> >&)::$_0::operator()() const /build/obj-x86_64-linux-gnu/../src/Interpreters/ClusterProxy/SelectStreamFactory.cpp:133:51
    ClickHouse#11 0x2abea85d in DB::ClusterProxy::SelectStreamFactory::createForShard(DB::Cluster::ShardInfo const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::shar
ed_ptr<DB::Throttler> const&, DB::SelectQueryInfo const&, std::__1::vector<DB::Pipe, std::__1::allocator<DB::Pipe> >&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ClusterProxy/SelectStreamFactory.cpp:189:13
    ClickHouse#12 0x2abe6d99 in DB::ClusterProxy::executeQuery(DB::ClusterProxy::IStreamFactory&, std::__1::shared_ptr<DB::Cluster> const&, Poco::Logger*, std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, DB::Settings const&, DB::SelectQueryInfo const&) /build/obj-x86_64-lin
ux-gnu/../src/Interpreters/ClusterProxy/executeQuery.cpp:107:24
    ClickHouse#13 0x2abc4b74 in DB::StorageDistributed::read(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, s
td::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, DB::Context const&, DB::QueryProcessingStage::Enum, unsigned long, unsigned int) /build/obj-x86_64-linux-gnu/../src/Storages/StorageDistributed.cpp:514:12
    ClickHouse#14 0x2bda1c5a in DB::ReadFromStorageStep::ReadFromStorageStep(std::__1::shared_ptr<DB::RWLockImpl::LockHolderImpl>, std::__1::shared_ptr<DB::StorageInMemoryMetadata const>&, DB::SelectQueryOptions, std::__1::shared_ptr<DB::IStorage>, std::__1::vector<std::__1::basic_st
ring<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context>, DB::QueryProcessingStage
::Enum, unsigned long, unsigned long) /build/obj-x86_64-linux-gnu/../src/Processors/QueryPlan/ReadFromStorageStep.cpp:39:26
    ClickHouse#15 0x2a01ca70 in std::__1::__unique_if<DB::ReadFromStorageStep>::__unique_single std::__1::make_unique<DB::ReadFromStorageStep, std::__1::shared_ptr<DB::RWLockImpl::LockHolderImpl>&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const>&, DB::SelectQueryOptions&, std
::__1::shared_ptr<DB::IStorage>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&, DB::SelectQueryInfo&, st
d::__1::shared_ptr<DB::Context>&, DB::QueryProcessingStage::Enum&, unsigned long&, unsigned long&>(std::__1::shared_ptr<DB::RWLockImpl::LockHolderImpl>&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const>&, DB::SelectQueryOptions&, std::__1::shared_ptr<DB::IStorage>&,
std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&, DB::SelectQueryInfo&, std::__1::shared_ptr<DB::Context>&,
DB::QueryProcessingStage::Enum&, unsigned long&, unsigned long&) /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3028:32
    ClickHouse#16 0x29ff556a in DB::InterpreterSelectQuery::executeFetchColumns(DB::QueryProcessingStage::Enum, DB::QueryPlan&, std::__1::shared_ptr<DB::PrewhereInfo> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::
__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:1383:26
    ClickHouse#17 0x29fe6b83 in DB::InterpreterSelectQuery::executeImpl(DB::QueryPlan&, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>) /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:795:9
    ClickHouse#18 0x29fe5771 in DB::InterpreterSelectQuery::buildQueryPlan(DB::QueryPlan&) /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:473:5
    ClickHouse#19 0x2a47d370 in DB::InterpreterSelectWithUnionQuery::buildQueryPlan(DB::QueryPlan&) /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectWithUnionQuery.cpp:182:38
    ClickHouse#20 0x29ee5bff in DB::InterpreterExplainQuery::executeImpl() /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterExplainQuery.cpp:265:21
    ClickHouse#21 0x29ee2e40 in DB::InterpreterExplainQuery::execute() /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterExplainQuery.cpp:73:14
    ClickHouse#22 0x2a7b44a2 in DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) /build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:389:28
    ClickHouse#23 0x2a7b1cb3 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) /build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:675:30
    ClickHouse#24 0x2b7993b2 in DB::TCPHandler::runImpl() /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:253:24
    ClickHouse#25 0x2b7b649a in DB::TCPHandler::run() /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1217:9
    ClickHouse#26 0x31d9c57e in Poco::Net::TCPServerConnection::start() /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3
    ClickHouse#27 0x31d9d281 in Poco::Net::TCPServerDispatcher::run() /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:114:20
    ClickHouse#28 0x3206b5d5 in Poco::PooledThread::run() /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199:14
    ClickHouse#29 0x320657ad in Poco::ThreadImpl::runnableEntry(void*) /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345:27
    ClickHouse#30 0x7ffff7f853e8 in start_thread (/usr/lib/libpthread.so.0+0x93e8)
```

</details>
alexey-milovidov pushed a commit that referenced this pull request Sep 19, 2020
@dreamerfable
Merge pull request #5 from dreamerfable/dreamerfable-docs-custom_pari…
f2b8e17 
…tioning_key

Update custom-partitioning-key.md
abyss7 added a commit to abyss7/ClickHouse that referenced this pull request Sep 21, 2020
@abyss7
Revert ASTTableIdentifier ClickHouse#5
e77635f
alesapin pushed a commit that referenced this pull request Oct 8, 2020
@alexelex
Merge pull request #5 from alexelex/quorum_inserts_2
1800762 
style check fix
@yanchoyanev yanchoyanev mentioned this pull request Apr 7, 2023
Closed
azat added a commit to azat/ClickHouse that referenced this pull request May 9, 2023
@azat
Fix stack-use-after-scope in gtest_resource_manager_static
9b3bcd0 
Reordering of the static variables should be enough to fix this:

    [ RUN      ] IOResourceStaticResourceManager.Prioritization
    =================================================================
    ==13==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7f7d8d621970 at pc 0x5636b80dcbb2 bp 0x7f7c48e47dd0 sp 0x7f7c48e47dc8
    READ of size 8 at 0x7f7d8d621970 thread T3975 (ThreadPool)
        #0 0x5636b80dcbb1 in IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_1::operator()(long) const build_docker/./src/IO/Resource/tests/gtest_resource_manager_static.cpp:78:13
        ClickHouse#1 0x5636b80dcbb1 in IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0::operator()() const build_docker/./src/IO/Resource/tests/gtest_resource_manager_static.cpp:95:21
        ClickHouse#2 0x5636b80dcbb1 in decltype(std::declval<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&>()()) std::__1::__invoke[abi:v15000]<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23
        ClickHouse#3 0x5636b80dcbb1 in decltype(auto) std::__1::__apply_tuple_impl[abi:v15000]<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&, std::__1::tuple<>&>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&, std::__1::tuple<>&, std::__1::__tuple_indices<>) build_docker/./contrib/llvm-project/libcxx/include/tuple:1789:1
        ClickHouse#4 0x5636b80dcbb1 in decltype(auto) std::__1::apply[abi:v15000]<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&, std::__1::tuple<>&>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&, std::__1::tuple<>&) build_docker/./contrib/llvm-project/libcxx/include/tuple:1798:1
        ClickHouse#5 0x5636b80dcbb1 in ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&)::'lambda'()::operator()() build_docker/./src/Common/ThreadPool.h:228:13
        ClickHouse#6 0x5636b80dcbb1 in decltype(std::declval<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>()()) std::__1::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&)::'lambda'()&>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23
        ClickHouse#7 0x5636b80dcbb1 in void std::__1::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&)::'lambda'()&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479:9
        ClickHouse#8 0x5636b80dcbb1 in std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12
        ClickHouse#9 0x5636b80dcbb1 in void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&)::'lambda'(), void ()>>(std::__1::__function::__policy_storage const*) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16
        ClickHouse#10 0x5636ea219310 in std::__1::__function::__policy_func<void ()>::operator()[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16
        ClickHouse#11 0x5636ea219310 in std::__1::function<void ()>::operator()() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12
        ClickHouse#12 0x5636ea219310 in ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) build_docker/./src/Common/ThreadPool.cpp:416:13
        ClickHouse#13 0x5636ea2258ac in void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()::operator()() const build_docker/./src/Common/ThreadPool.cpp:180:73
        ClickHouse#14 0x5636ea2258ac in decltype(std::declval<void>()()) std::__1::__invoke[abi:v15000]<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23
        ClickHouse#15 0x5636ea2258ac in void std::__1::__thread_execute[abi:v15000]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>&, std::__1::__tuple_indices<>) build_docker/./contrib/llvm-project/libcxx/include/thread:284:5
        ClickHouse#16 0x5636ea2258ac in void* std::__1::__thread_proxy[abi:v15000]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>>(void*) build_docker/./contrib/llvm-project/libcxx/include/thread:295:5
        ClickHouse#17 0x7f7d8fcf1608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8608) (BuildId: 7b4536f41cdaa5888408e82d0836e33dcf436466)
        ClickHouse#18 0x7f7d8fc16132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)

    Address 0x7f7d8d621970 is located in stack of thread T0 at offset 368 in frame
        #0 0x5636b80d9bef in IOResourceStaticResourceManager_Prioritization_Test::TestBody() build_docker/./src/IO/Resource/tests/gtest_resource_manager_static.cpp:46

      This frame has 10 object(s):
        [32, 36) 'requests_per_thead' (line 48)
        [48, 208) 't' (line 49)
        [272, 296) 'ref.tmp' (line 51)
        [336, 352) 'last_priority' (line 74)
        [368, 376) 'check' (line 75) <== Memory access at offset 368 is inside this variable
        [400, 424) 'name' (line 83)
        [464, 512) 'ref.tmp13' (line 87)
        [544, 560) 'c' (line 101)
        [576, 600) 'ref.tmp32' (line 101)
        [640, 664) 'ref.tmp41' (line 102)

Signed-off-by: Azat Khuzhin <[email protected]>
RunningXie pushed a commit to RunningXie/ClickHouse that referenced this pull request Jul 28, 2023
@young-scott
Merge pull request ClickHouse#5 from RunningXie/feature_handle_remove…
ac8df32 
…_error

handle remove error
rschu1ze added a commit to rschu1ze/ClickHouse that referenced this pull request Apr 8, 2024
@rschu1ze
Suppress tsan failures: use locks instead of atomics
d0a08aa 
Fixes integration test_reload_certificate/test.py::test_first_than_second_cert

---

E           Exception: Sanitizer assert found for instance ==================
E           WARNING: ThreadSanitizer: data race (pid=1)
E             Write of size 8 at 0x7b2800025d30 by thread T2 (mutexes: write M0, write M1):
E               #0 free <null> (clickhouse+0x709a3e5) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               #1 CRYPTO_free build_docker/./contrib/openssl/crypto/mem.c:282:5 (clickhouse+0x2015f8ea) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               #2 EVP_PKEY_free build_docker/./contrib/openssl/crypto/evp/p_lib.c:1809:5 (clickhouse+0x2012a751) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#3 Poco::Crypto::EVPPKey::~EVPPKey() build_docker/./base/poco/Crypto/src/EVPPKey.cpp:121:17 (clickhouse+0x1d00ffa9) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#4 DB::CertificateReloader::Data::~Data() build_docker/./src/Server/CertificateReloader.h:71:12 (clickhouse+0x194fb42d) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#5 std::__1::default_delete<DB::CertificateReloader::Data const>::operator()[abi:v15000](DB::CertificateReloader::Data const*) const build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48:5 (clickhouse+0x194fb42d)
E               ClickHouse#6 std::__1::__shared_ptr_pointer<DB::CertificateReloader::Data const*, std::__1::default_delete<DB::CertificateReloader::Data const>, std::__1::allocator<DB::CertificateReloader::Data const>>::__on_zero_shared() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:263:5 (clickhouse+0x194fb42d)
E               ClickHouse#7 std::__1::__shared_count::__release_shared[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:174:9 (clickhouse+0x194fade0) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#8 std::__1::__shared_weak_count::__release_shared[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:215:27 (clickhouse+0x194fade0)
E               ClickHouse#9 std::__1::shared_ptr<DB::CertificateReloader::Data const>::~shared_ptr[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:702:23 (clickhouse+0x194fade0)
E               ClickHouse#10 std::__1::shared_ptr<DB::CertificateReloader::Data const>::operator=[abi:v15000](std::__1::shared_ptr<DB::CertificateReloader::Data const>&&) build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723:9 (clickhouse+0x194fade0)
E               ClickHouse#11 MultiVersion<DB::CertificateReloader::Data>::set(std::__1::unique_ptr<DB::CertificateReloader::Data const, std::__1::default_delete<DB::CertificateReloader::Data const>>&&) build_docker/./src/Common/MultiVersion.h:76:25 (clickhouse+0x194fade0)
E               ClickHouse#12 DB::CertificateReloader::tryLoad(Poco::Util::AbstractConfiguration const&) build_docker/./src/Server/CertificateReloader.cpp:83:18 (clickhouse+0x194f94ca) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#13 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6::operator()(Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool) const build_docker/./programs/server/Server.cpp:1546:45 (clickhouse+0xf384df7) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#14 decltype(std::declval<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&>()(std::declval<Poco::AutoPtr<Poco::Util::AbstractConfiguration>>(), std::declval<bool>())) std::__1::__invoke[abi:v15000]<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&, Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool>(DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&, Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf3827a9) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#15 void std::__1::__invoke_void_return_wrapper<void, true>::__call<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&, Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool>(DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&, Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479:9 (clickhouse+0xf3827a9)
E               ClickHouse#16 std::__1::__function::__default_alloc_func<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6, void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::operator()[abi:v15000](Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 (clickhouse+0xf3827a9)
E               ClickHouse#17 void std::__1::__function::__policy_invoker<void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::__call_impl<std::__1::__function::__default_alloc_func<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6, void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>>(std::__1::__function::__policy_storage const*, Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16 (clickhouse+0xf3827a9)
E               ClickHouse#18 std::__1::__function::__policy_func<void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::operator()[abi:v15000](Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool&&) const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16 (clickhouse+0x19fd2cbe) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#19 std::__1::function<void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::operator()(Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool) const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12 (clickhouse+0x19fd2cbe)
E               ClickHouse#20 DB::ConfigReloader::reloadIfNewer(bool, bool, bool, bool) build_docker/./src/Common/Config/ConfigReloader.cpp:150:13 (clickhouse+0x19fd2cbe)
E               ClickHouse#21 DB::ConfigReloader::reload() build_docker/./src/Common/Config/ConfigReloader.h:51:21 (clickhouse+0xf38767c) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#22 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13::operator()() const build_docker/./programs/server/Server.cpp:1731:31 (clickhouse+0xf38767c)
E               ClickHouse#23 decltype(std::declval<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&>()()) std::__1::__invoke[abi:v15000]<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&>(DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf38767c)
E               ClickHouse#24 void std::__1::__invoke_void_return_wrapper<void, true>::__call<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&>(DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479:9 (clickhouse+0xf38767c)
E               ClickHouse#25 std::__1::__function::__default_alloc_func<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13, void ()>::operator()[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 (clickhouse+0xf38767c)
E               ClickHouse#26 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13, void ()>>(std::__1::__function::__policy_storage const*) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16 (clickhouse+0xf38767c)
E               ClickHouse#27 std::__1::__function::__policy_func<void ()>::operator()[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16 (clickhouse+0x16907aa0) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#28 std::__1::function<void ()>::operator()() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12 (clickhouse+0x16907aa0)
E               ClickHouse#29 DB::Context::reloadConfig() const build_docker/./src/Interpreters/Context.cpp:4357:5 (clickhouse+0x16907aa0)
E               ClickHouse#30 DB::InterpreterSystemQuery::execute() build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:577:29 (clickhouse+0x17e78c19) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#31 DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:1195:40 (clickhouse+0x17e3e462) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#32 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1374:26 (clickhouse+0x17e39837) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#33 DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:518:54 (clickhouse+0x195cc651) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#34 DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2329:9 (clickhouse+0x195e8707) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#35 Poco::Net::TCPServerConnection::start() build_docker/./base/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse+0x1d00d942) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#36 Poco::Net::TCPServerDispatcher::run() build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115:20 (clickhouse+0x1d00e1b1) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#37 Poco::PooledThread::run() build_docker/./base/poco/Foundation/src/ThreadPool.cpp:188:14 (clickhouse+0x1d20f2e6) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#38 Poco::(anonymous namespace)::RunnableHolder::run() build_docker/./base/poco/Foundation/src/Thread.cpp:45:11 (clickhouse+0x1d20d5af) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#39 Poco::ThreadImpl::runnableEntry(void*) build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:335:27 (clickhouse+0x1d20ba69) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E
E             Previous atomic write of size 4 at 0x7b2800025d30 by thread T3 (mutexes: write M2):
E               #0 CRYPTO_DOWN_REF build_docker/./contrib/openssl/include/internal/refcount.h:51:12 (clickhouse+0x2012a6e6) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               #1 EVP_PKEY_free build_docker/./contrib/openssl/crypto/evp/p_lib.c:1795:5 (clickhouse+0x2012a6e6)
E               #2 ssl_cert_clear_certs build_docker/./contrib/openssl/ssl/ssl_cert.c:246:9 (clickhouse+0x1ffafd37) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#3 ssl_cert_free build_docker/./contrib/openssl/ssl/ssl_cert.c:277:5 (clickhouse+0x1ffafd37)
E               ClickHouse#4 ossl_ssl_connection_free build_docker/./contrib/openssl/ssl/ssl_lib.c:1458:5 (clickhouse+0x1ffba6af) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#5 SSL_free build_docker/./contrib/openssl/ssl/ssl_lib.c:1417:9 (clickhouse+0x1ffb920e) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#6 Poco::Net::SecureSocketImpl::reset() build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:583:3 (clickhouse+0x1cfaac60) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#7 Poco::Net::SecureSocketImpl::~SecureSocketImpl() build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:80:3 (clickhouse+0x1cfaac60)
E               ClickHouse#8 Poco::Net::SecureStreamSocketImpl::~SecureStreamSocketImpl() build_docker/./base/poco/NetSSL_OpenSSL/src/SecureStreamSocketImpl.cpp:52:1 (clickhouse+0x1cfb15dd) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#9 Poco::Net::SecureStreamSocketImpl::~SecureStreamSocketImpl() build_docker/./base/poco/NetSSL_OpenSSL/src/SecureStreamSocketImpl.cpp:43:1 (clickhouse+0x1cfb15dd)
E               ClickHouse#10 Poco::RefCountedObject::release() const build_docker/./base/poco/Foundation/include/Poco/RefCountedObject.h:86:13 (clickhouse+0x1cffc81e) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#11 Poco::Net::Socket::~Socket() build_docker/./base/poco/Net/src/Socket.cpp:68:10 (clickhouse+0x1cffc81e)
E               ClickHouse#12 Poco::Net::StreamSocket::~StreamSocket() build_docker/./base/poco/Net/src/StreamSocket.cpp:63:1 (clickhouse+0x1d009c39) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#13 Poco::Net::TCPConnectionNotification::~TCPConnectionNotification() build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:43:2 (clickhouse+0x1d00ef50) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#14 Poco::Net::TCPConnectionNotification::~TCPConnectionNotification() build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:42:2 (clickhouse+0x1d00ef50)
E               ClickHouse#15 Poco::RefCountedObject::release() const build_docker/./base/poco/Foundation/include/Poco/RefCountedObject.h:86:13 (clickhouse+0x1d00e203) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#16 Poco::AutoPtr<Poco::Notification>::~AutoPtr() build_docker/./base/poco/Foundation/include/Poco/AutoPtr.h:91:19 (clickhouse+0x1d00e203)
E               ClickHouse#17 Poco::Net::TCPServerDispatcher::run() build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:122:3 (clickhouse+0x1d00e203)
E               ClickHouse#18 Poco::PooledThread::run() build_docker/./base/poco/Foundation/src/ThreadPool.cpp:188:14 (clickhouse+0x1d20f2e6) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#19 Poco::(anonymous namespace)::RunnableHolder::run() build_docker/./base/poco/Foundation/src/Thread.cpp:45:11 (clickhouse+0x1d20d5af) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E               ClickHouse#20 Poco::ThreadImpl::runnableEntry(void*) build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:335:27 (clickhouse+0x1d20ba69) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)
E
rschu1ze added a commit to rschu1ze/ClickHouse that referenced this pull request Apr 8, 2024
@rschu1ze
Fix leak-san false positive
9400fee 
This commit adresses below leaksan find.

I tried to reproduce this locally but 02802_clickhouse_disks_s3_copy.sh
but couldn't. clickhouse-disks did not do anything useful (it would not
even print logging), neither with a standard build nor with a leaksan
build. Could not find further documentation of it or even what this tool
is supposed to do, perhaps it is just for internal use. Also,
- line numbers in the leaksan report were partially missing,
- I am not really sure how Sha256HMACOpenSSLImpl::Calculate is calling
  into hmac_init (there must be some sort of static initialization
  somewhere but I did not find it), and
- my fix is in a weird place due to other restrictions (see the commit
  in the aws-sdk-cpp contrib repo).

The chance that this fix fixes the leak are low.
If it doesn't work, add "# Tag no-asan" + a comment to
02802_clickhouse_disks_s3_copy.sh and don't worry further.

EDIT: The commit fixes the issue, everything is good.

https://s3.amazonaws.com/clickhouse-test-reports/59870/b452e3d1ab87b8cc5810693aeea28f69ad28d671/stateless_tests__asan__[3_4].html

2024-03-19 08:34:03 =================================================================
2024-03-19 08:34:03 ==13149==ERROR: LeakSanitizer: detected memory leaks
2024-03-19 08:34:03
2024-03-19 08:34:03 Direct leak of 904 byte(s) in 1 object(s) allocated from:
2024-03-19 08:34:03     #0 0x55f9cb5a18ee in malloc (/usr/bin/clickhouse+0xa9a48ee) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     #1 0x55fa01e34070 in CRYPTO_malloc build_docker/./contrib/openssl/crypto/mem.c:202:11
2024-03-19 08:34:03     #2 0x55fa01e34070 in CRYPTO_zalloc build_docker/./contrib/openssl/crypto/mem.c:222:11
2024-03-19 08:34:03     ClickHouse#3 0x55fa01d6dcca in ossl_err_get_state_int build_docker/./contrib/openssl/crypto/err/err.c:691:17
2024-03-19 08:34:03     ClickHouse#4 0x55fa01d71748 in ERR_set_mark build_docker/./contrib/openssl/crypto/err/err_mark.c:19:10
2024-03-19 08:34:03     ClickHouse#5 0x55fa01f4735b in ossl_prov_digest_load_from_params build_docker/./contrib/openssl/providers/common/provider_util.c:194:5
2024-03-19 08:34:03     ClickHouse#6 0x55fa01ff467a in hmac_set_ctx_params build_docker/./contrib/openssl/providers/implementations/macs/hmac_prov.c:307:10
2024-03-19 08:34:03     ClickHouse#7 0x55fa01ff3ef2 in hmac_init build_docker/./contrib/openssl/providers/implementations/macs/hmac_prov.c:169:37
2024-03-19 08:34:03     ClickHouse#8 0x55f9fb83c75b in Aws::Utils::Crypto::Sha256HMACOpenSSLImpl::Calculate(Aws::Utils::Array<unsigned char> const&, Aws::Utils::Array<unsigned char> const&) (/usr/bin/clickhouse+0x3ac3f75b) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     ClickHouse#9 0x55f9fb82ebeb in Aws::Utils::Crypto::Sha256HMAC::Calculate(Aws::Utils::Array<unsigned char> const&, Aws::Utils::Array<unsigned char> const&) (/usr/bin/clickhouse+0x3ac31beb) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     ClickHouse#10 0x55f9fb6d5afd in Aws::Client::AWSAuthV4Signer::ComputeHash(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const (/usr/bin/clickhouse+0x3aad8afd) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     ClickHouse#11 0x55f9fb6e0bad in Aws::Client::AWSAuthV4Signer::SignRequest(Aws::Http::HttpRequest&, char const*, char const*, bool) const (/usr/bin/clickhouse+0x3aae3bad) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     ClickHouse#12 0x55f9fb72651d in bool smithy::components::tracing::TracingUtils::MakeCallWithTiming<bool>(std::__1::function<bool ()>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, smithy::components::tracing::Meter const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>>&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) (/usr/bin/clickhouse+0x3ab2951d) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     ClickHouse#13 0x55f9fb70adcb in Aws::Client::AWSClient::AttemptOneRequest(std::__1::shared_ptr<Aws::Http::HttpRequest> const&, Aws::AmazonWebServiceRequest const&, char const*, char const*, char const*) const (/usr/bin/clickhouse+0x3ab0ddcb) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     ClickHouse#14 0x55f9fb6fdb17 in Aws::Client::AWSClient::AttemptExhaustively(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char cons
allmazz referenced this pull request in allmazz/ClickHouse Apr 20, 2024
@allmazz
� This is a combination of 5 commits.
e7d2bde 
� This is the 1st commit message:

WIP

types

� The commit message #2 will be skipped:

� WIP

� The commit message #3 will be skipped:

� WIP

� The commit message #4 will be skipped:

� WIP

� The commit message #5 will be skipped:

� WIP
allmazz referenced this pull request in allmazz/ClickHouse Apr 20, 2024
@allmazz
� This is the 1st commit message:
9ff53ff 
WIP

types

� The commit message #2 will be skipped:

� WIP

� The commit message #3 will be skipped:

� WIP

� The commit message #4 will be skipped:

� WIP

� The commit message #5 will be skipped:

� WIP

� The commit message #6 will be skipped:

� WIP
@hanfei1991 hanfei1991 mentioned this pull request Apr 9, 2025
Closed
yahoNanJing pushed a commit to yahoNanJing/ClickHouse that referenced this pull request May 16, 2025
[COLUMNAR-3553][Iceberg] Support position deletes for Iceberg TableEn…
32b7c55 
…gine (ClickHouse#5)

* Add ChunkInfoReadRowsBefore for IInputFormat to record number of previously read rows

* Add support for position deletes

* Remove FileEntry = std::variant<DataFileEntry>

* Fix bug of setting base row number

* Filter out deletes files which belongs to the same partition of the data file

* Rename ChunkInfoRowNumBase to ChunkInfoRowNumOffset

* Refine UT for position deletes

* Refine the interface of getDataTransformer
azat added a commit to azat/ClickHouse that referenced this pull request Jun 19, 2025
@azat
Fix possible data-race between suggestion thread and main client thread
0a94cdc 
Found on CI [1]:

<details>

<summary>stacktrace</summary>

    WARNING: ThreadSanitizer: data race (pid=328014)
      Write of size 8 at 0x729800008140 by main thread:
        0 __tsan_memcpy <null> (clickhouse+0x8b061be) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        1 std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::operator=(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./contrib/llvm-project/libcxx/include/string:2658:22 (clickhouse+0x191548c3) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        2 DB::SettingFieldTimezone::operator=(DB::SettingFieldTimezone const&) ci/tmp/build/./src/Core/SettingsFields.h:468:8 (clickhouse+0x191548c3)
        3 DB::SettingsTraits::Data::operator=(DB::SettingsTraits::Data const&) ci/tmp/build/./src/Core/Settings.cpp:6957:1 (clickhouse+0x191548c3)
        4 DB::BaseSettings<DB::SettingsTraits>::operator=(DB::BaseSettings<DB::SettingsTraits> const&) ci/tmp/build/./src/Core/BaseSettings.h:109:60 (clickhouse+0x18f98aec) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        5 DB::SettingsImpl::operator=(DB::SettingsImpl const&) ci/tmp/build/./src/Core/Settings.cpp:6963:8 (clickhouse+0x18f98aec)
        6 DB::Settings::operator=(DB::Settings const&) ci/tmp/build/./src/Core/Settings.cpp:7170:11 (clickhouse+0x18f98aec)
        7 DB::Context::setSettings(DB::Settings const&) ci/tmp/build/./src/Interpreters/Context.cpp:2618:15 (clickhouse+0x1a6aac4a) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        8 DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long)::$_0::operator()() const ci/tmp/build/./src/Client/ClientBase.cpp:2180:9 (clickhouse+0x1db11597) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        9 BasicScopeGuard<DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long)::$_0>::invoke() ci/tmp/build/./base/base/../base/scope_guard.h:101:9 (clickhouse+0x1db11597)
        10 BasicScopeGuard<DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long)::$_0>::~BasicScopeGuard() ci/tmp/build/./base/base/../base/scope_guard.h:50:26 (clickhouse+0x1db11597)
        11 DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long) ci/tmp/build/./src/Client/ClientBase.cpp:2245:5 (clickhouse+0x1db0418a) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        12 DB::ClientBase::executeMultiQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./src/Client/ClientBase.cpp:2618:21 (clickhouse+0x1db12b5e) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        13 DB::ClientBase::processQueryText(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./src/Client/ClientBase.cpp:2812:12 (clickhouse+0x1db142a6) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        14 DB::ClientBase::runInteractive() ci/tmp/build/./src/Client/ClientBase.cpp:3367:18 (clickhouse+0x1db1d57b) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        15 DB::Client::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&) ci/tmp/build/./programs/client/Client.cpp:399:9 (clickhouse+0x128ba9c1) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        16 non-virtual thunk to DB::Client::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&) ci/tmp/build/./programs/client/Client.cpp (clickhouse+0x128bb0a0) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        17 Poco::Util::Application::run() ci/tmp/build/./base/poco/Util/src/Application.cpp:315:8 (clickhouse+0x23017cbe) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        18 mainEntryClickHouseClient(int, char**) ci/tmp/build/./programs/client/Client.cpp:1139:23 (clickhouse+0x128c8ea6) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        19 main ci/tmp/build/./programs/main.cpp:323:21 (clickhouse+0x8b8ea04) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)

      Previous read of size 8 at 0x729800008140 by thread T2:
        #0 __tsan_memcpy <null> (clickhouse+0x8b061be) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        ClickHouse#1 std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::operator=(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./contrib/llvm-project/libcxx/include/string:2658:22 (clickhouse+0x1259a1c7) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        ClickHouse#2 DateLUT::instance() ci/tmp/build/./src/Common/DateLUT.cpp:173:35 (clickhouse+0x1259a1c7)
        ClickHouse#3 DateLUT::instance(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./src/Common/DateLUT.h:29:20 (clickhouse+0x192bbab2) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        ClickHouse#4 TimezoneMixin::TimezoneMixin(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./src/DataTypes/TimezoneMixin.h:16:21 (clickhouse+0x192bbab2)
        ClickHouse#5 DB::DataTypeDateTime::DataTypeDateTime(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) ci/tmp/build/./src/DataTypes/DataTypeDateTime.cpp:13:7 (clickhouse+0x192bb645) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        6 std::__1::shared_ptr<DB::DataTypeDateTime> std::__1::allocate_shared[abi:ne190107]<DB::DataTypeDateTime, std::__1::allocator<DB::DataTypeDateTime>, 0>(std::__1::allocator<DB::DataTypeDateTime> const&) FunctionsExternalDictionaries.cpp (clickhouse+0x8ddfc9b) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        7 std::__1::shared_ptr<DB::DataTypeDateTime> std::__1::make_shared[abi:ne190107]<DB::DataTypeDateTime, 0>() ci/tmp/build/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:851:10 (clickhouse+0x193bbb51) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        8 DB::create(std::__1::shared_ptr<DB::IAST> const&) ci/tmp/build/./src/DataTypes/registerDataTypeDateTime.cpp:66:16 (clickhouse+0x193bbb51)
        9 decltype(std::declval<std::__1::shared_ptr<DB::IDataType const> (*&)(std::__1::shared_ptr<DB::IAST> const&)>()(std::declval<std::__1::shared_ptr<DB::IAST> const&>())) std::__1::__invoke[abi:ne190107]<std::__1::shared_ptr<DB::IDataType const> (*&)(std::__1::shared_ptr<DB::IAST> const&), std::__1::shared_ptr<DB::IAST> const&>(std::__1::shared_ptr<DB::IDataType const> (*&)(std::__1::shared_ptr<DB::IAST> const&), std::__1::shared_ptr<DB::IAST> const&) ci/tmp/build/./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149:25 (clickhouse+0x192b5ea3) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        10 std::__1::shared_ptr<DB::IDataType const> std::__1::__invoke_void_return_wrapper<std::__1::shared_ptr<DB::IDataType const>, false>::__call[abi:ne190107]<std::__1::shared_ptr<DB::IDataType const> (*&)(std::__1::shared_ptr<DB::IAST> const&), std::__1::shared_ptr<DB::IAST> const&>(std::__1::shared_ptr<DB::IDataType const> (*&)(std::__1::shared_ptr<DB::IAST> const&), std::__1::shared_ptr<DB::IAST> const&) ci/tmp/build/./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:216:12 (clickhouse+0x192b5ea3)
        11 std::__1::__function::__default_alloc_func<std::__1::shared_ptr<DB::IDataType const> (*)(std::__1::shared_ptr<DB::IAST> const&), std::__1::shared_ptr<DB::IDataType const> (std::__1::shared_ptr<DB::IAST> const&)>::operator()[abi:ne190107](std::__1::shared_ptr<DB::IAST> const&) ci/tmp/build/./contrib/llvm-project/libcxx/include/__functional/function.h:210:12 (clickhouse+0x192b5ea3)
        12 std::__1::shared_ptr<DB::IDataType const> std::__1::__function::__policy_invoker<std::__1::shared_ptr<DB::IDataType const> (std::__1::shared_ptr<DB::IAST> const&)>::__call_impl[abi:ne190107]<std::__1::__function::__default_alloc_func<std::__1::shared_ptr<DB::IDataType const> (*)(std::__1::shared_ptr<DB::IAST> const&), std::__1::shared_ptr<DB::IDataType const> (std::__1::shared_ptr<DB::IAST> const&)>>(std::__1::__function::__policy_storage const*, std::__1::shared_ptr<DB::IAST> const&) ci/tmp/build/./contrib/llvm-project/libcxx/include/__functional/function.h:610:12 (clickhouse+0x192b5ea3)
        13 std::__1::__function::__policy_func<std::__1::shared_ptr<DB::IDataType const> (std::__1::shared_ptr<DB::IAST> const&)>::operator()[abi:ne190107](std::__1::shared_ptr<DB::IAST> const&) const ci/tmp/build/./contrib/llvm-project/libcxx/include/__functional/function.h:716:12 (clickhouse+0x192cfcc6) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        14 std::__1::function<std::__1::shared_ptr<DB::IDataType const> (std::__1::shared_ptr<DB::IAST> const&)>::operator()(std::__1::shared_ptr<DB::IAST> const&) const ci/tmp/build/./contrib/llvm-project/libcxx/include/__functional/function.h:989:10 (clickhouse+0x192cfcc6)
        15 std::__1::shared_ptr<DB::IDataType const> DB::DataTypeFactory::getImpl<false>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::IAST> const&) const ci/tmp/build/./src/DataTypes/DataTypeFactory.cpp:144:21 (clickhouse+0x192cfcc6)
        16 std::__1::shared_ptr<DB::IDataType const> DB::DataTypeFactory::getImpl<false>(std::__1::shared_ptr<DB::IAST> const&) const ci/tmp/build/./src/DataTypes/DataTypeFactory.cpp:91:16 (clickhouse+0x192cf5f3) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        17 std::__1::shared_ptr<DB::IDataType const> DB::DataTypeFactory::getImpl<false>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const ci/tmp/build/./src/DataTypes/DataTypeFactory.cpp:73:12 (clickhouse+0x192cf18d) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        18 DB::DataTypeFactory::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const ci/tmp/build/./src/DataTypes/DataTypeFactory.cpp:35:12 (clickhouse+0x192ccfa0) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        19 DB::NativeReader::read() ci/tmp/build/./src/Formats/NativeReader.cpp:188:45 (clickhouse+0x1dde11c2) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        20 DB::Connection::receiveDataImpl(DB::NativeReader&) ci/tmp/build/./src/Client/Connection.cpp:1396:24 (clickhouse+0x1db4c3c9) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        21 DB::Connection::receiveProfileEvents() ci/tmp/build/./src/Client/Connection.cpp:1408:12 (clickhouse+0x1db4b471) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        22 DB::Connection::receivePacket() ci/tmp/build/./src/Client/Connection.cpp:1345:29 (clickhouse+0x1db4b471)
        23 DB::Suggest::fetch(DB::IServerConnection&, DB::ConnectionTimeouts const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, DB::ClientInfo const&) ci/tmp/build/./src/Client/Suggest.cpp:170:36 (clickhouse+0x1dbbfce0) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        24 void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()::operator()() const ci/tmp/build/./src/Client/Suggest.cpp:103:17 (clickhouse+0x1dbc236e) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        25 decltype(std::declval<void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()>()()) std::__1::__invoke[abi:ne190107]<void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()>(void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()&&) ci/tmp/build/./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149:25 (clickhouse+0x1dbc2038) (BuildId: 50a1015145bb9fb79089f4828db24f2dae0945e5)
        26 void std::__1::__thread_execute[abi:ne190107]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()>&, std::__1::__tuple_indices<...>) ci/tmp/build/./contrib/llvm-project/libcxx/include/__thread/thread.h:192:3 (clickhouse+0x1dbc2038)
        27 void* std::__1::__thread_proxy[abi:ne190107]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void DB::Suggest::load<DB::Connection>(std::__1::shared_ptr<DB::Context const>, DB::ConnectionParameters const&, int, bool)::'lambda'()>>(void*) ci/tmp/build/./contrib/llvm-project/libcxx/include/__thread/thread.h:201:3 (clickhouse+0x1dbc2038)

</details>

What is interesting that it has been pops up only after ClickHouse#81759

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/PRs/82168/8ab5ff7cd742626a263fd7a262d4ff86aa28880d//stateless_tests_amd_tsan_s3_storage_3_3/job.log
azat added a commit to azat/ClickHouse that referenced this pull request Jun 30, 2025
@azat
Destroy ThreadGroupSwitcher before set value in std::future
362026f 
Otherwise it may lead to data-race:

    WARNING: ThreadSanitizer: data race (pid=26127)
      Write of size 6 at 0x7298000099e8 by main thread:
        0 __tsan_memcpy <null> (clickhouse-82444-tsan+0x8be71be) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
        1 DB::SettingsTraits::Data::operator=(DB::SettingsTraits::Data const&) ci/tmp/build/./src/Core/Settings.cpp:6978:1 (clickhouse-82444-tsan+0x192ae4f9) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
        2 DB::BaseSettings<DB::SettingsTraits>::operator=(DB::BaseSettings<DB::SettingsTraits> const&) ci/tmp/build/./src/Core/BaseSettings.h:109:60 (clickhouse-82444-tsan+0x190f2bcc) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
        3 DB::SettingsImpl::operator=(DB::SettingsImpl const&) ci/tmp/build/./src/Core/Settings.cpp:6984:8 (clickhouse-82444-tsan+0x190f2bcc)
        4 DB::Settings::operator=(DB::Settings const&) ci/tmp/build/./src/Core/Settings.cpp:7191:11 (clickhouse-82444-tsan+0x190f2bcc)
        5 DB::Context::setSettings(DB::Settings const&) ci/tmp/build/./src/Interpreters/Context.cpp:2635:15 (clickhouse-82444-tsan+0x1aa94c0a) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
        6 DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long)::$_0::operator()() const ci/tmp/build/./src/Client/ClientBase.cpp:2184:9 (clickhouse-82444-tsan+0x1df0f4d7) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
        7 BasicScopeGuard<DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long)::$_0>::invoke() ci/tmp/build/./base/base/../base/scope_guard.h:101:9 (clickhouse-82444-tsan+0x1df0f4d7)
        8 BasicScopeGuard<DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long)::$_0>::~BasicScopeGuard() ci/tmp/build/./base/base/../base/scope_guard.h:50:26 (clickhouse-82444-tsan+0x1df0f4d7)
        9 DB::ClientBase::processParsedSingleQuery(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::shared_ptr<DB::IAST>, bool&, unsigned long) ci/tmp/build/./src/Client/ClientBase.cpp:2249:5 (clickhouse-82444-tsan+0x1df021ca) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)

      Previous read of size 1 at 0x7298000099e8 by thread T4:
        #0 DB::SettingFieldNumber<bool>::operator bool() const ci/tmp/build/./src/Core/SettingsFields.h:40:36 (clickhouse-82444-tsan+0x1b114e5f) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
        ClickHouse#1 DB::ThreadStatus::finalizePerformanceCounters() ci/tmp/build/./src/Interpreters/ThreadStatusExt.cpp:547:17 (clickhouse-82444-tsan+0x1b114e5f)
        ClickHouse#2 DB::ThreadStatus::detachFromGroup() ci/tmp/build/./src/Interpreters/ThreadStatusExt.cpp:375:5 (clickhouse-82444-tsan+0x1b113aec) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
        ClickHouse#3 DB::CurrentThread::detachFromGroupIfNotDetached() ci/tmp/build/./src/Interpreters/ThreadStatusExt.cpp:716:21 (clickhouse-82444-tsan+0x1b1119b3) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
        ClickHouse#4 DB::ThreadGroupSwitcher::~ThreadGroupSwitcher() ci/tmp/build/./src/Interpreters/ThreadStatusExt.cpp:261:9 (clickhouse-82444-tsan+0x1b1119b3)
        ClickHouse#5 DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>, std::__1::function<void ()>>::operator()(std::__1::function<void ()>&&, Priority, std::__1::optional<unsigned long>)::'lambda'()::operator()() ci/tmp/build/./src/Common/threadPoolCallbackRunner.h:179:9 (clickhouse-82444-tsan+0x14beafde) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)

      Thread T4 'ThreadPool' (tid=26133, running) created by thread T3 at:
        ...
        12 DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>, std::__1::function<void ()>>::operator()(std::__1::function<void ()>&&, Priority, std::__1::optional<unsigned long>) ci/tmp/build/./src/Common/threadPoolCallbackRunner.h:188:22 (clickhouse-82444-tsan+0x14be522d) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
        13 DB::ParallelParsingInputFormat::scheduleParserThreadForUnitWithNumber(unsigned long) ci/tmp/build/./src/Processors/Formats/Impl/ParallelParsingInputFormat.h:287:9 (clickhouse-82444-tsan+0x1e495627) (BuildId: e5eeeb8fc2a0ab65c098bb1f3db89853f2439448)
        14 DB::ParallelParsingInputFormat::segmentatorThreadFunction(std::__1::shared_ptr<DB::ThreadGroup>) ci/tmp/build/./src/Processors/Formats/Impl/ParallelParsingInputFormat.cpp:45:13 (clickhouse-82444-tsan+0x1e495627)

Refs: https://pastila.nl/?0001fdef/9a58e9d59c32d45100a481de26dccf68#T4iNrMFUnu4F2hWST5wLdQ==
azat added a commit to azat/ClickHouse that referenced this pull request Jul 16, 2025
@azat
Fix deadlock on shutdown due to recursive context locking during dict…
d9a26d6 
…ionary cleanup

<details>

<summary>stacktrace</summary>

    ClickHouse#5  DB::ContextSharedMutex::lockSharedImpl (this=this@entry=0x7ffff5b96040) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1036
            watch = {start_ns = 98678735300141, stop_ns = 0, clock_type = 4, is_running = true}
            increment = {what = <optimized out>, amount = 1}
    ClickHouse#6  0x00005555699352d6 in DB::SharedMutexHelper<DB::ContextSharedMutex, DB::SharedMutex>::lock_shared (this=0x7ffff5b96040) at /src/ch/clickhouse/src/Common/SharedMutexHelper.h:70
    ClickHouse#7  DB::SharedLockGuard<DB::ContextSharedMutex>::SharedLockGuard (mutex_=..., this=<optimized out>) at /src/ch/clickhouse/src/Common/SharedLockGuard.h:19
    ClickHouse#8  DB::ContextSharedPart::getConfigRef (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:760
    ClickHouse#9  DB::Context::getConfigRef (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1590
    ClickHouse#10 0x000055556450941c in DB::ProxyConfigurationResolverProvider::get (request_protocol=DB::ProxyConfiguration::Protocol::HTTP) at /src/ch/clickhouse/src/Common/ProxyConfigurationResolverProvider.cpp:161
    ClickHouse#11 0x00005555669ea75f in DB::BuilderRWBufferFromHTTP::create (this=0x7fffffffa848, credentials_=...) at /src/ch/clickhouse/src/IO/ReadWriteBufferFromHTTP.cpp:814
    ClickHouse#12 0x000055556cea0035 in DB::ExternalDictionaryLibraryBridgeHelper::bridgeHandShake (this=0x7ffd60c4ca18) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:79
    ClickHouse#13 0x000055556cea1f34 in DB::ExternalDictionaryLibraryBridgeHelper::removeLibrary (this=0x7ffff5b96044) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:186
    ClickHouse#14 0x0000555566cb0248 in DB::LibraryDictionarySource::~LibraryDictionarySource (this=0x7ffd60c78318) at /src/ch/clickhouse/src/Dictionaries/LibraryDictionarySource.cpp:70
    ClickHouse#15 0x000055556708d775 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156
    ClickHouse#16 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185
    ClickHouse#17 std::__1::shared_ptr<DB::IDictionarySource>::~shared_ptr[abi:ne190107]() (this=0x7ffd60c605b0) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668
    ClickHouse#18 DB::HashedDictionary<(DB::DictionaryKeyType)0, false, false>::~HashedDictionary (this=0x7ffd60c60398) at /src/ch/clickhouse/src/Dictionaries/HashedDictionary.h:378
    ClickHouse#19 0x0000555569a9c477 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156
    ClickHouse#20 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185
    ClickHouse#21 std::__1::shared_ptr<DB::IExternalLoadable>::~shared_ptr[abi:ne190107]() (this=0x7ffff4832560) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668
    ClickHouse#22 DB::ExternalLoader::LoadingDispatcher::Info::~Info (this=0x7ffff4832548) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:719
    ClickHouse#23 std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>::~pair (this=0x7ffff4832530, this@entry=0x7ffd60c60390) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__utility/pair.h:65
    ClickHouse#24 __destroy_at<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, 0> (__loc=__loc@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/construct_at.h:67
    ClickHouse#25 0x0000555569a9c628 in destroy<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, void, 0> (__p=0x7ffff5b96044, __p@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:339
    ClickHouse#28 std::__1::unique_ptr<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*>, std::__1::__hash_node_destructor<std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*> > > >::~unique_ptr[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:261
    ClickHouse#31 DB::ExternalLoader::LoadingDispatcher::setConfiguration (this=this@entry=0x7ffecad4c300, new_configs=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:495
    ClickHouse#32 0x0000555569a9116d in DB::ExternalLoader::reloadConfig (this=0x7ffecb3fd740, repository_name=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1533
    ClickHouse#33 DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0::operator()() const (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1334
    ClickHouse#34 std::__1::__invoke[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__f=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149
    ClickHouse#35 std::__1::__invoke_void_return_wrapper<void, true>::__call[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__args=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:224
    ClickHouse#36 std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()>::operator()[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:210
    ClickHouse#37 std::__1::__function::__policy_invoker<void ()>::__call_impl[abi:ne190107]<std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()> ClickHouse#38 0x0000555569966aaa in std::__1::__function::__policy_func<void ()>::operator()[abi:ne190107]() const (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:716
    ClickHouse#39 std::__1::function<void()>::operator() (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:989
    ClickHouse#40 BasicScopeGuard<std::__1::function<void()> >::invoke (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101
    ClickHouse#41 BasicScopeGuard<std::__1::function<void()> >::reset (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:63
    ClickHouse#42 DB::ContextSharedPart::shutdown (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:909
    ClickHouse#43 0x00005555644cf5cb in DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3::operator()() const (this=this@entry=0x7fffffffc870) at /src/ch/clickhouse/programs/server/Server.cpp:1257
    ClickHouse#44 0x00005555644b4439 in BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::invoke (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101
    ClickHouse#45 BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::~BasicScopeGuard (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:50
    ClickHouse#46 DB::Server::main (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:2788
    ClickHouse#47 0x0000555572cbbee9 in Poco::Util::Application::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/base/poco/Util/src/Application.cpp:315
    ClickHouse#48 0x000055556449c390 in DB::Server::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:610
    ClickHouse#49 0x0000555564499aec in mainEntryClickHouseServer (argc=6, argv=0x7ffff67ecdc0) at /src/ch/clickhouse/programs/server/Server.cpp:397
    ClickHouse#50 0x000055555dc994c2 in main (argc_=<optimized out>, argv_=<optimized out>) at /src/ch/clickhouse/programs/main.cpp:340

</details>
azat added a commit to azat/ClickHouse that referenced this pull request Jul 16, 2025
@azat
Fix deadlock on shutdown due to recursive context locking during libr…
8286e0d 
…ary bridge cleanup

<details>

<summary>stacktrace</summary>

    ClickHouse#5  DB::ContextSharedMutex::lockSharedImpl (this=this@entry=0x7ffff5b96040) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1036
            watch = {start_ns = 98678735300141, stop_ns = 0, clock_type = 4, is_running = true}
            increment = {what = <optimized out>, amount = 1}
    ClickHouse#6  0x00005555699352d6 in DB::SharedMutexHelper<DB::ContextSharedMutex, DB::SharedMutex>::lock_shared (this=0x7ffff5b96040) at /src/ch/clickhouse/src/Common/SharedMutexHelper.h:70
    ClickHouse#7  DB::SharedLockGuard<DB::ContextSharedMutex>::SharedLockGuard (mutex_=..., this=<optimized out>) at /src/ch/clickhouse/src/Common/SharedLockGuard.h:19
    ClickHouse#8  DB::ContextSharedPart::getConfigRef (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:760
    ClickHouse#9  DB::Context::getConfigRef (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1590
    ClickHouse#10 0x000055556450941c in DB::ProxyConfigurationResolverProvider::get (request_protocol=DB::ProxyConfiguration::Protocol::HTTP) at /src/ch/clickhouse/src/Common/ProxyConfigurationResolverProvider.cpp:161
    ClickHouse#11 0x00005555669ea75f in DB::BuilderRWBufferFromHTTP::create (this=0x7fffffffa848, credentials_=...) at /src/ch/clickhouse/src/IO/ReadWriteBufferFromHTTP.cpp:814
    ClickHouse#12 0x000055556cea0035 in DB::ExternalDictionaryLibraryBridgeHelper::bridgeHandShake (this=0x7ffd60c4ca18) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:79
    ClickHouse#13 0x000055556cea1f34 in DB::ExternalDictionaryLibraryBridgeHelper::removeLibrary (this=0x7ffff5b96044) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:186
    ClickHouse#14 0x0000555566cb0248 in DB::LibraryDictionarySource::~LibraryDictionarySource (this=0x7ffd60c78318) at /src/ch/clickhouse/src/Dictionaries/LibraryDictionarySource.cpp:70
    ClickHouse#15 0x000055556708d775 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156
    ClickHouse#16 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185
    ClickHouse#17 std::__1::shared_ptr<DB::IDictionarySource>::~shared_ptr[abi:ne190107]() (this=0x7ffd60c605b0) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668
    ClickHouse#18 DB::HashedDictionary<(DB::DictionaryKeyType)0, false, false>::~HashedDictionary (this=0x7ffd60c60398) at /src/ch/clickhouse/src/Dictionaries/HashedDictionary.h:378
    ClickHouse#19 0x0000555569a9c477 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156
    ClickHouse#20 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185
    ClickHouse#21 std::__1::shared_ptr<DB::IExternalLoadable>::~shared_ptr[abi:ne190107]() (this=0x7ffff4832560) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668
    ClickHouse#22 DB::ExternalLoader::LoadingDispatcher::Info::~Info (this=0x7ffff4832548) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:719
    ClickHouse#23 std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>::~pair (this=0x7ffff4832530, this@entry=0x7ffd60c60390) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__utility/pair.h:65
    ClickHouse#24 __destroy_at<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, 0> (__loc=__loc@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/construct_at.h:67
    ClickHouse#25 0x0000555569a9c628 in destroy<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, void, 0> (__p=0x7ffff5b96044, __p@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:339
    ClickHouse#28 std::__1::unique_ptr<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*>, std::__1::__hash_node_destructor<std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*> > > >::~unique_ptr[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:261
    ClickHouse#31 DB::ExternalLoader::LoadingDispatcher::setConfiguration (this=this@entry=0x7ffecad4c300, new_configs=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:495
    ClickHouse#32 0x0000555569a9116d in DB::ExternalLoader::reloadConfig (this=0x7ffecb3fd740, repository_name=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1533
    ClickHouse#33 DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0::operator()() const (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1334
    ClickHouse#34 std::__1::__invoke[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__f=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149
    ClickHouse#35 std::__1::__invoke_void_return_wrapper<void, true>::__call[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__args=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:224
    ClickHouse#36 std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()>::operator()[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:210
    ClickHouse#37 std::__1::__function::__policy_invoker<void ()>::__call_impl[abi:ne190107]<std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()> ClickHouse#38 0x0000555569966aaa in std::__1::__function::__policy_func<void ()>::operator()[abi:ne190107]() const (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:716
    ClickHouse#39 std::__1::function<void()>::operator() (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:989
    ClickHouse#40 BasicScopeGuard<std::__1::function<void()> >::invoke (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101
    ClickHouse#41 BasicScopeGuard<std::__1::function<void()> >::reset (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:63
    ClickHouse#42 DB::ContextSharedPart::shutdown (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:909
    ClickHouse#43 0x00005555644cf5cb in DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3::operator()() const (this=this@entry=0x7fffffffc870) at /src/ch/clickhouse/programs/server/Server.cpp:1257
    ClickHouse#44 0x00005555644b4439 in BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::invoke (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101
    ClickHouse#45 BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::~BasicScopeGuard (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:50
    ClickHouse#46 DB::Server::main (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:2788
    ClickHouse#47 0x0000555572cbbee9 in Poco::Util::Application::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/base/poco/Util/src/Application.cpp:315
    ClickHouse#48 0x000055556449c390 in DB::Server::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:610
    ClickHouse#49 0x0000555564499aec in mainEntryClickHouseServer (argc=6, argv=0x7ffff67ecdc0) at /src/ch/clickhouse/programs/server/Server.cpp:397
    ClickHouse#50 0x000055555dc994c2 in main (argc_=<optimized out>, argv_=<optimized out>) at /src/ch/clickhouse/programs/main.cpp:340

</details>
azat added a commit to azat/ClickHouse that referenced this pull request Jul 16, 2025
@azat
Fix deadlock on shutdown due to recursive context locking during libr…
19aaa11 
…ary bridge cleanup

<details>

<summary>stacktrace</summary>

    ClickHouse#5  DB::ContextSharedMutex::lockSharedImpl (this=this@entry=0x7ffff5b96040) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1036
            watch = {start_ns = 98678735300141, stop_ns = 0, clock_type = 4, is_running = true}
            increment = {what = <optimized out>, amount = 1}
    ClickHouse#6  0x00005555699352d6 in DB::SharedMutexHelper<DB::ContextSharedMutex, DB::SharedMutex>::lock_shared (this=0x7ffff5b96040) at /src/ch/clickhouse/src/Common/SharedMutexHelper.h:70
    ClickHouse#7  DB::SharedLockGuard<DB::ContextSharedMutex>::SharedLockGuard (mutex_=..., this=<optimized out>) at /src/ch/clickhouse/src/Common/SharedLockGuard.h:19
    ClickHouse#8  DB::ContextSharedPart::getConfigRef (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:760
    ClickHouse#9  DB::Context::getConfigRef (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1590
    ClickHouse#10 0x000055556450941c in DB::ProxyConfigurationResolverProvider::get (request_protocol=DB::ProxyConfiguration::Protocol::HTTP) at /src/ch/clickhouse/src/Common/ProxyConfigurationResolverProvider.cpp:161
    ClickHouse#11 0x00005555669ea75f in DB::BuilderRWBufferFromHTTP::create (this=0x7fffffffa848, credentials_=...) at /src/ch/clickhouse/src/IO/ReadWriteBufferFromHTTP.cpp:814
    ClickHouse#12 0x000055556cea0035 in DB::ExternalDictionaryLibraryBridgeHelper::bridgeHandShake (this=0x7ffd60c4ca18) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:79
    ClickHouse#13 0x000055556cea1f34 in DB::ExternalDictionaryLibraryBridgeHelper::removeLibrary (this=0x7ffff5b96044) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:186
    ClickHouse#14 0x0000555566cb0248 in DB::LibraryDictionarySource::~LibraryDictionarySource (this=0x7ffd60c78318) at /src/ch/clickhouse/src/Dictionaries/LibraryDictionarySource.cpp:70
    ClickHouse#15 0x000055556708d775 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156
    ClickHouse#16 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185
    ClickHouse#17 std::__1::shared_ptr<DB::IDictionarySource>::~shared_ptr[abi:ne190107]() (this=0x7ffd60c605b0) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668
    ClickHouse#18 DB::HashedDictionary<(DB::DictionaryKeyType)0, false, false>::~HashedDictionary (this=0x7ffd60c60398) at /src/ch/clickhouse/src/Dictionaries/HashedDictionary.h:378
    ClickHouse#19 0x0000555569a9c477 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156
    ClickHouse#20 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185
    ClickHouse#21 std::__1::shared_ptr<DB::IExternalLoadable>::~shared_ptr[abi:ne190107]() (this=0x7ffff4832560) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668
    ClickHouse#22 DB::ExternalLoader::LoadingDispatcher::Info::~Info (this=0x7ffff4832548) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:719
    ClickHouse#23 std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>::~pair (this=0x7ffff4832530, this@entry=0x7ffd60c60390) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__utility/pair.h:65
    ClickHouse#24 __destroy_at<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, 0> (__loc=__loc@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/construct_at.h:67
    ClickHouse#25 0x0000555569a9c628 in destroy<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, void, 0> (__p=0x7ffff5b96044, __p@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:339
    ClickHouse#28 std::__1::unique_ptr<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*>, std::__1::__hash_node_destructor<std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*> > > >::~unique_ptr[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:261
    ClickHouse#31 DB::ExternalLoader::LoadingDispatcher::setConfiguration (this=this@entry=0x7ffecad4c300, new_configs=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:495
    ClickHouse#32 0x0000555569a9116d in DB::ExternalLoader::reloadConfig (this=0x7ffecb3fd740, repository_name=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1533
    ClickHouse#33 DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0::operator()() const (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1334
    ClickHouse#34 std::__1::__invoke[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__f=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149
    ClickHouse#35 std::__1::__invoke_void_return_wrapper<void, true>::__call[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__args=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:224
    ClickHouse#36 std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()>::operator()[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:210
    ClickHouse#37 std::__1::__function::__policy_invoker<void ()>::__call_impl[abi:ne190107]<std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()> ClickHouse#38 0x0000555569966aaa in std::__1::__function::__policy_func<void ()>::operator()[abi:ne190107]() const (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:716
    ClickHouse#39 std::__1::function<void()>::operator() (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:989
    ClickHouse#40 BasicScopeGuard<std::__1::function<void()> >::invoke (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101
    ClickHouse#41 BasicScopeGuard<std::__1::function<void()> >::reset (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:63
    ClickHouse#42 DB::ContextSharedPart::shutdown (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:909
    ClickHouse#43 0x00005555644cf5cb in DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3::operator()() const (this=this@entry=0x7fffffffc870) at /src/ch/clickhouse/programs/server/Server.cpp:1257
    ClickHouse#44 0x00005555644b4439 in BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::invoke (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101
    ClickHouse#45 BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::~BasicScopeGuard (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:50
    ClickHouse#46 DB::Server::main (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:2788
    ClickHouse#47 0x0000555572cbbee9 in Poco::Util::Application::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/base/poco/Util/src/Application.cpp:315
    ClickHouse#48 0x000055556449c390 in DB::Server::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:610
    ClickHouse#49 0x0000555564499aec in mainEntryClickHouseServer (argc=6, argv=0x7ffff67ecdc0) at /src/ch/clickhouse/programs/server/Server.cpp:397
    ClickHouse#50 0x000055555dc994c2 in main (argc_=<optimized out>, argv_=<optimized out>) at /src/ch/clickhouse/programs/main.cpp:340

</details>
Felixoid pushed a commit that referenced this pull request Jul 16, 2025
@azat @Felixoid
Fix deadlock on shutdown due to recursive context locking during libr…
44f6d18 
…ary bridge cleanup

<details>

<summary>stacktrace</summary>

    #5  DB::ContextSharedMutex::lockSharedImpl (this=this@entry=0x7ffff5b96040) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1036
            watch = {start_ns = 98678735300141, stop_ns = 0, clock_type = 4, is_running = true}
            increment = {what = <optimized out>, amount = 1}
    #6  0x00005555699352d6 in DB::SharedMutexHelper<DB::ContextSharedMutex, DB::SharedMutex>::lock_shared (this=0x7ffff5b96040) at /src/ch/clickhouse/src/Common/SharedMutexHelper.h:70
    #7  DB::SharedLockGuard<DB::ContextSharedMutex>::SharedLockGuard (mutex_=..., this=<optimized out>) at /src/ch/clickhouse/src/Common/SharedLockGuard.h:19
    #8  DB::ContextSharedPart::getConfigRef (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:760
    #9  DB::Context::getConfigRef (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1590
    #10 0x000055556450941c in DB::ProxyConfigurationResolverProvider::get (request_protocol=DB::ProxyConfiguration::Protocol::HTTP) at /src/ch/clickhouse/src/Common/ProxyConfigurationResolverProvider.cpp:161
    #11 0x00005555669ea75f in DB::BuilderRWBufferFromHTTP::create (this=0x7fffffffa848, credentials_=...) at /src/ch/clickhouse/src/IO/ReadWriteBufferFromHTTP.cpp:814
    #12 0x000055556cea0035 in DB::ExternalDictionaryLibraryBridgeHelper::bridgeHandShake (this=0x7ffd60c4ca18) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:79
    #13 0x000055556cea1f34 in DB::ExternalDictionaryLibraryBridgeHelper::removeLibrary (this=0x7ffff5b96044) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:186
    #14 0x0000555566cb0248 in DB::LibraryDictionarySource::~LibraryDictionarySource (this=0x7ffd60c78318) at /src/ch/clickhouse/src/Dictionaries/LibraryDictionarySource.cpp:70
    #15 0x000055556708d775 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156
    #16 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185
    #17 std::__1::shared_ptr<DB::IDictionarySource>::~shared_ptr[abi:ne190107]() (this=0x7ffd60c605b0) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668
    #18 DB::HashedDictionary<(DB::DictionaryKeyType)0, false, false>::~HashedDictionary (this=0x7ffd60c60398) at /src/ch/clickhouse/src/Dictionaries/HashedDictionary.h:378
    #19 0x0000555569a9c477 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156
    #20 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185
    #21 std::__1::shared_ptr<DB::IExternalLoadable>::~shared_ptr[abi:ne190107]() (this=0x7ffff4832560) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668
    #22 DB::ExternalLoader::LoadingDispatcher::Info::~Info (this=0x7ffff4832548) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:719
    #23 std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>::~pair (this=0x7ffff4832530, this@entry=0x7ffd60c60390) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__utility/pair.h:65
    #24 __destroy_at<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, 0> (__loc=__loc@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/construct_at.h:67
    #25 0x0000555569a9c628 in destroy<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, void, 0> (__p=0x7ffff5b96044, __p@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:339
    #28 std::__1::unique_ptr<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*>, std::__1::__hash_node_destructor<std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*> > > >::~unique_ptr[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:261
    #31 DB::ExternalLoader::LoadingDispatcher::setConfiguration (this=this@entry=0x7ffecad4c300, new_configs=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:495
    #32 0x0000555569a9116d in DB::ExternalLoader::reloadConfig (this=0x7ffecb3fd740, repository_name=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1533
    #33 DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0::operator()() const (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1334
    #34 std::__1::__invoke[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__f=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149
    #35 std::__1::__invoke_void_return_wrapper<void, true>::__call[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__args=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:224
    #36 std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()>::operator()[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:210
    #37 std::__1::__function::__policy_invoker<void ()>::__call_impl[abi:ne190107]<std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()> #38 0x0000555569966aaa in std::__1::__function::__policy_func<void ()>::operator()[abi:ne190107]() const (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:716
    #39 std::__1::function<void()>::operator() (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:989
    #40 BasicScopeGuard<std::__1::function<void()> >::invoke (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101
    #41 BasicScopeGuard<std::__1::function<void()> >::reset (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:63
    #42 DB::ContextSharedPart::shutdown (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:909
    #43 0x00005555644cf5cb in DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3::operator()() const (this=this@entry=0x7fffffffc870) at /src/ch/clickhouse/programs/server/Server.cpp:1257
    #44 0x00005555644b4439 in BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::invoke (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101
    #45 BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::~BasicScopeGuard (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:50
    #46 DB::Server::main (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:2788
    #47 0x0000555572cbbee9 in Poco::Util::Application::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/base/poco/Util/src/Application.cpp:315
    #48 0x000055556449c390 in DB::Server::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:610
    #49 0x0000555564499aec in mainEntryClickHouseServer (argc=6, argv=0x7ffff67ecdc0) at /src/ch/clickhouse/programs/server/Server.cpp:397
    #50 0x000055555dc994c2 in main (argc_=<optimized out>, argv_=<optimized out>) at /src/ch/clickhouse/programs/main.cpp:340

</details>
Felixoid pushed a commit that referenced this pull request Jul 16, 2025
@azat @Felixoid
Fix deadlock on shutdown due to recursive context locking during libr…
eab9cb7 
…ary bridge cleanup

<details>

<summary>stacktrace</summary>

    #5  DB::ContextSharedMutex::lockSharedImpl (this=this@entry=0x7ffff5b96040) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1036
            watch = {start_ns = 98678735300141, stop_ns = 0, clock_type = 4, is_running = true}
            increment = {what = <optimized out>, amount = 1}
    #6  0x00005555699352d6 in DB::SharedMutexHelper<DB::ContextSharedMutex, DB::SharedMutex>::lock_shared (this=0x7ffff5b96040) at /src/ch/clickhouse/src/Common/SharedMutexHelper.h:70
    #7  DB::SharedLockGuard<DB::ContextSharedMutex>::SharedLockGuard (mutex_=..., this=<optimized out>) at /src/ch/clickhouse/src/Common/SharedLockGuard.h:19
    #8  DB::ContextSharedPart::getConfigRef (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:760
    #9  DB::Context::getConfigRef (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1590
    #10 0x000055556450941c in DB::ProxyConfigurationResolverProvider::get (request_protocol=DB::ProxyConfiguration::Protocol::HTTP) at /src/ch/clickhouse/src/Common/ProxyConfigurationResolverProvider.cpp:161
    #11 0x00005555669ea75f in DB::BuilderRWBufferFromHTTP::create (this=0x7fffffffa848, credentials_=...) at /src/ch/clickhouse/src/IO/ReadWriteBufferFromHTTP.cpp:814
    #12 0x000055556cea0035 in DB::ExternalDictionaryLibraryBridgeHelper::bridgeHandShake (this=0x7ffd60c4ca18) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:79
    #13 0x000055556cea1f34 in DB::ExternalDictionaryLibraryBridgeHelper::removeLibrary (this=0x7ffff5b96044) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:186
    #14 0x0000555566cb0248 in DB::LibraryDictionarySource::~LibraryDictionarySource (this=0x7ffd60c78318) at /src/ch/clickhouse/src/Dictionaries/LibraryDictionarySource.cpp:70
    #15 0x000055556708d775 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156
    #16 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185
    #17 std::__1::shared_ptr<DB::IDictionarySource>::~shared_ptr[abi:ne190107]() (this=0x7ffd60c605b0) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668
    #18 DB::HashedDictionary<(DB::DictionaryKeyType)0, false, false>::~HashedDictionary (this=0x7ffd60c60398) at /src/ch/clickhouse/src/Dictionaries/HashedDictionary.h:378
    #19 0x0000555569a9c477 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156
    #20 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185
    #21 std::__1::shared_ptr<DB::IExternalLoadable>::~shared_ptr[abi:ne190107]() (this=0x7ffff4832560) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668
    #22 DB::ExternalLoader::LoadingDispatcher::Info::~Info (this=0x7ffff4832548) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:719
    #23 std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>::~pair (this=0x7ffff4832530, this@entry=0x7ffd60c60390) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__utility/pair.h:65
    #24 __destroy_at<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, 0> (__loc=__loc@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/construct_at.h:67
    #25 0x0000555569a9c628 in destroy<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, void, 0> (__p=0x7ffff5b96044, __p@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:339
    #28 std::__1::unique_ptr<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*>, std::__1::__hash_node_destructor<std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*> > > >::~unique_ptr[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:261
    #31 DB::ExternalLoader::LoadingDispatcher::setConfiguration (this=this@entry=0x7ffecad4c300, new_configs=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:495
    #32 0x0000555569a9116d in DB::ExternalLoader::reloadConfig (this=0x7ffecb3fd740, repository_name=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1533
    #33 DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0::operator()() const (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1334
    #34 std::__1::__invoke[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__f=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149
    #35 std::__1::__invoke_void_return_wrapper<void, true>::__call[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__args=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:224
    #36 std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()>::operator()[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:210
    #37 std::__1::__function::__policy_invoker<void ()>::__call_impl[abi:ne190107]<std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()> #38 0x0000555569966aaa in std::__1::__function::__policy_func<void ()>::operator()[abi:ne190107]() const (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:716
    #39 std::__1::function<void()>::operator() (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:989
    #40 BasicScopeGuard<std::__1::function<void()> >::invoke (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101
    #41 BasicScopeGuard<std::__1::function<void()> >::reset (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:63
    #42 DB::ContextSharedPart::shutdown (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:909
    #43 0x00005555644cf5cb in DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3::operator()() const (this=this@entry=0x7fffffffc870) at /src/ch/clickhouse/programs/server/Server.cpp:1257
    #44 0x00005555644b4439 in BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::invoke (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101
    #45 BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::~BasicScopeGuard (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:50
    #46 DB::Server::main (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:2788
    #47 0x0000555572cbbee9 in Poco::Util::Application::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/base/poco/Util/src/Application.cpp:315
    #48 0x000055556449c390 in DB::Server::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:610
    #49 0x0000555564499aec in mainEntryClickHouseServer (argc=6, argv=0x7ffff67ecdc0) at /src/ch/clickhouse/programs/server/Server.cpp:397
    #50 0x000055555dc994c2 in main (argc_=<optimized out>, argv_=<optimized out>) at /src/ch/clickhouse/programs/main.cpp:340

</details>
Felixoid pushed a commit that referenced this pull request Jul 16, 2025
@azat @Felixoid
Fix deadlock on shutdown due to recursive context locking during libr…
9a90d0b 
…ary bridge cleanup

<details>

<summary>stacktrace</summary>

    #5  DB::ContextSharedMutex::lockSharedImpl (this=this@entry=0x7ffff5b96040) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1036
            watch = {start_ns = 98678735300141, stop_ns = 0, clock_type = 4, is_running = true}
            increment = {what = <optimized out>, amount = 1}
    #6  0x00005555699352d6 in DB::SharedMutexHelper<DB::ContextSharedMutex, DB::SharedMutex>::lock_shared (this=0x7ffff5b96040) at /src/ch/clickhouse/src/Common/SharedMutexHelper.h:70
    #7  DB::SharedLockGuard<DB::ContextSharedMutex>::SharedLockGuard (mutex_=..., this=<optimized out>) at /src/ch/clickhouse/src/Common/SharedLockGuard.h:19
    #8  DB::ContextSharedPart::getConfigRef (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:760
    #9  DB::Context::getConfigRef (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/Context.cpp:1590
    #10 0x000055556450941c in DB::ProxyConfigurationResolverProvider::get (request_protocol=DB::ProxyConfiguration::Protocol::HTTP) at /src/ch/clickhouse/src/Common/ProxyConfigurationResolverProvider.cpp:161
    #11 0x00005555669ea75f in DB::BuilderRWBufferFromHTTP::create (this=0x7fffffffa848, credentials_=...) at /src/ch/clickhouse/src/IO/ReadWriteBufferFromHTTP.cpp:814
    #12 0x000055556cea0035 in DB::ExternalDictionaryLibraryBridgeHelper::bridgeHandShake (this=0x7ffd60c4ca18) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:79
    #13 0x000055556cea1f34 in DB::ExternalDictionaryLibraryBridgeHelper::removeLibrary (this=0x7ffff5b96044) at /src/ch/clickhouse/src/BridgeHelper/ExternalDictionaryLibraryBridgeHelper.cpp:186
    #14 0x0000555566cb0248 in DB::LibraryDictionarySource::~LibraryDictionarySource (this=0x7ffd60c78318) at /src/ch/clickhouse/src/Dictionaries/LibraryDictionarySource.cpp:70
    #15 0x000055556708d775 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156
    #16 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c78300) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185
    #17 std::__1::shared_ptr<DB::IDictionarySource>::~shared_ptr[abi:ne190107]() (this=0x7ffd60c605b0) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668
    #18 DB::HashedDictionary<(DB::DictionaryKeyType)0, false, false>::~HashedDictionary (this=0x7ffd60c60398) at /src/ch/clickhouse/src/Dictionaries/HashedDictionary.h:378
    #19 0x0000555569a9c477 in std::__1::__shared_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:156
    #20 std::__1::__shared_weak_count::__release_shared[abi:ne190107]() (this=0x7ffd60c60380) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:185
    #21 std::__1::shared_ptr<DB::IExternalLoadable>::~shared_ptr[abi:ne190107]() (this=0x7ffff4832560) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:668
    #22 DB::ExternalLoader::LoadingDispatcher::Info::~Info (this=0x7ffff4832548) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:719
    #23 std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>::~pair (this=0x7ffff4832530, this@entry=0x7ffd60c60390) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__utility/pair.h:65
    #24 __destroy_at<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, 0> (__loc=__loc@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/construct_at.h:67
    #25 0x0000555569a9c628 in destroy<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::ExternalLoader::LoadingDispatcher::Info>, void, 0> (__p=0x7ffff5b96044, __p@entry=0x7ffff4832530) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:339
    #28 std::__1::unique_ptr<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*>, std::__1::__hash_node_destructor<std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::ExternalLoader::LoadingDispatcher::Info>, void*> > > >::~unique_ptr[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:261
    #31 DB::ExternalLoader::LoadingDispatcher::setConfiguration (this=this@entry=0x7ffecad4c300, new_configs=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:495
    #32 0x0000555569a9116d in DB::ExternalLoader::reloadConfig (this=0x7ffecb3fd740, repository_name=...) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1533
    #33 DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0::operator()() const (this=<optimized out>) at /src/ch/clickhouse/src/Interpreters/ExternalLoader.cpp:1334
    #34 std::__1::__invoke[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__f=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149
    #35 std::__1::__invoke_void_return_wrapper<void, true>::__call[abi:ne190107]<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&>(DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0&) (__args=...) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__type_traits/invoke.h:224
    #36 std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()>::operator()[abi:ne190107]() (this=<optimized out>) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:210
    #37 std::__1::__function::__policy_invoker<void ()>::__call_impl[abi:ne190107]<std::__1::__function::__default_alloc_func<DB::ExternalLoader::addConfigRepository(std::__1::unique_ptr<DB::IExternalLoaderConfigRepository, std::__1::default_delete<DB::IExternalLoaderConfigRepository> >) const::$_0, void ()> #38 0x0000555569966aaa in std::__1::__function::__policy_func<void ()>::operator()[abi:ne190107]() const (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:716
    #39 std::__1::function<void()>::operator() (this=0x7ffff5b96044) at /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:989
    #40 BasicScopeGuard<std::__1::function<void()> >::invoke (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101
    #41 BasicScopeGuard<std::__1::function<void()> >::reset (this=0x7ffff5b96044) at /src/ch/clickhouse/base/base/../base/scope_guard.h:63
    #42 DB::ContextSharedPart::shutdown (this=0x7ffff5b96000) at /src/ch/clickhouse/src/Interpreters/Context.cpp:909
    #43 0x00005555644cf5cb in DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3::operator()() const (this=this@entry=0x7fffffffc870) at /src/ch/clickhouse/programs/server/Server.cpp:1257
    #44 0x00005555644b4439 in BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::invoke (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:101
    #45 BasicScopeGuard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_3>::~BasicScopeGuard (this=0x7fffffffc870) at /src/ch/clickhouse/base/base/../base/scope_guard.h:50
    #46 DB::Server::main (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:2788
    #47 0x0000555572cbbee9 in Poco::Util::Application::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/base/poco/Util/src/Application.cpp:315
    #48 0x000055556449c390 in DB::Server::run (this=0x7fffffffe4c8) at /src/ch/clickhouse/programs/server/Server.cpp:610
    #49 0x0000555564499aec in mainEntryClickHouseServer (argc=6, argv=0x7ffff67ecdc0) at /src/ch/clickhouse/programs/server/Server.cpp:397
    #50 0x000055555dc994c2 in main (argc_=<optimized out>, argv_=<optimized out>) at /src/ch/clickhouse/programs/main.cpp:340

</details>
@serxa serxa mentioned this pull request Aug 27, 2025
Closed
@karthik-thiyagarajan karthik-thiyagarajan mentioned this pull request Oct 9, 2025
Open
AVMusorin added a commit to AVMusorin/ClickHouse that referenced this pull request Nov 17, 2025
@AVMusorin
Fix crash in StorageDistributed when parsing malformed shard director…
8cd469f 
…y names

When ClickHouse encounters a distributed table directory with a name that:
- starts with the pattern `shardN_all_replicas` (e.g., `shard3_all_replicas_bkp`)
- but doesn't end with exactly `_all_replicas`

It would crash with a segmentation fault due to out-of-bounds array access.

Trace example:
```
* thread ClickHouse#1, name = 'clickhouse-serv', stop reason = signal SIGSEGV
  * frame #0: 0x000000001194481b clickhouse-server`DB::Cluster::Address::Address(DB::Cluster::Address const&) + 27
    frame ClickHouse#1: 0x0000000011944550 clickhouse-server`void std::__1::vector<DB::Cluster::Address, std::__1::allocator<DB::Cluster::Address> >::__push_back_slow_path<DB::Cluster::Address const&>(DB::Cluster::Address const&&&) + 208
    frame ClickHouse#2: 0x00000000129a6c35 clickhouse-server`DB::StorageDistributed::parseAddresses(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const + 3349
    frame ClickHouse#3: 0x00000000129a589b clickhouse-server`DB::StorageDistributed::getDirectoryQueue(std::__1::shared_ptr<DB::IDisk> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 3675
    frame ClickHouse#4: 0x00000000129a3f22 clickhouse-server`DB::StorageDistributed::initializeDirectoryQueuesForDisk(std::__1::shared_ptr<DB::IDisk> const&) + 2082
    frame ClickHouse#5: 0x000000000f681c48 clickhouse-server`DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true> >, std::__1::function<void ()> >::operator()(std::__1::function<void ()>&&, Priority)::'lambda'()::operator()() + 136
    frame ClickHouse#6: 0x000000000f68286c clickhouse-server`std::__1::packaged_task<void ()>::operator()() + 44
    frame ClickHouse#7: 0x000000000d83335b clickhouse-server`ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true> >::ThreadFromThreadPool::worker() + 2395
    frame ClickHouse#8: 0x000000000d839e23 clickhouse-server`void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false, true>::ThreadFromGlobalPoolImpl<void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true> >::ThreadFromThreadPool::*)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true> >::ThreadFromThre
adPool*>(void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true> >::ThreadFromThreadPool::*&&)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true> >::ThreadFromThreadPool*&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) + 99
    frame ClickHouse#9: 0x000000000d830602 clickhouse-server`ThreadPoolImpl<std::__1::thread>::ThreadFromThreadPool::worker() + 2242
    frame ClickHouse#10: 0x000000000d8379ba clickhouse-server`_ZNSt3__114__thread_proxyB6v15007INS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEMN14ThreadPoolImplINS_6threadEE20ThreadFromThreadPoolEFvvEPSA_EEEEEPvSF_ + 90
```
@kssenii kssenii mentioned this pull request Dec 4, 2025
Closed
@Blargian Blargian mentioned this pull request Dec 22, 2025
Merged
1 task
github-merge-queue bot pushed a commit that referenced this pull request Dec 24, 2025
@Blargian
Merge pull request #92811 from Blargian/aggregate_functions_5
0c359ba 
Docs: move aggregate functions docs to source (#5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@smagellan @alexey-milovidov