Thanks to visit codestin.com
Credit goes to github.com

Skip to content

SOLR-17877: Move distributedCollectionCommandRunner from CoreContainer to ZkController #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 14 commits into from
Closed

SOLR-17877: Move distributedCollectionCommandRunner from CoreContainer to ZkController #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
2818ad1
Initial plan
Copilot Aug 24, 2025
0b21071
Move distributedCollectionCommandRunner from CoreContainer to ZkContr…
Copilot Aug 24, 2025
1e5d216
Fix remaining compilation errors after moving distributedCollectionCo…
Copilot Aug 24, 2025
4bf80ab
Update test files and apply code formatting after moving distributedC…
Copilot Aug 24, 2025
1d31b1b
Address PR review comments: remove conditionals, improve method organ…
Copilot Aug 25, 2025
befe22b
getDistributedCollectionCommandRunner -> getDistribCommandRunner
dsmiley Aug 25, 2025
c5bf0b3
tweaks
dsmiley Aug 25, 2025
af0871b
rename to getDistributedCommandRunner
dsmiley Aug 25, 2025
e7430f2
Fix: can't access ZkController too early
dsmiley Aug 25, 2025
07b5495
fix mock tests
dsmiley Aug 25, 2025
24fad22
Tag renovate PRs with 'exempt-stale' label (#3483)
janhoy Aug 25, 2025
3a3e6b2
SOLR-17871: fix error-prone warning
dsmiley Aug 26, 2025
cfc9e5b
Merge branch 'main' into fork/dsmiley/copilot/fix-6ebc3776-b156-40ce-…
dsmiley Aug 26, 2025
653165d
Move initialization sooner; make final.
dsmiley Aug 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/renovate.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"enabled": true,
"dependencyDashboard": false,
"enabledManagers": ["gradle", "github-actions"],
"labels": ["exempt-stale"],
"includePaths": ["gradle/libs.versions.toml", "versions.*", "build.gradle", ".github/workflows/*"],
"postUpgradeTasks": {
"commands": [
Expand Down
29 changes: 28 additions & 1 deletion solr/core/src/java/org/apache/solr/cloud/ZkController.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@
import org.apache.solr.client.solrj.impl.SolrZkClientTimeout;
import org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider;
import org.apache.solr.client.solrj.request.CoreAdminRequest.WaitForState;
import org.apache.solr.cloud.api.collections.DistributedCollectionConfigSetCommandRunner;
import org.apache.solr.cloud.overseer.ClusterStateMutator;
import org.apache.solr.cloud.overseer.OverseerAction;
import org.apache.solr.cloud.overseer.SliceMutator;
Expand Down Expand Up @@ -225,6 +226,8 @@ public String toString() {

private final DistributedClusterStateUpdater distributedClusterStateUpdater;

private final Optional<DistributedCollectionConfigSetCommandRunner> distributedCommandRunner;

private LeaderElector overseerElector;

private Map<String, ReplicateFromLeader> replicateFromLeaders = new ConcurrentHashMap<>();
Expand Down Expand Up @@ -372,6 +375,11 @@ public ZkController(
// Refuse to start if ZK has a non empty /clusterstate.json or a /solr.xml file
checkNoOldClusterstate(zkClient);

this.distributedCommandRunner =
cloudConfig.getDistributedCollectionConfigSetExecution()
? Optional.of(new DistributedCollectionConfigSetCommandRunner(cc, zkClient))
: Optional.empty();

this.overseerRunningMap = Overseer.getRunningMap(zkClient);
this.overseerCompletedMap = Overseer.getCompletedMap(zkClient);
this.overseerFailureMap = Overseer.getFailureMap(zkClient);
Expand Down Expand Up @@ -686,6 +694,22 @@ public NodesSysPropsCacher getSysPropsCacher() {
return sysPropsCacher;
}

/** Non-empty if the Collection API is executed in a distributed way (Overseer is disabled). */
public Optional<DistributedCollectionConfigSetCommandRunner> getDistributedCommandRunner() {
return this.distributedCommandRunner;
}

/** Waits for pending tasks to complete. Should be called before {@link #close()}. */
public void waitForPendingTasksToComplete() {
if (distributedCommandRunner.isPresent()) {
// Local (i.e. distributed) Collection API processing
distributedCommandRunner.get().stopAndWaitForPendingTasksToComplete();
} else {
// Overseer based processing
getOverseerCollectionQueue().allowOverseerPendingTasksToComplete();
}
}

private ContextKey closeExistingElectionContext(CoreDescriptor cd, boolean sessionExpired) {
// look for old context - if we find it, cancel it
String collection = cd.getCloudDescriptor().getCollectionName();
Expand Down Expand Up @@ -784,6 +808,7 @@ public void close() {
} finally {

sysPropsCacher.close();

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot No need to do this here since stopAndWaitForPendingTasksToComplete is already called earlier before close is called.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the duplicate cleanup logic since stopAndWaitForPendingTasksToComplete is already called earlier. (1d31b1b)

customThreadPool.execute(() -> IOUtils.closeQuietly(cloudManager));
customThreadPool.execute(() -> IOUtils.closeQuietly(cloudSolrClient));

Expand Down Expand Up @@ -1003,7 +1028,9 @@ private void init() {
checkForExistingEphemeralNode();
registerLiveNodesListener();

// start the overseer first as following code may need it's processing
// Start the overseer now since the following code may need it's processing.
// Note: even when using distributed processing, we still create an Overseer anyway since
// cluster singleton processing is linked to the elected Overseer.
if (!zkRunOnly) {
overseerElector = new LeaderElector(zkClient);
this.overseer =
Expand Down
18 changes: 6 additions & 12 deletions ...e/src/java/org/apache/solr/cloud/api/collections/DistributedCollectionCommandContext.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,23 +27,17 @@

public class DistributedCollectionCommandContext implements CollectionCommandContext {
private final CoreContainer coreContainer;
private final DistributedClusterStateUpdater getDistributedClusterStateUpdater;
private final DistributedClusterStateUpdater distributedClusterStateUpdater;
private final ExecutorService executorService;

private final SolrCloudManager solrCloudManager;
private final ZkStateReader zkStateReader;

public DistributedCollectionCommandContext(
CoreContainer coreContainer, ExecutorService executorService) {
// note: coreContainer.getZkController() is not yet instantiated; don't call it right now
this.coreContainer = coreContainer;
this.getDistributedClusterStateUpdater =
this.distributedClusterStateUpdater =
new DistributedClusterStateUpdater(
coreContainer.getConfig().getCloudConfig().getDistributedClusterStateUpdates());
;
this.executorService = executorService;

solrCloudManager = this.coreContainer.getZkController().getSolrCloudManager();
zkStateReader = this.coreContainer.getZkController().getZkStateReader();
}

@Override
Expand All @@ -60,7 +54,7 @@ public ShardHandler newShardHandler() {

@Override
public SolrCloudManager getSolrCloudManager() {
return solrCloudManager;
return this.coreContainer.getZkController().getSolrCloudManager();
}

@Override
Expand All @@ -70,12 +64,12 @@ public CoreContainer getCoreContainer() {

@Override
public ZkStateReader getZkStateReader() {
return zkStateReader;
return this.coreContainer.getZkController().getZkStateReader();
}

@Override
public DistributedClusterStateUpdater getDistributedClusterStateUpdater() {
return this.getDistributedClusterStateUpdater;
return this.distributedClusterStateUpdater;
}

@Override
Expand Down
8 changes: 5 additions & 3 deletions ...va/org/apache/solr/cloud/api/collections/DistributedCollectionConfigSetCommandRunner.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
import org.apache.solr.cloud.ZkDistributedCollectionLockFactory;
import org.apache.solr.cloud.ZkDistributedConfigSetLockFactory;
import org.apache.solr.common.SolrException;
import org.apache.solr.common.cloud.SolrZkClient;
import org.apache.solr.common.cloud.ZkNodeProps;
import org.apache.solr.common.params.CollectionParams;
import org.apache.solr.common.params.ConfigSetParams;
Expand Down Expand Up @@ -106,7 +107,9 @@ public class DistributedCollectionConfigSetCommandRunner {

private volatile boolean shuttingDown = false;

public DistributedCollectionConfigSetCommandRunner(CoreContainer coreContainer) {
public DistributedCollectionConfigSetCommandRunner(
CoreContainer coreContainer, SolrZkClient zkClient) {
// note: coreContainer.getZkController() is not yet instantiated; don't call it right now
this.coreContainer = coreContainer;

if (log.isInfoEnabled()) {
Expand Down Expand Up @@ -144,8 +147,7 @@ public DistributedCollectionConfigSetCommandRunner(CoreContainer coreContainer)
new DistributedCollectionCommandContext(
this.coreContainer, this.distributedCollectionApiExecutorService);
commandMapper = new CollApiCmds.CommandMap(ccc);
asyncTaskTracker =
new DistributedApiAsyncTracker(ccc.getZkStateReader().getZkClient(), ZK_ASYNC_ROOT);
asyncTaskTracker = new DistributedApiAsyncTracker(zkClient, ZK_ASYNC_ROOT);
}

/** See {@link DistributedApiAsyncTracker#getAsyncTaskRequestStatus(String)} */
Expand Down
2 changes: 1 addition & 1 deletion solr/core/src/java/org/apache/solr/cloud/api/collections/OverseerStatusCmd.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ public void call(ClusterState state, ZkNodeProps message, NamedList<Object> resu
throws Exception {
// If Collection API execution is distributed, we're not running on the Overseer node so can't
// return any Overseer stats.
if (ccc.getCoreContainer().getDistributedCollectionCommandRunner().isPresent()) {
if (ccc.getCoreContainer().getZkController().getDistributedCommandRunner().isPresent()) {
// TODO: introduce a per node status command allowing insight into how Cluster state updates,
// Collection API and config set API execution went on that node...
return;
Expand Down
43 changes: 2 additions & 41 deletions solr/core/src/java/org/apache/solr/core/CoreContainer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Optional;
import java.util.Properties;
import java.util.Set;
import java.util.UUID;
Expand Down Expand Up @@ -77,9 +76,7 @@
import org.apache.solr.client.solrj.util.SolrIdentifierValidator;
import org.apache.solr.cloud.CloudDescriptor;
import org.apache.solr.cloud.ClusterSingleton;
import org.apache.solr.cloud.OverseerTaskQueue;
import org.apache.solr.cloud.ZkController;
import org.apache.solr.cloud.api.collections.DistributedCollectionConfigSetCommandRunner;
import org.apache.solr.cluster.events.ClusterEventProducer;
import org.apache.solr.cluster.events.impl.ClusterEventProducerFactory;
import org.apache.solr.cluster.placement.PlacementPluginConfig;
Expand Down Expand Up @@ -326,14 +323,6 @@ && getZkController().getOverseer() != null
private ExecutorService coreContainerAsyncTaskExecutor =
ExecutorUtil.newMDCAwareCachedThreadPool("Core Container Async Task");

/**
* Non-empty if the Collection API is executed in a distributed way and not on Overseer, once the
* CoreContainer has been initialized properly, i.e. method {@link #load()} called. Until then it
* is null, and it is not expected to be read.
*/
private volatile Optional<DistributedCollectionConfigSetCommandRunner>
distributedCollectionCommandRunner;

private enum CoreInitFailedAction {
fromleader,
none
Expand Down Expand Up @@ -686,7 +675,6 @@ protected CoreContainer(Object testConstructor) {
cfg = null;
containerProperties = null;
replayUpdatesExecutor = null;
distributedCollectionCommandRunner = Optional.empty();
allowPaths = null;
allowListUrlChecker = null;
indexSearcherExecutor = null;
Expand Down Expand Up @@ -900,19 +888,6 @@ private void loadInternal() {
createHandler(
ZK_STATUS_PATH, ZookeeperStatusHandler.class.getName(), ZookeeperStatusHandler.class);

// CoreContainer is initialized enough at this stage so we can set
// distributedCollectionCommandRunner (the construction of
// DistributedCollectionConfigSetCommandRunner uses Zookeeper so can't be done from the
// CoreContainer constructor because there Zookeeper is not yet ready). Given this is used in
// the CollectionsHandler created next line, this is the latest point where
// distributedCollectionCommandRunner can be initialized without refactoring this method...
// TODO: manage to completely build CoreContainer in the constructor and not in the load()
// method... Requires some test refactoring.
this.distributedCollectionCommandRunner =
isZooKeeperAware() && cfg.getCloudConfig().getDistributedCollectionConfigSetExecution()
? Optional.of(new DistributedCollectionConfigSetCommandRunner(this))
: Optional.empty();

collectionsHandler =
createHandler(
COLLECTIONS_HANDLER_PATH, cfg.getCollectionsHandlerClass(), CollectionsHandler.class);
Expand Down Expand Up @@ -1246,9 +1221,7 @@ protected void configure() {
throw new SolrException(ErrorCode.SERVER_ERROR, e);
}
}
if (!distributedCollectionCommandRunner.isPresent()) {
zkSys.getZkController().checkOverseerDesignate();
}
zkSys.getZkController().checkOverseerDesignate();
}

// This is a bit redundant but these are two distinct concepts for all they're accomplished at
Expand Down Expand Up @@ -1318,14 +1291,7 @@ public void shutdown() {

ZkController zkController = getZkController();
if (zkController != null) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot This logic of waiting for pending tasks to complete should be moved to ZkController -- a new method maybe named waitForPendingTasksToComplete

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added waitForPendingTasksToComplete() method to ZkController and updated CoreContainer.shutdown() to use it. This cleanly encapsulates the pending task logic within ZkController. (1d31b1b)

if (distributedCollectionCommandRunner.isPresent()) {
// Local (i.e. distributed) Collection API processing
distributedCollectionCommandRunner.get().stopAndWaitForPendingTasksToComplete();
} else {
// Overseer based processing
OverseerTaskQueue overseerCollectionQueue = zkController.getOverseerCollectionQueue();
overseerCollectionQueue.allowOverseerPendingTasksToComplete();
}
zkController.waitForPendingTasksToComplete();
}
if (log.isInfoEnabled()) {
log.info("Shutting down CoreContainer instance={}", System.identityHashCode(this));
Expand Down Expand Up @@ -2595,11 +2561,6 @@ public PlacementPluginFactory<? extends PlacementPluginConfig> getPlacementPlugi
return placementPluginFactory;
}

public Optional<DistributedCollectionConfigSetCommandRunner>
getDistributedCollectionCommandRunner() {
return this.distributedCollectionCommandRunner;
}

/**
* A general-purpose HTTP/2 Solr client.
*
Expand Down
Loading
Loading