feature: Introduce Cleanup API for `TableMetaRefreshHolder` Instance #7559

YongGoose · 2025-07-27T08:02:30Z

I have registered the PR changes.

Ⅰ. Describe what this PR did

Ⅱ. Does this pull request fix one issue?

Signed-off-by: yongjunhong <[email protected]>

codecov · 2025-07-27T08:17:33Z

Codecov Report

❌ Patch coverage is 85.71429% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.03%. Comparing base (d1ae495) to head (e4e0473).
⚠️ Report is 1 commits behind head on 2.x.

Files with missing lines	Patch %	Lines
...m/datasource/sql/struct/TableMetaCacheFactory.java	83.33%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##                2.x    #7559      +/-   ##
============================================
- Coverage     61.05%   61.03%   -0.02%     
+ Complexity      670      666       -4     
============================================
  Files          1316     1316              
  Lines         49804    49819      +15     
  Branches       5855     5858       +3     
============================================
+ Hits          30407    30408       +1     
- Misses        16689    16700      +11     
- Partials       2708     2711       +3

Files with missing lines	Coverage Δ
...rg/apache/seata/rm/datasource/DataSourceProxy.java	`67.87% <100.00%> (+0.33%)`	⬆️
...m/datasource/sql/struct/TableMetaCacheFactory.java	`49.31% <83.33%> (+7.64%)`	⬆️

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

funky-eyes · 2025-07-31T03:11:23Z

...atasource/src/main/java/org/apache/seata/rm/datasource/sql/struct/TableMetaCacheFactory.java

+    /**
+     * Shutdown all TableMetaRefreshHolder threads.
+     */
+    public static void shutdown() {


Is it possible for us to use the publish-subscribe pattern, so that after the rm close message is published, TableMetaCache proceeds to shut down?

Is it possible for us to use the publish-subscribe pattern, so that after the rm close message is published, TableMetaCache proceeds to shut down?

Something like this could work.

However, compared to explicitly calling the shutdown API when the RM is destroyed, I don’t see a significant advantage.

If we go with the approach where TableMetaCacheFactory implements a listener, we could consider making shutdown a private API.

What do you think?

When registering itself with the Seata-Server (TC), I believe that DataSourceProxy should proxy the close method. Upon closing, it should remove itself from both DefaultResourceManager and TableMetaCacheFactory. Currently, TableMetaCacheFactory#removeHolderFromMap is executed passively—only when refreshing table metadata encounters an exception due to a closed datasource does the removal occur. This approach, however, feels inelegant.

When registering itself with the Seata-Server (TC), I believe that DataSourceProxy should proxy the close method. Upon closing, it should remove itself from both DefaultResourceManager and TableMetaCacheFactory. Currently, TableMetaCacheFactory#removeHolderFromMap is executed passively—only when refreshing table metadata encounters an exception due to a closed datasource does the removal occur. This approach, however, feels inelegant.

Out of curiosity, I don't seem to see any class that actually implements org.apache.seata.core.model.ResourceManager#unregisterResource(org.apache.seata.core.model.Resource).

org.apache.seata.common.exception.NotSupportYetException: unregister a resource at org.apache.seata.rm.datasource.DataSourceManager.unregisterResource(DataSourceManager.java:96) at org.apache.seata.rm.DefaultResourceManager.unregisterResource(DefaultResourceManager.java:125)

I think so too

When registering itself with the Seata-Server (TC), I believe that DataSourceProxy should proxy the close method. Upon closing, it should remove itself from both DefaultResourceManager and TableMetaCacheFactory. Currently, TableMetaCacheFactory#removeHolderFromMap is executed passively—only when refreshing table metadata encounters an exception due to a closed datasource does the removal occur. This approach, however, feels inelegant.

Out of curiosity, I don't seem to see any class that actually implements org.apache.seata.core.model.ResourceManager#unregisterResource(org.apache.seata.core.model.Resource).

org.apache.seata.common.exception.NotSupportYetException: unregister a resource at org.apache.seata.rm.datasource.DataSourceManager.unregisterResource(DataSourceManager.java:96) at org.apache.seata.rm.DefaultResourceManager.unregisterResource(DefaultResourceManager.java:125)

Yes, that’s why we need to actually perform the unregisterResource operation on it.

Yes, that’s why we need to actually perform the unregisterResource operation on it.

Shall we handle this together in this PR?

Yes, that’s why we need to actually perform the unregisterResource operation on it.

Shall we handle this together in this PR?

Ok

Could you please let me know if there’s anything I’m missing or need to improve in eea1fdd !

…eManager and TableMetaCacheFactory Signed-off-by: yongjunhong <[email protected]>

Signed-off-by: yongjunhong <[email protected]>

YongGoose · 2025-08-15T14:31:39Z

rm-datasource/src/main/java/org/apache/seata/rm/datasource/DataSourceProxy.java

 *
 */
-public class DataSourceProxy extends AbstractDataSourceProxy implements Resource {
+public class DataSourceProxy extends AbstractDataSourceProxy implements Resource, AutoCloseable {


How about implementing AutoCloseable so that it can be automatically closed using a try-with-resources statement?

WDYT?

How about implementing AutoCloseable so that it can be automatically closed using a try-with-resources statement?

WDYT?

I don’t think it’s necessary, because the datasource itself already provides a close interface.

Done in f111e69 !

Signed-off-by: yongjunhong <[email protected]>

funky-eyes · 2025-08-19T09:13:11Z

core/src/main/java/org/apache/seata/core/protocol/UnregisterRMRequest.java

+/**
+ * Represents a request to unregister a resource from the Resource Manager (RM).
+ */
+public class UnregisterRMRequest extends AbstractIdentifyRequest implements Serializable {


It seems that I haven’t found anywhere on the server side that handles this request.

funky-eyes · 2025-08-23T13:06:36Z

The functionality of UnregisterRMRequest should be separated into an individual PR, with the current PR focusing solely on resolving the issue where tableMetaRefresh does not terminate.
In NettyClientChannelManager, cache the rpcContext for a channel based on the response from RegisterRMResponse, and also cache the server version number. You may refer to the design of the RpcContext class for this implementation.
When checking the server version, if it is lower than 2.6.0, output a log indicating that UnregisterRMRequest is not supported by older server versions, and therefore, resources cannot be unregistered.
RMClient should provide both destroy and destroyAll interfaces. The destroy interface should accept resource information, construct an UnregisterRMRequest, and send it to the server to unregister the resource. The destroyAll interface should proactively notify the server that the client node is going offline.
These points can be split into multiple PRs to improve review efficiency and ensure that each PR is clear and focused.

Hi @slievrly, The above is a summary of our previous meeting. Do you have any additional comments or suggestions?

YongGoose · 2025-08-23T13:10:07Z

5. These points can be split into multiple PRs to improve review efficiency and ensure that each PR is clear and focused.

Agree

YongGoose · 2025-08-24T05:28:30Z

The functionality of UnregisterRMRequest should be separated into an individual PR, with the current PR focusing solely on resolving the issue where tableMetaRefresh does not terminate.

Please take another look!
Reverted the commit to f111e69!

YongGoose · 2025-08-24T05:29:03Z

In NettyClientChannelManager, cache the rpcContext for a channel based on the response from RegisterRMResponse, and also cache the server version number. You may refer to the design of the RpcContext class for this implementation.

When checking the server version, if it is lower than 2.6.0, output a log indicating that UnregisterRMRequest is not supported by older server versions, and therefore, resources cannot be unregistered.

RMClient should provide both destroy and destroyAll interfaces. The destroy interface should accept resource information, construct an UnregisterRMRequest, and send it to the server to unregister the resource. The destroyAll interface should proactively notify the server that the client node is going offline.

These points can be split into multiple PRs to improve review efficiency and ensure that each PR is clear and focused.

For this tasks, I'll open a issue and get to work :)

rm-datasource/src/main/java/org/apache/seata/rm/datasource/DataSourceProxy.java

funky-eyes · 2025-08-25T06:40:03Z

rm-datasource/src/main/java/org/apache/seata/rm/datasource/DataSourceProxy.java

    }
+
+    public void close() throws Exception {
+        DefaultResourceManager.get().unregisterResource(this);


The unregisterResource method of DataSourceManager throws a NotSupportYetException with the message "unregister a resource".

That’s correct.
But wasn’t the implementation for that part planned to be done in a separate PR?

That’s correct. But wasn’t the implementation for that part planned to be done in a separate PR?

So I think this line of code should be added in a separate PR, not in this one.

Done in 189b5c6 !

Done in 189b5c6 !

I think there may have been a misunderstanding. I meant that the line
DefaultResourceManager.get().unregisterResource(this);
should be removed from this PR and only reintroduced once unregisterResource has been properly implemented.

OK 👍🏻

…aSourceProxy.java Co-authored-by: funkye <[email protected]>

Signed-off-by: yongjunhong <[email protected]>

funky-eyes

I think we should rewrite the PR title and then register the pull request.

funky-eyes

One more question: should the TABLE_META_CACHE in AbstractTableMetaCache be cleared when a data source is closed? I also noticed TABLE_META_CACHE is shared globally — under multiple data sources, if different databases contain tables with the same name, could that cause metadata conflicts? In addition, when calling tableMetaCache.refresh the connection used is tied to a single database, and that connection cannot retrieve metadata for tables in other databases. @slievrly, could you please confirm whether the issues I described actually exist?

slievrly · 2025-09-01T13:39:08Z

5. These points can be split into multiple PRs to improve review efficiency and ensure that each PR is clear and focused.

OK, go ahead.

slievrly

LGTM

YongGoose · 2025-09-09T11:49:00Z

@slievrly

One more question: should the TABLE_META_CACHE in AbstractTableMetaCache be cleared when a data source is closed? I also noticed TABLE_META_CACHE is shared globally — under multiple data sources, if different databases contain tables with the same name, could that cause metadata conflicts? In addition, when calling tableMetaCache.refresh the connection used is tied to a single database, and that connection cannot retrieve metadata for tables in other databases.

Please take a look at this question :)

…beforeeach Signed-off-by: yongjunhong <[email protected]>

Signed-off-by: yongjunhong <[email protected]>

funky-eyes

LGTM

TODO:

One more question: should the TABLE_META_CACHE in AbstractTableMetaCache be cleared when a data source is closed? I also noticed TABLE_META_CACHE is shared globally — under multiple data sources, if different databases contain tables with the same name, could that cause metadata conflicts? In addition, when calling tableMetaCache.refresh the connection used is tied to a single database, and that connection cannot retrieve metadata for tables in other databases. @slievrly, could you please confirm whether the issues I described actually exist?

…pache#7559)

Add Public API to Clear All TableMetaRefreshHolder Instances

407a681

Signed-off-by: yongjunhong <[email protected]>

YongGoose mentioned this pull request Jul 27, 2025

When destroying RmNettyRemotingClient, all TableMetaRefreshHolder should actively close the process #7523

Open

2 tasks

Merge branch '2.x' into feature/7523

3d1b551

funky-eyes reviewed Jul 31, 2025

View reviewed changes

YongGoose added 5 commits August 15, 2025 22:30

Merge branch '2.x' into feature/7523

f6706be

Proxy DataSourceProxy#close to proactively remove from DefaultResourc…

28fa63a

…eManager and TableMetaCacheFactory Signed-off-by: yongjunhong <[email protected]>

Make shutdown method private

ab31516

Signed-off-by: yongjunhong <[email protected]>

test

04f37ff

Signed-off-by: yongjunhong <[email protected]>

Apply spotless

30d4a98

Signed-off-by: yongjunhong <[email protected]>

YongGoose requested a review from funky-eyes August 15, 2025 14:24

YongGoose commented Aug 15, 2025

View reviewed changes

Remove autocloseable interface

f111e69

Signed-off-by: yongjunhong <[email protected]>

YongGoose force-pushed the feature/7523 branch from a9a67e4 to eea1fdd Compare August 19, 2025 07:55

funky-eyes reviewed Aug 19, 2025

View reviewed changes

YongGoose force-pushed the feature/7523 branch from eea1fdd to f111e69 Compare August 24, 2025 05:26

Merge branch '2.x' into feature/7523

4654291

YongGoose requested a review from funky-eyes August 24, 2025 05:53

funky-eyes added this to the 2.6.0 milestone Aug 25, 2025

funky-eyes reviewed Aug 25, 2025

View reviewed changes

rm-datasource/src/main/java/org/apache/seata/rm/datasource/DataSourceProxy.java Show resolved Hide resolved

funky-eyes reviewed Aug 25, 2025

View reviewed changes

YongGoose and others added 2 commits August 25, 2025 15:55

Update rm-datasource/src/main/java/org/apache/seata/rm/datasource/Dat…

45220f2

…aSourceProxy.java Co-authored-by: funkye <[email protected]>

Revert changes

a884d71

Signed-off-by: yongjunhong <[email protected]>

YongGoose force-pushed the feature/7523 branch from 189b5c6 to 7d94fd1 Compare August 26, 2025 02:28

Abort the close method implementation

479cf7b

Signed-off-by: yongjunhong <[email protected]>

YongGoose force-pushed the feature/7523 branch from 7d94fd1 to 479cf7b Compare August 26, 2025 02:39

funky-eyes added module/rm-datasource rm-datasource module type: optimize labels Aug 26, 2025

funky-eyes reviewed Aug 26, 2025

View reviewed changes

funky-eyes requested a review from slievrly August 26, 2025 13:41

funky-eyes added 2 commits August 27, 2025 13:36

Merge branch '2.x' into feature/7523

5f8ff5c

Merge branch '2.x' into feature/7523

999649d

slievrly approved these changes Sep 1, 2025

View reviewed changes

YongGoose added 3 commits September 9, 2025 20:49

Merge branch '2.x' into feature/7523

74df959

Merge branch '2.x' into feature/7523

878c683

Ensure test isolation by clearing TableMetaCacheFactory cache map in @…

08fc422

…beforeeach Signed-off-by: yongjunhong <[email protected]>

YongGoose changed the title ~~Add Public API to Clear All TableMetaRefreshHolder Instances~~ feature: Introduce Cleanup API for TableMetaRefreshHolder Instance Sep 13, 2025

Add changes

f648b23

Signed-off-by: yongjunhong <[email protected]>

YongGoose requested review from funky-eyes and slievrly September 13, 2025 02:52

Merge branch '2.x' into feature/7523

e4e0473

funky-eyes approved these changes Sep 22, 2025

View reviewed changes

funky-eyes merged commit 7accdf8 into apache:2.x Sep 22, 2025
16 of 18 checks passed

slievrly pushed a commit to slievrly/fescar that referenced this pull request Oct 21, 2025

feature: Introduce Cleanup API for TableMetaRefreshHolder Instance (a…

c5d7dee

…pache#7559)

YvCeung pushed a commit to YvCeung/incubator-seata that referenced this pull request Dec 25, 2025

feature: Introduce Cleanup API for TableMetaRefreshHolder Instance (a…

3c7ded7

…pache#7559)

feature: Introduce Cleanup API for TableMetaRefreshHolder Instance #7559

feature: Introduce Cleanup API for TableMetaRefreshHolder Instance #7559

Uh oh!

Conversation

YongGoose commented Jul 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Ⅰ. Describe what this PR did

Ⅱ. Does this pull request fix one issue?

Uh oh!

codecov bot commented Jul 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YongGoose Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

funky-eyes commented Aug 23, 2025

Uh oh!

YongGoose commented Aug 23, 2025

Uh oh!

YongGoose commented Aug 24, 2025

Uh oh!

YongGoose commented Aug 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

funky-eyes left a comment

Choose a reason for hiding this comment

Uh oh!

funky-eyes left a comment

Choose a reason for hiding this comment

Uh oh!

slievrly commented Sep 1, 2025

Uh oh!

slievrly left a comment

Choose a reason for hiding this comment

Uh oh!

YongGoose commented Sep 9, 2025

Uh oh!

funky-eyes left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

feature: Introduce Cleanup API for `TableMetaRefreshHolder` Instance #7559

feature: Introduce Cleanup API for `TableMetaRefreshHolder` Instance #7559

YongGoose commented Jul 27, 2025 •

edited

Loading

codecov bot commented Jul 27, 2025 •

edited

Loading

YongGoose Aug 19, 2025 •

edited

Loading

YongGoose commented Aug 24, 2025 •

edited

Loading