支持不同topic指定专属partitionNum #2479
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
各位好，这儿能不能不指定分区数目。 Kafka的API中 可以指定写入哪个分区 在这儿看到的是 没有设置分区 就写到第0个分区 这样做的目的是 保证消息顺序么？ |
Member
|
@GuangYaoLee92 挺好的想法，目前MQ结构做了下SPI设计，代码和现在的结构略有冲突，请先解决一下冲突 |
ca146cc to
621b2cc
Compare
Codecov Report
@@ Coverage Diff @@
## master #2479 +/- ##
===========================================
+ Coverage 4.77% 4.89% +0.11%
Complexity 239 239
===========================================
Files 392 365 -27
Lines 35614 34624 -990
Branches 5546 5392 -154
===========================================
- Hits 1702 1694 -8
+ Misses 33663 32681 -982
Partials 249 249
Continue to review full report at Codecov.
|
fd2f2da to
3643588
Compare
Contributor
Author
|
@agapple 冲突已解决，殷盼管理员百忙之中抽空 review 一下。 |
|
|
bef08c8 to
b79af46
Compare
Member
|
tks |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
issue: #2173
canal 向 mq 发送消息时，可以根据不同的 topic 配置相应的 partitionNum
在 instance.properties 配置文件中新增如下配置项：
#canal.mq.dynamicTopicPartitionNum=test.*:4,mycanal:6
支持以正则表达式的方式，为不同的 topic 指定专属 partitionNum
一条规则中，分号前是 topic 或者 topic 的正则表达式，分号后是 partitionNum
多条规则以逗号分隔
大家可以结合自己的业务需求，设置匹配规则，多条匹配规则之间是按照顺序进行匹配(命中一条规则就返回)
例子1: test.*:4,mycanal:6 前缀为test的topic分区数是4，topic mycanal指定分区数为6
例子2: test1|mycanal:6 指定 topic test1 或者 topic mycanal 分区数为6
例子3: 匹配规则啥都不写，或者未匹配上，则 topic 的默认分区数是 canal.mq.partitionsNum