Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@Shmillerov
Copy link
Contributor

@Shmillerov Shmillerov commented Sep 20, 2024

1. Why is this pull request needed and what does it do?

Added the ability to run multiple servers on the same port using the multisocket plugin. This increases the throughput of CoreDNS in environments with a large number of CPU cores.

Syntax:

multisocket [NUM_SOCKETS]

default value of NUM_SOCKETS is equal to GOMAXPROCS.

Performance testing:
#6882 (comment)

2. Which issues (if any) are related?

#5595

3. Which documentation changes (if any) need to be made?

I added a readme for the plugin. After merging to CoreDNS repo, need to re-generate https://coredns.io/plugins/ using https://github.com/coredns/coredns.io

4. Does this introduce a backward incompatible change or deprecation?

Changes are backward compatible

SuperQ
SuperQ previously requested changes Sep 20, 2024
Copy link
Collaborator

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we want to implement this. Multi-process makes a number of things much worse. For example monitoring metrics and similar things.

I think we should put more effort into making multi-threaded scaling better rather than paper over the issue with multiple processes.

@chrisohaver
Copy link
Member

This is more or less doable currently by running multiple instances of coredns.

@johnbelamaric
Copy link
Member

I think we should put more effort into making multi-threaded scaling better rather than paper over the issue with multiple processes.

Isn't that what this does? It allows multiple internal servers (same process) to run on the same address and port. Basically, in #5595 it was determined that the single socket used to read the queries was a bottleneck; this alleviates that, IIUC.

@Shmillerov
Copy link
Contributor Author

Let me tell you about our scenario.
We want to run СoreDNS as a DaemonSet in Kubernetes. This means that there is only one pod per node. We can't scale CoreDNS horizontally in that case. And vertically too, because no matter how many resources you add, the CoreDNS will not perform better (#5595)

This fix allows to scale CoreDNS vertically.

This is more or less doable currently by running multiple instances of coredns.

In general, I agree. But it is more difficult to control these instances.

If we run several processes in a container, it is not clear how to detect main process. If the instance goes down, who will restart it? We would like to have one process so that Kubernetes itself controls the lifecycle of our container.

@SuperQ
Copy link
Collaborator

SuperQ commented Sep 23, 2024

Sorry, I clearly didn't read the code / proposal well enough. Setting up multiple listeners from the same process is the goal.

I think this is fine for now, at least until Go has better support for this.

@chrisohaver
Copy link
Member

Perhaps we can consider an alternate plugin name? “reuseport” is a bit confusing given that coredns already binds using the reuseport option. Perhaps “multiprocess” or “multiserver” or something.

@johnbelamaric
Copy link
Member

Perhaps we can consider an alternate plugin name? “reuseport” is a bit confusing given that coredns already binds using the reuseport option. Perhaps “multiprocess” or “multiserver” or something.

"numsockets" would most clearly describe what it does, I think. It's not about multiple processes.

@glebkin
Copy link
Contributor

glebkin commented Sep 23, 2024

Hi!

Please take into account, that we also need additional changes in Caddy server - coredns/caddy#6 to address this issue - #5595 (comment) with performance degradation after server reload.

@Shmillerov Shmillerov changed the title reuseport plugin numsockets plugin Sep 23, 2024
@SuperQ
Copy link
Collaborator

SuperQ commented Sep 23, 2024

Maybe this should be a command line flag and not something reloadable?

@Shmillerov
Copy link
Contributor Author

Maybe this should be a command line flag and not something reloadable?

During reloading, the server restarts. When restarting, all servers start listening to a single socket because here: https://github.com/coredns/caddy/blob/master/caddy.go#L221, we have a map of server addresses to PacketConn.

With numsockets, we are starting multiple servers on the same address. Therefore, reloading is a a problem for us.

If we want this functionality with numsockets, we need to fix this somehow. It doesn't matter whether it is a plugin or a CLI flag.

@glebkin
Copy link
Contributor

glebkin commented Sep 26, 2024

Can anyone please take a look at this PR - coredns/caddy#6?

@johnbelamaric
Copy link
Member

@Shmillerov if you happen to be at KubeCon NA in Salt Lake City next month, would be great if you want to talk about this in the CoreDNS maintainer track session. Otherwise I will cover it. Great stuff.

@Shmillerov Shmillerov mentioned this pull request Oct 30, 2024
@Shmillerov
Copy link
Contributor Author

@johnbelamaric, thank you, but I can't participate :) I believe you will cover me!

@Shmillerov Shmillerov changed the title numsockets plugin multisocket plugin Nov 6, 2024
Shmillerov and others added 12 commits November 6, 2024 14:54
Signed-off-by: Viktor Rodionov <[email protected]>
- update doc

Signed-off-by: Viktor Rodionov <[email protected]>
Signed-off-by: Viktor Rodionov <[email protected]>
Signed-off-by: Viktor Rodionov <[email protected]>
Signed-off-by: Viktor Rodionov <[email protected]>
- default as GOMAXPROCS
- update README

Signed-off-by: Viktor Rodionov <[email protected]>
Signed-off-by: Viktor Rodionov <[email protected]>
@johnbelamaric
Copy link
Member

@Shmillerov just FYI since I don't think you are on the CoreDNS Slack. We are hoping to do the following in the next few days:

  • merge low risk PRs and cut a patch release
  • merge this (and maybe others?) and cut a minor release

Copy link
Member

@johnbelamaric johnbelamaric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not merge until we cut a patch release with low-risk changes first.

@johnbelamaric johnbelamaric requested a review from SuperQ November 6, 2024 17:21
@johnbelamaric johnbelamaric dismissed SuperQ’s stale review November 13, 2024 17:38

Already resolved

@johnbelamaric johnbelamaric merged commit 6c39f4b into coredns:master Nov 13, 2024
12 checks passed
thevilledev added a commit to thevilledev/coredns that referenced this pull request May 14, 2025
Missed from coredns#6882

Signed-off-by: Ville Vesilehto <[email protected]>
yongtang pushed a commit that referenced this pull request May 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants