Netmap bindings for libuv #3
Merged
+1,020
−1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Netmap is a project that gives user-space networking to linux applications. Netmap binds to the network interface, and traffic steering rules (generated by ethtool) drive packets into user-specific receive queues. This completely bypasses the kernel except for NIC IRQs. Because only some queues are controlled in userspace, the kernel continues to run other functionality like pings and ssh. In practice we find this yields approximately 5x more efficient packet receives. https://blog.cloudflare.com/single-rx-queue-kernel-bypass-with-netmap/ This patch brings netmap to libuv for UDP packets. We connect the event loop to a single netmap queue. When the `use_netmap` flag is turned on for new udp sockets, these sockets bypass the kernel calls and instead operate on top of the netmap rings.
adill
reviewed
Aug 7, 2019
adill
reviewed
Aug 7, 2019
adill
requested changes
Aug 8, 2019
There was a problem hiding this comment.
I think this generally looks good 👍 with the following caveats / notes:
- It's difficult to thoroughly review this for correctness without deeper knowledge of both libuv and netmap
- A lot of potential cases are not handled (
NULLinputs,memcpybounds validation) and we should consider how pedantic we need to be / should be - The
uv_udp_interface seems incomplete -- maybe that is fine because it looks like in most cases you're failing with-1and what is implemented fits our use case (should we explicitlyassertwhen some functionality is not intended to be used?) - Is it worth your time to add tests which exercise the netmap functionality?
src/unix/netmap-udp.c
Outdated
|
|
||
| udp_chksum = udp->check; | ||
| if (udp_chksum != udp_checksum(ip, udp, payload, payload_len)) { | ||
| printf("udp checksum failed, %x != %x\n", udp_chksum, udp_checksum(ip, udp, payload, payload_len)); |
There was a problem hiding this comment.
didn't mean to leave this printf in, will remove
adill
approved these changes
Aug 12, 2019
| if (intopt > UCHAR_MAX) { | ||
| return -1; | ||
| } | ||
| if (option4 == IP_TOS) { |
There was a problem hiding this comment.
Are you not using option6? It's not clear to me what option4 and option6 are supposed to be. setsockopt calls them level and option_name, yea? Now it looks like option4 == option_name and option6 is not used?
There was a problem hiding this comment.
For some confusing reason, uv__udp_setsockopt is actually setsockopt(IPPROTO_IP, ...) or setsockopt(IPPROTO_IPV6, ...), depending on whether the socket is ipv4 or ipv6. I completely missed this the first time around. Callers are supposed to invoke with the IPv4 and IPv6 options and uv__udp_setsockopt() sorts out which to use.
This fun detail means it's only valid to use uv__udp_setsockopt() when one wants to set IP parameters, and the calling signature is completely different from setsockopt()
snowyote
pushed a commit
that referenced
this pull request
May 24, 2022
ERROR: LeakSanitizer: detected memory leaks
```
Direct leak of 432 byte(s) in 9 object(s) allocated from:
#0 0x1062eedc2 in __sanitizer_mz_calloc+0x92 (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x46dc2)
#1 0x7fff20171eb6 in _malloc_zone_calloc+0x3a (libsystem_malloc.dylib:x86_64+0x1beb6)
#2 0x7fff203ac180 in _CFRuntimeCreateInstance+0x124 (CoreFoundation:x86_64h+0x4180)
#3 0x7fff203ab906 in __CFStringCreateImmutableFunnel3+0x84d (CoreFoundation:x86_64h+0x3906)
#4 0x7fff203ab0a1 in CFStringCreateWithCString+0x48 (CoreFoundation:x86_64h+0x30a1)
#5 0x1056f63e1 in uv__get_cpu_speed darwin.c:267
#6 0x1056f491e in uv_cpu_info darwin.c:338
```
PR-URL: libuv#3098
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Santiago Gimeno <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Netmap is a project that gives user-space networking to linux
applications. Netmap binds to the network interface, and traffic
steering rules (generated by ethtool) drive packets into user-specific
receive queues. This completely bypasses the kernel except for NIC IRQs.
Because only some queues are controlled in userspace, the kernel
continues to run other functionality like pings and ssh. In practice we
find this yields approximately 5x more efficient packet receives.
https://blog.cloudflare.com/single-rx-queue-kernel-bypass-with-netmap/
This patch brings netmap to libuv for UDP packets. We connect the event
loop to a single netmap queue. When the
use_netmapflag is turned onfor new udp sockets, these sockets bypass the kernel calls and instead
operate on top of the netmap rings.