gh-128881: Do not ignore `address` and `flags` parameters in `socket.{send,recv}_fds` #128882

GalaxySnail · 2025-01-15T14:43:18Z

fix 8d120f7

Issue: address and flags are ignored in socket.send_fds and socket.recv_fds #128881

GalaxySnail · 2025-01-15T15:21:29Z

I have no idea what's happening on macOS. Should we just skip it on macOS?

Zheaoli · 2025-01-16T04:39:00Z

MSG_PEEK may got different action on macOS/FreeBSD, let's figure it out.

BTW, we can skip the macOS test first

picnixz

Can we also have a test that combine both address and flags as well for send_fds please? ~~and see if we can have test coverage for Windows (just using a 0 flag for instance would be sufficient).~~

According to https://docs.python.org/3/library/socket.html#socket.send_fds, send_fds and recv_fds are is available on Windows and Unix, so we should be able to have tests for them as well.

Misc/NEWS.d/next/Library/2025-01-15-22-50-41.gh-issue-128881.JBL_9E.rst

Lib/test/test_socket.py

Co-authored-by: Bénédikt Tran <[email protected]>

GalaxySnail · 2025-06-02T04:21:50Z

I found that FreeBSD doesn't seem to support receiving only 1 file descriptor with recvmsg. Using CMSG_LEN(sizeof(int)) leads to MSG_CTRUNC, while CMSG_LEN(2 * sizeof(int)) is required to successfully receive a single file descriptor.

Here is a POC: https://fars.ee/Ih-8/c

$ curl https://fars.ee/Ih-8 -o foo.c
$ cc -Wall -Wextra -Wpedantic -std=c17 foo.c && truss -o out.txt ./a.out && ggrep -E 'sendmsg|recvmsg' out.txt
sendmsg(3,{NULL,0,[{"Hello world!",12}],1,{{level=SOL_SOCKET,type=SCM_RIGHTS,data={0x01,0x00,0x00,0x00}}},20,0},0) = 12 (0xc)
recvmsg(4,{NULL,0,[{"Hello world!",12}],1,{},0,MSG_CTRUNC},0) = 12 (0xc)
$ gsed -i 's|fds\[\] = {1}|fds[] = {1, 2}|' foo.c
$ cc -Wall -Wextra -Wpedantic -std=c17 foo.c && truss -o out.txt ./a.out && ggrep -E 'sendmsg|recvmsg' out.txt
sendmsg(3,{NULL,0,[{"Hello world!",12}],1,{{level=SOL_SOCKET,type=SCM_RIGHTS,data={0x01,0x00,0x00,0x00,0x02,0x00,0x00,0x00}}},24,0},0) = 12 (0xc)
recvmsg(4,{NULL,0,[{"Hello world!",12}],1,{{level=SOL_SOCKET,type=SCM_RIGHTS,data={0x05,0x00,0x00,0x00,0x06,0x00,0x00,0x00}}},24,0},0) = 12 (0xc)

This reverts commit b130108.

Lib/test/test_socket.py

update comments Co-authored-by: Bénédikt Tran <[email protected]>

picnixz

We are close to the end!

picnixz · 2025-06-22T08:01:52Z

Lib/test/test_socket.py

+        if sys.platform.startswith("freebsd"):
+            # FreeBSD requires at least CMSG_LEN(2*sizeof(int)),
+            # otherwise the access control message is truncated.
+            max_fds = 2
+        else:
+            max_fds = 1


Suggested change

if sys.platform.startswith("freebsd"):

# FreeBSD requires at least CMSG_LEN(2*sizeof(int)),

# otherwise the access control message is truncated.

max_fds = 2

else:

max_fds = 1

# FreeBSD requires at least CMSG_LEN(2*sizeof(int)),

# otherwise the access control message is truncated.

max_fds = 2 if sys.platform.startswith("freebsd") else 1

picnixz · 2025-06-22T08:05:42Z

Lib/socket.py

@@ -579,7 +580,7 @@ def recv_fds(sock, bufsize, maxfds, flags=0):
        # Array of ints
        fds = array.array("i")
        msg, ancdata, flags, addr = sock.recvmsg(bufsize,


Let's rename the result flags as msg_flags to be consistent with the signature (don't forget to update return ...)

picnixz · 2025-06-22T08:10:38Z

Lib/test/test_socket.py

+        assert len(msg) < 512
+        os.write(wfd, msg)
+        data = os.read(rfd, 512)
+        self.assertEqual(data, msg)


Suggested change

assert len(msg) < 512

os.write(wfd, msg)

data = os.read(rfd, 512)

self.assertEqual(data, msg)

os.write(wfd, msg)

data = os.read(rfd, len(msg))

self.assertEqual(data, msg)

I think we can do it like this right?

picnixz · 2025-06-22T08:10:59Z

Lib/test/test_socket.py

-        for index, fds in enumerate(pipes):
-            rfd, wfd = fds
-            os.write(wfd, str(index).encode())
+        for index, ((_, wfd), rfd) in enumerate(zip(pipes, fds2)):


Suggested change

for index, ((_, wfd), rfd) in enumerate(zip(pipes, fds2)):

for index, ((_, wfd), rfd) in enumerate(zip(pipes, fds2, strict=True)):

picnixz · 2025-06-22T08:11:27Z

Lib/test/test_socket.py

+            peek_len = len(MSG) // 2
+            msg, fds, flags, addr = socket.recv_fds(sock2, peek_len, 1,


Suggested change

peek_len = len(MSG) // 2

msg, fds, flags, addr = socket.recv_fds(sock2, peek_len, 1,

peek_len = len(MSG) // 2

self.assertGreater(peek_len, 0)

msg, fds, flags, addr = socket.recv_fds(sock2, peek_len, 1,

picnixz · 2025-06-22T08:12:46Z

Lib/test/test_socket.py

+                for _ in range(64 * 1024):
+                    socket.send_fds(sock1, [MSG], [rfd], socket.MSG_DONTWAIT)
+
+            msg, fds, flags, addr = self._recv_one_fd(sock2, len(MSG))


Let's also check flags below

picnixz · 2025-06-22T08:16:16Z

Lib/test/test_socket.py

+            self._test_pipe(fds[0], wfd, MSG)
+
+    @requireAttrs(socket, "MSG_DONTWAIT")
+    @unittest.skipIf(sys.platform in ("darwin",),


Above, the test for sendmsg only enabled it for @skipWithClientIf(sys.platform not in {"linux", "android", "freebsd"}. I think it's better if we check this here as well because there might be more platforms than {"linux", "android", "freebsd", "darwin"} in the future.

Or: we change the test above with @skipWithClientIf(sys.platform not in ("darwin",) instead and add @requireAttrs(socket, "MSG_DONTWAIT")

picnixz · 2025-06-22T08:17:48Z

Lib/test/test_socket.py


+    def testSendAndRecvFds(self):


Suggested change

def testSendAndRecvFds(self):

def test_send_and_recv_fds(self):

picnixz · 2025-06-22T08:19:13Z

Lib/test/test_socket.py

+        self._test_pipe(fds[0], wfd, MSG)
+
+    @requireAttrs(socket, "MSG_PEEK")
+    @unittest.skipUnless(sys.platform in ("linux", "android"), "works on Linux")


I'm back here, but does it work on freebsd?

Fix address and flags for send_fds/recv_fds

b9ae8e2

bedevere-app bot mentioned this pull request Jan 15, 2025

address and flags are ignored in socket.send_fds and socket.recv_fds #128881

Open

bedevere-app bot added the awaiting review label Jan 15, 2025

GalaxySnail added 2 commits January 15, 2025 22:51

Add NEWS

45b28f6

Only test MSG_DONTWAIT on Linux

fbb2668

GalaxySnail added 2 commits January 17, 2025 00:30

Fix cleanup fds

c58d2de

Skip platform-dependent tests

aefccca

picnixz reviewed Jan 17, 2025

View reviewed changes

picnixz changed the title ~~gh-128881: Fix address and flags for send_fds/recv_fds~~ gh-128881: Do not ignore address and flags parameters in socket.{send,recv}_fds Jan 17, 2025

Apply suggestions from code review

6b58f4b

Co-authored-by: Bénédikt Tran <[email protected]>

GalaxySnail mentioned this pull request May 31, 2025

gh-122153: indicate that Windows does not support socket.{send,recv}_fds #134960

Merged

GalaxySnail added 3 commits June 2, 2025 11:15

Merge branch 'main' into fix_send_recv_fds

486e18c

fix test on FreeBSD

0134f01

fix test_send_fds_dontwait on *BSD

5cca1b4

GalaxySnail added 3 commits June 2, 2025 15:30

skip sendmsg with MSG_DONTWAIT test on macOS

6c226c4

try enabling test_recv_fds_peek on macOS

b130108

Revert "try enabling test_recv_fds_peek on macOS"

bb1c5ad

This reverts commit b130108.

GalaxySnail requested a review from picnixz June 2, 2025 08:18

picnixz reviewed Jun 2, 2025

View reviewed changes

Lib/test/test_socket.py Outdated Show resolved Hide resolved

Lib/test/test_socket.py Outdated Show resolved Hide resolved

Lib/test/test_socket.py Outdated Show resolved Hide resolved

Lib/test/test_socket.py Outdated Show resolved Hide resolved

Lib/test/test_socket.py Show resolved Hide resolved

GalaxySnail and others added 3 commits June 2, 2025 18:46

Apply suggestions from code review

02b0ddd

update comments Co-authored-by: Bénédikt Tran <[email protected]>

extract the logic into _recv_one_fd method

7368656

make sockets live only as little as possible

4f8d1ca

GalaxySnail requested a review from picnixz June 2, 2025 13:41

picnixz reviewed Jun 22, 2025

View reviewed changes

	for index, ((_, wfd), rfd) in enumerate(zip(pipes, fds2)):
	for index, ((_, wfd), rfd) in enumerate(zip(pipes, fds2, strict=True)):

		peek_len = len(MSG) // 2
		msg, fds, flags, addr = socket.recv_fds(sock2, peek_len, 1,

	def testSendAndRecvFds(self):
	def test_send_and_recv_fds(self):

Uh oh!

gh-128881: Do not ignore address and flags parameters in socket.{send,recv}_fds #128882

Are you sure you want to change the base?

gh-128881: Do not ignore address and flags parameters in socket.{send,recv}_fds #128882

Conversation

GalaxySnail commented Jan 15, 2025 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

GalaxySnail commented Jan 15, 2025

Uh oh!

Zheaoli commented Jan 16, 2025

Uh oh!

picnixz left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

GalaxySnail commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gh-128881: Do not ignore `address` and `flags` parameters in `socket.{send,recv}_fds` #128882

gh-128881: Do not ignore `address` and `flags` parameters in `socket.{send,recv}_fds` #128882

GalaxySnail commented Jan 15, 2025 •

edited by bedevere-app bot

Loading

picnixz left a comment •

edited

Loading

GalaxySnail commented Jun 2, 2025 •

edited

Loading