gh-128881: Do not ignore `address` and `flags` parameters in `socket.{send,recv}_fds` #128882

GalaxySnail · 2025-01-15T14:43:18Z

fix 8d120f7

Issue: address and flags are ignored in socket.send_fds and socket.recv_fds #128881

GalaxySnail · 2025-01-15T15:21:29Z

I have no idea what's happening on macOS. Should we just skip it on macOS?

Zheaoli · 2025-01-16T04:39:00Z

MSG_PEEK may got different action on macOS/FreeBSD, let's figure it out.

BTW, we can skip the macOS test first

picnixz

Can we also have a test that combine both address and flags as well for send_fds please? ~~and see if we can have test coverage for Windows (just using a 0 flag for instance would be sufficient).~~

According to https://docs.python.org/3/library/socket.html#socket.send_fds, send_fds and recv_fds are is available on Windows and Unix, so we should be able to have tests for them as well.

Misc/NEWS.d/next/Library/2025-01-15-22-50-41.gh-issue-128881.JBL_9E.rst

Lib/test/test_socket.py

Co-authored-by: Bénédikt Tran <[email protected]>

GalaxySnail · 2025-06-02T04:21:50Z

I found that FreeBSD doesn't seem to support receiving only 1 file descriptor with recvmsg. Using CMSG_LEN(sizeof(int)) leads to MSG_CTRUNC, while CMSG_LEN(2 * sizeof(int)) is required to successfully receive a single file descriptor.

Here is a POC: https://fars.ee/Ih-8/c

$ curl https://fars.ee/Ih-8 -o foo.c
$ cc -Wall -Wextra -Wpedantic -std=c17 foo.c && truss -o out.txt ./a.out && ggrep -E 'sendmsg|recvmsg' out.txt
sendmsg(3,{NULL,0,[{"Hello world!",12}],1,{{level=SOL_SOCKET,type=SCM_RIGHTS,data={0x01,0x00,0x00,0x00}}},20,0},0) = 12 (0xc)
recvmsg(4,{NULL,0,[{"Hello world!",12}],1,{},0,MSG_CTRUNC},0) = 12 (0xc)
$ gsed -i 's|fds\[\] = {1}|fds[] = {1, 2}|' foo.c
$ cc -Wall -Wextra -Wpedantic -std=c17 foo.c && truss -o out.txt ./a.out && ggrep -E 'sendmsg|recvmsg' out.txt
sendmsg(3,{NULL,0,[{"Hello world!",12}],1,{{level=SOL_SOCKET,type=SCM_RIGHTS,data={0x01,0x00,0x00,0x00,0x02,0x00,0x00,0x00}}},24,0},0) = 12 (0xc)
recvmsg(4,{NULL,0,[{"Hello world!",12}],1,{{level=SOL_SOCKET,type=SCM_RIGHTS,data={0x05,0x00,0x00,0x00,0x06,0x00,0x00,0x00}}},24,0},0) = 12 (0xc)

This reverts commit b130108.

Lib/test/test_socket.py

update comments Co-authored-by: Bénédikt Tran <[email protected]>

picnixz

We are close to the end!

Lib/test/test_socket.py

picnixz · 2025-06-22T08:05:42Z

Lib/socket.py

@@ -579,7 +580,7 @@ def recv_fds(sock, bufsize, maxfds, flags=0):
        # Array of ints
        fds = array.array("i")
        msg, ancdata, flags, addr = sock.recvmsg(bufsize,


Let's rename the result flags as msg_flags to be consistent with the signature (don't forget to update return ...)

picnixz · 2025-06-22T08:10:38Z

Lib/test/test_socket.py

+        assert len(msg) < 512
+        os.write(wfd, msg)
+        data = os.read(rfd, 512)
+        self.assertEqual(data, msg)


Suggested change

assert len(msg) < 512

os.write(wfd, msg)

data = os.read(rfd, 512)

self.assertEqual(data, msg)

os.write(wfd, msg)

data = os.read(rfd, len(msg))

self.assertEqual(data, msg)

I think we can do it like this right?

man 7 pipe says:

POSIX.1 says that writes of less than PIPE_BUF bytes must be atomic: the output data is written to the pipe as a contiguous sequence. [...] POSIX.1 requires PIPE_BUF to be at least 512 bytes.

I hardcoded 512 here, which may look confusing. I've updated it to make it more clear.

Lib/test/test_socket.py

picnixz · 2025-06-22T08:12:46Z

Lib/test/test_socket.py

+                for _ in range(64 * 1024):
+                    socket.send_fds(sock1, [MSG], [rfd], socket.MSG_DONTWAIT)
+
+            msg, fds, flags, addr = self._recv_one_fd(sock2, len(MSG))


Let's also check flags below

picnixz · 2025-06-22T08:16:16Z

Lib/test/test_socket.py

+            self._test_pipe(fds[0], wfd, MSG)
+
+    @requireAttrs(socket, "MSG_DONTWAIT")
+    @unittest.skipIf(sys.platform in ("darwin",),


Above, the test for sendmsg only enabled it for @skipWithClientIf(sys.platform not in {"linux", "android", "freebsd"}. I think it's better if we check this here as well because there might be more platforms than {"linux", "android", "freebsd", "darwin"} in the future.

Or: we change the test above with @skipWithClientIf(sys.platform not in ("darwin",) instead and add @requireAttrs(socket, "MSG_DONTWAIT")

According to PEP 11, it should be safe to replace @skipWithClientIf(sys.platform not in {"linux", "android", "freebsd"}) with @requireAttrs(socket, "MSG_DONTWAIT") and @skipWithClientIf(sys.platform not in ("darwin",)) on all supported platforms. If this test fails on any other platform, we can update it as needed.

Lib/test/test_socket.py

picnixz · 2025-06-22T08:19:13Z

Lib/test/test_socket.py

+        self._test_pipe(fds[0], wfd, MSG)
+
+    @requireAttrs(socket, "MSG_PEEK")
+    @unittest.skipUnless(sys.platform in ("linux", "android"), "works on Linux")


I'm back here, but does it work on freebsd?

Unfortunately no. On FreeBSD recvmsg with MSG_PEEK returns random bytes in the ancillary data, which looks like some kind of uninitialized memory.

Co-authored-by: Bénédikt Tran <[email protected]>

Fix address and flags for send_fds/recv_fds

b9ae8e2

bedevere-app bot mentioned this pull request Jan 15, 2025

address and flags are ignored in socket.send_fds and socket.recv_fds #128881

Open

bedevere-app bot added the awaiting review label Jan 15, 2025

GalaxySnail added 2 commits January 15, 2025 22:51

Add NEWS

45b28f6

Only test MSG_DONTWAIT on Linux

fbb2668

GalaxySnail added 2 commits January 17, 2025 00:30

Fix cleanup fds

c58d2de

Skip platform-dependent tests

aefccca

picnixz reviewed Jan 17, 2025

View reviewed changes

picnixz changed the title ~~gh-128881: Fix address and flags for send_fds/recv_fds~~ gh-128881: Do not ignore address and flags parameters in socket.{send,recv}_fds Jan 17, 2025

Apply suggestions from code review

6b58f4b

Co-authored-by: Bénédikt Tran <[email protected]>

GalaxySnail mentioned this pull request May 31, 2025

gh-122153: indicate that Windows does not support socket.{send,recv}_fds #134960

Merged

GalaxySnail added 3 commits June 2, 2025 11:15

Merge branch 'main' into fix_send_recv_fds

486e18c

fix test on FreeBSD

0134f01

fix test_send_fds_dontwait on *BSD

5cca1b4

GalaxySnail added 3 commits June 2, 2025 15:30

skip sendmsg with MSG_DONTWAIT test on macOS

6c226c4

try enabling test_recv_fds_peek on macOS

b130108

Revert "try enabling test_recv_fds_peek on macOS"

bb1c5ad

This reverts commit b130108.

GalaxySnail requested a review from picnixz June 2, 2025 08:18

picnixz reviewed Jun 2, 2025

View reviewed changes

Lib/test/test_socket.py Outdated Show resolved Hide resolved

Lib/test/test_socket.py Outdated Show resolved Hide resolved

Lib/test/test_socket.py Outdated Show resolved Hide resolved

Lib/test/test_socket.py Outdated Show resolved Hide resolved

Lib/test/test_socket.py Show resolved Hide resolved

GalaxySnail and others added 3 commits June 2, 2025 18:46

Apply suggestions from code review

02b0ddd

update comments Co-authored-by: Bénédikt Tran <[email protected]>

extract the logic into _recv_one_fd method

7368656

make sockets live only as little as possible

4f8d1ca

GalaxySnail requested a review from picnixz June 2, 2025 13:41

picnixz reviewed Jun 22, 2025

View reviewed changes

GalaxySnail and others added 3 commits June 22, 2025 17:56

Apply suggestions from code review

c423723

Co-authored-by: Bénédikt Tran <[email protected]>

rename flags to msg_flags

1dcd26f

also check flags in test_send_fds_dontwait

5d3e896

GalaxySnail added 2 commits June 22, 2025 18:27

make 2 tests of MSG_DONTWAIT to be consistent

d37ba07

clarify PIPE_BUF in _test_pipe

f4c9d67

Uh oh!

gh-128881: Do not ignore address and flags parameters in socket.{send,recv}_fds #128882

Are you sure you want to change the base?

gh-128881: Do not ignore address and flags parameters in socket.{send,recv}_fds #128882

Conversation

GalaxySnail commented Jan 15, 2025 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

GalaxySnail commented Jan 15, 2025

Uh oh!

Zheaoli commented Jan 16, 2025

Uh oh!

picnixz left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

GalaxySnail commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gh-128881: Do not ignore `address` and `flags` parameters in `socket.{send,recv}_fds` #128882

gh-128881: Do not ignore `address` and `flags` parameters in `socket.{send,recv}_fds` #128882

GalaxySnail commented Jan 15, 2025 •

edited by bedevere-app bot

Loading

picnixz left a comment •

edited

Loading

GalaxySnail commented Jun 2, 2025 •

edited

Loading